2024-03-22T06:47:25.889 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-22T06:47:25.889 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012 --owner scheduled_teuthology@teuthology --description upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012/orig.config.yaml 2024-03-22T06:47:25.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T06:47:26.053 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012 branch: squid description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7616012' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 768f6cd6e95d0c7a626042719c230df59bdba9d3 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 3446 sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 sleep_before_teardown: 0 subset: 21/32 suite: upgrade suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_768f6cd6e95d0c7a626042719c230df59bdba9d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 768f6cd6e95d0c7a626042719c230df59bdba9d3 targets: smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwNgaw9IvHANiy323QcUSEv0eajn27jRpU8ksYOYRdnRnSKvguX+wrRKylK2EcXxprsG1eAliPTRlvrgTCpyAU= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVRdHgXkB1iKextmBEkFFRabpyekhTPCapgDe+3fOyrSJRkuY90+1wptaQFt/JnWt2FpZO/lWsSAOcgDApsGmY= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_02:08:13 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-22T06:47:26.053 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_768f6cd6e95d0c7a626042719c230df59bdba9d3/qa; will attempt to use it 2024-03-22T06:47:26.054 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_768f6cd6e95d0c7a626042719c230df59bdba9d3/qa/tasks 2024-03-22T06:47:26.054 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-22T06:47:26.056 INFO:teuthology.task.internal:Checking packages... 2024-03-22T06:47:26.081 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'db0330b1e4e2470d52b750e251e55a522b4f7d69' 2024-03-22T06:47:26.081 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:47:26.081 INFO:teuthology.packaging:ref: None 2024-03-22T06:47:26.081 INFO:teuthology.packaging:tag: None 2024-03-22T06:47:26.081 INFO:teuthology.packaging:branch: squid 2024-03-22T06:47:26.081 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T06:47:26.082 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-22T06:47:26.306 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1667.gdb0330b1 2024-03-22T06:47:26.307 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-22T06:47:26.322 INFO:teuthology.task.internal:no buildpackages task found 2024-03-22T06:47:26.322 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-22T06:47:26.337 INFO:teuthology.task.internal:Saving configuration 2024-03-22T06:47:26.358 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-22T06:47:26.368 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T06:47:26.395 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 06:42:18.102830', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwNgaw9IvHANiy323QcUSEv0eajn27jRpU8ksYOYRdnRnSKvguX+wrRKylK2EcXxprsG1eAliPTRlvrgTCpyAU='} 2024-03-22T06:47:26.418 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 06:42:18.104714', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVRdHgXkB1iKextmBEkFFRabpyekhTPCapgDe+3fOyrSJRkuY90+1wptaQFt/JnWt2FpZO/lWsSAOcgDApsGmY='} 2024-03-22T06:47:26.418 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-22T06:47:26.429 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-22T06:47:26.429 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-22T06:47:26.429 INFO:teuthology.run_tasks:Running task console_log... 2024-03-22T06:47:26.514 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcce062c0d0>, signals=[15]) 2024-03-22T06:47:26.515 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-22T06:47:26.525 INFO:teuthology.task.internal:Opening connections... 2024-03-22T06:47:26.526 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T06:47:26.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:47:26.623 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-03-22T06:47:26.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:47:26.716 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-22T06:47:26.727 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-03-22T06:47:26.745 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-03-22T06:47:26.746 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-03-22T06:47:26.802 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-03-22T06:47:26.803 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T06:47:26.804 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-22T06:47:26.804 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-22T06:47:26.804 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-03-22T06:47:26.826 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-03-22T06:47:26.844 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-03-22T06:47:26.844 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-03-22T06:47:26.900 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-03-22T06:47:26.900 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-03-22T06:47:26.900 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-03-22T06:47:26.900 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-03-22T06:47:26.900 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-03-22T06:47:26.900 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-03-22T06:47:26.900 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-22T06:47:26.900 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-03-22T06:47:26.901 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-03-22T06:47:26.901 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-22T06:47:26.901 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-03-22T06:47:26.901 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T06:47:26.901 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-22T06:47:26.901 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-22T06:47:26.901 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-03-22T06:47:26.926 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-22T06:47:26.941 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-22T06:47:26.954 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-22T06:47:26.955 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-03-22T06:47:26.957 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-03-22T06:47:26.973 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-22T06:47:26.986 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-22T06:47:26.986 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-03-22T06:47:27.014 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-03-22T06:47:27.320 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-22T06:47:27.381 INFO:teuthology.run_tasks:Running task kernel... 2024-03-22T06:47:27.402 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-22T06:47:27.402 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-22T06:47:27.402 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T06:47:27.403 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T06:47:27.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:47:27.419 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-03-22T06:47:27.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:47:27.420 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-03-22T06:47:27.476 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-361.el9.x86_64 2024-03-22T06:47:27.476 INFO:teuthology.task.kernel:Running kernel on smithi081: 5.14.0-361.el9.x86_64 2024-03-22T06:47:27.476 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum install -y kernel 2024-03-22T06:47:27.477 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-361.el9.x86_64 2024-03-22T06:47:27.478 INFO:teuthology.task.kernel:Running kernel on smithi149: 5.14.0-361.el9.x86_64 2024-03-22T06:47:27.478 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum install -y kernel 2024-03-22T06:47:29.614 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-22T06:47:29.654 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-22T06:47:31.231 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-03-22T06:47:31.280 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-22T06:47:32.214 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-22T06:47:32.498 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.0 MB 00:00 2024-03-22T06:47:39.739 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 2.9 MB/s | 19 MB 00:06 2024-03-22T06:47:43.606 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.1 MB 00:00 2024-03-22T06:47:45.049 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-03-22T06:47:45.901 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 28 MB/s | 21 MB 00:00 2024-03-22T06:47:50.851 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-03-22T06:47:52.907 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-22T06:47:52.908 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-22T06:47:53.006 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:47:53.015 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:47:53.015 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-03-22T06:47:53.015 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:47:53.015 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout:Install 4 Packages 2024-03-22T06:47:53.016 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:47:53.017 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 76 M 2024-03-22T06:47:53.017 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 123 M 2024-03-22T06:47:53.017 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:47:53.411 INFO:teuthology.orchestra.run.smithi081.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 159 kB/s | 44 kB 00:00 2024-03-22T06:47:54.952 INFO:teuthology.orchestra.run.smithi081.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 8.2 MB/s | 15 MB 00:01 2024-03-22T06:47:55.786 INFO:teuthology.orchestra.run.smithi081.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 12 MB/s | 27 MB 00:02 2024-03-22T06:47:56.094 INFO:teuthology.orchestra.run.smithi081.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 11 MB/s | 33 MB 00:02 2024-03-22T06:47:56.097 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:47:56.098 INFO:teuthology.orchestra.run.smithi081.stdout:Total 25 MB/s | 76 MB 00:03 2024-03-22T06:47:56.764 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:47:57.123 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:47:57.123 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:47:58.139 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:47:58.140 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:48:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:48:01.282 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-22T06:48:01.362 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:48:04.080 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:48:04.216 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:48:08.946 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:48:08.983 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:48:13.716 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:48:18.524 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 430 kB/s | 19 MB 00:44 2024-03-22T06:48:24.133 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.1 MB 00:02 2024-03-22T06:48:25.683 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-03-22T06:48:27.878 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 21 MB 00:02 2024-03-22T06:48:32.818 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-03-22T06:48:34.933 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-22T06:48:34.933 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-22T06:48:35.019 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:48:35.026 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:48:35.026 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-03-22T06:48:35.026 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:48:35.026 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:48:35.026 INFO:teuthology.orchestra.run.smithi149.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-22T06:48:35.026 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-22T06:48:35.026 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout:Install 4 Packages 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 76 M 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 123 M 2024-03-22T06:48:35.027 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:48:35.702 INFO:teuthology.orchestra.run.smithi149.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 121 kB/s | 44 kB 00:00 2024-03-22T06:48:38.244 INFO:teuthology.orchestra.run.smithi149.stdout:(2/4): kernel-modules-core-5.14.0-430.el9.x86_6 11 MB/s | 27 MB 00:02 2024-03-22T06:48:38.419 INFO:teuthology.orchestra.run.smithi149.stdout:(3/4): kernel-core-5.14.0-430.el9.x86_64.rpm 4.8 MB/s | 15 MB 00:03 2024-03-22T06:48:39.977 INFO:teuthology.orchestra.run.smithi149.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 7.2 MB/s | 33 MB 00:04 2024-03-22T06:48:39.980 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:48:39.981 INFO:teuthology.orchestra.run.smithi149.stdout:Total 15 MB/s | 76 MB 00:04 2024-03-22T06:48:40.643 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:48:41.019 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:48:41.020 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:48:42.044 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:48:42.044 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:48:42.983 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:48:42.983 INFO:teuthology.orchestra.run.smithi081.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T06:48:42.983 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:48:43.029 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:48:43.380 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:48:43.380 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-22T06:48:43.380 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:48:43.380 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:48:43.803 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:48:43.804 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:48:43.804 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:48:43.804 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-22T06:48:43.804 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-22T06:48:43.804 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-22T06:48:43.804 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-22T06:48:43.804 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:48:43.804 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:48:44.111 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T06:48:44.227 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T06:48:44.227 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T06:48:44.228 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-22T06:48:44.228 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-22T06:48:44.228 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-22T06:48:44.228 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-22T06:48:44.228 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=None, version=distro) 2024-03-22T06:48:44.228 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T06:48:44.308 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T06:48:44.308 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T06:48:44.308 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2024-03-22T06:48:44.351 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2024-03-22T06:48:44.351 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2024-03-22T06:48:44.351 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.06 2024-03-22T06:48:44.351 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 68.el9 2024-03-22T06:48:44.351 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2024-03-22T06:48:44.351 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Group : Unspecified 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 8264254 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : builder@centos.org 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:48:44.352 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-22T06:48:44.353 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-22T06:48:44.353 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-22T06:48:44.353 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2024-03-22T06:48:44.353 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:48:44.353 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2024-03-22T06:48:44.354 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-22T06:48:44.354 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 5.14.0-430.el9.x86_64 2024-03-22T06:48:44.354 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-22T06:48:44.525 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:48:45.005 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2024-03-22T06:48:45.226 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-22T06:48:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:48:46.390 INFO:teuthology.orchestra.run.smithi081.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-22T06:48:46.423 INFO:teuthology.orchestra.run.smithi081.stderr:done 2024-03-22T06:48:46.426 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-03-22T06:48:46.441 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.71HGD5hlec 2024-03-22T06:48:46.441 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.71HGD5hlec 2024-03-22T06:48:46.511 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.71HGD5hlec 2024-03-22T06:48:46.654 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.71HGD5hlec is 6KB 2024-03-22T06:48:46.701 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.71HGD5hlec 2024-03-22T06:48:46.715 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2024-03-22T06:48:46.781 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-22T06:48:46.781 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-22T06:48:46.781 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-22T06:48:46.781 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-22T06:48:46.782 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-22T06:48:46.926 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2024-03-22T06:48:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:48:48.280 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:48:53.027 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:48:53.063 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:48:57.776 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:49:16.930 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-22T06:49:16.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-22T06:49:16.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:49:27.167 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:49:27.167 INFO:teuthology.orchestra.run.smithi149.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T06:49:27.167 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:49:27.212 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:49:27.583 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:49:27.583 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-22T06:49:27.583 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:49:27.583 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:49:28.004 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:49:28.005 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:49:28.005 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:49:28.005 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-22T06:49:28.005 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-22T06:49:28.005 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-22T06:49:28.005 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-22T06:49:28.006 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:49:28.006 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:49:28.322 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T06:49:28.423 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T06:49:28.424 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T06:49:28.424 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-22T06:49:28.424 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-22T06:49:28.424 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-22T06:49:28.424 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-22T06:49:28.424 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi149.front.sepia.ceph.com, path=None, version=distro) 2024-03-22T06:49:28.425 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T06:49:28.505 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T06:49:28.506 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T06:49:28.506 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -qi grub2-tools 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:Name : grub2-tools 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:Epoch : 1 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:Version : 2.06 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:Release : 68.el9 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:Architecture: x86_64 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:Group : Unspecified 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:Size : 8264254 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:License : GPLv3+ 2024-03-22T06:49:28.541 INFO:teuthology.orchestra.run.smithi149.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:Packager : builder@centos.org 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:Vendor : CentOS 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:Summary : Support tools for GRUB. 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:Description : 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout:hardware devices. 2024-03-22T06:49:28.542 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:49:28.543 INFO:teuthology.orchestra.run.smithi149.stdout:This subpackage provides tools for support of all platforms. 2024-03-22T06:49:28.543 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-22T06:49:28.544 INFO:teuthology.task.kernel:Updating grub on smithi149 to boot 5.14.0-430.el9.x86_64 2024-03-22T06:49:28.544 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-22T06:49:29.169 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-03-22T06:49:30.513 INFO:teuthology.orchestra.run.smithi149.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-22T06:49:30.549 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-03-22T06:49:30.551 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-03-22T06:49:30.565 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.kXfPgxCyzJ 2024-03-22T06:49:30.565 DEBUG:teuthology.orchestra.run.smithi149:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kXfPgxCyzJ 2024-03-22T06:49:30.634 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.kXfPgxCyzJ 2024-03-22T06:49:30.774 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.kXfPgxCyzJ is 6KB 2024-03-22T06:49:30.822 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.kXfPgxCyzJ 2024-03-22T06:49:30.836 DEBUG:teuthology.orchestra.run.smithi149:> sudo /bin/ls /boot/loader/entries || true 2024-03-22T06:49:30.902 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-22T06:49:30.903 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-22T06:49:30.903 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-22T06:49:30.903 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-22T06:49:30.904 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-22T06:49:31.060 DEBUG:teuthology.orchestra.run.smithi149:> sudo shutdown -r now 2024-03-22T06:49:35.262 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-03-22T06:49:44.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-22T06:49:44.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:49:47.326 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-03-22T06:49:59.338 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-22T06:49:59.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:50:01.063 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-22T06:50:01.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-22T06:50:01.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:50:19.555 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-03-22T06:50:28.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-22T06:50:28.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:50:31.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-03-22T06:50:43.659 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-22T06:50:43.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:50:59.356 DEBUG:teuthology.orchestra.remote:timed out 2024-03-22T06:51:14.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-22T06:51:14.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:51:14.799 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-03-22T06:51:15.169 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-22T06:51:15.169 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-430.el9.x86_64"... 2024-03-22T06:51:15.169 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-03-22T06:51:15.185 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-430.el9.x86_64 2024-03-22T06:51:15.185 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-22T06:51:15.185 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T06:51:15.186 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-22T06:51:16.186 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-22T06:51:16.186 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-22T06:51:16.268 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2024-03-22T06:51:16.300 DEBUG:teuthology.parallel:result is None 2024-03-22T06:51:43.662 DEBUG:teuthology.orchestra.remote:timed out 2024-03-22T06:51:58.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-22T06:51:58.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:51:59.147 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-03-22T06:51:59.543 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-22T06:51:59.543 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-430.el9.x86_64"... 2024-03-22T06:51:59.543 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-03-22T06:51:59.559 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-430.el9.x86_64 2024-03-22T06:51:59.559 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-22T06:51:59.559 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T06:51:59.559 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-22T06:52:00.560 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-22T06:52:00.560 DEBUG:teuthology.orchestra.run.smithi149:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-22T06:52:00.627 INFO:teuthology.orchestra.run.smithi149.stdout:ttyS1 2024-03-22T06:52:00.662 DEBUG:teuthology.parallel:result is None 2024-03-22T06:52:00.662 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-22T06:52:00.672 INFO:teuthology.task.internal:Creating test directory... 2024-03-22T06:52:00.672 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T06:52:00.675 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T06:52:00.691 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-22T06:52:00.700 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-22T06:52:00.709 INFO:teuthology.task.internal:Creating archive directory... 2024-03-22T06:52:00.709 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T06:52:00.733 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T06:52:00.762 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-22T06:52:00.772 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-22T06:52:00.772 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T06:52:00.794 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T06:52:00.828 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T06:52:00.837 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T06:52:00.842 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T06:52:00.850 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T06:52:00.852 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-22T06:52:00.861 INFO:teuthology.task.internal:Configuring sudo... 2024-03-22T06:52:00.861 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T06:52:00.887 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T06:52:00.921 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-22T06:52:00.937 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-22T06:52:00.937 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T06:52:00.956 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T06:52:00.976 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T06:52:01.052 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T06:52:01.100 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T06:52:01.183 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T06:52:01.224 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T06:52:01.224 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T06:52:01.291 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T06:52:01.335 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T06:52:01.397 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T06:52:01.437 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T06:52:01.484 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T06:52:01.484 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T06:52:01.554 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-03-22T06:52:01.556 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-03-22T06:52:01.602 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T06:52:01.640 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T06:52:02.105 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-22T06:52:02.115 INFO:teuthology.task.internal:Starting timer... 2024-03-22T06:52:02.115 INFO:teuthology.run_tasks:Running task pcp... 2024-03-22T06:52:02.131 INFO:teuthology.run_tasks:Running task selinux... 2024-03-22T06:52:02.140 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-22T06:52:02.140 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-03-22T06:52:02.189 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-03-22T06:52:02.191 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-03-22T06:52:02.264 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-03-22T06:52:02.266 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-22T06:52:02.266 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-03-22T06:52:02.300 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-03-22T06:52:02.301 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-03-22T06:52:02.324 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-03-22T06:52:02.324 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi081.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-03-22T06:52:02.324 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T06:52:02.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:52:02.368 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T06:52:02.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:52:02.397 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-22T06:52:02.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-03-22T06:52:02.468 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-03-22T06:52:02.500 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-22T06:52:02.513 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-22T06:52:02.518 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-22T06:52:02.671 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-22T06:52:02.688 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-22T06:52:02.689 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi081.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-22T06:56:29.099 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi081.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-03-22T06:56:29.100 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-22T06:56:29.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:56:29.192 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-03-22T06:56:29.268 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-03-22T06:56:29.268 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-22T06:56:29.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:56:29.360 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-03-22T06:56:29.395 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-22T06:56:29.395 INFO:teuthology.run_tasks:Running task clock... 2024-03-22T06:56:29.405 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-22T06:56:29.406 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T06:56:29.406 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T06:56:29.409 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T06:56:29.409 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T06:56:29.437 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T06:56:29.453 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T06:56:29.467 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T06:56:29.482 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-03-22T06:56:29.483 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T06:56:29.495 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-03-22T06:56:29.512 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T06:56:29.514 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-03-22T06:56:29.528 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-03-22T06:56:29.528 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T06:56:29.544 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T06:56:29.561 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T06:56:29.583 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-03-22T06:56:29.586 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T06:56:29.586 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-03-22T06:56:29.586 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:56:29.587 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:56:29.587 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:56:29.587 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:56:29.613 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-22T06:56:29.616 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T06:56:29.616 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-22T06:56:29.617 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:56:29.617 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:56:29.617 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:56:29.617 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T06:56:29.618 INFO:teuthology.run_tasks:Running task install... 2024-03-22T06:56:29.626 DEBUG:teuthology.task.install:project ceph 2024-03-22T06:56:29.627 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69'}} 2024-03-22T06:56:29.627 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69'} 2024-03-22T06:56:29.627 INFO:teuthology.task.install:Using flavor: default 2024-03-22T06:56:29.635 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-22T06:56:29.635 INFO:teuthology.task.install:extra packages: [] 2024-03-22T06:56:29.636 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69', 'tag': None, 'wait_for_package': False} 2024-03-22T06:56:29.636 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:56:29.636 INFO:teuthology.packaging:ref: None 2024-03-22T06:56:29.636 INFO:teuthology.packaging:tag: None 2024-03-22T06:56:29.636 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:56:29.636 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T06:56:29.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T06:56:29.639 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69', 'tag': None, 'wait_for_package': False} 2024-03-22T06:56:29.639 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:56:29.639 INFO:teuthology.packaging:ref: None 2024-03-22T06:56:29.639 INFO:teuthology.packaging:tag: None 2024-03-22T06:56:29.639 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:56:29.639 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T06:56:29.639 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T06:56:29.963 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/ 2024-03-22T06:56:29.964 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-22T06:56:29.978 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/ 2024-03-22T06:56:29.979 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-22T06:56:30.098 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T06:56:30.098 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T06:56:30.098 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T06:56:30.128 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-22T06:56:30.128 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:56:30.128 INFO:teuthology.packaging:ref: None 2024-03-22T06:56:30.128 INFO:teuthology.packaging:tag: None 2024-03-22T06:56:30.128 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:56:30.128 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T06:56:30.128 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T06:56:30.133 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T06:56:30.133 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T06:56:30.133 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T06:56:30.162 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-22T06:56:30.162 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:56:30.162 INFO:teuthology.packaging:ref: None 2024-03-22T06:56:30.162 INFO:teuthology.packaging:tag: None 2024-03-22T06:56:30.162 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:56:30.162 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T06:56:30.162 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T06:56:30.197 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T06:56:30.230 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T06:56:30.277 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T06:56:30.310 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T06:56:30.344 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2024-03-22T06:56:30.345 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2024-03-22T06:56:30.376 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-03-22T06:56:30.378 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-03-22T06:56:30.588 INFO:teuthology.orchestra.run.smithi081.stdout:74 files removed 2024-03-22T06:56:30.618 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw 2024-03-22T06:56:30.631 INFO:teuthology.orchestra.run.smithi149.stdout:74 files removed 2024-03-22T06:56:30.659 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw 2024-03-22T06:56:31.369 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 191 kB/s | 77 kB 00:00 2024-03-22T06:56:31.377 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 167 kB/s | 77 kB 00:00 2024-03-22T06:56:31.729 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 29 kB/s | 9.2 kB 00:00 2024-03-22T06:56:31.744 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 29 kB/s | 9.2 kB 00:00 2024-03-22T06:56:32.062 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 6.4 kB/s | 1.8 kB 00:00 2024-03-22T06:56:32.079 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 5.7 kB/s | 1.8 kB 00:00 2024-03-22T06:56:32.547 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-22T06:56:32.548 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-22T06:56:33.964 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - BaseOS 6.7 MB/s | 2.0 MB 00:00 2024-03-22T06:56:34.523 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - BaseOS 2.3 MB/s | 2.0 MB 00:00 2024-03-22T06:56:35.497 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.0 MB 00:01 2024-03-22T06:56:35.714 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-03-22T06:56:37.676 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-03-22T06:56:42.041 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.1 MB 00:00 2024-03-22T06:56:44.067 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-03-22T06:56:46.120 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-22T06:56:51.101 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-03-22T06:56:53.695 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:56:53.706 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:56:53.706 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-22T06:56:53.706 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:56:53.706 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:56:53.706 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 14 M 2024-03-22T06:56:53.706 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.1 M 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 5.0 M 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 23 M 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 636 k 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-22T06:56:53.707 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 457 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 4.9 M 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 45 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 117 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 156 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 320 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 293 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 99 k 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-22T06:56:53.708 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:56:53.709 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:56:53.709 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:56:53.709 INFO:teuthology.orchestra.run.smithi149.stdout:Install 22 Packages 2024-03-22T06:56:53.709 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 2 Packages 2024-03-22T06:56:53.709 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:56:53.709 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 58 M 2024-03-22T06:56:53.709 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:56:55.344 INFO:teuthology.orchestra.run.smithi149.stdout:(1/24): ceph-radosgw-17.2.7-726.g9c4e0c34.el9.x 11 MB/s | 14 MB 00:01 2024-03-22T06:56:55.419 INFO:teuthology.orchestra.run.smithi149.stdout:(2/24): ceph-base-17.2.7-726.g9c4e0c34.el9.x86_ 3.7 MB/s | 5.0 MB 00:01 2024-03-22T06:56:55.444 INFO:teuthology.orchestra.run.smithi149.stdout:(3/24): ceph-selinux-17.2.7-726.g9c4e0c34.el9.x 251 kB/s | 25 kB 00:00 2024-03-22T06:56:55.503 INFO:teuthology.orchestra.run.smithi149.stdout:(4/24): libradosstriper1-17.2.7-726.g9c4e0c34.e 7.7 MB/s | 457 kB 00:00 2024-03-22T06:56:55.536 INFO:teuthology.orchestra.run.smithi149.stdout:(5/24): libcephfs2-17.2.7-726.g9c4e0c34.el9.x86 5.3 MB/s | 636 kB 00:00 2024-03-22T06:56:55.629 INFO:teuthology.orchestra.run.smithi149.stdout:(6/24): python3-ceph-argparse-17.2.7-726.g9c4e0 491 kB/s | 45 kB 00:00 2024-03-22T06:56:55.712 INFO:teuthology.orchestra.run.smithi149.stdout:(7/24): python3-ceph-common-17.2.7-726.g9c4e0c3 1.4 MB/s | 117 kB 00:00 2024-03-22T06:56:55.812 INFO:teuthology.orchestra.run.smithi149.stdout:(8/24): librgw2-17.2.7-726.g9c4e0c34.el9.x86_64 16 MB/s | 4.9 MB 00:00 2024-03-22T06:56:55.838 INFO:teuthology.orchestra.run.smithi149.stdout:(9/24): python3-cephfs-17.2.7-726.g9c4e0c34.el9 1.2 MB/s | 156 kB 00:00 2024-03-22T06:56:55.863 INFO:teuthology.orchestra.run.smithi149.stdout:(10/24): python3-rados-17.2.7-726.g9c4e0c34.el9 6.2 MB/s | 320 kB 00:00 2024-03-22T06:56:55.905 INFO:teuthology.orchestra.run.smithi149.stdout:(11/24): python3-rgw-17.2.7-726.g9c4e0c34.el9.x 2.3 MB/s | 99 kB 00:00 2024-03-22T06:56:55.938 INFO:teuthology.orchestra.run.smithi149.stdout:(12/24): python3-rbd-17.2.7-726.g9c4e0c34.el9.x 2.8 MB/s | 293 kB 00:00 2024-03-22T06:56:56.047 INFO:teuthology.orchestra.run.smithi149.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 408 kB/s | 44 kB 00:00 2024-03-22T06:56:56.072 INFO:teuthology.orchestra.run.smithi149.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 952 kB/s | 159 kB 00:00 2024-03-22T06:56:56.114 INFO:teuthology.orchestra.run.smithi149.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-22T06:56:56.139 INFO:teuthology.orchestra.run.smithi149.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-22T06:56:56.156 INFO:teuthology.orchestra.run.smithi149.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 993 kB/s | 41 kB 00:00 2024-03-22T06:56:56.290 INFO:teuthology.orchestra.run.smithi149.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 365 kB/s | 49 kB 00:00 2024-03-22T06:56:56.390 INFO:teuthology.orchestra.run.smithi149.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-03-22T06:56:56.415 INFO:teuthology.orchestra.run.smithi149.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 537 kB/s | 67 kB 00:00 2024-03-22T06:56:56.541 INFO:teuthology.orchestra.run.smithi149.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-22T06:56:56.750 INFO:teuthology.orchestra.run.smithi149.stdout:(22/24): ceph-common-17.2.7-726.g9c4e0c34.el9.x 8.4 MB/s | 23 MB 00:02 2024-03-22T06:56:56.808 INFO:teuthology.orchestra.run.smithi149.stdout:(23/24): librbd1-17.2.7-726.g9c4e0c34.el9.x86_6 11 MB/s | 2.9 MB 00:00 2024-03-22T06:56:56.967 INFO:teuthology.orchestra.run.smithi149.stdout:(24/24): librados2-17.2.7-726.g9c4e0c34.el9.x86 5.6 MB/s | 3.1 MB 00:00 2024-03-22T06:56:56.970 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:56:56.971 INFO:teuthology.orchestra.run.smithi149.stdout:Total 18 MB/s | 58 MB 00:03 2024-03-22T06:56:56.971 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:56:57.000 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:56:57.000 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:56:57.129 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 936 kB/s | 19 MB 00:20 2024-03-22T06:56:57.464 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:56:57.464 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:56:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:56:59.021 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-22T06:56:59.085 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:56:59.177 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:56:59.255 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T06:56:59.278 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:56:59.358 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:56:59.413 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-22T06:56:59.441 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-22T06:56:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-22T06:56:59.906 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T06:56:59.969 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T06:56:59.998 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-22T06:57:00.040 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 10/26 2024-03-22T06:57:00.077 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T06:57:00.139 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T06:57:00.167 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:57:00.216 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:57:00.256 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-22T06:57:00.284 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-22T06:57:00.317 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-22T06:57:00.613 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-22T06:57:00.638 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T06:57:00.711 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T06:57:00.784 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 19/26 2024-03-22T06:57:00.815 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 20/26 2024-03-22T06:57:01.224 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.1 MB 00:00 2024-03-22T06:57:02.956 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-03-22T06:57:03.222 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:57:03.270 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:57:03.708 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 32 MB/s | 21 MB 00:00 2024-03-22T06:57:03.807 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:57:03.840 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T06:57:03.933 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T06:57:03.933 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-22T06:57:03.934 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-22T06:57:03.934 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:03.966 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:57:08.733 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-22T06:57:11.188 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:57:11.191 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:57:11.191 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 14 M 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.1 M 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 5.0 M 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 23 M 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-22T06:57:11.192 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 636 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 457 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 4.9 M 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 45 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 117 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 156 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 320 k 2024-03-22T06:57:11.193 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 293 k 2024-03-22T06:57:11.194 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 99 k 2024-03-22T06:57:11.194 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-22T06:57:11.194 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:11.194 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:57:11.194 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:57:11.194 INFO:teuthology.orchestra.run.smithi081.stdout:Install 22 Packages 2024-03-22T06:57:11.194 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 2 Packages 2024-03-22T06:57:11.194 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:11.195 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 58 M 2024-03-22T06:57:11.195 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:57:12.784 INFO:teuthology.orchestra.run.smithi081.stdout:(1/24): ceph-common-17.2.7-726.g9c4e0c34.el9.x8 15 MB/s | 23 MB 00:01 2024-03-22T06:57:12.835 INFO:teuthology.orchestra.run.smithi081.stdout:(2/24): ceph-selinux-17.2.7-726.g9c4e0c34.el9.x 504 kB/s | 25 kB 00:00 2024-03-22T06:57:12.918 INFO:teuthology.orchestra.run.smithi081.stdout:(3/24): libcephfs2-17.2.7-726.g9c4e0c34.el9.x86 7.5 MB/s | 636 kB 00:00 2024-03-22T06:57:12.977 INFO:teuthology.orchestra.run.smithi081.stdout:(4/24): libradosstriper1-17.2.7-726.g9c4e0c34.e 7.7 MB/s | 457 kB 00:00 2024-03-22T06:57:13.094 INFO:teuthology.orchestra.run.smithi081.stdout:(5/24): ceph-base-17.2.7-726.g9c4e0c34.el9.x86_ 2.8 MB/s | 5.0 MB 00:01 2024-03-22T06:57:13.194 INFO:teuthology.orchestra.run.smithi081.stdout:(6/24): python3-ceph-argparse-17.2.7-726.g9c4e0 451 kB/s | 45 kB 00:00 2024-03-22T06:57:13.261 INFO:teuthology.orchestra.run.smithi081.stdout:(7/24): librgw2-17.2.7-726.g9c4e0c34.el9.x86_64 17 MB/s | 4.9 MB 00:00 2024-03-22T06:57:13.287 INFO:teuthology.orchestra.run.smithi081.stdout:(8/24): python3-ceph-common-17.2.7-726.g9c4e0c3 1.2 MB/s | 117 kB 00:00 2024-03-22T06:57:13.312 INFO:teuthology.orchestra.run.smithi081.stdout:(9/24): python3-cephfs-17.2.7-726.g9c4e0c34.el9 3.0 MB/s | 156 kB 00:00 2024-03-22T06:57:13.362 INFO:teuthology.orchestra.run.smithi081.stdout:(10/24): python3-rbd-17.2.7-726.g9c4e0c34.el9.x 5.8 MB/s | 293 kB 00:00 2024-03-22T06:57:13.404 INFO:teuthology.orchestra.run.smithi081.stdout:(11/24): python3-rados-17.2.7-726.g9c4e0c34.el9 2.7 MB/s | 320 kB 00:00 2024-03-22T06:57:13.429 INFO:teuthology.orchestra.run.smithi081.stdout:(12/24): python3-rgw-17.2.7-726.g9c4e0c34.el9.x 1.4 MB/s | 99 kB 00:00 2024-03-22T06:57:13.546 INFO:teuthology.orchestra.run.smithi081.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 379 kB/s | 44 kB 00:00 2024-03-22T06:57:13.571 INFO:teuthology.orchestra.run.smithi081.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 952 kB/s | 159 kB 00:00 2024-03-22T06:57:13.622 INFO:teuthology.orchestra.run.smithi081.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-22T06:57:13.647 INFO:teuthology.orchestra.run.smithi081.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-22T06:57:13.673 INFO:teuthology.orchestra.run.smithi081.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 821 kB/s | 41 kB 00:00 2024-03-22T06:57:13.713 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:57:13.713 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-03-22T06:57:13.713 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-03-22T06:57:13.713 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-03-22T06:57:13.713 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-03-22T06:57:13.714 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-03-22T06:57:13.714 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-03-22T06:57:13.714 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-03-22T06:57:13.714 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:13.756 INFO:teuthology.orchestra.run.smithi081.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 585 kB/s | 49 kB 00:00 2024-03-22T06:57:13.840 INFO:teuthology.orchestra.run.smithi081.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.6 MB/s | 309 kB 00:00 2024-03-22T06:57:13.982 INFO:teuthology.orchestra.run.smithi081.stdout:(20/24): ceph-radosgw-17.2.7-726.g9c4e0c34.el9. 5.2 MB/s | 14 MB 00:02 2024-03-22T06:57:14.024 INFO:teuthology.orchestra.run.smithi081.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 252 kB/s | 67 kB 00:00 2024-03-22T06:57:14.090 INFO:teuthology.orchestra.run.smithi081.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-03-22T06:57:14.207 INFO:teuthology.orchestra.run.smithi081.stdout:(23/24): librbd1-17.2.7-726.g9c4e0c34.el9.x86_6 16 MB/s | 2.9 MB 00:00 2024-03-22T06:57:14.549 INFO:teuthology.orchestra.run.smithi081.stdout:(24/24): librados2-17.2.7-726.g9c4e0c34.el9.x86 5.4 MB/s | 3.1 MB 00:00 2024-03-22T06:57:14.554 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:14.554 INFO:teuthology.orchestra.run.smithi081.stdout:Total 17 MB/s | 58 MB 00:03 2024-03-22T06:57:14.554 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:57:14.586 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:57:14.587 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:57:14.987 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T06:57:15.052 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:57:15.052 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:57:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T06:57:15.064 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:15.064 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-22T06:57:15.065 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T06:57:15.065 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T06:57:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T06:57:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T06:57:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:57:16.283 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:57:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:57:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/26 2024-03-22T06:57:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 7/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 8/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 9/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 10/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-22T06:57:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-22T06:57:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 25/26 2024-03-22T06:57:16.621 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-22T06:57:16.685 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:57:16.777 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:57:16.855 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T06:57:16.886 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:57:16.957 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:57:17.008 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-22T06:57:17.032 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-22T06:57:17.262 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:57:17.262 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:17.262 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-03-22T06:57:17.262 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.263 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:17.264 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:57:17.463 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-22T06:57:17.498 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T06:57:17.561 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T06:57:17.589 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-22T06:57:17.617 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-test 2024-03-22T06:57:17.631 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 10/26 2024-03-22T06:57:17.669 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T06:57:17.731 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T06:57:17.759 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:57:17.808 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:57:17.848 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-22T06:57:17.900 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-22T06:57:17.933 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-22T06:57:18.208 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:27 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:57:18.226 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-22T06:57:18.255 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T06:57:18.335 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T06:57:18.364 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:57:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:18.365 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-22T06:57:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:57:18.365 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 48 M 2024-03-22T06:57:18.365 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-22T06:57:18.365 INFO:teuthology.orchestra.run.smithi149.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-22T06:57:18.366 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 163 k 2024-03-22T06:57:18.366 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-22T06:57:18.366 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-22T06:57:18.366 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-22T06:57:18.366 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:57:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:18.366 INFO:teuthology.orchestra.run.smithi149.stdout:Install 6 Packages 2024-03-22T06:57:18.366 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 49 M 2024-03-22T06:57:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 196 M 2024-03-22T06:57:18.367 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:57:18.408 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 19/26 2024-03-22T06:57:18.432 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 20/26 2024-03-22T06:57:18.810 INFO:teuthology.orchestra.run.smithi149.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.2 MB/s | 217 kB 00:00 2024-03-22T06:57:18.852 INFO:teuthology.orchestra.run.smithi149.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-22T06:57:18.877 INFO:teuthology.orchestra.run.smithi149.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.5 MB/s | 63 kB 00:00 2024-03-22T06:57:19.019 INFO:teuthology.orchestra.run.smithi149.stdout:(4/6): libcephsqlite-17.2.7-726.g9c4e0c34.el9.x 427 kB/s | 163 kB 00:00 2024-03-22T06:57:19.103 INFO:teuthology.orchestra.run.smithi149.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 831 kB/s | 187 kB 00:00 2024-03-22T06:57:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:57:20.870 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:57:21.011 INFO:teuthology.orchestra.run.smithi149.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el9.x86_6 20 MB/s | 48 MB 00:02 2024-03-22T06:57:21.015 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:21.015 INFO:teuthology.orchestra.run.smithi149.stdout:Total 18 MB/s | 49 MB 00:02 2024-03-22T06:57:21.029 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:57:21.050 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:57:21.050 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:57:21.407 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T06:57:21.428 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:57:21.429 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:57:21.440 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T06:57:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T06:57:21.534 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-22T06:57:21.535 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-22T06:57:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:21.567 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:57:22.014 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:57:22.078 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-22T06:57:22.112 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-22T06:57:22.144 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T06:57:22.186 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-22T06:57:22.223 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T06:57:26.659 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T06:57:26.738 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T06:57:29.087 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T06:57:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/6 2024-03-22T06:57:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/6 2024-03-22T06:57:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T06:57:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-22T06:57:29.088 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-22T06:57:29.454 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-22T06:57:29.454 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:29.454 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:57:29.454 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:29.454 INFO:teuthology.orchestra.run.smithi149.stdout: jq-1.6-16.el9.x86_64 2024-03-22T06:57:29.454 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:29.454 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-22T06:57:29.455 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-22T06:57:29.455 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-22T06:57:29.455 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:29.455 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:57:29.738 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph 2024-03-22T06:57:30.334 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:39 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:57:30.501 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================== 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================== 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 6.5 k 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.1 M 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 1.4 M 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 240 k 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.7 M 2024-03-22T06:57:30.504 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 15 M 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-22T06:57:30.505 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-22T06:57:30.506 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-22T06:57:30.507 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-22T06:57:30.507 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-22T06:57:30.507 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:30.507 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:57:30.507 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================== 2024-03-22T06:57:30.507 INFO:teuthology.orchestra.run.smithi149.stdout:Install 33 Packages 2024-03-22T06:57:30.507 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:30.507 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 27 M 2024-03-22T06:57:30.508 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 100 M 2024-03-22T06:57:30.508 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:57:30.846 INFO:teuthology.orchestra.run.smithi149.stdout:(1/33): ceph-17.2.7-726.g9c4e0c34.el9.x86_64.rp 29 kB/s | 6.5 kB 00:00 2024-03-22T06:57:30.963 INFO:teuthology.orchestra.run.smithi149.stdout:(2/33): ceph-mgr-17.2.7-726.g9c4e0c34.el9.x86_6 4.1 MB/s | 1.4 MB 00:00 2024-03-22T06:57:31.005 INFO:teuthology.orchestra.run.smithi149.stdout:(3/33): ceph-mds-17.2.7-726.g9c4e0c34.el9.x86_6 5.4 MB/s | 2.1 MB 00:00 2024-03-22T06:57:31.080 INFO:teuthology.orchestra.run.smithi149.stdout:(4/33): ceph-mgr-modules-core-17.2.7-726.g9c4e0 3.1 MB/s | 240 kB 00:00 2024-03-22T06:57:31.188 INFO:teuthology.orchestra.run.smithi149.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-03-22T06:57:31.199 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:57:31.199 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2024-03-22T06:57:31.199 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2024-03-22T06:57:31.199 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2024-03-22T06:57:31.199 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2024-03-22T06:57:31.199 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2024-03-22T06:57:31.199 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2024-03-22T06:57:31.200 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2024-03-22T06:57:31.200 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:31.224 INFO:teuthology.orchestra.run.smithi149.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 889 kB/s | 32 kB 00:00 2024-03-22T06:57:31.258 INFO:teuthology.orchestra.run.smithi149.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-22T06:57:31.292 INFO:teuthology.orchestra.run.smithi149.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-22T06:57:31.358 INFO:teuthology.orchestra.run.smithi149.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2024-03-22T06:57:31.426 INFO:teuthology.orchestra.run.smithi149.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-03-22T06:57:31.459 INFO:teuthology.orchestra.run.smithi149.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-03-22T06:57:31.493 INFO:teuthology.orchestra.run.smithi149.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 3.6 MB/s | 124 kB 00:00 2024-03-22T06:57:31.527 INFO:teuthology.orchestra.run.smithi149.stdout:(13/33): python3-urllib3-1.26.5-5.el9.noarch.rp 5.5 MB/s | 187 kB 00:00 2024-03-22T06:57:31.694 INFO:teuthology.orchestra.run.smithi149.stdout:(14/33): ceph-osd-17.2.7-726.g9c4e0c34.el9.x86_ 21 MB/s | 15 MB 00:00 2024-03-22T06:57:31.735 INFO:teuthology.orchestra.run.smithi149.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 600 kB/s | 125 kB 00:00 2024-03-22T06:57:31.986 INFO:teuthology.orchestra.run.smithi149.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 149 kB/s | 43 kB 00:00 2024-03-22T06:57:32.078 INFO:teuthology.orchestra.run.smithi149.stdout:(17/33): ceph-mon-17.2.7-726.g9c4e0c34.el9.x86_ 3.0 MB/s | 3.7 MB 00:01 2024-03-22T06:57:32.178 INFO:teuthology.orchestra.run.smithi149.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 388 kB/s | 172 kB 00:00 2024-03-22T06:57:32.211 INFO:teuthology.orchestra.run.smithi149.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-03-22T06:57:32.245 INFO:teuthology.orchestra.run.smithi149.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 64 kB/s | 11 kB 00:00 2024-03-22T06:57:32.278 INFO:teuthology.orchestra.run.smithi149.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 176 kB/s | 18 kB 00:00 2024-03-22T06:57:32.304 INFO:teuthology.orchestra.run.smithi149.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 251 kB/s | 23 kB 00:00 2024-03-22T06:57:32.337 INFO:teuthology.orchestra.run.smithi149.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2024-03-22T06:57:32.371 INFO:teuthology.orchestra.run.smithi149.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 213 kB/s | 20 kB 00:00 2024-03-22T06:57:32.397 INFO:teuthology.orchestra.run.smithi149.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 501 kB/s | 46 kB 00:00 2024-03-22T06:57:32.469 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T06:57:32.488 INFO:teuthology.orchestra.run.smithi149.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 179 kB/s | 16 kB 00:00 2024-03-22T06:57:32.514 INFO:teuthology.orchestra.run.smithi149.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 448 kB/s | 79 kB 00:00 2024-03-22T06:57:32.539 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T06:57:32.539 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:32.539 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-22T06:57:32.539 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T06:57:32.539 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T06:57:32.539 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:32.540 INFO:teuthology.orchestra.run.smithi149.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-03-22T06:57:32.541 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T06:57:32.581 INFO:teuthology.orchestra.run.smithi149.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 973 kB/s | 90 kB 00:00 2024-03-22T06:57:32.606 INFO:teuthology.orchestra.run.smithi149.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 388 kB/s | 36 kB 00:00 2024-03-22T06:57:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T06:57:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:57:32.640 INFO:teuthology.orchestra.run.smithi149.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-03-22T06:57:32.698 INFO:teuthology.orchestra.run.smithi149.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 217 kB/s | 20 kB 00:00 2024-03-22T06:57:32.740 INFO:teuthology.orchestra.run.smithi149.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.6 MB/s | 427 kB 00:00 2024-03-22T06:57:32.744 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:32.744 INFO:teuthology.orchestra.run.smithi149.stdout:Total 12 MB/s | 27 MB 00:02 2024-03-22T06:57:32.757 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:57:32.796 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:57:32.796 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:57:33.146 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:57:33.146 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:57:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 7/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 8/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 9/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 10/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T06:57:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-22T06:57:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 25/26 2024-03-22T06:57:34.379 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:57:34.460 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-22T06:57:34.492 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-22T06:57:34.525 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-22T06:57:34.608 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-22T06:57:34.695 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-22T06:57:34.736 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-22T06:57:34.815 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-22T06:57:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:34.864 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:57:34.870 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-22T06:57:34.908 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-22T06:57:34.949 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-22T06:57:35.011 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-22T06:57:35.204 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-test 2024-03-22T06:57:35.206 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-22T06:57:35.263 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-22T06:57:35.302 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-22T06:57:35.336 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-22T06:57:35.366 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-22T06:57:35.442 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-22T06:57:35.477 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T06:57:35.510 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-22T06:57:35.550 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-22T06:57:35.585 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-22T06:57:35.620 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-22T06:57:35.710 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-22T06:57:35.743 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-22T06:57:35.754 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:27 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:57:35.815 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-22T06:57:35.908 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:57:35.909 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:57:35.909 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-22T06:57:35.909 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:57:35.909 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:57:35.909 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 48 M 2024-03-22T06:57:35.909 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-22T06:57:35.909 INFO:teuthology.orchestra.run.smithi081.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-22T06:57:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 163 k 2024-03-22T06:57:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-22T06:57:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-22T06:57:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-22T06:57:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:35.910 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:57:35.910 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:57:35.910 INFO:teuthology.orchestra.run.smithi081.stdout:Install 6 Packages 2024-03-22T06:57:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:35.911 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 49 M 2024-03-22T06:57:35.911 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 196 M 2024-03-22T06:57:35.911 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:57:35.953 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-22T06:57:36.076 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T06:57:36.113 INFO:teuthology.orchestra.run.smithi081.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.3 MB/s | 217 kB 00:00 2024-03-22T06:57:36.155 INFO:teuthology.orchestra.run.smithi081.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-22T06:57:36.189 INFO:teuthology.orchestra.run.smithi081.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-22T06:57:36.224 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 28/33 2024-03-22T06:57:36.248 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T06:57:36.273 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T06:57:36.273 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:36.273 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-22T06:57:36.273 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T06:57:36.273 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T06:57:36.274 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:36.314 INFO:teuthology.orchestra.run.smithi081.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-22T06:57:36.373 INFO:teuthology.orchestra.run.smithi081.stdout:(5/6): libcephsqlite-17.2.7-726.g9c4e0c34.el9.x 386 kB/s | 163 kB 00:00 2024-03-22T06:57:37.615 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T06:57:37.642 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T06:57:37.643 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:37.643 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-22T06:57:37.643 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T06:57:37.643 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T06:57:37.643 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:38.024 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T06:57:38.048 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T06:57:38.048 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:38.048 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-22T06:57:38.048 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T06:57:38.048 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T06:57:38.048 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:38.250 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T06:57:38.273 INFO:teuthology.orchestra.run.smithi081.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el9.x86_6 21 MB/s | 48 MB 00:02 2024-03-22T06:57:38.273 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T06:57:38.273 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:38.273 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-22T06:57:38.273 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T06:57:38.274 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T06:57:38.274 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:38.276 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:38.276 INFO:teuthology.orchestra.run.smithi081.stdout:Total 21 MB/s | 49 MB 00:02 2024-03-22T06:57:38.290 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:57:38.310 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:57:38.310 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:57:38.339 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T06:57:38.696 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:57:38.696 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:57:39.272 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:57:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-22T06:57:39.374 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-22T06:57:39.406 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T06:57:39.456 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-22T06:57:39.484 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T06:57:39.951 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 6/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-22T06:57:39.952 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-22T06:57:39.953 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-22T06:57:39.954 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-22T06:57:39.954 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T06:57:39.954 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-22T06:57:39.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-22T06:57:39.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-22T06:57:39.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-22T06:57:39.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-22T06:57:40.728 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-22T06:57:40.728 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:40.728 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:57:40.728 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:40.728 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:40.728 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:40.728 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-22T06:57:40.729 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-22T06:57:40.730 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-22T06:57:40.731 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-22T06:57:40.731 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-22T06:57:40.731 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-22T06:57:40.731 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-22T06:57:40.731 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-22T06:57:40.731 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:40.731 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:57:41.085 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-base 2024-03-22T06:57:41.680 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:50 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:57:41.776 INFO:teuthology.orchestra.run.smithi149.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:57:41.827 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:57:41.828 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-03-22T06:57:41.828 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:57:41.890 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install cephadm 2024-03-22T06:57:42.466 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:51 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:57:42.615 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 82 k 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 82 k 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 372 k 2024-03-22T06:57:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:57:42.925 INFO:teuthology.orchestra.run.smithi149.stdout:cephadm-17.2.7-726.g9c4e0c34.el9.noarch.rpm 266 kB/s | 82 kB 00:00 2024-03-22T06:57:42.926 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:42.926 INFO:teuthology.orchestra.run.smithi149.stdout:Total 265 kB/s | 82 kB 00:00 2024-03-22T06:57:42.926 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:57:42.931 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:57:42.931 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:57:42.933 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:57:42.933 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:57:43.008 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:57:43.439 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:57:43.538 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:57:43.921 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:57:43.937 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T06:57:44.011 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T06:57:44.216 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:57:44.217 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:44.217 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:57:44.217 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:57:44.217 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:44.217 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:57:44.422 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-immutable-object-cache 2024-03-22T06:57:45.010 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:54 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:57:45.162 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 137 k 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-03-22T06:57:45.163 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:45.164 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 137 k 2024-03-22T06:57:45.164 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 379 k 2024-03-22T06:57:45.164 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:57:45.368 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 672 kB/s | 137 kB 00:00 2024-03-22T06:57:45.369 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:45.369 INFO:teuthology.orchestra.run.smithi149.stdout:Total 667 kB/s | 137 kB 00:00 2024-03-22T06:57:45.369 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:57:45.376 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:57:45.377 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:57:45.428 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:57:45.429 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:57:45.567 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:57:45.636 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:57:45.660 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:57:45.661 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:45.661 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-22T06:57:45.661 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T06:57:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/6 2024-03-22T06:57:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/6 2024-03-22T06:57:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T06:57:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-22T06:57:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-22T06:57:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-22T06:57:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:46.537 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:57:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: jq-1.6-16.el9.x86_64 2024-03-22T06:57:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:46.538 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-22T06:57:46.538 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-22T06:57:46.538 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-22T06:57:46.538 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:46.538 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:57:46.636 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:57:46.636 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:46.636 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:57:46.636 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:46.636 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:46.637 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:57:46.818 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph 2024-03-22T06:57:46.863 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr 2024-03-22T06:57:47.382 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:39 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:57:47.449 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:56 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:57:47.532 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:57:47.534 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================== 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================== 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 6.5 k 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.1 M 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 1.4 M 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 240 k 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.7 M 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 15 M 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-22T06:57:47.535 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-22T06:57:47.536 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-22T06:57:47.537 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:47.538 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:57:47.538 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================== 2024-03-22T06:57:47.538 INFO:teuthology.orchestra.run.smithi081.stdout:Install 33 Packages 2024-03-22T06:57:47.538 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:47.538 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 27 M 2024-03-22T06:57:47.538 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 100 M 2024-03-22T06:57:47.538 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:57:47.545 INFO:teuthology.orchestra.run.smithi149.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:57:47.596 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:57:47.597 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-03-22T06:57:47.597 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:57:47.663 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-dashboard 2024-03-22T06:57:47.977 INFO:teuthology.orchestra.run.smithi081.stdout:(1/33): ceph-17.2.7-726.g9c4e0c34.el9.x86_64.rp 41 kB/s | 6.5 kB 00:00 2024-03-22T06:57:48.210 INFO:teuthology.orchestra.run.smithi081.stdout:(2/33): ceph-mgr-17.2.7-726.g9c4e0c34.el9.x86_6 3.6 MB/s | 1.4 MB 00:00 2024-03-22T06:57:48.232 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:57 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:57:48.385 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:57:48.385 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:48.385 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 1.7 M 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 23 k 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 15 k 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:48.386 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:57:48.387 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:57:48.387 INFO:teuthology.orchestra.run.smithi149.stdout:Install 7 Packages 2024-03-22T06:57:48.387 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:48.387 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 2.0 M 2024-03-22T06:57:48.387 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 8.1 M 2024-03-22T06:57:48.387 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:57:48.410 INFO:teuthology.orchestra.run.smithi081.stdout:(3/33): ceph-mon-17.2.7-726.g9c4e0c34.el9.x86_6 8.5 MB/s | 3.7 MB 00:00 2024-03-22T06:57:48.486 INFO:teuthology.orchestra.run.smithi081.stdout:(4/33): ceph-mgr-modules-core-17.2.7-726.g9c4e0 3.1 MB/s | 240 kB 00:00 2024-03-22T06:57:48.594 INFO:teuthology.orchestra.run.smithi081.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-03-22T06:57:48.628 INFO:teuthology.orchestra.run.smithi081.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 947 kB/s | 32 kB 00:00 2024-03-22T06:57:48.662 INFO:teuthology.orchestra.run.smithi081.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-22T06:57:48.695 INFO:teuthology.orchestra.run.smithi081.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-22T06:57:48.745 INFO:teuthology.orchestra.run.smithi081.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-22T06:57:48.812 INFO:teuthology.orchestra.run.smithi081.stdout:(10/33): ceph-mds-17.2.7-726.g9c4e0c34.el9.x86_ 2.1 MB/s | 2.1 MB 00:00 2024-03-22T06:57:48.822 INFO:teuthology.orchestra.run.smithi149.stdout:(1/7): ceph-prometheus-alerts-17.2.7-726.g9c4e0 49 kB/s | 15 kB 00:00 2024-03-22T06:57:48.847 INFO:teuthology.orchestra.run.smithi149.stdout:(2/7): ceph-grafana-dashboards-17.2.7-726.g9c4e 72 kB/s | 23 kB 00:00 2024-03-22T06:57:48.855 INFO:teuthology.orchestra.run.smithi081.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 11 MB/s | 1.1 MB 00:00 2024-03-22T06:57:48.880 INFO:teuthology.orchestra.run.smithi081.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-22T06:57:48.905 INFO:teuthology.orchestra.run.smithi081.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-03-22T06:57:48.930 INFO:teuthology.orchestra.run.smithi081.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-03-22T06:57:49.089 INFO:teuthology.orchestra.run.smithi081.stdout:(15/33): ceph-osd-17.2.7-726.g9c4e0c34.el9.x86_ 17 MB/s | 15 MB 00:00 2024-03-22T06:57:49.148 INFO:teuthology.orchestra.run.smithi149.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 28 kB/s | 9.0 kB 00:00 2024-03-22T06:57:49.189 INFO:teuthology.orchestra.run.smithi081.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 440 kB/s | 125 kB 00:00 2024-03-22T06:57:49.206 INFO:teuthology.orchestra.run.smithi081.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 158 kB/s | 43 kB 00:00 2024-03-22T06:57:49.290 INFO:teuthology.orchestra.run.smithi081.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 858 kB/s | 172 kB 00:00 2024-03-22T06:57:49.307 INFO:teuthology.orchestra.run.smithi081.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 106 kB/s | 11 kB 00:00 2024-03-22T06:57:49.315 INFO:teuthology.orchestra.run.smithi149.stdout:(4/7): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 2.1 MB/s | 1.7 MB 00:00 2024-03-22T06:57:49.340 INFO:teuthology.orchestra.run.smithi149.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 83 kB/s | 41 kB 00:00 2024-03-22T06:57:49.341 INFO:teuthology.orchestra.run.smithi081.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-03-22T06:57:49.365 INFO:teuthology.orchestra.run.smithi149.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 141 kB/s | 31 kB 00:00 2024-03-22T06:57:49.374 INFO:teuthology.orchestra.run.smithi081.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.9 MB/s | 358 kB 00:00 2024-03-22T06:57:49.391 INFO:teuthology.orchestra.run.smithi081.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 276 kB/s | 23 kB 00:00 2024-03-22T06:57:49.408 INFO:teuthology.orchestra.run.smithi081.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 289 kB/s | 19 kB 00:00 2024-03-22T06:57:49.425 INFO:teuthology.orchestra.run.smithi081.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 389 kB/s | 20 kB 00:00 2024-03-22T06:57:49.442 INFO:teuthology.orchestra.run.smithi081.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-03-22T06:57:49.492 INFO:teuthology.orchestra.run.smithi081.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-03-22T06:57:49.517 INFO:teuthology.orchestra.run.smithi081.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 722 kB/s | 79 kB 00:00 2024-03-22T06:57:49.542 INFO:teuthology.orchestra.run.smithi081.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-03-22T06:57:49.568 INFO:teuthology.orchestra.run.smithi081.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-03-22T06:57:49.593 INFO:teuthology.orchestra.run.smithi081.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-03-22T06:57:49.640 INFO:teuthology.orchestra.run.smithi149.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 580 kB/s | 188 kB 00:00 2024-03-22T06:57:49.644 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:49.644 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2024-03-22T06:57:49.644 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:57:49.652 INFO:teuthology.orchestra.run.smithi081.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-03-22T06:57:49.652 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:57:49.652 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:57:49.668 INFO:teuthology.orchestra.run.smithi081.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 266 kB/s | 20 kB 00:00 2024-03-22T06:57:49.694 INFO:teuthology.orchestra.run.smithi081.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-03-22T06:57:49.697 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:49.698 INFO:teuthology.orchestra.run.smithi081.stdout:Total 12 MB/s | 27 MB 00:02 2024-03-22T06:57:49.711 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:57:49.716 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:57:49.716 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:57:49.750 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:57:49.750 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:57:50.021 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:57:50.083 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-22T06:57:50.095 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:57:50.095 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:57:50.117 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-22T06:57:50.164 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-22T06:57:50.196 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-22T06:57:50.225 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 5/7 2024-03-22T06:57:50.452 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 6/7 2024-03-22T06:57:50.478 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T06:57:50.980 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T06:57:50.980 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 1/7 2024-03-22T06:57:50.980 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2/7 2024-03-22T06:57:50.980 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 3/7 2024-03-22T06:57:50.980 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-22T06:57:50.980 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-22T06:57:50.981 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-22T06:57:51.284 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-22T06:57:51.285 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:51.285 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:57:51.385 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:57:51.455 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-22T06:57:51.487 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-22T06:57:51.509 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-22T06:57:51.520 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-22T06:57:51.611 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-22T06:57:51.691 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-22T06:57:51.731 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-22T06:57:51.802 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-22T06:57:51.857 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-22T06:57:51.895 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-22T06:57:51.944 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-22T06:57:52.005 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-22T06:57:52.073 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:01 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:57:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-22T06:57:52.224 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout:=========================================================================================== 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout:=========================================================================================== 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 7.4 M 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-22T06:57:52.225 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout:=========================================================================================== 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout:Install 12 Packages 2024-03-22T06:57:52.226 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:57:52.227 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 37 M 2024-03-22T06:57:52.227 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 209 M 2024-03-22T06:57:52.227 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:57:52.241 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-22T06:57:52.288 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-22T06:57:52.323 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-22T06:57:52.353 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-22T06:57:52.430 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-22T06:57:52.463 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T06:57:52.505 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-22T06:57:52.545 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-22T06:57:52.581 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-22T06:57:52.598 INFO:teuthology.orchestra.run.smithi149.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 184 kB/s | 32 kB 00:00 2024-03-22T06:57:52.615 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-22T06:57:52.623 INFO:teuthology.orchestra.run.smithi149.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 694 kB/s | 17 kB 00:00 2024-03-22T06:57:52.648 INFO:teuthology.orchestra.run.smithi149.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2024-03-22T06:57:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-22T06:57:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-22T06:57:52.818 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-22T06:57:52.832 INFO:teuthology.orchestra.run.smithi149.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2024-03-22T06:57:52.956 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-22T06:57:52.965 INFO:teuthology.orchestra.run.smithi149.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-22T06:57:53.033 INFO:teuthology.orchestra.run.smithi149.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.9 MB/s | 3.0 MB 00:00 2024-03-22T06:57:53.057 INFO:teuthology.orchestra.run.smithi149.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 217 kB 00:00 2024-03-22T06:57:53.081 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T06:57:53.236 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 28/33 2024-03-22T06:57:53.268 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T06:57:53.292 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T06:57:53.292 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:53.293 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-22T06:57:53.293 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T06:57:53.293 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T06:57:53.293 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:53.483 INFO:teuthology.orchestra.run.smithi149.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-22T06:57:53.550 INFO:teuthology.orchestra.run.smithi149.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-03-22T06:57:53.592 INFO:teuthology.orchestra.run.smithi149.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-03-22T06:57:54.034 INFO:teuthology.orchestra.run.smithi149.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 507 kB/s | 245 kB 00:00 2024-03-22T06:57:54.627 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T06:57:54.654 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T06:57:54.654 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:54.654 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-22T06:57:54.654 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T06:57:54.655 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T06:57:54.655 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:54.684 INFO:teuthology.orchestra.run.smithi149.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-7 3.3 MB/s | 7.4 MB 00:02 2024-03-22T06:57:54.687 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:54.687 INFO:teuthology.orchestra.run.smithi149.stdout:Total 15 MB/s | 37 MB 00:02 2024-03-22T06:57:54.726 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:57:54.751 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:57:54.751 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:57:55.036 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T06:57:55.059 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T06:57:55.060 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:55.060 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-22T06:57:55.060 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T06:57:55.060 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T06:57:55.060 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:55.191 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:57:55.191 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:57:55.253 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T06:57:55.276 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T06:57:55.276 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:57:55.276 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-22T06:57:55.276 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T06:57:55.277 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T06:57:55.277 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T06:57:55.907 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:57:55.978 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-22T06:57:56.005 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-22T06:57:56.099 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-22T06:57:56.141 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-22T06:57:56.321 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T06:57:56.348 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T06:57:56.425 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-22T06:57:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T06:57:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/33 2024-03-22T06:57:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/33 2024-03-22T06:57:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/33 2024-03-22T06:57:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 6/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-22T06:57:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-22T06:57:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-22T06:57:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-22T06:57:57.186 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-22T06:57:57.920 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-22T06:57:57.920 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:57.920 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:57:57.920 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:57.920 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-22T06:57:57.921 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-22T06:57:57.922 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-22T06:57:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-22T06:57:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-22T06:57:57.923 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:57.923 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:57:58.231 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-base 2024-03-22T06:57:58.454 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-22T06:57:58.793 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:50 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:57:58.889 INFO:teuthology.orchestra.run.smithi081.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:57:58.939 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:57:58.940 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-22T06:57:58.940 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:57:59.012 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install cephadm 2024-03-22T06:57:59.473 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-22T06:57:59.507 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T06:57:59.570 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:51 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:57:59.718 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 82 k 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 82 k 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 372 k 2024-03-22T06:57:59.719 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:57:59.935 INFO:teuthology.orchestra.run.smithi081.stdout:cephadm-17.2.7-726.g9c4e0c34.el9.noarch.rpm 380 kB/s | 82 kB 00:00 2024-03-22T06:57:59.936 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:57:59.936 INFO:teuthology.orchestra.run.smithi081.stdout:Total 378 kB/s | 82 kB 00:00 2024-03-22T06:57:59.936 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:57:59.941 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:57:59.941 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:57:59.943 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:57:59.943 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:00.019 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:00.449 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:58:00.531 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:58:00.919 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:58:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T06:58:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:01.183 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:01.183 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:01.384 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-immutable-object-cache 2024-03-22T06:58:01.954 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:53 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:02.029 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 1/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-22T06:58:02.030 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-22T06:58:02.106 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:02.107 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:02.107 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-03-22T06:58:02.107 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:02.107 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:02.108 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 137 k 2024-03-22T06:58:02.108 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:02.108 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:02.108 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:02.108 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-03-22T06:58:02.108 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:02.108 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 137 k 2024-03-22T06:58:02.108 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 379 k 2024-03-22T06:58:02.108 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:02.335 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 606 kB/s | 137 kB 00:00 2024-03-22T06:58:02.336 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:02.336 INFO:teuthology.orchestra.run.smithi081.stdout:Total 602 kB/s | 137 kB 00:00 2024-03-22T06:58:02.336 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:02.344 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:02.344 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:02.395 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:02.395 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-22T06:58:02.492 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-22T06:58:02.493 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-22T06:58:02.493 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-22T06:58:02.493 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-22T06:58:02.493 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:02.493 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:02.527 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:02.586 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:58:02.610 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:58:02.610 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:58:02.611 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-22T06:58:02.611 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:02.877 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-rook 2024-03-22T06:58:03.483 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:12 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:03.587 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T06:58:03.587 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:03.587 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:03.587 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:03.587 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:03.587 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:03.647 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:03.648 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-03-22T06:58:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-22T06:58:03.648 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-03-22T06:58:03.648 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 47 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-22T06:58:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-22T06:58:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-22T06:58:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:03.650 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:58:03.650 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-03-22T06:58:03.650 INFO:teuthology.orchestra.run.smithi149.stdout:Install 13 Packages 2024-03-22T06:58:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:03.650 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 2.0 M 2024-03-22T06:58:03.651 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 26 M 2024-03-22T06:58:03.651 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:58:03.826 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr 2024-03-22T06:58:04.116 INFO:teuthology.orchestra.run.smithi149.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 236 kB/s | 24 kB 00:00 2024-03-22T06:58:04.166 INFO:teuthology.orchestra.run.smithi149.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 118 kB/s | 18 kB 00:00 2024-03-22T06:58:04.208 INFO:teuthology.orchestra.run.smithi149.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.5 MB/s | 174 kB 00:00 2024-03-22T06:58:04.233 INFO:teuthology.orchestra.run.smithi149.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-22T06:58:04.258 INFO:teuthology.orchestra.run.smithi149.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-22T06:58:04.284 INFO:teuthology.orchestra.run.smithi149.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 856 kB/s | 43 kB 00:00 2024-03-22T06:58:04.334 INFO:teuthology.orchestra.run.smithi149.stdout:(7/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el9. 147 kB/s | 47 kB 00:00 2024-03-22T06:58:04.398 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:56 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:04.476 INFO:teuthology.orchestra.run.smithi149.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 148 kB/s | 32 kB 00:00 2024-03-22T06:58:04.494 INFO:teuthology.orchestra.run.smithi081.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:04.501 INFO:teuthology.orchestra.run.smithi149.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 69 kB/s | 15 kB 00:00 2024-03-22T06:58:04.546 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:04.547 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-22T06:58:04.547 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:04.602 INFO:teuthology.orchestra.run.smithi149.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 591 kB/s | 59 kB 00:00 2024-03-22T06:58:04.622 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-dashboard 2024-03-22T06:58:04.652 INFO:teuthology.orchestra.run.smithi149.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 751 kB/s | 203 kB 00:00 2024-03-22T06:58:04.677 INFO:teuthology.orchestra.run.smithi149.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-03-22T06:58:04.711 INFO:teuthology.orchestra.run.smithi149.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.4 MB/s | 1.0 MB 00:00 2024-03-22T06:58:04.715 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:04.715 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2024-03-22T06:58:04.715 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:58:04.728 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:58:04.728 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:58:04.820 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:58:04.820 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:58:05.177 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:57 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:05.326 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 1.7 M 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 23 k 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 15 k 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-22T06:58:05.327 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-22T06:58:05.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-22T06:58:05.328 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2024-03-22T06:58:05.328 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-22T06:58:05.328 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:05.328 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:05.328 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:05.328 INFO:teuthology.orchestra.run.smithi081.stdout:Install 7 Packages 2024-03-22T06:58:05.328 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:05.329 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 2.0 M 2024-03-22T06:58:05.329 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 8.1 M 2024-03-22T06:58:05.329 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:05.330 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:58:05.502 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-22T06:58:05.554 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-22T06:58:05.609 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-22T06:58:05.646 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-22T06:58:05.688 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-22T06:58:05.783 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-22T06:58:05.880 INFO:teuthology.orchestra.run.smithi081.stdout:(1/7): ceph-prometheus-alerts-17.2.7-726.g9c4e0 74 kB/s | 15 kB 00:00 2024-03-22T06:58:05.892 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-22T06:58:05.906 INFO:teuthology.orchestra.run.smithi081.stdout:(2/7): ceph-grafana-dashboards-17.2.7-726.g9c4e 105 kB/s | 23 kB 00:00 2024-03-22T06:58:05.939 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-22T06:58:05.947 INFO:teuthology.orchestra.run.smithi081.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-03-22T06:58:05.990 INFO:teuthology.orchestra.run.smithi081.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 488 kB/s | 41 kB 00:00 2024-03-22T06:58:06.006 INFO:teuthology.orchestra.run.smithi081.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 521 kB/s | 31 kB 00:00 2024-03-22T06:58:06.082 INFO:teuthology.orchestra.run.smithi081.stdout:(6/7): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 4.3 MB/s | 1.7 MB 00:00 2024-03-22T06:58:06.107 INFO:teuthology.orchestra.run.smithi081.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.6 MB/s | 188 kB 00:00 2024-03-22T06:58:06.109 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:06.109 INFO:teuthology.orchestra.run.smithi081.stdout:Total 2.6 MB/s | 2.0 MB 00:00 2024-03-22T06:58:06.109 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:06.117 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:06.117 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:06.178 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:06.179 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:06.472 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:06.533 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-22T06:58:06.568 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-22T06:58:06.623 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-22T06:58:06.654 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-22T06:58:06.673 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-22T06:58:06.683 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 5/7 2024-03-22T06:58:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-22T06:58:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-22T06:58:06.808 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-22T06:58:06.831 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T06:58:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 6/7 2024-03-22T06:58:06.936 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T06:58:07.462 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T06:58:07.462 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 1/7 2024-03-22T06:58:07.462 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2/7 2024-03-22T06:58:07.462 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 3/7 2024-03-22T06:58:07.462 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-22T06:58:07.462 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-22T06:58:07.462 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-22T06:58:07.612 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T06:58:07.612 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 1/13 2024-03-22T06:58:07.612 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-22T06:58:07.612 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-22T06:58:07.612 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-22T06:58:07.613 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-22T06:58:07.613 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-22T06:58:07.613 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-22T06:58:07.613 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-22T06:58:07.613 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-22T06:58:07.613 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-22T06:58:07.614 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-22T06:58:07.614 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-22T06:58:07.764 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-22T06:58:07.764 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:07.764 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:07.764 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:07.764 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:07.765 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:07.765 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-22T06:58:07.765 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-22T06:58:07.765 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-22T06:58:07.765 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-22T06:58:07.765 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:07.765 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-22T06:58:07.973 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-22T06:58:07.974 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-22T06:58:07.974 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-22T06:58:07.974 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-22T06:58:07.974 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-22T06:58:07.974 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-22T06:58:07.974 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-22T06:58:07.974 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:07.974 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:07.991 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-22T06:58:08.220 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-cephadm 2024-03-22T06:58:08.542 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:00 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:08.692 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:08.693 INFO:teuthology.orchestra.run.smithi081.stdout:=========================================================================================== 2024-03-22T06:58:08.693 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-22T06:58:08.693 INFO:teuthology.orchestra.run.smithi081.stdout:=========================================================================================== 2024-03-22T06:58:08.693 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 7.4 M 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-22T06:58:08.694 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-22T06:58:08.695 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-22T06:58:08.695 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-22T06:58:08.695 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-22T06:58:08.695 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:08.695 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:08.695 INFO:teuthology.orchestra.run.smithi081.stdout:=========================================================================================== 2024-03-22T06:58:08.695 INFO:teuthology.orchestra.run.smithi081.stdout:Install 12 Packages 2024-03-22T06:58:08.695 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:08.696 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 37 M 2024-03-22T06:58:08.696 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 209 M 2024-03-22T06:58:08.696 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:08.799 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:17 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:08.949 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 123 k 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-22T06:58:08.950 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-22T06:58:08.951 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-22T06:58:08.951 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:08.951 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:58:08.951 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-03-22T06:58:08.951 INFO:teuthology.orchestra.run.smithi149.stdout:Install 6 Packages 2024-03-22T06:58:08.951 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:08.951 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 6.8 M 2024-03-22T06:58:08.951 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 33 M 2024-03-22T06:58:08.952 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:58:09.222 INFO:teuthology.orchestra.run.smithi081.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 32 kB 00:00 2024-03-22T06:58:09.248 INFO:teuthology.orchestra.run.smithi081.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 714 kB/s | 17 kB 00:00 2024-03-22T06:58:09.273 INFO:teuthology.orchestra.run.smithi081.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.4 MB/s | 34 kB 00:00 2024-03-22T06:58:09.333 INFO:teuthology.orchestra.run.smithi081.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-03-22T06:58:09.490 INFO:teuthology.orchestra.run.smithi149.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-22T06:58:09.522 INFO:teuthology.orchestra.run.smithi149.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 796 kB/s | 248 kB 00:00 2024-03-22T06:58:09.574 INFO:teuthology.orchestra.run.smithi081.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2024-03-22T06:58:09.641 INFO:teuthology.orchestra.run.smithi081.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2024-03-22T06:58:09.655 INFO:teuthology.orchestra.run.smithi149.stdout:(3/6): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 277 kB/s | 123 kB 00:00 2024-03-22T06:58:09.666 INFO:teuthology.orchestra.run.smithi081.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-03-22T06:58:10.008 INFO:teuthology.orchestra.run.smithi081.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-03-22T06:58:10.039 INFO:teuthology.orchestra.run.smithi149.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 112 kB/s | 58 kB 00:00 2024-03-22T06:58:10.130 INFO:teuthology.orchestra.run.smithi149.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 107 kB/s | 51 kB 00:00 2024-03-22T06:58:10.150 INFO:teuthology.orchestra.run.smithi081.stdout:(9/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-03-22T06:58:10.225 INFO:teuthology.orchestra.run.smithi081.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 902 kB/s | 195 kB 00:00 2024-03-22T06:58:10.273 INFO:teuthology.orchestra.run.smithi149.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 701 kB/s | 548 kB 00:00 2024-03-22T06:58:10.276 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:10.276 INFO:teuthology.orchestra.run.smithi149.stdout:Total 5.1 MB/s | 6.8 MB 00:01 2024-03-22T06:58:10.289 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:58:10.292 INFO:teuthology.orchestra.run.smithi081.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-03-22T06:58:10.300 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:58:10.301 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:58:10.393 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:58:10.393 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:58:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:58:10.784 INFO:teuthology.orchestra.run.smithi081.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-7 4.4 MB/s | 7.4 MB 00:01 2024-03-22T06:58:10.787 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:10.788 INFO:teuthology.orchestra.run.smithi081.stdout:Total 18 MB/s | 37 MB 00:02 2024-03-22T06:58:10.828 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:10.851 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:10.852 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:10.873 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-22T06:58:10.924 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-22T06:58:11.282 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:11.282 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:11.439 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-22T06:58:11.535 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-22T06:58:11.590 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-22T06:58:11.636 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T06:58:11.993 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:12.067 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-22T06:58:12.096 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-22T06:58:12.206 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-22T06:58:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-22T06:58:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T06:58:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/6 2024-03-22T06:58:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-22T06:58:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-22T06:58:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-22T06:58:12.392 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-22T06:58:12.418 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T06:58:12.448 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T06:58:12.525 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-22T06:58:12.690 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-22T06:58:12.690 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:12.690 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:58:12.690 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:12.690 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-22T06:58:12.690 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-22T06:58:12.690 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-22T06:58:12.691 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-22T06:58:12.691 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-22T06:58:12.691 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:12.691 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:12.946 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-fuse 2024-03-22T06:58:13.163 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-22T06:58:13.278 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-22T06:58:13.520 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:22 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:13.670 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:13.671 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:13.671 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-22T06:58:13.671 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:13.671 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 812 k 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout:Install 2 Packages 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 891 k 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 2.6 M 2024-03-22T06:58:13.672 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:58:13.807 INFO:teuthology.orchestra.run.smithi149.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 582 kB/s | 79 kB 00:00 2024-03-22T06:58:14.125 INFO:teuthology.orchestra.run.smithi149.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el9.x86_6 1.8 MB/s | 812 kB 00:00 2024-03-22T06:58:14.126 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:14.127 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.9 MB/s | 891 kB 00:00 2024-03-22T06:58:14.127 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:58:14.147 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:58:14.147 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:58:14.218 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:58:14.219 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:58:14.411 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:58:14.527 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-22T06:58:14.564 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-22T06:58:14.632 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T06:58:15.283 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T06:58:15.283 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/2 2024-03-22T06:58:15.543 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-22T06:58:15.582 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T06:58:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-22T06:58:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:15.604 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:58:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-22T06:58:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:15.605 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:15.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librados-devel 2024-03-22T06:58:16.470 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:25 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:16.623 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:16.624 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:16.624 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-03-22T06:58:16.624 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:16.624 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:58:16.624 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 126 k 2024-03-22T06:58:16.624 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:16.624 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:58:16.624 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:16.625 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-03-22T06:58:16.625 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:16.625 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 126 k 2024-03-22T06:58:16.625 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 448 k 2024-03-22T06:58:16.625 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:58:16.929 INFO:teuthology.orchestra.run.smithi149.stdout:librados-devel-17.2.7-726.g9c4e0c34.el9.x86_64. 415 kB/s | 126 kB 00:00 2024-03-22T06:58:16.930 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:16.930 INFO:teuthology.orchestra.run.smithi149.stdout:Total 413 kB/s | 126 kB 00:00 2024-03-22T06:58:16.930 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:58:16.936 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:58:16.936 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:58:16.991 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:58:16.991 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:58:17.134 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:58:17.243 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:17.693 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:18.061 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:18.061 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:18.061 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:58:18.061 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:18.061 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:18.061 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:18.154 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T06:58:18.154 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 1/12 2024-03-22T06:58:18.154 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-22T06:58:18.154 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-22T06:58:18.154 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-22T06:58:18.154 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T06:58:18.154 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T06:58:18.155 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-22T06:58:18.155 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-22T06:58:18.155 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-22T06:58:18.155 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-22T06:58:18.155 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-22T06:58:18.309 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install libcephfs2 2024-03-22T06:58:18.571 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-22T06:58:18.572 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-22T06:58:18.573 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-22T06:58:18.573 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:18.573 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:18.896 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:27 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:18.941 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-rook 2024-03-22T06:58:18.997 INFO:teuthology.orchestra.run.smithi149.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:19.049 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:19.050 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-03-22T06:58:19.050 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:19.123 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install libcephfs-devel 2024-03-22T06:58:19.514 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:11 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:19.664 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 47 k 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-22T06:58:19.666 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout:Install 13 Packages 2024-03-22T06:58:19.667 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:19.668 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 2.0 M 2024-03-22T06:58:19.668 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 26 M 2024-03-22T06:58:19.668 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:19.698 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:28 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:19.850 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:19.850 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:19.851 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-03-22T06:58:19.851 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:19.851 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:58:19.851 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T06:58:19.851 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:19.851 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:58:19.852 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:19.852 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-03-22T06:58:19.852 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:19.852 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 25 k 2024-03-22T06:58:19.852 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 110 k 2024-03-22T06:58:19.852 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:58:19.874 INFO:teuthology.orchestra.run.smithi081.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 184 kB/s | 24 kB 00:00 2024-03-22T06:58:19.900 INFO:teuthology.orchestra.run.smithi081.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 114 kB/s | 18 kB 00:00 2024-03-22T06:58:19.950 INFO:teuthology.orchestra.run.smithi081.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-22T06:58:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-03-22T06:58:19.993 INFO:teuthology.orchestra.run.smithi081.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.9 MB/s | 211 kB 00:00 2024-03-22T06:58:20.017 INFO:teuthology.orchestra.run.smithi081.stdout:(6/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el9. 172 kB/s | 47 kB 00:00 2024-03-22T06:58:20.043 INFO:teuthology.orchestra.run.smithi081.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 568 kB/s | 43 kB 00:00 2024-03-22T06:58:20.108 INFO:teuthology.orchestra.run.smithi149.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el9.x86_64 97 kB/s | 25 kB 00:00 2024-03-22T06:58:20.109 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:20.109 INFO:teuthology.orchestra.run.smithi149.stdout:Total 96 kB/s | 25 kB 00:00 2024-03-22T06:58:20.109 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:58:20.112 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:58:20.112 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:58:20.114 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:58:20.114 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:58:20.205 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:58:20.293 INFO:teuthology.orchestra.run.smithi081.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 60 kB/s | 15 kB 00:00 2024-03-22T06:58:20.314 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:20.318 INFO:teuthology.orchestra.run.smithi081.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 99 kB/s | 32 kB 00:00 2024-03-22T06:58:20.535 INFO:teuthology.orchestra.run.smithi081.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 273 kB/s | 59 kB 00:00 2024-03-22T06:58:20.560 INFO:teuthology.orchestra.run.smithi081.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 392 kB/s | 203 kB 00:00 2024-03-22T06:58:20.650 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:20.702 INFO:teuthology.orchestra.run.smithi081.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 538 kB/s | 90 kB 00:00 2024-03-22T06:58:20.836 INFO:teuthology.orchestra.run.smithi081.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2024-03-22T06:58:20.837 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:20.838 INFO:teuthology.orchestra.run.smithi081.stdout:Total 1.7 MB/s | 2.0 MB 00:01 2024-03-22T06:58:20.838 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:20.851 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:20.851 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:20.940 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:20.940 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:20.960 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:20.960 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:20.960 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:58:20.960 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:20.960 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:20.960 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:21.107 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librados2 2024-03-22T06:58:21.445 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:21.614 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-22T06:58:21.669 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-22T06:58:21.679 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:30 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:21.734 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-22T06:58:21.772 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-22T06:58:21.778 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:21.813 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-22T06:58:21.830 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:21.830 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-03-22T06:58:21.831 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:21.901 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librbd1 2024-03-22T06:58:21.907 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-22T06:58:22.013 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-22T06:58:22.072 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-22T06:58:22.479 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:31 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:22.577 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:22.631 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:22.632 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-03-22T06:58:22.632 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:22.694 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rados 2024-03-22T06:58:22.782 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-22T06:58:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-22T06:58:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-22T06:58:22.925 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-22T06:58:22.948 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T06:58:23.265 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:32 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:23.363 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:23.415 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:23.416 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-03-22T06:58:23.416 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:23.479 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rgw 2024-03-22T06:58:23.705 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T06:58:23.705 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 1/13 2024-03-22T06:58:23.705 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-22T06:58:23.705 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-22T06:58:23.705 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-22T06:58:23.705 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-22T06:58:23.705 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-22T06:58:23.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-22T06:58:23.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-22T06:58:23.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-22T06:58:23.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-22T06:58:23.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-22T06:58:23.706 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-22T06:58:24.049 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:33 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:24.053 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-22T06:58:24.053 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:24.053 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:24.053 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:24.053 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-22T06:58:24.053 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:24.054 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:24.147 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:24.199 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:24.200 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-03-22T06:58:24.200 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:24.269 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-cephfs 2024-03-22T06:58:24.299 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-cephadm 2024-03-22T06:58:24.846 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:33 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:24.857 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:16 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:24.943 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:24.995 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:24.996 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-03-22T06:58:24.996 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:25.005 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:25.006 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-03-22T06:58:25.006 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-22T06:58:25.007 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-03-22T06:58:25.007 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:25.007 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 123 k 2024-03-22T06:58:25.007 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-22T06:58:25.007 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-22T06:58:25.007 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-22T06:58:25.008 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-22T06:58:25.008 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-22T06:58:25.008 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-22T06:58:25.008 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:25.008 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:25.008 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-03-22T06:58:25.009 INFO:teuthology.orchestra.run.smithi081.stdout:Install 6 Packages 2024-03-22T06:58:25.009 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:25.010 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 6.8 M 2024-03-22T06:58:25.010 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 33 M 2024-03-22T06:58:25.010 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:25.062 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rbd 2024-03-22T06:58:25.605 INFO:teuthology.orchestra.run.smithi081.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-03-22T06:58:25.637 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:34 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:25.737 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:25.747 INFO:teuthology.orchestra.run.smithi081.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2024-03-22T06:58:25.789 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:25.790 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-03-22T06:58:25.790 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:25.797 INFO:teuthology.orchestra.run.smithi081.stdout:(3/6): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 380 kB/s | 123 kB 00:00 2024-03-22T06:58:25.854 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-fuse 2024-03-22T06:58:26.289 INFO:teuthology.orchestra.run.smithi081.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 111 kB/s | 58 kB 00:00 2024-03-22T06:58:26.314 INFO:teuthology.orchestra.run.smithi081.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 99 kB/s | 51 kB 00:00 2024-03-22T06:58:26.430 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:35 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:26.580 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:26.581 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-03-22T06:58:26.581 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:26.581 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:58:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 82 k 2024-03-22T06:58:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:26.581 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:58:26.581 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:26.581 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-03-22T06:58:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:26.582 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 82 k 2024-03-22T06:58:26.582 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 222 k 2024-03-22T06:58:26.582 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:58:26.765 INFO:teuthology.orchestra.run.smithi081.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 473 kB/s | 548 kB 00:01 2024-03-22T06:58:26.769 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:26.769 INFO:teuthology.orchestra.run.smithi081.stdout:Total 3.8 MB/s | 6.8 MB 00:01 2024-03-22T06:58:26.782 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:26.793 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:26.793 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:26.881 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:26.882 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:26.911 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 248 kB/s | 82 kB 00:00 2024-03-22T06:58:26.912 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:26.912 INFO:teuthology.orchestra.run.smithi149.stdout:Total 247 kB/s | 82 kB 00:00 2024-03-22T06:58:26.912 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:58:26.919 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:58:26.919 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:58:26.974 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:58:26.974 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:58:27.111 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:58:27.175 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:27.209 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:27.355 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-22T06:58:27.399 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-22T06:58:27.588 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:27.877 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:27.877 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:27.877 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:58:27.877 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:27.877 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:27.877 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:27.908 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-22T06:58:27.984 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-22T06:58:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-22T06:58:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T06:58:28.088 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-mirror 2024-03-22T06:58:28.670 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:37 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:28.825 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-03-22T06:58:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:58:28.825 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T06:58:28.825 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:58:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-03-22T06:58:28.826 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 2.9 M 2024-03-22T06:58:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 12 M 2024-03-22T06:58:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:58:28.973 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T06:58:28.973 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/6 2024-03-22T06:58:28.973 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-22T06:58:28.973 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-22T06:58:28.974 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-22T06:58:28.974 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-22T06:58:29.198 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 7.9 MB/s | 2.9 MB 00:00 2024-03-22T06:58:29.198 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:29.201 INFO:teuthology.orchestra.run.smithi149.stdout:Total 7.9 MB/s | 2.9 MB 00:00 2024-03-22T06:58:29.201 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:58:29.208 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:58:29.208 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:58:29.252 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-22T06:58:29.252 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:29.252 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:29.252 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T06:58:29.252 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-22T06:58:29.252 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-22T06:58:29.252 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-22T06:58:29.253 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-22T06:58:29.253 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-22T06:58:29.253 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:29.253 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:29.280 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:58:29.280 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:58:29.529 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-fuse 2024-03-22T06:58:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:58:29.768 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:29.793 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:29.793 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:58:29.793 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-22T06:58:29.793 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T06:58:29.793 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T06:58:29.793 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:30.085 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:22 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:30.235 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:30.235 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:30.235 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-03-22T06:58:30.236 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:30.236 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:30.236 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 812 k 2024-03-22T06:58:30.236 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-03-22T06:58:30.236 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-22T06:58:30.236 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:30.237 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:30.237 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:30.237 INFO:teuthology.orchestra.run.smithi081.stdout:Install 2 Packages 2024-03-22T06:58:30.237 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:30.237 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 891 k 2024-03-22T06:58:30.237 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 2.6 M 2024-03-22T06:58:30.238 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:30.358 INFO:teuthology.orchestra.run.smithi081.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 636 kB/s | 79 kB 00:00 2024-03-22T06:58:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:30.917 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:58:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:30.917 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:31.108 INFO:teuthology.orchestra.run.smithi081.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el9.x86_6 930 kB/s | 812 kB 00:00 2024-03-22T06:58:31.109 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:31.110 INFO:teuthology.orchestra.run.smithi081.stdout:Total 1.0 MB/s | 891 kB 00:00 2024-03-22T06:58:31.110 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:31.131 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:31.131 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:31.170 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-nbd 2024-03-22T06:58:31.204 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:31.204 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:31.381 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:31.512 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-22T06:58:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T06:58:31.756 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:40 ago on Fri 22 Mar 2024 06:56:51 AM UTC. 2024-03-22T06:58:31.905 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 165 k 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 165 k 2024-03-22T06:58:31.906 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 477 k 2024-03-22T06:58:31.907 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-22T06:58:32.139 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 709 kB/s | 165 kB 00:00 2024-03-22T06:58:32.140 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:32.140 INFO:teuthology.orchestra.run.smithi149.stdout:Total 705 kB/s | 165 kB 00:00 2024-03-22T06:58:32.140 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-22T06:58:32.148 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-22T06:58:32.148 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-22T06:58:32.202 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-22T06:58:32.203 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-22T06:58:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T06:58:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/2 2024-03-22T06:58:32.352 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-22T06:58:32.460 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:32.549 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-22T06:58:32.549 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:32.549 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:32.549 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-22T06:58:32.549 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:32.550 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:32.845 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install librados-devel 2024-03-22T06:58:32.864 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:33.191 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:33.192 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:33.192 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-22T06:58:33.192 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:33.192 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T06:58:33.192 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-22T06:58:33.408 DEBUG:teuthology.parallel:result is None 2024-03-22T06:58:33.423 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:25 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:33.573 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 126 k 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-03-22T06:58:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:33.575 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 126 k 2024-03-22T06:58:33.575 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 448 k 2024-03-22T06:58:33.575 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:34.204 INFO:teuthology.orchestra.run.smithi081.stdout:librados-devel-17.2.7-726.g9c4e0c34.el9.x86_64. 201 kB/s | 126 kB 00:00 2024-03-22T06:58:34.205 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:34.205 INFO:teuthology.orchestra.run.smithi081.stdout:Total 200 kB/s | 126 kB 00:00 2024-03-22T06:58:34.205 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:34.211 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:34.211 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:34.266 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:34.266 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:34.406 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:34.502 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:34.992 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:35.273 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:35.273 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:35.273 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:35.273 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:35.273 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:35.274 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:35.525 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install libcephfs2 2024-03-22T06:58:36.095 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:28 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:36.198 INFO:teuthology.orchestra.run.smithi081.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:36.250 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:36.251 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-22T06:58:36.251 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:36.322 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install libcephfs-devel 2024-03-22T06:58:36.884 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:28 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:37.034 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:37.034 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 25 k 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 110 k 2024-03-22T06:58:37.035 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:37.276 INFO:teuthology.orchestra.run.smithi081.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el9.x86_64 104 kB/s | 25 kB 00:00 2024-03-22T06:58:37.277 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:37.277 INFO:teuthology.orchestra.run.smithi081.stdout:Total 103 kB/s | 25 kB 00:00 2024-03-22T06:58:37.277 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:37.280 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:37.281 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:37.282 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:37.282 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:37.367 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:37.465 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:37.859 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:38.101 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:38.101 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:38.101 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:38.101 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:38.101 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:38.101 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:38.300 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install librados2 2024-03-22T06:58:38.860 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:30 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:38.956 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:39.007 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:39.008 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-22T06:58:39.008 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:39.082 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install librbd1 2024-03-22T06:58:39.639 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:31 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:39.735 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:39.788 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:39.789 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-22T06:58:39.789 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:39.857 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-rados 2024-03-22T06:58:40.414 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:32 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:40.511 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:40.564 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:40.565 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-22T06:58:40.565 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:40.632 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-rgw 2024-03-22T06:58:41.186 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:33 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:41.284 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:41.338 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:41.339 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-22T06:58:41.339 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:41.404 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-cephfs 2024-03-22T06:58:41.968 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:33 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:42.064 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:42.116 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:42.117 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-22T06:58:42.117 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:42.191 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-rbd 2024-03-22T06:58:42.747 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:34 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:42.845 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T06:58:42.897 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:42.898 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-03-22T06:58:42.899 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:42.965 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-fuse 2024-03-22T06:58:43.521 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:35 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:43.669 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:43.670 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:43.670 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-03-22T06:58:43.670 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:43.670 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:43.670 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 82 k 2024-03-22T06:58:43.670 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:43.670 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:43.670 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:43.671 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-03-22T06:58:43.671 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:43.671 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 82 k 2024-03-22T06:58:43.671 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 222 k 2024-03-22T06:58:43.671 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:43.955 INFO:teuthology.orchestra.run.smithi081.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 288 kB/s | 82 kB 00:00 2024-03-22T06:58:43.955 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:43.955 INFO:teuthology.orchestra.run.smithi081.stdout:Total 287 kB/s | 82 kB 00:00 2024-03-22T06:58:43.955 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:43.962 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:43.962 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:44.016 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:44.016 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:44.277 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:44.975 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:44.975 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:44.975 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:44.975 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:44.976 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:44.976 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:45.235 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-mirror 2024-03-22T06:58:45.802 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:37 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:45.954 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:45.955 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:45.955 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-03-22T06:58:45.955 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:45.955 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:45.955 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T06:58:45.955 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:45.955 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:45.955 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:45.955 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-03-22T06:58:45.956 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:45.956 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 2.9 M 2024-03-22T06:58:45.956 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 12 M 2024-03-22T06:58:45.956 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:46.333 INFO:teuthology.orchestra.run.smithi081.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 7.8 MB/s | 2.9 MB 00:00 2024-03-22T06:58:46.334 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:46.334 INFO:teuthology.orchestra.run.smithi081.stdout:Total 7.7 MB/s | 2.9 MB 00:00 2024-03-22T06:58:46.334 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:46.343 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:46.344 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:46.418 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:46.419 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:46.841 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:46.901 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:46.927 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:46.927 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T06:58:46.927 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-22T06:58:46.927 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T06:58:46.927 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T06:58:46.927 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:48.073 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:48.073 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:48.073 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:48.073 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:48.074 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:48.074 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:48.352 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-nbd 2024-03-22T06:58:48.930 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:40 ago on Fri 22 Mar 2024 06:57:08 AM UTC. 2024-03-22T06:58:49.078 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-03-22T06:58:49.079 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:49.079 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-03-22T06:58:49.079 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:49.079 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-03-22T06:58:49.079 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 165 k 2024-03-22T06:58:49.079 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:49.079 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-03-22T06:58:49.080 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-03-22T06:58:49.080 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-03-22T06:58:49.080 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:49.080 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 165 k 2024-03-22T06:58:49.080 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 477 k 2024-03-22T06:58:49.080 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-03-22T06:58:49.429 INFO:teuthology.orchestra.run.smithi081.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 473 kB/s | 165 kB 00:00 2024-03-22T06:58:49.430 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:58:49.430 INFO:teuthology.orchestra.run.smithi081.stdout:Total 471 kB/s | 165 kB 00:00 2024-03-22T06:58:49.430 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-03-22T06:58:49.437 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-03-22T06:58:49.437 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-03-22T06:58:49.492 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-03-22T06:58:49.492 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-03-22T06:58:49.649 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-03-22T06:58:49.752 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:50.156 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:50.440 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T06:58:50.440 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:50.440 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-03-22T06:58:50.441 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T06:58:50.441 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:58:50.441 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-03-22T06:58:50.704 DEBUG:teuthology.parallel:result is None 2024-03-22T06:58:50.705 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:58:50.705 INFO:teuthology.packaging:ref: None 2024-03-22T06:58:50.705 INFO:teuthology.packaging:tag: None 2024-03-22T06:58:50.705 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:58:50.705 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T06:58:50.706 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T06:58:50.963 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T06:58:50.983 INFO:teuthology.orchestra.run.smithi081.stdout:17.2.7-726.g9c4e0c34.el9 2024-03-22T06:58:50.984 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el9 2024-03-22T06:58:50.984 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-22T06:58:50.986 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:58:50.986 INFO:teuthology.packaging:ref: None 2024-03-22T06:58:50.986 INFO:teuthology.packaging:tag: None 2024-03-22T06:58:50.986 INFO:teuthology.packaging:branch: quincy 2024-03-22T06:58:50.986 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T06:58:50.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T06:58:51.323 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T06:58:51.348 INFO:teuthology.orchestra.run.smithi149.stdout:17.2.7-726.g9c4e0c34.el9 2024-03-22T06:58:51.349 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el9 2024-03-22T06:58:51.349 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-22T06:58:51.351 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-22T06:58:51.351 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T06:58:51.351 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T06:58:51.395 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T06:58:51.395 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T06:58:51.428 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-22T06:58:51.429 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T06:58:51.429 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T06:58:51.461 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T06:58:51.529 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T06:58:51.529 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T06:58:51.559 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T06:58:51.628 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-22T06:58:51.628 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T06:58:51.628 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T06:58:51.655 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T06:58:51.722 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T06:58:51.722 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T06:58:51.751 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T06:58:51.818 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-22T06:58:51.819 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T06:58:51.819 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T06:58:51.845 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T06:58:51.911 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T06:58:51.912 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T06:58:51.939 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T06:58:52.007 INFO:teuthology.run_tasks:Running task print... 2024-03-22T06:58:52.016 INFO:teuthology.task.print:**** done install task... 2024-03-22T06:58:52.016 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-22T06:58:52.139 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69'} 2024-03-22T06:58:52.140 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-03-22T06:58:52.140 INFO:tasks.cephadm:Cluster fsid is a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T06:58:52.140 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-22T06:58:52.140 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-22T06:58:52.140 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi081': '172.21.15.81', 'mon.smithi149': '172.21.15.149'} 2024-03-22T06:58:52.140 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-22T06:58:52.141 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname $(hostname -s) 2024-03-22T06:58:52.168 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-03-22T06:58:52.196 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-03-22T06:58:52.197 DEBUG:teuthology.orchestra.run.smithi081:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T06:58:52.421 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 22 06:58 /home/ubuntu/cephtest/cephadm 2024-03-22T06:58:52.421 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T06:58:52.583 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 22 06:58 /home/ubuntu/cephtest/cephadm 2024-03-22T06:58:52.583 DEBUG:teuthology.orchestra.run.smithi081:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T06:58:52.599 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T06:58:52.624 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-03-22T06:58:52.624 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-22T06:58:52.641 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-22T06:58:53.498 INFO:teuthology.orchestra.run.smithi081.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T06:58:53.504 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T06:59:24.585 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-22T06:59:24.585 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-22T06:59:24.585 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-22T06:59:24.585 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-03-22T06:59:24.585 INFO:teuthology.orchestra.run.smithi149.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-22T06:59:24.586 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-03-22T06:59:24.586 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-22T06:59:26.949 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T06:59:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-22T06:59:26.949 INFO:teuthology.orchestra.run.smithi081.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-22T06:59:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "repo_digests": [ 2024-03-22T06:59:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-22T06:59:26.950 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-03-22T06:59:26.950 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T06:59:26.968 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph 2024-03-22T06:59:27.007 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-03-22T06:59:27.042 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /etc/ceph 2024-03-22T06:59:27.074 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-03-22T06:59:27.114 INFO:tasks.cephadm:Writing seed config... 2024-03-22T06:59:27.115 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-22T06:59:27.115 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-22T06:59:27.115 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-22T06:59:27.116 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-22T06:59:27.117 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-22T06:59:27.118 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-22T06:59:27.118 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-22T06:59:27.118 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-22T06:59:27.118 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-22T06:59:27.118 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-22T06:59:27.118 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-22T06:59:27.118 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-22T06:59:27.118 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-22T06:59:27.119 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T06:59:27.119 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-22T06:59:27.134 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a42e8632-e819-11ee-95cd-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-22T06:59:27.134 DEBUG:teuthology.orchestra.run.smithi081:mon.smithi081> sudo journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service 2024-03-22T06:59:27.176 INFO:tasks.cephadm:Bootstrapping... 2024-03-22T06:59:27.176 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid a42e8632-e819-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.81 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-22T06:59:27.379 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:59:27.379 INFO:teuthology.orchestra.run.smithi081.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'a42e8632-e819-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.81', '--skip-admin-label'] 2024-03-22T06:59:27.405 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T06:59:27.406 INFO:teuthology.orchestra.run.smithi081.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-22T06:59:27.406 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying podman|docker is present... 2024-03-22T06:59:27.430 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T06:59:27.430 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying lvm2 is present... 2024-03-22T06:59:27.430 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying time synchronization is in place... 2024-03-22T06:59:27.438 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T06:59:27.438 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T06:59:27.444 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T06:59:27.444 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout inactive 2024-03-22T06:59:27.450 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout enabled 2024-03-22T06:59:27.456 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout active 2024-03-22T06:59:27.456 INFO:teuthology.orchestra.run.smithi081.stdout:Unit chronyd.service is enabled and running 2024-03-22T06:59:27.457 INFO:teuthology.orchestra.run.smithi081.stdout:Repeating the final host check... 2024-03-22T06:59:27.481 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T06:59:27.481 INFO:teuthology.orchestra.run.smithi081.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-22T06:59:27.481 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl is present 2024-03-22T06:59:27.481 INFO:teuthology.orchestra.run.smithi081.stdout:lvcreate is present 2024-03-22T06:59:27.487 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T06:59:27.487 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T06:59:27.493 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T06:59:27.493 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout inactive 2024-03-22T06:59:27.500 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout enabled 2024-03-22T06:59:27.506 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout active 2024-03-22T06:59:27.506 INFO:teuthology.orchestra.run.smithi081.stdout:Unit chronyd.service is enabled and running 2024-03-22T06:59:27.506 INFO:teuthology.orchestra.run.smithi081.stdout:Host looks OK 2024-03-22T06:59:27.507 INFO:teuthology.orchestra.run.smithi081.stdout:Cluster fsid: a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T06:59:27.507 INFO:teuthology.orchestra.run.smithi081.stdout:Acquiring lock 140189020942496 on /run/cephadm/a42e8632-e819-11ee-95cd-87774f69a715.lock 2024-03-22T06:59:27.507 INFO:teuthology.orchestra.run.smithi081.stdout:Lock 140189020942496 acquired on /run/cephadm/a42e8632-e819-11ee-95cd-87774f69a715.lock 2024-03-22T06:59:27.507 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying IP 172.21.15.81 port 3300 ... 2024-03-22T06:59:27.507 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying IP 172.21.15.81 port 6789 ... 2024-03-22T06:59:27.507 INFO:teuthology.orchestra.run.smithi081.stdout:Base mon IP(s) is [172.21.15.81:3300, 172.21.15.81:6789], mon addrv is [v2:172.21.15.81:3300,v1:172.21.15.81:6789] 2024-03-22T06:59:27.511 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.81 metric 100 2024-03-22T06:59:27.511 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.81 metric 100 2024-03-22T06:59:27.513 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-22T06:59:27.513 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-22T06:59:27.513 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2024-03-22T06:59:27.516 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-22T06:59:27.516 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-22T06:59:27.516 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T06:59:27.516 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-22T06:59:27.516 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-03-22T06:59:27.516 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T06:59:27.517 INFO:teuthology.orchestra.run.smithi081.stdout:Mon IP `172.21.15.81` is in CIDR network `172.21.0.0/20` 2024-03-22T06:59:27.517 INFO:teuthology.orchestra.run.smithi081.stdout:Mon IP `172.21.15.81` is in CIDR network `172.21.0.0/20` 2024-03-22T06:59:27.517 INFO:teuthology.orchestra.run.smithi081.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-22T06:59:27.517 INFO:teuthology.orchestra.run.smithi081.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-22T06:59:27.518 INFO:teuthology.orchestra.run.smithi081.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T06:59:28.000 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stdout 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-22T06:59:28.000 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T06:59:28.000 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stderr Getting image source signatures 2024-03-22T06:59:28.000 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stderr Copying blob sha256:601ec453fe24458667695dda21742a4f3ea719c31ad8aa8ec48bf585cc42a7d7 2024-03-22T06:59:28.000 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-22T06:59:28.000 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stderr Copying config sha256:837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-22T06:59:28.000 INFO:teuthology.orchestra.run.smithi081.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-22T06:59:28.780 INFO:teuthology.orchestra.run.smithi081.stdout:ceph: stdout ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-22T06:59:28.780 INFO:teuthology.orchestra.run.smithi081.stdout:Ceph version: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-22T06:59:28.780 INFO:teuthology.orchestra.run.smithi081.stdout:Extracting ceph user uid/gid from container image... 2024-03-22T06:59:29.610 INFO:teuthology.orchestra.run.smithi081.stdout:stat: stdout 167 167 2024-03-22T06:59:29.610 INFO:teuthology.orchestra.run.smithi081.stdout:Creating initial keys... 2024-03-22T06:59:30.490 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph-authtool: stdout AQBRLP1lodIbNxAAAPkUusXLkBAtMTgvm6c9Yw== 2024-03-22T06:59:31.422 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph-authtool: stdout AQBSLP1lzzj/LRAA8ZoV3+76qNHP3JwYidS8+A== 2024-03-22T06:59:32.336 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph-authtool: stdout AQBTLP1lgbLwLxAAuRZFm+R5oZCMiOJamCyXmw== 2024-03-22T06:59:32.336 INFO:teuthology.orchestra.run.smithi081.stdout:Creating initial monmap... 2024-03-22T06:59:33.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T06:59:33.187 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-03-22T06:59:33.187 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T06:59:33.187 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T06:59:33.187 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool for smithi081 [v2:172.21.15.81:3300,v1:172.21.15.81:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T06:59:33.187 INFO:teuthology.orchestra.run.smithi081.stdout:setting min_mon_release = octopus 2024-03-22T06:59:33.187 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/monmaptool: set fsid to a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T06:59:33.187 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T06:59:33.187 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T06:59:33.187 INFO:teuthology.orchestra.run.smithi081.stdout:Creating mon... 2024-03-22T06:59:34.126 INFO:teuthology.orchestra.run.smithi081.stdout:create mon.smithi081 on 2024-03-22T06:59:34.316 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-22T06:59:34.469 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-22T06:59:34.615 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a42e8632-e819-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-a42e8632-e819-11ee-95cd-87774f69a715.target. 2024-03-22T06:59:34.616 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a42e8632-e819-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-a42e8632-e819-11ee-95cd-87774f69a715.target. 2024-03-22T06:59:34.791 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081 2024-03-22T06:59:34.791 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service: Unit ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service not loaded. 2024-03-22T06:59:34.964 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a42e8632-e819-11ee-95cd-87774f69a715.target.wants/ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service → /etc/systemd/system/ceph-a42e8632-e819-11ee-95cd-87774f69a715@.service. 2024-03-22T06:59:35.467 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T06:59:35.468 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout disabled 2024-03-22T06:59:35.475 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T06:59:35.475 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout inactive 2024-03-22T06:59:35.475 INFO:teuthology.orchestra.run.smithi081.stdout:firewalld.service is not enabled 2024-03-22T06:59:35.475 INFO:teuthology.orchestra.run.smithi081.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T06:59:35.475 INFO:teuthology.orchestra.run.smithi081.stdout:Waiting for mon to start... 2024-03-22T06:59:35.475 INFO:teuthology.orchestra.run.smithi081.stdout:Waiting for mon... 2024-03-22T06:59:36.402 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout cluster: 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout id: a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout services: 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi081 (age 0.526867s) 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout data: 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-22T06:59:36.403 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout pgs: 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.865+0000 7f9b0ecea700 1 Processor -- start 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.865+0000 7f9b0ecea700 1 -- start start 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.865+0000 7f9b0ecea700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b081019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.865+0000 7f9b0ecea700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b08101f00 con 0x7f9b081015b0 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.866+0000 7f9b0ca86700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b081019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.866+0000 7f9b0ca86700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b081019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46266/0 (socket says 172.21.15.81:46266) 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.866+0000 7f9b0ca86700 1 -- 172.21.15.81:0/552937135 learned_addr learned my addr 172.21.15.81:0/552937135 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.866+0000 7f9b0ca86700 1 -- 172.21.15.81:0/552937135 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b08102040 con 0x7f9b081015b0 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.866+0000 7f9b0ca86700 1 --2- 172.21.15.81:0/552937135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b081019c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0098e0 tx=0x7f9afc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3cbae1a9335d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.866+0000 7f9b077fe700 1 -- 172.21.15.81:0/552937135 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9afc00b860 con 0x7f9b081015b0 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.866+0000 7f9b077fe700 1 -- 172.21.15.81:0/552937135 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc00b9c0 con 0x7f9b081015b0 2024-03-22T06:59:36.404 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.866+0000 7f9b077fe700 1 -- 172.21.15.81:0/552937135 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9afc00bc90 con 0x7f9b081015b0 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.867+0000 7f9b0ecea700 1 -- 172.21.15.81:0/552937135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 msgr2=0x7f9b081019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.867+0000 7f9b0ecea700 1 --2- 172.21.15.81:0/552937135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b081019c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0098e0 tx=0x7f9afc009250 comp rx=0 tx=0).stop 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.867+0000 7f9b0ecea700 1 -- 172.21.15.81:0/552937135 shutdown_connections 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.867+0000 7f9b0ecea700 1 --2- 172.21.15.81:0/552937135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b081019c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.867+0000 7f9b0ecea700 1 -- 172.21.15.81:0/552937135 >> 172.21.15.81:0/552937135 conn(0x7f9b080fcd50 msgr2=0x7f9b080ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.867+0000 7f9b0ecea700 1 -- 172.21.15.81:0/552937135 shutdown_connections 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.867+0000 7f9b0ecea700 1 -- 172.21.15.81:0/552937135 wait complete. 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.867+0000 7f9b0ecea700 1 Processor -- start 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.867+0000 7f9b0ecea700 1 -- start start 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b0ecea700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b08110d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b0ecea700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b08111280 con 0x7f9b081015b0 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b0ca86700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b08110d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:36.405 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b0ca86700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b08110d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46274/0 (socket says 172.21.15.81:46274) 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b0ca86700 1 -- 172.21.15.81:0/2673242804 learned_addr learned my addr 172.21.15.81:0/2673242804 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b0ca86700 1 -- 172.21.15.81:0/2673242804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc0094f0 con 0x7f9b081015b0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b0ca86700 1 --2- 172.21.15.81:0/2673242804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b08110d40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9afc003080 tx=0x7f9afc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b057fa700 1 -- 172.21.15.81:0/2673242804 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9afc01c5c0 con 0x7f9b081015b0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b057fa700 1 -- 172.21.15.81:0/2673242804 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc01c720 con 0x7f9b081015b0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b0ecea700 1 -- 172.21.15.81:0/2673242804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b08111480 con 0x7f9b081015b0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.868+0000 7f9b057fa700 1 -- 172.21.15.81:0/2673242804 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9afc01ca30 con 0x7f9b081015b0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.869+0000 7f9b0ecea700 1 -- 172.21.15.81:0/2673242804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b08111ff0 con 0x7f9b081015b0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.869+0000 7f9b057fa700 1 -- 172.21.15.81:0/2673242804 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9afc01a450 con 0x7f9b081015b0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.869+0000 7f9b057fa700 1 -- 172.21.15.81:0/2673242804 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9afc02e790 con 0x7f9b081015b0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.869+0000 7f9b0ecea700 1 -- 172.21.15.81:0/2673242804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0805f5f0 con 0x7f9b081015b0 2024-03-22T06:59:36.406 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.872+0000 7f9b057fa700 1 -- 172.21.15.81:0/2673242804 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9afc01a890 con 0x7f9b081015b0 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.940+0000 7f9b0ecea700 1 -- 172.21.15.81:0/2673242804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f9b081122f0 con 0x7f9b081015b0 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.940+0000 7f9b057fa700 1 -- 172.21.15.81:0/2673242804 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f9afc01aa30 con 0x7f9b081015b0 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.941+0000 7f9b0ecea700 1 -- 172.21.15.81:0/2673242804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 msgr2=0x7f9b08110d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.941+0000 7f9b0ecea700 1 --2- 172.21.15.81:0/2673242804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b08110d40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9afc003080 tx=0x7f9afc016000 comp rx=0 tx=0).stop 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.941+0000 7f9b0ecea700 1 -- 172.21.15.81:0/2673242804 shutdown_connections 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.941+0000 7f9b0ecea700 1 --2- 172.21.15.81:0/2673242804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b081015b0 0x7f9b08110d40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.941+0000 7f9b0ecea700 1 -- 172.21.15.81:0/2673242804 >> 172.21.15.81:0/2673242804 conn(0x7f9b080fcd50 msgr2=0x7f9b080767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.941+0000 7f9b0ecea700 1 -- 172.21.15.81:0/2673242804 shutdown_connections 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:35.941+0000 7f9b0ecea700 1 -- 172.21.15.81:0/2673242804 wait complete. 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:mon is available 2024-03-22T06:59:36.407 INFO:teuthology.orchestra.run.smithi081.stdout:Assimilating anything we can from ceph.conf... 2024-03-22T06:59:37.458 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:37 smithi081 ceph-mon[28874]: from='client.? 172.21.15.81:0/2560410373' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-22T06:59:37.466 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout fsid = a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.81:3300,v1:172.21.15.81:6789] 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:37.467 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.816+0000 7f96f276d700 1 Processor -- start 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.816+0000 7f96f276d700 1 -- start start 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.816+0000 7f96f276d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec1053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.816+0000 7f96f276d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec105930 con 0x7f96ec076b70 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.816+0000 7f96ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec1053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.816+0000 7f96ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec1053f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46288/0 (socket says 172.21.15.81:46288) 2024-03-22T06:59:37.468 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.816+0000 7f96ebfff700 1 -- 172.21.15.81:0/745341636 learned_addr learned my addr 172.21.15.81:0/745341636 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.816+0000 7f96ebfff700 1 -- 172.21.15.81:0/745341636 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec105a70 con 0x7f96ec076b70 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.816+0000 7f96ebfff700 1 --2- 172.21.15.81:0/745341636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec1053f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f96d40098e0 tx=0x7f96d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcc7abe4bebe88 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96eaffd700 1 -- 172.21.15.81:0/745341636 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96d400b860 con 0x7f96ec076b70 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96eaffd700 1 -- 172.21.15.81:0/745341636 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d400b9c0 con 0x7f96ec076b70 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96eaffd700 1 -- 172.21.15.81:0/745341636 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96d400bc90 con 0x7f96ec076b70 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96f276d700 1 -- 172.21.15.81:0/745341636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 msgr2=0x7f96ec1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96f276d700 1 --2- 172.21.15.81:0/745341636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec1053f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f96d40098e0 tx=0x7f96d4009250 comp rx=0 tx=0).stop 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96f276d700 1 -- 172.21.15.81:0/745341636 shutdown_connections 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96f276d700 1 --2- 172.21.15.81:0/745341636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec1053f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96f276d700 1 -- 172.21.15.81:0/745341636 >> 172.21.15.81:0/745341636 conn(0x7f96ec0fce40 msgr2=0x7f96ec0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96f276d700 1 -- 172.21.15.81:0/745341636 shutdown_connections 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.817+0000 7f96f276d700 1 -- 172.21.15.81:0/745341636 wait complete. 2024-03-22T06:59:37.469 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.818+0000 7f96f276d700 1 Processor -- start 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.818+0000 7f96f276d700 1 -- start start 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.818+0000 7f96f276d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec110e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.818+0000 7f96f276d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec111360 con 0x7f96ec076b70 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.818+0000 7f96ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec110e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.818+0000 7f96ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec110e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46296/0 (socket says 172.21.15.81:46296) 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.818+0000 7f96ebfff700 1 -- 172.21.15.81:0/2560410373 learned_addr learned my addr 172.21.15.81:0/2560410373 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.818+0000 7f96ebfff700 1 -- 172.21.15.81:0/2560410373 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d40094f0 con 0x7f96ec076b70 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.818+0000 7f96ebfff700 1 --2- 172.21.15.81:0/2560410373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec110e20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009290 tx=0x7f96d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.819+0000 7f96e8ff9700 1 -- 172.21.15.81:0/2560410373 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96d401c5c0 con 0x7f96ec076b70 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.819+0000 7f96e8ff9700 1 -- 172.21.15.81:0/2560410373 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d401c740 con 0x7f96ec076b70 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.819+0000 7f96f276d700 1 -- 172.21.15.81:0/2560410373 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec111560 con 0x7f96ec076b70 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.819+0000 7f96e8ff9700 1 -- 172.21.15.81:0/2560410373 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96d401ca50 con 0x7f96ec076b70 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.819+0000 7f96f276d700 1 -- 172.21.15.81:0/2560410373 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec111f90 con 0x7f96ec076b70 2024-03-22T06:59:37.470 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.819+0000 7f96e8ff9700 1 -- 172.21.15.81:0/2560410373 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f96d401a450 con 0x7f96ec076b70 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.819+0000 7f96e8ff9700 1 -- 172.21.15.81:0/2560410373 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f96d402e790 con 0x7f96ec076b70 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.820+0000 7f96f276d700 1 -- 172.21.15.81:0/2560410373 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec05f5f0 con 0x7f96ec076b70 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.822+0000 7f96e8ff9700 1 -- 172.21.15.81:0/2560410373 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f96d401a840 con 0x7f96ec076b70 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.887+0000 7f96f276d700 1 -- 172.21.15.81:0/2560410373 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f96ec10aad0 con 0x7f96ec076b70 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.891+0000 7f96e8ff9700 1 -- 172.21.15.81:0/2560410373 <== mon.0 v2:172.21.15.81:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f96d401a9e0 con 0x7f96ec076b70 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.891+0000 7f96e8ff9700 1 -- 172.21.15.81:0/2560410373 <== mon.0 v2:172.21.15.81:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f96d4024e30 con 0x7f96ec076b70 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.892+0000 7f96f276d700 1 -- 172.21.15.81:0/2560410373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 msgr2=0x7f96ec110e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.892+0000 7f96f276d700 1 --2- 172.21.15.81:0/2560410373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec110e20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009290 tx=0x7f96d4016000 comp rx=0 tx=0).stop 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.892+0000 7f96f276d700 1 -- 172.21.15.81:0/2560410373 shutdown_connections 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.892+0000 7f96f276d700 1 --2- 172.21.15.81:0/2560410373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96ec076b70 0x7f96ec110e20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.892+0000 7f96f276d700 1 -- 172.21.15.81:0/2560410373 >> 172.21.15.81:0/2560410373 conn(0x7f96ec0fce40 msgr2=0x7f96ec0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.892+0000 7f96f276d700 1 -- 172.21.15.81:0/2560410373 shutdown_connections 2024-03-22T06:59:37.471 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:36.892+0000 7f96f276d700 1 -- 172.21.15.81:0/2560410373 wait complete. 2024-03-22T06:59:37.472 INFO:teuthology.orchestra.run.smithi081.stdout:Generating new minimal ceph.conf... 2024-03-22T06:59:38.515 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:38 smithi081 ceph-mon[28874]: from='client.? 172.21.15.81:0/4088011191' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T06:59:38.523 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.894+0000 7f3f3bb3b700 1 Processor -- start 2024-03-22T06:59:38.523 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.895+0000 7f3f3bb3b700 1 -- start start 2024-03-22T06:59:38.523 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.895+0000 7f3f3bb3b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:38.523 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.895+0000 7f3f3bb3b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f34103080 con 0x7f3f34102730 2024-03-22T06:59:38.523 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.895+0000 7f3f398d7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:38.523 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.895+0000 7f3f398d7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46300/0 (socket says 172.21.15.81:46300) 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.895+0000 7f3f398d7700 1 -- 172.21.15.81:0/247214104 learned_addr learned my addr 172.21.15.81:0/247214104 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.895+0000 7f3f398d7700 1 -- 172.21.15.81:0/247214104 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f341031c0 con 0x7f3f34102730 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.895+0000 7f3f398d7700 1 --2- 172.21.15.81:0/247214104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34102b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3f300098e0 tx=0x7f3f30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f5a75be1aa4513 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f388d5700 1 -- 172.21.15.81:0/247214104 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f3000b860 con 0x7f3f34102730 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f388d5700 1 -- 172.21.15.81:0/247214104 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3f3000be50 con 0x7f3f34102730 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f388d5700 1 -- 172.21.15.81:0/247214104 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f30010380 con 0x7f3f34102730 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/247214104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 msgr2=0x7f3f34102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f3bb3b700 1 --2- 172.21.15.81:0/247214104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34102b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3f300098e0 tx=0x7f3f30009250 comp rx=0 tx=0).stop 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/247214104 shutdown_connections 2024-03-22T06:59:38.524 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f3bb3b700 1 --2- 172.21.15.81:0/247214104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34102b40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/247214104 >> 172.21.15.81:0/247214104 conn(0x7f3f340fdf10 msgr2=0x7f3f34100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/247214104 shutdown_connections 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.896+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/247214104 wait complete. 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.897+0000 7f3f3bb3b700 1 Processor -- start 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.897+0000 7f3f3bb3b700 1 -- start start 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.897+0000 7f3f3bb3b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.897+0000 7f3f3bb3b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f34079110 con 0x7f3f34102730 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.897+0000 7f3f398d7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.897+0000 7f3f398d7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46312/0 (socket says 172.21.15.81:46312) 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.897+0000 7f3f398d7700 1 -- 172.21.15.81:0/4088011191 learned_addr learned my addr 172.21.15.81:0/4088011191 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.897+0000 7f3f398d7700 1 -- 172.21.15.81:0/4088011191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300094f0 con 0x7f3f34102730 2024-03-22T06:59:38.525 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.897+0000 7f3f398d7700 1 --2- 172.21.15.81:0/4088011191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34078bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000b440 tx=0x7f3f30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.898+0000 7f3f2a7fc700 1 -- 172.21.15.81:0/4088011191 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f30010760 con 0x7f3f34102730 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.898+0000 7f3f2a7fc700 1 -- 172.21.15.81:0/4088011191 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3f30010d50 con 0x7f3f34102730 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.898+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/4088011191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f34077180 con 0x7f3f34102730 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.898+0000 7f3f2a7fc700 1 -- 172.21.15.81:0/4088011191 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f3001adc0 con 0x7f3f34102730 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.898+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/4088011191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f34077670 con 0x7f3f34102730 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.898+0000 7f3f2a7fc700 1 -- 172.21.15.81:0/4088011191 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3f30019690 con 0x7f3f34102730 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.898+0000 7f3f2a7fc700 1 -- 172.21.15.81:0/4088011191 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3f30024900 con 0x7f3f34102730 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.899+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/4088011191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3404bed0 con 0x7f3f34102730 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.901+0000 7f3f2a7fc700 1 -- 172.21.15.81:0/4088011191 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3f30029060 con 0x7f3f34102730 2024-03-22T06:59:38.526 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.966+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/4088011191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3f34077b20 con 0x7f3f34102730 2024-03-22T06:59:38.527 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.966+0000 7f3f2a7fc700 1 -- 172.21.15.81:0/4088011191 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f3f3001a5f0 con 0x7f3f34102730 2024-03-22T06:59:38.527 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.967+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/4088011191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 msgr2=0x7f3f34078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:38.527 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.967+0000 7f3f3bb3b700 1 --2- 172.21.15.81:0/4088011191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34078bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000b440 tx=0x7f3f30009250 comp rx=0 tx=0).stop 2024-03-22T06:59:38.527 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.967+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/4088011191 shutdown_connections 2024-03-22T06:59:38.527 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.967+0000 7f3f3bb3b700 1 --2- 172.21.15.81:0/4088011191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f34102730 0x7f3f34078bd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:38.527 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.967+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/4088011191 >> 172.21.15.81:0/4088011191 conn(0x7f3f340fdf10 msgr2=0x7f3f340feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:38.527 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.967+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/4088011191 shutdown_connections 2024-03-22T06:59:38.527 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:37.967+0000 7f3f3bb3b700 1 -- 172.21.15.81:0/4088011191 wait complete. 2024-03-22T06:59:38.527 INFO:teuthology.orchestra.run.smithi081.stdout:Restarting the monitor... 2024-03-22T06:59:38.794 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:38 smithi081 systemd[1]: Stopping Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T06:59:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:38 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081[28870]: 2024-03-22T06:59:38.793+0000 7f43864b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi081 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T06:59:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:38 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081[28870]: 2024-03-22T06:59:38.793+0000 7f43864b6700 -1 mon.smithi081@0(leader) e1 *** Got Signal Terminated *** 2024-03-22T06:59:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:38 smithi081 podman[29117]: 2024-03-22 06:59:38.977911735 +0000 UTC m=+0.283554786 container died e545d94bf655478c6f1a426220103e04a9a6348e8a70e9708c9bdec05162ecc9 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9c4e0c3, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-22T06:59:39.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:39 smithi081 podman[29117]: 2024-03-22 06:59:39.23509604 +0000 UTC m=+0.540739099 container cleanup e545d94bf655478c6f1a426220103e04a9a6348e8a70e9708c9bdec05162ecc9 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=quincy-9c4e0c3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T06:59:39.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:39 smithi081 bash[29117]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081 2024-03-22T06:59:39.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:39 smithi081 podman[29129]: 2024-03-22 06:59:39.494390844 +0000 UTC m=+0.516114352 container remove e545d94bf655478c6f1a426220103e04a9a6348e8a70e9708c9bdec05162ecc9 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T06:59:39.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:39 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service: Deactivated successfully. 2024-03-22T06:59:39.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:39 smithi081 systemd[1]: Stopped Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T06:59:39.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:39 smithi081 systemd[1]: Starting Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T06:59:40.118 INFO:teuthology.orchestra.run.smithi081.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-22T06:59:40.151 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:39 smithi081 podman[29226]: 2024-03-22 06:59:39.870573092 +0000 UTC m=+0.113947556 container create 32e741b91680caae3d4735d27fea7aaec08f7c1cf189869ce1e6193b880f1fa2 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-22T06:59:40.151 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:39 smithi081 podman[29226]: 2024-03-22 06:59:39.773725522 +0000 UTC m=+0.017099995 image pull 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de quay.ceph.io/ceph-ci/ceph:quincy 2024-03-22T06:59:40.151 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 podman[29226]: 2024-03-22 06:59:40.048139534 +0000 UTC m=+0.291514009 container init 32e741b91680caae3d4735d27fea7aaec08f7c1cf189869ce1e6193b880f1fa2 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 podman[29226]: 2024-03-22 06:59:40.052077769 +0000 UTC m=+0.295452234 container start 32e741b91680caae3d4735d27fea7aaec08f7c1cf189869ce1e6193b880f1fa2 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable), process ceph-mon, pid 2 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: pidfile_write: ignore empty --pid-file 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: load: jerasure load: lrc 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: RocksDB version: 6.15.5 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Compile date Mar 21 2024 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: DB SUMMARY 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: DB Session ID: KTSH0AA4C6T39MKOFJL7 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: CURRENT file: CURRENT 2024-03-22T06:59:40.152 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi081/store.db dir, Total Num: 1, files: 000008.sst 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi081/store.db: 000010.log size: 97282 ; 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.error_if_exists: 0 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.create_if_missing: 0 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.paranoid_checks: 1 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.env: 0x56145fb75a80 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.fs: Posix File System 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.info_log: 0x561461b77c00 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.statistics: (nil) 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.use_fsync: 0 2024-03-22T06:59:40.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_log_file_size: 0 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.allow_fallocate: 1 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.use_direct_reads: 0 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.db_log_dir: 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi081/store.db 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T06:59:40.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.write_buffer_manager: 0x5614628ecab0 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T06:59:40.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.unordered_write: 0 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.row_cache: None 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.wal_filter: None 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.preserve_deletes: 0 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.two_write_queues: 0 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.atomic_flush: 0 2024-03-22T06:59:40.156 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.log_readahead_size: 0 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_background_jobs: 2 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_background_compactions: -1 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_subcompactions: 1 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T06:59:40.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_open_files: -1 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_background_flushes: -1 2024-03-22T06:59:40.158 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Compression algorithms supported: 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: kZSTD supported: 0 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: kXpressCompression supported: 0 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: kZlibCompression supported: 1 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: kSnappyCompression supported: 1 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: kLZ4Compression supported: 1 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: kBZip2Compression supported: 0 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi081/store.db/MANIFEST-000009 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.merge_operator: 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_filter: None 2024-03-22T06:59:40.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561461bccf58) 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: cache_index_and_filter_blocks: 1 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: pin_top_level_index_and_filter: 1 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: index_type: 0 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: data_block_index_type: 0 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: index_shortening: 1 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: hash_index_allow_collision: 1 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: checksum: 1 2024-03-22T06:59:40.160 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: no_block_cache: 0 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_cache: 0x561461bff090 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_cache_name: BinnedLRUCache 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_cache_options: 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: capacity : 536870912 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: num_shard_bits : 4 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: strict_capacity_limit : 0 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: high_pri_pool_ratio: 0.000 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_cache_compressed: (nil) 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: persistent_cache: (nil) 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_size: 4096 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_size_deviation: 10 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_restart_interval: 16 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: index_block_restart_interval: 1 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: metadata_block_size: 4096 2024-03-22T06:59:40.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: partition_filters: 0 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: use_delta_encoding: 1 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: whole_key_filtering: 1 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: verify_compression: 0 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: read_amp_bytes_per_bit: 0 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: format_version: 4 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: enable_index_compression: 1 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_align: 0 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compression: NoCompression 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T06:59:40.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.num_levels: 7 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T06:59:40.163 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T06:59:40.164 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.arena_block_size: 4194304 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T06:59:40.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.table_properties_collectors: 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.inplace_update_support: 0 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.bloom_locality: 0 2024-03-22T06:59:40.166 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.max_successive_merges: 0 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.ttl: 2592000 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.enable_blob_files: false 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.min_blob_size: 0 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi081/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T06:59:40.167 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711090780078128, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711090780078961, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92554, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90649, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83364, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711090780, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "6901592a-6807-4c27-9bd3-b26e6c3aea42", "db_session_id": "KTSH0AA4C6T39MKOFJL7"}} 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711090780079305, "job": 1, "event": "recovery_finished"} 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi081/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x561461b93180 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: DB pointer 0x561461c68000 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** DB Stats ** 2024-03-22T06:59:40.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** Compaction Stats [default] ** 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: L0 2/0 92.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.4 0.00 0.00 1 0.001 0 0 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Sum 2/0 92.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.4 0.00 0.00 1 0.001 0 0 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.4 0.00 0.00 1 0.001 0 0 2024-03-22T06:59:40.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** Compaction Stats [default] ** 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 151.4 0.00 0.00 1 0.001 0 0 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Cumulative compaction: 0.00 GB write, 29.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Interval compaction: 0.00 GB write, 29.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T06:59:40.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** Compaction Stats [default] ** 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: L0 2/0 92.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.4 0.00 0.00 1 0.001 0 0 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Sum 2/0 92.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.4 0.00 0.00 1 0.001 0 0 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** Compaction Stats [default] ** 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 151.4 0.00 0.00 1 0.001 0 0 2024-03-22T06:59:40.171 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Cumulative compaction: 0.00 GB write, 29.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: starting mon.smithi081 rank 0 at public addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] at bind addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi081 fsid a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mon.smithi081@-1(???) e1 preinit fsid a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mon.smithi081@-1(???).mds e1 new map 2024-03-22T06:59:40.172 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mon.smithi081@-1(???).mds e1 print_map 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: e1 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: legacy client fscid: -1 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: No filesystems configured 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mon.smithi081@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mon.smithi081@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mon.smithi081@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mon.smithi081@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mon.smithi081@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 bash[29226]: 32e741b91680caae3d4735d27fea7aaec08f7c1cf189869ce1e6193b880f1fa2 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 systemd[1]: Started Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T06:59:40.173 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mon.smithi081 is new leader, mons smithi081 in quorum (ranks 0) 2024-03-22T06:59:40.174 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: monmap e1: 1 mons at {smithi081=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T06:59:40.174 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: fsmap 2024-03-22T06:59:40.174 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: osdmap e1: 0 total, 0 up, 0 in 2024-03-22T06:59:40.174 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:40 smithi081 ceph-mon[29245]: mgrmap e1: no daemons active 2024-03-22T06:59:41.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.580+0000 7f0590ab7700 1 Processor -- start 2024-03-22T06:59:41.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.580+0000 7f0590ab7700 1 -- start start 2024-03-22T06:59:41.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.581+0000 7f0590ab7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:41.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.581+0000 7f0590ab7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c1034d0 con 0x7f058c102b80 2024-03-22T06:59:41.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.581+0000 7f058a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:41.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.581+0000 7f058a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46318/0 (socket says 172.21.15.81:46318) 2024-03-22T06:59:41.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.581+0000 7f058a59c700 1 -- 172.21.15.81:0/1463780092 learned_addr learned my addr 172.21.15.81:0/1463780092 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.581+0000 7f058a59c700 1 -- 172.21.15.81:0/1463780092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c103610 con 0x7f058c102b80 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.581+0000 7f058a59c700 1 --2- 172.21.15.81:0/1463780092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c102f90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f058000bb80 tx=0x7f058000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2871c68be97db541 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f058959a700 1 -- 172.21.15.81:0/1463780092 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f058000dd00 con 0x7f058c102b80 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f058959a700 1 -- 172.21.15.81:0/1463780092 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f058000de60 con 0x7f058c102b80 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f058959a700 1 -- 172.21.15.81:0/1463780092 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05800188a0 con 0x7f058c102b80 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f0590ab7700 1 -- 172.21.15.81:0/1463780092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 msgr2=0x7f058c102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f0590ab7700 1 --2- 172.21.15.81:0/1463780092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c102f90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f058000bb80 tx=0x7f058000b4f0 comp rx=0 tx=0).stop 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f0590ab7700 1 -- 172.21.15.81:0/1463780092 shutdown_connections 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f0590ab7700 1 --2- 172.21.15.81:0/1463780092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c102f90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f0590ab7700 1 -- 172.21.15.81:0/1463780092 >> 172.21.15.81:0/1463780092 conn(0x7f058c0fc4d0 msgr2=0x7f058c0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f0590ab7700 1 -- 172.21.15.81:0/1463780092 shutdown_connections 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.582+0000 7f0590ab7700 1 -- 172.21.15.81:0/1463780092 wait complete. 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.583+0000 7f0590ab7700 1 Processor -- start 2024-03-22T06:59:41.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.583+0000 7f0590ab7700 1 -- start start 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.583+0000 7f0590ab7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c10f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.583+0000 7f0590ab7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c10f880 con 0x7f058c102b80 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.583+0000 7f058a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c10f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.583+0000 7f058a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c10f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46328/0 (socket says 172.21.15.81:46328) 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.583+0000 7f058a59c700 1 -- 172.21.15.81:0/1035693048 learned_addr learned my addr 172.21.15.81:0/1035693048 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.583+0000 7f058a59c700 1 -- 172.21.15.81:0/1035693048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058000b790 con 0x7f058c102b80 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.583+0000 7f058a59c700 1 --2- 172.21.15.81:0/1035693048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c10f340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0580018600 tx=0x7f058000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.584+0000 7f057affd700 1 -- 172.21.15.81:0/1035693048 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0580010bd0 con 0x7f058c102b80 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.584+0000 7f057affd700 1 -- 172.21.15.81:0/1035693048 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f058001d050 con 0x7f058c102b80 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.584+0000 7f0590ab7700 1 -- 172.21.15.81:0/1035693048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058c10fa80 con 0x7f058c102b80 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.584+0000 7f057affd700 1 -- 172.21.15.81:0/1035693048 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0580022590 con 0x7f058c102b80 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.584+0000 7f0590ab7700 1 -- 172.21.15.81:0/1035693048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c1105f0 con 0x7f058c102b80 2024-03-22T06:59:41.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.584+0000 7f057affd700 1 -- 172.21.15.81:0/1035693048 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f05800227d0 con 0x7f058c102b80 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.584+0000 7f057affd700 1 -- 172.21.15.81:0/1035693048 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0580014070 con 0x7f058c102b80 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.585+0000 7f0590ab7700 1 -- 172.21.15.81:0/1035693048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c05f5f0 con 0x7f058c102b80 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.587+0000 7f057affd700 1 -- 172.21.15.81:0/1035693048 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0580010d30 con 0x7f058c102b80 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.652+0000 7f0590ab7700 1 -- 172.21.15.81:0/1035693048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f058c101350 con 0x7f058c102b80 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.655+0000 7f057affd700 1 -- 172.21.15.81:0/1035693048 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0580022c10 con 0x7f058c102b80 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.656+0000 7f0590ab7700 1 -- 172.21.15.81:0/1035693048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 msgr2=0x7f058c10f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.656+0000 7f0590ab7700 1 --2- 172.21.15.81:0/1035693048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c10f340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0580018600 tx=0x7f058000b4f0 comp rx=0 tx=0).stop 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.656+0000 7f0590ab7700 1 -- 172.21.15.81:0/1035693048 shutdown_connections 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.656+0000 7f0590ab7700 1 --2- 172.21.15.81:0/1035693048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f058c102b80 0x7f058c10f340 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.656+0000 7f0590ab7700 1 -- 172.21.15.81:0/1035693048 >> 172.21.15.81:0/1035693048 conn(0x7f058c0fc4d0 msgr2=0x7f058c105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.656+0000 7f0590ab7700 1 -- 172.21.15.81:0/1035693048 shutdown_connections 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:40.656+0000 7f0590ab7700 1 -- 172.21.15.81:0/1035693048 wait complete. 2024-03-22T06:59:41.199 INFO:teuthology.orchestra.run.smithi081.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-22T06:59:41.200 INFO:teuthology.orchestra.run.smithi081.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-22T06:59:41.200 INFO:teuthology.orchestra.run.smithi081.stdout:Creating mgr... 2024-03-22T06:59:41.200 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying port 9283 ... 2024-03-22T06:59:41.364 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a42e8632-e819-11ee-95cd-87774f69a715@mgr.smithi081.bcxarw 2024-03-22T06:59:41.364 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a42e8632-e819-11ee-95cd-87774f69a715@mgr.smithi081.bcxarw.service: Unit ceph-a42e8632-e819-11ee-95cd-87774f69a715@mgr.smithi081.bcxarw.service not loaded. 2024-03-22T06:59:41.510 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a42e8632-e819-11ee-95cd-87774f69a715.target.wants/ceph-a42e8632-e819-11ee-95cd-87774f69a715@mgr.smithi081.bcxarw.service → /etc/systemd/system/ceph-a42e8632-e819-11ee-95cd-87774f69a715@.service. 2024-03-22T06:59:41.933 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:41 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1035693048' entity='client.admin' 2024-03-22T06:59:42.030 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T06:59:42.031 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout disabled 2024-03-22T06:59:42.038 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T06:59:42.038 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout inactive 2024-03-22T06:59:42.038 INFO:teuthology.orchestra.run.smithi081.stdout:firewalld.service is not enabled 2024-03-22T06:59:42.038 INFO:teuthology.orchestra.run.smithi081.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T06:59:42.046 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T06:59:42.046 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout disabled 2024-03-22T06:59:42.054 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T06:59:42.054 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout inactive 2024-03-22T06:59:42.054 INFO:teuthology.orchestra.run.smithi081.stdout:firewalld.service is not enabled 2024-03-22T06:59:42.054 INFO:teuthology.orchestra.run.smithi081.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-22T06:59:42.054 INFO:teuthology.orchestra.run.smithi081.stdout:Waiting for mgr to start... 2024-03-22T06:59:42.054 INFO:teuthology.orchestra.run.smithi081.stdout:Waiting for mgr... 2024-03-22T06:59:42.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:42 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3294601778' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T06:59:43.120 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "fsid": "a42e8632-e819-11ee-95cd-87774f69a715", 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 0 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "smithi081" 2024-03-22T06:59:43.121 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T06:59:43.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T06:59:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T06:59:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T06:59:43.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T06:59:43.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T06:59:43.133 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T06:59:35.415586+0000", 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.475+0000 7f3ea2508700 1 Processor -- start 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.475+0000 7f3ea2508700 1 -- start start 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.475+0000 7f3ea2508700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:43.134 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.475+0000 7f3ea2508700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1058f0 con 0x7f3e9c076bb0 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.475+0000 7f3e9bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.475+0000 7f3e9bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46348/0 (socket says 172.21.15.81:46348) 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.475+0000 7f3e9bfff700 1 -- 172.21.15.81:0/2883681893 learned_addr learned my addr 172.21.15.81:0/2883681893 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.476+0000 7f3e9bfff700 1 -- 172.21.15.81:0/2883681893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c105a30 con 0x7f3e9c076bb0 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.476+0000 7f3e9bfff700 1 --2- 172.21.15.81:0/2883681893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1053b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3e900098e0 tx=0x7f3e90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a366dbeadfbc65 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.476+0000 7f3e9affd700 1 -- 172.21.15.81:0/2883681893 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e9000b860 con 0x7f3e9c076bb0 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.476+0000 7f3e9affd700 1 -- 172.21.15.81:0/2883681893 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3e9000be50 con 0x7f3e9c076bb0 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.476+0000 7f3e9affd700 1 -- 172.21.15.81:0/2883681893 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e90010380 con 0x7f3e9c076bb0 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.477+0000 7f3ea2508700 1 -- 172.21.15.81:0/2883681893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 msgr2=0x7f3e9c1053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.477+0000 7f3ea2508700 1 --2- 172.21.15.81:0/2883681893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1053b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3e900098e0 tx=0x7f3e90009250 comp rx=0 tx=0).stop 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.477+0000 7f3ea2508700 1 -- 172.21.15.81:0/2883681893 shutdown_connections 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.477+0000 7f3ea2508700 1 --2- 172.21.15.81:0/2883681893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1053b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.477+0000 7f3ea2508700 1 -- 172.21.15.81:0/2883681893 >> 172.21.15.81:0/2883681893 conn(0x7f3e9c0fce40 msgr2=0x7f3e9c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:43.135 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.477+0000 7f3ea2508700 1 -- 172.21.15.81:0/2883681893 shutdown_connections 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.477+0000 7f3ea2508700 1 -- 172.21.15.81:0/2883681893 wait complete. 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.477+0000 7f3ea2508700 1 Processor -- start 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.478+0000 7f3ea2508700 1 -- start start 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.478+0000 7f3ea2508700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.478+0000 7f3ea2508700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c111b10 con 0x7f3e9c076bb0 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.478+0000 7f3e9bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.478+0000 7f3e9bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46362/0 (socket says 172.21.15.81:46362) 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.478+0000 7f3e9bfff700 1 -- 172.21.15.81:0/3294601778 learned_addr learned my addr 172.21.15.81:0/3294601778 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.478+0000 7f3e9bfff700 1 -- 172.21.15.81:0/3294601778 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900094f0 con 0x7f3e9c076bb0 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.478+0000 7f3e9bfff700 1 --2- 172.21.15.81:0/3294601778 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1115d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e900090b0 tx=0x7f3e90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.478+0000 7f3e98ff9700 1 -- 172.21.15.81:0/3294601778 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e90010760 con 0x7f3e9c076bb0 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.479+0000 7f3e98ff9700 1 -- 172.21.15.81:0/3294601778 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3e90010d50 con 0x7f3e9c076bb0 2024-03-22T06:59:43.136 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.479+0000 7f3e98ff9700 1 -- 172.21.15.81:0/3294601778 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e9001ace0 con 0x7f3e9c076bb0 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.479+0000 7f3ea2508700 1 -- 172.21.15.81:0/3294601778 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c111d10 con 0x7f3e9c076bb0 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.479+0000 7f3ea2508700 1 -- 172.21.15.81:0/3294601778 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c112880 con 0x7f3e9c076bb0 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.479+0000 7f3e98ff9700 1 -- 172.21.15.81:0/3294601778 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3e9002d670 con 0x7f3e9c076bb0 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.479+0000 7f3e98ff9700 1 -- 172.21.15.81:0/3294601778 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3e900247d0 con 0x7f3e9c076bb0 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.480+0000 7f3ea2508700 1 -- 172.21.15.81:0/3294601778 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c05f5f0 con 0x7f3e9c076bb0 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.482+0000 7f3e98ff9700 1 -- 172.21.15.81:0/3294601778 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3e90029070 con 0x7f3e9c076bb0 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.553+0000 7f3ea2508700 1 -- 172.21.15.81:0/3294601778 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3e9c1067e0 con 0x7f3e9c076bb0 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.554+0000 7f3e98ff9700 1 -- 172.21.15.81:0/3294601778 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f3e90010ec0 con 0x7f3e9c076bb0 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.555+0000 7f3ea2508700 1 -- 172.21.15.81:0/3294601778 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 msgr2=0x7f3e9c1115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.555+0000 7f3ea2508700 1 --2- 172.21.15.81:0/3294601778 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1115d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3e900090b0 tx=0x7f3e90009250 comp rx=0 tx=0).stop 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.555+0000 7f3ea2508700 1 -- 172.21.15.81:0/3294601778 shutdown_connections 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.555+0000 7f3ea2508700 1 --2- 172.21.15.81:0/3294601778 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c076bb0 0x7f3e9c1115d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.555+0000 7f3ea2508700 1 -- 172.21.15.81:0/3294601778 >> 172.21.15.81:0/3294601778 conn(0x7f3e9c0fce40 msgr2=0x7f3e9c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:43.137 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.555+0000 7f3ea2508700 1 -- 172.21.15.81:0/3294601778 shutdown_connections 2024-03-22T06:59:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:42.555+0000 7f3ea2508700 1 -- 172.21.15.81:0/3294601778 wait complete. 2024-03-22T06:59:43.138 INFO:teuthology.orchestra.run.smithi081.stdout:mgr not available, waiting (1/15)... 2024-03-22T06:59:46.005 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:45 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/928541459' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T06:59:46.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:46.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:59:46.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "fsid": "a42e8632-e819-11ee-95cd-87774f69a715", 2024-03-22T06:59:46.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T06:59:46.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T06:59:46.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 0 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "smithi081" 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T06:59:46.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T06:59:46.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:46.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T06:59:46.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:46.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T06:59:46.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T06:59:46.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T06:59:46.205 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T06:59:46.206 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T06:59:46.207 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T06:59:35.415586+0000", 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.513+0000 7faa512a9700 1 Processor -- start 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.513+0000 7faa512a9700 1 -- start start 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.513+0000 7faa512a9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.513+0000 7faa512a9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c102030 con 0x7faa4c1016e0 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.514+0000 7faa4affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:46.208 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.514+0000 7faa4affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45590/0 (socket says 172.21.15.81:45590) 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.514+0000 7faa4affd700 1 -- 172.21.15.81:0/429190890 learned_addr learned my addr 172.21.15.81:0/429190890 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.514+0000 7faa4affd700 1 -- 172.21.15.81:0/429190890 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c102170 con 0x7faa4c1016e0 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.514+0000 7faa4affd700 1 --2- 172.21.15.81:0/429190890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c101af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0098e0 tx=0x7faa3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12290ba84109c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.514+0000 7faa49ffb700 1 -- 172.21.15.81:0/429190890 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa3c00b860 con 0x7faa4c1016e0 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.514+0000 7faa49ffb700 1 -- 172.21.15.81:0/429190890 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7faa3c00be50 con 0x7faa4c1016e0 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.514+0000 7faa49ffb700 1 -- 172.21.15.81:0/429190890 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa3c010380 con 0x7faa4c1016e0 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.515+0000 7faa512a9700 1 -- 172.21.15.81:0/429190890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 msgr2=0x7faa4c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.515+0000 7faa512a9700 1 --2- 172.21.15.81:0/429190890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c101af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0098e0 tx=0x7faa3c009250 comp rx=0 tx=0).stop 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.515+0000 7faa512a9700 1 -- 172.21.15.81:0/429190890 shutdown_connections 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.515+0000 7faa512a9700 1 --2- 172.21.15.81:0/429190890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c101af0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.515+0000 7faa512a9700 1 -- 172.21.15.81:0/429190890 >> 172.21.15.81:0/429190890 conn(0x7faa4c0fce40 msgr2=0x7faa4c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.515+0000 7faa512a9700 1 -- 172.21.15.81:0/429190890 shutdown_connections 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.515+0000 7faa512a9700 1 -- 172.21.15.81:0/429190890 wait complete. 2024-03-22T06:59:46.209 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.516+0000 7faa512a9700 1 Processor -- start 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.516+0000 7faa512a9700 1 -- start start 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.516+0000 7faa512a9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c10f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.516+0000 7faa512a9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c10f8d0 con 0x7faa4c1016e0 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.516+0000 7faa4affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c10f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.516+0000 7faa4affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c10f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45606/0 (socket says 172.21.15.81:45606) 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.516+0000 7faa4affd700 1 -- 172.21.15.81:0/928541459 learned_addr learned my addr 172.21.15.81:0/928541459 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.516+0000 7faa4affd700 1 -- 172.21.15.81:0/928541459 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c0094f0 con 0x7faa4c1016e0 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.517+0000 7faa4affd700 1 --2- 172.21.15.81:0/928541459 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c10f390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006b00 tx=0x7faa3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.517+0000 7faa337fe700 1 -- 172.21.15.81:0/928541459 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa3c010760 con 0x7faa4c1016e0 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.517+0000 7faa337fe700 1 -- 172.21.15.81:0/928541459 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7faa3c010d50 con 0x7faa4c1016e0 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.517+0000 7faa512a9700 1 -- 172.21.15.81:0/928541459 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c10fad0 con 0x7faa4c1016e0 2024-03-22T06:59:46.210 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.517+0000 7faa337fe700 1 -- 172.21.15.81:0/928541459 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa3c01ad40 con 0x7faa4c1016e0 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.517+0000 7faa512a9700 1 -- 172.21.15.81:0/928541459 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c110640 con 0x7faa4c1016e0 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.517+0000 7faa337fe700 1 -- 172.21.15.81:0/928541459 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7faa3c02e6c0 con 0x7faa4c1016e0 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.517+0000 7faa337fe700 1 -- 172.21.15.81:0/928541459 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7faa3c024790 con 0x7faa4c1016e0 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.518+0000 7faa512a9700 1 -- 172.21.15.81:0/928541459 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c05f5f0 con 0x7faa4c1016e0 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.520+0000 7faa337fe700 1 -- 172.21.15.81:0/928541459 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7faa3c029070 con 0x7faa4c1016e0 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.591+0000 7faa512a9700 1 -- 172.21.15.81:0/928541459 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7faa4c1034b0 con 0x7faa4c1016e0 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.591+0000 7faa337fe700 1 -- 172.21.15.81:0/928541459 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7faa3c01a5c0 con 0x7faa4c1016e0 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.592+0000 7faa512a9700 1 -- 172.21.15.81:0/928541459 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 msgr2=0x7faa4c10f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.592+0000 7faa512a9700 1 --2- 172.21.15.81:0/928541459 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c10f390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006b00 tx=0x7faa3c009250 comp rx=0 tx=0).stop 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.593+0000 7faa512a9700 1 -- 172.21.15.81:0/928541459 shutdown_connections 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.593+0000 7faa512a9700 1 --2- 172.21.15.81:0/928541459 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa4c1016e0 0x7faa4c10f390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.593+0000 7faa512a9700 1 -- 172.21.15.81:0/928541459 >> 172.21.15.81:0/928541459 conn(0x7faa4c0fce40 msgr2=0x7faa4c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.593+0000 7faa512a9700 1 -- 172.21.15.81:0/928541459 shutdown_connections 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:45.593+0000 7faa512a9700 1 -- 172.21.15.81:0/928541459 wait complete. 2024-03-22T06:59:46.211 INFO:teuthology.orchestra.run.smithi081.stdout:mgr not available, waiting (2/15)... 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: Activating manager daemon smithi081.bcxarw 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: mgrmap e2: smithi081.bcxarw(active, starting, since 0.00278693s) 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: Manager daemon smithi081.bcxarw is now available 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/mirror_snapshot_schedule"}]: dispatch 2024-03-22T06:59:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/trash_purge_schedule"}]: dispatch 2024-03-22T06:59:48.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' 2024-03-22T06:59:48.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' 2024-03-22T06:59:48.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:48 smithi081 ceph-mon[29245]: from='mgr.14100 172.21.15.81:0/1537121440' entity='mgr.smithi081.bcxarw' 2024-03-22T06:59:49.184 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:49.184 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:59:49.184 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "fsid": "a42e8632-e819-11ee-95cd-87774f69a715", 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 0 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "smithi081" 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:49.185 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T06:59:49.186 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T06:59:49.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T06:59:49.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T06:59:49.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T06:59:49.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:49.195 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:49.196 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:49.197 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T06:59:35.415586+0000", 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.566+0000 7fd778a75700 1 Processor -- start 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.566+0000 7fd778a75700 1 -- start start 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.566+0000 7fd778a75700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd774102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.566+0000 7fd778a75700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd774103080 con 0x7fd774102730 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.567+0000 7fd77259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd774102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:49.198 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.567+0000 7fd77259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd774102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45678/0 (socket says 172.21.15.81:45678) 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.567+0000 7fd77259c700 1 -- 172.21.15.81:0/3229440537 learned_addr learned my addr 172.21.15.81:0/3229440537 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.567+0000 7fd77259c700 1 -- 172.21.15.81:0/3229440537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7741031c0 con 0x7fd774102730 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.567+0000 7fd77259c700 1 --2- 172.21.15.81:0/3229440537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd774102b40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd7640098e0 tx=0x7fd764009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e318a89dc36d111b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.567+0000 7fd77159a700 1 -- 172.21.15.81:0/3229440537 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd76400b860 con 0x7fd774102730 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.567+0000 7fd77159a700 1 -- 172.21.15.81:0/3229440537 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd76400be50 con 0x7fd774102730 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.567+0000 7fd77159a700 1 -- 172.21.15.81:0/3229440537 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd764010380 con 0x7fd774102730 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 -- 172.21.15.81:0/3229440537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 msgr2=0x7fd774102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 --2- 172.21.15.81:0/3229440537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd774102b40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd7640098e0 tx=0x7fd764009250 comp rx=0 tx=0).stop 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 -- 172.21.15.81:0/3229440537 shutdown_connections 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 --2- 172.21.15.81:0/3229440537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd774102b40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 -- 172.21.15.81:0/3229440537 >> 172.21.15.81:0/3229440537 conn(0x7fd7740fdf10 msgr2=0x7fd774100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:49.199 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 -- 172.21.15.81:0/3229440537 shutdown_connections 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 -- 172.21.15.81:0/3229440537 wait complete. 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 Processor -- start 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 -- start start 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd7741115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd778a75700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd774111b30 con 0x7fd774102730 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd77259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd7741115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd77259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd7741115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45680/0 (socket says 172.21.15.81:45680) 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.568+0000 7fd77259c700 1 -- 172.21.15.81:0/387296252 learned_addr learned my addr 172.21.15.81:0/387296252 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.569+0000 7fd77259c700 1 -- 172.21.15.81:0/387296252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7640094f0 con 0x7fd774102730 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.569+0000 7fd77259c700 1 --2- 172.21.15.81:0/387296252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd7741115f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd764006fd0 tx=0x7fd764009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:49.200 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.569+0000 7fd762ffd700 1 -- 172.21.15.81:0/387296252 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7640107c0 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.569+0000 7fd762ffd700 1 -- 172.21.15.81:0/387296252 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd764010db0 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.569+0000 7fd778a75700 1 -- 172.21.15.81:0/387296252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd774111d30 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.569+0000 7fd762ffd700 1 -- 172.21.15.81:0/387296252 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd76401ada0 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.569+0000 7fd778a75700 1 -- 172.21.15.81:0/387296252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7741128a0 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.570+0000 7fd762ffd700 1 -- 172.21.15.81:0/387296252 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 2) v1 ==== 43124+0+0 (secure 0 0 0) 0x7fd764019410 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.570+0000 7fd762ffd700 1 -- 172.21.15.81:0/387296252 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd764049a40 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.570+0000 7fd778a75700 1 -- 172.21.15.81:0/387296252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7741121c0 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.572+0000 7fd762ffd700 1 -- 172.21.15.81:0/387296252 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd764022310 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.641+0000 7fd778a75700 1 -- 172.21.15.81:0/387296252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd77405f5f0 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.641+0000 7fd762ffd700 1 -- 172.21.15.81:0/387296252 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fd764047090 con 0x7fd774102730 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.642+0000 7fd778a75700 1 -- 172.21.15.81:0/387296252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 msgr2=0x7fd7741115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:49.201 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.642+0000 7fd778a75700 1 --2- 172.21.15.81:0/387296252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd7741115f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd764006fd0 tx=0x7fd764009250 comp rx=0 tx=0).stop 2024-03-22T06:59:49.202 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.642+0000 7fd778a75700 1 -- 172.21.15.81:0/387296252 shutdown_connections 2024-03-22T06:59:49.202 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.642+0000 7fd778a75700 1 --2- 172.21.15.81:0/387296252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd774102730 0x7fd7741115f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:49.202 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.642+0000 7fd778a75700 1 -- 172.21.15.81:0/387296252 >> 172.21.15.81:0/387296252 conn(0x7fd7740fdf10 msgr2=0x7fd7740feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:49.202 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.642+0000 7fd778a75700 1 -- 172.21.15.81:0/387296252 shutdown_connections 2024-03-22T06:59:49.202 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:48.642+0000 7fd778a75700 1 -- 172.21.15.81:0/387296252 wait complete. 2024-03-22T06:59:49.202 INFO:teuthology.orchestra.run.smithi081.stdout:mgr not available, waiting (3/15)... 2024-03-22T06:59:49.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:49 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/387296252' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T06:59:49.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:49 smithi081 ceph-mon[29245]: mgrmap e3: smithi081.bcxarw(active, since 1.00667s) 2024-03-22T06:59:51.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:51 smithi081 ceph-mon[29245]: mgrmap e4: smithi081.bcxarw(active, since 2s) 2024-03-22T06:59:52.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:52.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout { 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "fsid": "a42e8632-e819-11ee-95cd-87774f69a715", 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 0 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "smithi081" 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-22T06:59:52.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T06:59:52.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:52.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T06:59:52.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T06:59:52.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:52.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T06:59:52.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:52.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T06:59:52.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T06:59:52.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T06:59:52.426 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T06:59:52.426 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T06:59:52.426 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T06:59:52.426 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:52.426 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T06:59:52.426 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T06:59:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ], 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T06:59:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T06:59:35.415586+0000", 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout }, 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout } 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.555+0000 7fdcf98fc700 1 Processor -- start 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.555+0000 7fdcf98fc700 1 -- start start 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.555+0000 7fdcf98fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4105360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.555+0000 7fdcf98fc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf41058a0 con 0x7fdcf40712e0 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.556+0000 7fdcf2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4105360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.556+0000 7fdcf2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4105360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45696/0 (socket says 172.21.15.81:45696) 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.556+0000 7fdcf2ffd700 1 -- 172.21.15.81:0/3601876252 learned_addr learned my addr 172.21.15.81:0/3601876252 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.556+0000 7fdcf2ffd700 1 -- 172.21.15.81:0/3601876252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf41059e0 con 0x7fdcf40712e0 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.556+0000 7fdcf2ffd700 1 --2- 172.21.15.81:0/3601876252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4105360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009e50 tx=0x7fdce4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8145b8f88bd0f17a server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:52.429 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.556+0000 7fdcf1ffb700 1 -- 172.21.15.81:0/3601876252 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdce40092e0 con 0x7fdcf40712e0 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.556+0000 7fdcf1ffb700 1 -- 172.21.15.81:0/3601876252 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdce4012430 con 0x7fdcf40712e0 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.556+0000 7fdcf1ffb700 1 -- 172.21.15.81:0/3601876252 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdce40075c0 con 0x7fdcf40712e0 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.557+0000 7fdcf98fc700 1 -- 172.21.15.81:0/3601876252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 msgr2=0x7fdcf4105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.557+0000 7fdcf98fc700 1 --2- 172.21.15.81:0/3601876252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4105360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009e50 tx=0x7fdce4009840 comp rx=0 tx=0).stop 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.557+0000 7fdcf98fc700 1 -- 172.21.15.81:0/3601876252 shutdown_connections 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.557+0000 7fdcf98fc700 1 --2- 172.21.15.81:0/3601876252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4105360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.557+0000 7fdcf98fc700 1 -- 172.21.15.81:0/3601876252 >> 172.21.15.81:0/3601876252 conn(0x7fdcf40fcd50 msgr2=0x7fdcf40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.557+0000 7fdcf98fc700 1 -- 172.21.15.81:0/3601876252 shutdown_connections 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.557+0000 7fdcf98fc700 1 -- 172.21.15.81:0/3601876252 wait complete. 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.558+0000 7fdcf98fc700 1 Processor -- start 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.558+0000 7fdcf98fc700 1 -- start start 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.558+0000 7fdcf98fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4111650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:52.430 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.558+0000 7fdcf98fc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4111b90 con 0x7fdcf40712e0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.558+0000 7fdcf2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.558+0000 7fdcf2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4111650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45706/0 (socket says 172.21.15.81:45706) 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.558+0000 7fdcf2ffd700 1 -- 172.21.15.81:0/1786351694 learned_addr learned my addr 172.21.15.81:0/1786351694 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.558+0000 7fdcf2ffd700 1 -- 172.21.15.81:0/1786351694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce4009ae0 con 0x7fdcf40712e0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.558+0000 7fdcf2ffd700 1 --2- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4111650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdce4012da0 tx=0x7fdce4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.560+0000 7fdceb7fe700 1 -- 172.21.15.81:0/1786351694 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdce40092e0 con 0x7fdcf40712e0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.560+0000 7fdceb7fe700 1 -- 172.21.15.81:0/1786351694 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdce4007d10 con 0x7fdcf40712e0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.560+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf4111d90 con 0x7fdcf40712e0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.560+0000 7fdceb7fe700 1 -- 172.21.15.81:0/1786351694 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdce401c910 con 0x7fdcf40712e0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.560+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf4112900 con 0x7fdcf40712e0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.561+0000 7fdceb7fe700 1 -- 172.21.15.81:0/1786351694 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fdce4019070 con 0x7fdcf40712e0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.561+0000 7fdceb7fe700 1 --2- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fdce0036790 0x7fdce0038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.561+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf405f5f0 con 0x7fdcf40712e0 2024-03-22T06:59:52.431 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.561+0000 7fdceb7fe700 1 -- 172.21.15.81:0/1786351694 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdce4049df0 con 0x7fdcf40712e0 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.561+0000 7fdcf27fc700 1 --2- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fdce0036790 0x7fdce0038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.562+0000 7fdcf27fc700 1 --2- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fdce0036790 0x7fdce0038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009990 tx=0x7fdcdc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.566+0000 7fdceb7fe700 1 -- 172.21.15.81:0/1786351694 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdce4017080 con 0x7fdcf40712e0 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.828+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fdcf4112c80 con 0x7fdcf40712e0 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.828+0000 7fdceb7fe700 1 -- 172.21.15.81:0/1786351694 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fdce4019910 con 0x7fdcf40712e0 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fdce0036790 msgr2=0x7fdce0038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 --2- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fdce0036790 0x7fdce0038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009990 tx=0x7fdcdc006e40 comp rx=0 tx=0).stop 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 msgr2=0x7fdcf4111650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 --2- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4111650 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdce4012da0 tx=0x7fdce4009840 comp rx=0 tx=0).stop 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 shutdown_connections 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 --2- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fdce0036790 0x7fdce0038c40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 --2- 172.21.15.81:0/1786351694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf40712e0 0x7fdcf4111650 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 >> 172.21.15.81:0/1786351694 conn(0x7fdcf40fcd50 msgr2=0x7fdcf40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:52.432 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 shutdown_connections 2024-03-22T06:59:52.433 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:51.830+0000 7fdcf98fc700 1 -- 172.21.15.81:0/1786351694 wait complete. 2024-03-22T06:59:52.433 INFO:teuthology.orchestra.run.smithi081.stdout:mgr is available 2024-03-22T06:59:52.811 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:52 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1786351694' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T06:59:53.491 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:53.491 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T06:59:53.491 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout fsid = a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T06:59:53.491 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T06:59:53.491 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T06:59:53.491 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T06:59:53.492 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T06:59:53.492 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:53.492 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T06:59:53.492 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T06:59:53.492 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 2024-03-22T06:59:53.492 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T06:59:53.493 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T06:59:53.493 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T06:59:53.493 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T06:59:53.493 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T06:59:53.493 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T06:59:53.493 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.801+0000 7ff3f8fb7700 1 Processor -- start 2024-03-22T06:59:53.494 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.801+0000 7ff3f8fb7700 1 -- start start 2024-03-22T06:59:53.494 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.801+0000 7ff3f8fb7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f41030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:53.494 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.801+0000 7ff3f8fb7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f4103600 con 0x7ff3f4102cb0 2024-03-22T06:59:53.494 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.801+0000 7ff3f259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f41030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:53.494 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.802+0000 7ff3f259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f41030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45708/0 (socket says 172.21.15.81:45708) 2024-03-22T06:59:53.495 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.802+0000 7ff3f259c700 1 -- 172.21.15.81:0/4006622492 learned_addr learned my addr 172.21.15.81:0/4006622492 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:53.495 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.802+0000 7ff3f259c700 1 -- 172.21.15.81:0/4006622492 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4103740 con 0x7ff3f4102cb0 2024-03-22T06:59:53.495 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.802+0000 7ff3f259c700 1 --2- 172.21.15.81:0/4006622492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f41030c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40098e0 tx=0x7ff3e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718741d787cc0b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:53.495 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.802+0000 7ff3f159a700 1 -- 172.21.15.81:0/4006622492 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3e400b860 con 0x7ff3f4102cb0 2024-03-22T06:59:53.495 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.802+0000 7ff3f159a700 1 -- 172.21.15.81:0/4006622492 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff3e400be50 con 0x7ff3f4102cb0 2024-03-22T06:59:53.495 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.802+0000 7ff3f159a700 1 -- 172.21.15.81:0/4006622492 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3e4010380 con 0x7ff3f4102cb0 2024-03-22T06:59:53.495 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/4006622492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 msgr2=0x7ff3f41030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:53.496 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 --2- 172.21.15.81:0/4006622492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f41030c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40098e0 tx=0x7ff3e4009250 comp rx=0 tx=0).stop 2024-03-22T06:59:53.496 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/4006622492 shutdown_connections 2024-03-22T06:59:53.496 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 --2- 172.21.15.81:0/4006622492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f41030c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:53.496 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/4006622492 >> 172.21.15.81:0/4006622492 conn(0x7ff3f40fc4a0 msgr2=0x7ff3f40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:53.496 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/4006622492 shutdown_connections 2024-03-22T06:59:53.496 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/4006622492 wait complete. 2024-03-22T06:59:53.497 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 Processor -- start 2024-03-22T06:59:53.497 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 -- start start 2024-03-22T06:59:53.497 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f4113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:53.497 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.803+0000 7ff3f8fb7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f4113cd0 con 0x7ff3f4102cb0 2024-03-22T06:59:53.497 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3f259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f4113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:53.497 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3f259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f4113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45714/0 (socket says 172.21.15.81:45714) 2024-03-22T06:59:53.498 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3f259c700 1 -- 172.21.15.81:0/1534830139 learned_addr learned my addr 172.21.15.81:0/1534830139 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T06:59:53.498 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3f259c700 1 -- 172.21.15.81:0/1534830139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e40094f0 con 0x7ff3f4102cb0 2024-03-22T06:59:53.498 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3f259c700 1 --2- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f4113790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400b440 tx=0x7ff3e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:53.498 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3e2ffd700 1 -- 172.21.15.81:0/1534830139 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3e4010760 con 0x7ff3f4102cb0 2024-03-22T06:59:53.498 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f4113ed0 con 0x7ff3f4102cb0 2024-03-22T06:59:53.498 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3e2ffd700 1 -- 172.21.15.81:0/1534830139 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff3e4010d50 con 0x7ff3f4102cb0 2024-03-22T06:59:53.499 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f4114a40 con 0x7ff3f4102cb0 2024-03-22T06:59:53.499 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.804+0000 7ff3e2ffd700 1 -- 172.21.15.81:0/1534830139 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3e401ae20 con 0x7ff3f4102cb0 2024-03-22T06:59:53.499 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.805+0000 7ff3e2ffd700 1 -- 172.21.15.81:0/1534830139 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7ff3e40108c0 con 0x7ff3f4102cb0 2024-03-22T06:59:53.499 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.805+0000 7ff3e2ffd700 1 --2- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7ff3dc036720 0x7ff3dc038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T06:59:53.499 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.805+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f405f5f0 con 0x7ff3f4102cb0 2024-03-22T06:59:53.499 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.805+0000 7ff3e2ffd700 1 -- 172.21.15.81:0/1534830139 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff3e404a5c0 con 0x7ff3f4102cb0 2024-03-22T06:59:53.500 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.805+0000 7ff3f1d9b700 1 --2- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7ff3dc036720 0x7ff3dc038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T06:59:53.500 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.806+0000 7ff3f1d9b700 1 --2- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7ff3dc036720 0x7ff3dc038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009990 tx=0x7ff3e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T06:59:53.500 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.810+0000 7ff3e2ffd700 1 -- 172.21.15.81:0/1534830139 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff3e402e440 con 0x7ff3f4102cb0 2024-03-22T06:59:53.500 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.994+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff3f404bed0 con 0x7ff3f4102cb0 2024-03-22T06:59:53.500 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.995+0000 7ff3e2ffd700 1 -- 172.21.15.81:0/1534830139 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff3e4048020 con 0x7ff3f4102cb0 2024-03-22T06:59:53.501 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7ff3dc036720 msgr2=0x7ff3dc038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:53.501 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 --2- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7ff3dc036720 0x7ff3dc038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009990 tx=0x7ff3e8006e40 comp rx=0 tx=0).stop 2024-03-22T06:59:53.501 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 msgr2=0x7ff3f4113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T06:59:53.501 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 --2- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f4113790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400b440 tx=0x7ff3e4009250 comp rx=0 tx=0).stop 2024-03-22T06:59:53.501 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 shutdown_connections 2024-03-22T06:59:53.501 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 --2- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7ff3dc036720 0x7ff3dc038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:53.501 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 --2- 172.21.15.81:0/1534830139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3f4102cb0 0x7ff3f4113790 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T06:59:53.502 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 >> 172.21.15.81:0/1534830139 conn(0x7ff3f40fc4a0 msgr2=0x7ff3f40fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T06:59:53.502 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 shutdown_connections 2024-03-22T06:59:53.502 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:52.997+0000 7ff3f8fb7700 1 -- 172.21.15.81:0/1534830139 wait complete. 2024-03-22T06:59:53.502 INFO:teuthology.orchestra.run.smithi081.stdout:Enabling cephadm module... 2024-03-22T06:59:53.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:53 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1534830139' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-22T06:59:54.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:54 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3447297062' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-22T06:59:55.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:55 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3447297062' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-22T06:59:55.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 06:59:55 smithi081 ceph-mon[29245]: mgrmap e5: smithi081.bcxarw(active, since 6s) 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.890+0000 7fb6d2b40700 1 Processor -- start 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6d2b40700 1 -- start start 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6d2b40700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6d2b40700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc103090 con 0x7fb6cc102740 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6d08dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6d08dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45724/0 (socket says 172.21.15.81:45724) 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6d08dc700 1 -- 172.21.15.81:0/577926311 learned_addr learned my addr 172.21.15.81:0/577926311 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6d08dc700 1 -- 172.21.15.81:0/577926311 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc1031d0 con 0x7fb6cc102740 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6d08dc700 1 --2- 172.21.15.81:0/577926311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc102b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00098e0 tx=0x7fb6c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9056b35216d52b4d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6cb7fe700 1 -- 172.21.15.81:0/577926311 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6c000b860 con 0x7fb6cc102740 2024-03-22T07:00:00.115 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6cb7fe700 1 -- 172.21.15.81:0/577926311 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb6c000be50 con 0x7fb6cc102740 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.891+0000 7fb6cb7fe700 1 -- 172.21.15.81:0/577926311 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6c0010380 con 0x7fb6cc102740 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.892+0000 7fb6d2b40700 1 -- 172.21.15.81:0/577926311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 msgr2=0x7fb6cc102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.892+0000 7fb6d2b40700 1 --2- 172.21.15.81:0/577926311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc102b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00098e0 tx=0x7fb6c0009250 comp rx=0 tx=0).stop 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.892+0000 7fb6d2b40700 1 -- 172.21.15.81:0/577926311 shutdown_connections 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.892+0000 7fb6d2b40700 1 --2- 172.21.15.81:0/577926311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc102b50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.892+0000 7fb6d2b40700 1 -- 172.21.15.81:0/577926311 >> 172.21.15.81:0/577926311 conn(0x7fb6cc0fdea0 msgr2=0x7fb6cc1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.892+0000 7fb6d2b40700 1 -- 172.21.15.81:0/577926311 shutdown_connections 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.892+0000 7fb6d2b40700 1 -- 172.21.15.81:0/577926311 wait complete. 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.892+0000 7fb6d2b40700 1 Processor -- start 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.892+0000 7fb6d2b40700 1 -- start start 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6d2b40700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc1115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6d2b40700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc111af0 con 0x7fb6cc102740 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6d08dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc1115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:00.116 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6d08dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc1115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45730/0 (socket says 172.21.15.81:45730) 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6d08dc700 1 -- 172.21.15.81:0/3447297062 learned_addr learned my addr 172.21.15.81:0/3447297062 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6d08dc700 1 -- 172.21.15.81:0/3447297062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c00094f0 con 0x7fb6cc102740 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6d08dc700 1 --2- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc1115b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0006b00 tx=0x7fb6c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6c97fa700 1 -- 172.21.15.81:0/3447297062 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6c0010760 con 0x7fb6cc102740 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6c97fa700 1 -- 172.21.15.81:0/3447297062 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fb6c0010d50 con 0x7fb6cc102740 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc111cf0 con 0x7fb6cc102740 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.893+0000 7fb6c97fa700 1 -- 172.21.15.81:0/3447297062 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb6c001ad40 con 0x7fb6cc102740 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.894+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc112860 con 0x7fb6cc102740 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.894+0000 7fb6c97fa700 1 -- 172.21.15.81:0/3447297062 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fb6c002e440 con 0x7fb6cc102740 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.894+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6cc05f5f0 con 0x7fb6cc102740 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.894+0000 7fb6c97fa700 1 --2- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fb6bc036740 0x7fb6bc038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:00.117 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.895+0000 7fb6c97fa700 1 -- 172.21.15.81:0/3447297062 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb6c004a580 con 0x7fb6cc102740 2024-03-22T07:00:00.118 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.895+0000 7fb6cbfff700 1 --2- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fb6bc036740 0x7fb6bc038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:00.118 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.895+0000 7fb6cbfff700 1 --2- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fb6bc036740 0x7fb6bc038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009990 tx=0x7fb6b8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:00.118 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:53.899+0000 7fb6c97fa700 1 -- 172.21.15.81:0/3447297062 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb6c0018940 con 0x7fb6cc102740 2024-03-22T07:00:00.118 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.138+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb6cc04bed0 con 0x7fb6cc102740 2024-03-22T07:00:00.118 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.566+0000 7fb6c97fa700 1 -- 172.21.15.81:0/3447297062 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fb6c00243f0 con 0x7fb6cc102740 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.567+0000 7fb6c97fa700 1 -- 172.21.15.81:0/3447297062 <== mon.0 v2:172.21.15.81:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb6c004c490 con 0x7fb6cc102740 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.569+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fb6bc036740 msgr2=0x7fb6bc038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.569+0000 7fb6d2b40700 1 --2- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fb6bc036740 0x7fb6bc038bf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009990 tx=0x7fb6b8006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.569+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 msgr2=0x7fb6cc1115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.569+0000 7fb6d2b40700 1 --2- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc1115b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0006b00 tx=0x7fb6c0009250 comp rx=0 tx=0).stop 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.570+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 shutdown_connections 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.570+0000 7fb6d2b40700 1 --2- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:6800/4111281059,v1:172.21.15.81:6801/4111281059] conn(0x7fb6bc036740 0x7fb6bc038bf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.570+0000 7fb6d2b40700 1 --2- 172.21.15.81:0/3447297062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6cc102740 0x7fb6cc1115b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.570+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 >> 172.21.15.81:0/3447297062 conn(0x7fb6cc0fdea0 msgr2=0x7fb6cc0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:00.119 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.570+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 shutdown_connections 2024-03-22T07:00:00.120 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T06:59:54.570+0000 7fb6d2b40700 1 -- 172.21.15.81:0/3447297062 wait complete. 2024-03-22T07:00:01.571 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: Active manager daemon smithi081.bcxarw restarted 2024-03-22T07:00:01.571 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: Activating manager daemon smithi081.bcxarw 2024-03-22T07:00:01.571 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: osdmap e2: 0 total, 0 up, 0 in 2024-03-22T07:00:01.571 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: mgrmap e6: smithi081.bcxarw(active, starting, since 0.00314442s) 2024-03-22T07:00:01.571 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:00:01.571 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:00:01.571 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:00:01.571 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:00:01.571 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:00:01.572 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: Manager daemon smithi081.bcxarw is now available 2024-03-22T07:00:01.572 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:01.572 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:01.572 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:00:01.572 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:00:01.572 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:01 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:00:02.450 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:02 smithi081 ceph-mon[29245]: Found migration_current of "None". Setting to last migration. 2024-03-22T07:00:02.450 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:02 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:00:02.450 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:02 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/trash_purge_schedule"}]: dispatch 2024-03-22T07:00:02.451 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:02 smithi081 ceph-mon[29245]: [22/Mar/2024:07:00:01] ENGINE Bus STARTING 2024-03-22T07:00:02.451 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:02 smithi081 ceph-mon[29245]: [22/Mar/2024:07:00:01] ENGINE Serving on https://172.21.15.81:7150 2024-03-22T07:00:02.451 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:02 smithi081 ceph-mon[29245]: [22/Mar/2024:07:00:01] ENGINE Bus STARTED 2024-03-22T07:00:02.451 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:02 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:02.451 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:02 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:00:02.451 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:02 smithi081 ceph-mon[29245]: mgrmap e7: smithi081.bcxarw(active, since 1.00589s) 2024-03-22T07:00:04.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:04 smithi081 ceph-mon[29245]: mgrmap e8: smithi081.bcxarw(active, since 2s) 2024-03-22T07:00:04.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:04 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/790919415' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T07:00:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:00:05.018 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-03-22T07:00:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T07:00:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "active_name": "smithi081.bcxarw", 2024-03-22T07:00:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T07:00:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:00:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.398+0000 7f37d40bb700 1 Processor -- start 2024-03-22T07:00:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.398+0000 7f37d40bb700 1 -- start start 2024-03-22T07:00:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.399+0000 7f37d40bb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc1040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:05.019 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.399+0000 7f37d40bb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc104600 con 0x7f37cc103cb0 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.399+0000 7f37d1e57700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc1040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.399+0000 7f37d1e57700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc1040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46516/0 (socket says 172.21.15.81:46516) 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.399+0000 7f37d1e57700 1 -- 172.21.15.81:0/3320356183 learned_addr learned my addr 172.21.15.81:0/3320356183 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.399+0000 7f37d1e57700 1 -- 172.21.15.81:0/3320356183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc104740 con 0x7f37cc103cb0 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.399+0000 7f37d1e57700 1 --2- 172.21.15.81:0/3320356183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc1040c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37c80098e0 tx=0x7f37c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d66d223fd75cb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d0e55700 1 -- 172.21.15.81:0/3320356183 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37c800b860 con 0x7f37cc103cb0 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d0e55700 1 -- 172.21.15.81:0/3320356183 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f37c800be50 con 0x7f37cc103cb0 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d0e55700 1 -- 172.21.15.81:0/3320356183 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37c8010380 con 0x7f37cc103cb0 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d40bb700 1 -- 172.21.15.81:0/3320356183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 msgr2=0x7f37cc1040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d40bb700 1 --2- 172.21.15.81:0/3320356183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc1040c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f37c80098e0 tx=0x7f37c8009250 comp rx=0 tx=0).stop 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d40bb700 1 -- 172.21.15.81:0/3320356183 shutdown_connections 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d40bb700 1 --2- 172.21.15.81:0/3320356183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc1040c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:05.020 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d40bb700 1 -- 172.21.15.81:0/3320356183 >> 172.21.15.81:0/3320356183 conn(0x7f37cc0fc4a0 msgr2=0x7f37cc0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d40bb700 1 -- 172.21.15.81:0/3320356183 shutdown_connections 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.400+0000 7f37d40bb700 1 -- 172.21.15.81:0/3320356183 wait complete. 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.401+0000 7f37d40bb700 1 Processor -- start 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.401+0000 7f37d40bb700 1 -- start start 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.401+0000 7f37d40bb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc111570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.401+0000 7f37d40bb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc111ab0 con 0x7f37cc103cb0 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.401+0000 7f37d1e57700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc111570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.401+0000 7f37d1e57700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc111570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46528/0 (socket says 172.21.15.81:46528) 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.401+0000 7f37d1e57700 1 -- 172.21.15.81:0/790919415 learned_addr learned my addr 172.21.15.81:0/790919415 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.401+0000 7f37d1e57700 1 -- 172.21.15.81:0/790919415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80094f0 con 0x7f37cc103cb0 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.402+0000 7f37d1e57700 1 --2- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc111570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f37c8000c00 tx=0x7f37c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.402+0000 7f37c27fc700 1 -- 172.21.15.81:0/790919415 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37c80107c0 con 0x7f37cc103cb0 2024-03-22T07:00:05.021 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.402+0000 7f37c27fc700 1 -- 172.21.15.81:0/790919415 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f37c8010db0 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.402+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc111cb0 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.402+0000 7f37c27fc700 1 -- 172.21.15.81:0/790919415 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37c801ad00 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.402+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc112820 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.403+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc05f5f0 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.403+0000 7f37c27fc700 1 -- 172.21.15.81:0/790919415 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f37c80193c0 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.403+0000 7f37c27fc700 1 --2- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f37b8036790 0x7f37b8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.403+0000 7f37c27fc700 1 -- 172.21.15.81:0/790919415 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f37c804a800 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.403+0000 7f37d1656700 1 --2- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f37b8036790 0x7f37b8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.403+0000 7f37d1656700 1 --2- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f37b8036790 0x7f37b8038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009990 tx=0x7f37bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.408+0000 7f37c27fc700 1 -- 172.21.15.81:0/790919415 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f37c8010b00 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.645+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f37cc112bc0 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.645+0000 7f37c27fc700 1 -- 172.21.15.81:0/790919415 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v8) v1 ==== 56+0+103 (secure 0 0 0) 0x7f37c8022340 con 0x7f37cc103cb0 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f37b8036790 msgr2=0x7f37b8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:05.022 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 --2- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f37b8036790 0x7f37b8038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009990 tx=0x7f37bc006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 msgr2=0x7f37cc111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 --2- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc111570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f37c8000c00 tx=0x7f37c8009250 comp rx=0 tx=0).stop 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 shutdown_connections 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 --2- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f37b8036790 0x7f37b8038c40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 --2- 172.21.15.81:0/790919415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37cc103cb0 0x7f37cc111570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 >> 172.21.15.81:0/790919415 conn(0x7f37cc0fc4a0 msgr2=0x7f37cc0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 shutdown_connections 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:03.647+0000 7f37d40bb700 1 -- 172.21.15.81:0/790919415 wait complete. 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:Waiting for the mgr to restart... 2024-03-22T07:00:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:Waiting for mgr epoch 8... 2024-03-22T07:00:05.781 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:05 smithi081 ceph-mon[29245]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-22T07:00:05.781 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:05 smithi081 ceph-mon[29245]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.479+0000 7f9e42eb1700 1 Processor -- start 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.479+0000 7f9e42eb1700 1 -- start start 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.479+0000 7f9e42eb1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c077220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.479+0000 7f9e42eb1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c079130 con 0x7f9e3c078d20 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.479+0000 7f9e40c4d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c077220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:06.091 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.479+0000 7f9e40c4d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c077220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34508/0 (socket says 172.21.15.81:34508) 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.479+0000 7f9e40c4d700 1 -- 172.21.15.81:0/2810065096 learned_addr learned my addr 172.21.15.81:0/2810065096 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.480+0000 7f9e40c4d700 1 -- 172.21.15.81:0/2810065096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c077760 con 0x7f9e3c078d20 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.480+0000 7f9e40c4d700 1 --2- 172.21.15.81:0/2810065096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c077220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e300098e0 tx=0x7f9e30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfc814b772a6514 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.480+0000 7f9e3b7fe700 1 -- 172.21.15.81:0/2810065096 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e3000b860 con 0x7f9e3c078d20 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.480+0000 7f9e3b7fe700 1 -- 172.21.15.81:0/2810065096 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9e3000be50 con 0x7f9e3c078d20 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.480+0000 7f9e3b7fe700 1 -- 172.21.15.81:0/2810065096 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e30010380 con 0x7f9e3c078d20 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.480+0000 7f9e42eb1700 1 -- 172.21.15.81:0/2810065096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 msgr2=0x7f9e3c077220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.480+0000 7f9e42eb1700 1 --2- 172.21.15.81:0/2810065096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c077220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9e300098e0 tx=0x7f9e30009250 comp rx=0 tx=0).stop 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.481+0000 7f9e42eb1700 1 -- 172.21.15.81:0/2810065096 shutdown_connections 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.481+0000 7f9e42eb1700 1 --2- 172.21.15.81:0/2810065096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c077220 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.481+0000 7f9e42eb1700 1 -- 172.21.15.81:0/2810065096 >> 172.21.15.81:0/2810065096 conn(0x7f9e3c0fdf10 msgr2=0x7f9e3c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.481+0000 7f9e42eb1700 1 -- 172.21.15.81:0/2810065096 shutdown_connections 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.481+0000 7f9e42eb1700 1 -- 172.21.15.81:0/2810065096 wait complete. 2024-03-22T07:00:06.092 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.481+0000 7f9e42eb1700 1 Processor -- start 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.481+0000 7f9e42eb1700 1 -- start start 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.481+0000 7f9e42eb1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c1137b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.481+0000 7f9e42eb1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c113cf0 con 0x7f9e3c078d20 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e40c4d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c1137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e40c4d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c1137b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34524/0 (socket says 172.21.15.81:34524) 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e40c4d700 1 -- 172.21.15.81:0/1591041293 learned_addr learned my addr 172.21.15.81:0/1591041293 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e40c4d700 1 -- 172.21.15.81:0/1591041293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e300094f0 con 0x7f9e3c078d20 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e40c4d700 1 --2- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c1137b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9e30003050 tx=0x7f9e30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e397fa700 1 -- 172.21.15.81:0/1591041293 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e30010760 con 0x7f9e3c078d20 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e42eb1700 1 -- 172.21.15.81:0/1591041293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c113ef0 con 0x7f9e3c078d20 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e397fa700 1 -- 172.21.15.81:0/1591041293 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9e30010d50 con 0x7f9e3c078d20 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e397fa700 1 -- 172.21.15.81:0/1591041293 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e3001ae20 con 0x7f9e3c078d20 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.482+0000 7f9e42eb1700 1 -- 172.21.15.81:0/1591041293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c114a60 con 0x7f9e3c078d20 2024-03-22T07:00:06.093 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.483+0000 7f9e397fa700 1 -- 172.21.15.81:0/1591041293 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f9e3002e440 con 0x7f9e3c078d20 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.483+0000 7f9e397fa700 1 --2- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f9e2c036770 0x7f9e2c038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.483+0000 7f9e397fa700 1 -- 172.21.15.81:0/1591041293 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e2c039330 con 0x7f9e2c036770 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.483+0000 7f9e397fa700 1 -- 172.21.15.81:0/1591041293 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9e3004a6a0 con 0x7f9e3c078d20 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.483+0000 7f9e3bfff700 1 --2- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f9e2c036770 0x7f9e2c038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.483+0000 7f9e3bfff700 1 --2- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f9e2c036770 0x7f9e2c038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009990 tx=0x7f9e28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.484+0000 7f9e397fa700 1 -- 172.21.15.81:0/1591041293 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f9e2c039330 con 0x7f9e2c036770 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.489+0000 7f9e42eb1700 1 -- 172.21.15.81:0/1591041293 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f9e3c114380 con 0x7f9e2c036770 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e397fa700 1 -- 172.21.15.81:0/1591041293 <== mgr.14120 v2:172.21.15.81:6800/4051369418 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9e3c114380 con 0x7f9e2c036770 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 -- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f9e2c036770 msgr2=0x7f9e2c038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 --2- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f9e2c036770 0x7f9e2c038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009990 tx=0x7f9e28006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 -- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 msgr2=0x7f9e3c1137b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 --2- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c1137b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9e30003050 tx=0x7f9e30009250 comp rx=0 tx=0).stop 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 -- 172.21.15.81:0/1591041293 shutdown_connections 2024-03-22T07:00:06.094 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 --2- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f9e2c036770 0x7f9e2c038c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:06.095 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 --2- 172.21.15.81:0/1591041293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e3c078d20 0x7f9e3c1137b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:06.095 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 -- 172.21.15.81:0/1591041293 >> 172.21.15.81:0/1591041293 conn(0x7f9e3c0fdf10 msgr2=0x7f9e3c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:06.095 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 -- 172.21.15.81:0/1591041293 shutdown_connections 2024-03-22T07:00:06.095 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:05.490+0000 7f9e42eb1700 1 -- 172.21.15.81:0/1591041293 wait complete. 2024-03-22T07:00:06.095 INFO:teuthology.orchestra.run.smithi081.stdout:mgr epoch 8 is available 2024-03-22T07:00:06.095 INFO:teuthology.orchestra.run.smithi081.stdout:Setting orchestrator backend to cephadm... 2024-03-22T07:00:07.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.482+0000 7f63a0551700 1 Processor -- start 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.483+0000 7f63a0551700 1 -- start start 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.483+0000 7f63a0551700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398101a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.483+0000 7f63a0551700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398101f60 con 0x7f6398101610 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.483+0000 7f639e2ed700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.483+0000 7f639e2ed700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34540/0 (socket says 172.21.15.81:34540) 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.483+0000 7f639e2ed700 1 -- 172.21.15.81:0/2850023935 learned_addr learned my addr 172.21.15.81:0/2850023935 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.483+0000 7f639e2ed700 1 -- 172.21.15.81:0/2850023935 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63981020a0 con 0x7f6398101610 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.483+0000 7f639e2ed700 1 --2- 172.21.15.81:0/2850023935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398101a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f639400bb80 tx=0x7f639400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319adf4917079c48 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f639d2eb700 1 -- 172.21.15.81:0/2850023935 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f639400dd00 con 0x7f6398101610 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f639d2eb700 1 -- 172.21.15.81:0/2850023935 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f639400de60 con 0x7f6398101610 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f639d2eb700 1 -- 172.21.15.81:0/2850023935 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63940188a0 con 0x7f6398101610 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f63a0551700 1 -- 172.21.15.81:0/2850023935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 msgr2=0x7f6398101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:07.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f63a0551700 1 --2- 172.21.15.81:0/2850023935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398101a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f639400bb80 tx=0x7f639400b4f0 comp rx=0 tx=0).stop 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f63a0551700 1 -- 172.21.15.81:0/2850023935 shutdown_connections 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f63a0551700 1 --2- 172.21.15.81:0/2850023935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398101a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f63a0551700 1 -- 172.21.15.81:0/2850023935 >> 172.21.15.81:0/2850023935 conn(0x7f63980fcd90 msgr2=0x7f63980ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f63a0551700 1 -- 172.21.15.81:0/2850023935 shutdown_connections 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.484+0000 7f63a0551700 1 -- 172.21.15.81:0/2850023935 wait complete. 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.485+0000 7f63a0551700 1 Processor -- start 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.485+0000 7f63a0551700 1 -- start start 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.485+0000 7f63a0551700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.485+0000 7f63a0551700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398113c40 con 0x7f6398101610 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.485+0000 7f639e2ed700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.485+0000 7f639e2ed700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398113700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34546/0 (socket says 172.21.15.81:34546) 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.485+0000 7f639e2ed700 1 -- 172.21.15.81:0/816031298 learned_addr learned my addr 172.21.15.81:0/816031298 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.485+0000 7f639e2ed700 1 -- 172.21.15.81:0/816031298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639400b790 con 0x7f6398101610 2024-03-22T07:00:07.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.485+0000 7f639e2ed700 1 --2- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398113700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6394010fa0 tx=0x7f639400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.486+0000 7f638effd700 1 -- 172.21.15.81:0/816031298 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6394018e10 con 0x7f6398101610 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.486+0000 7f638effd700 1 -- 172.21.15.81:0/816031298 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6394010b80 con 0x7f6398101610 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.486+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6398113e40 con 0x7f6398101610 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.486+0000 7f638effd700 1 -- 172.21.15.81:0/816031298 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6394021550 con 0x7f6398101610 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.486+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63981149b0 con 0x7f6398101610 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.486+0000 7f638effd700 1 -- 172.21.15.81:0/816031298 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f6394022a10 con 0x7f6398101610 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.487+0000 7f638effd700 1 --2- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f6384036770 0x7f6384038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.487+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639805f5f0 con 0x7f6398101610 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.487+0000 7f638effd700 1 -- 172.21.15.81:0/816031298 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6394014070 con 0x7f6398101610 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.487+0000 7f639daec700 1 --2- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f6384036770 0x7f6384038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.487+0000 7f639daec700 1 --2- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f6384036770 0x7f6384038c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6388009990 tx=0x7f6388006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.492+0000 7f638effd700 1 -- 172.21.15.81:0/816031298 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f639400eea0 con 0x7f6398101610 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.687+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f6398114dc0 con 0x7f6384036770 2024-03-22T07:00:07.316 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.695+0000 7f638effd700 1 -- 172.21.15.81:0/816031298 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6398114dc0 con 0x7f6384036770 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.697+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f6384036770 msgr2=0x7f6384038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.697+0000 7f63a0551700 1 --2- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f6384036770 0x7f6384038c20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6388009990 tx=0x7f6388006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.697+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 msgr2=0x7f6398113700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.697+0000 7f63a0551700 1 --2- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398113700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6394010fa0 tx=0x7f639400b4f0 comp rx=0 tx=0).stop 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.698+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 shutdown_connections 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.698+0000 7f63a0551700 1 --2- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f6384036770 0x7f6384038c20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.698+0000 7f63a0551700 1 --2- 172.21.15.81:0/816031298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6398101610 0x7f6398113700 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.698+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 >> 172.21.15.81:0/816031298 conn(0x7f63980fcd90 msgr2=0x7f6398105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.698+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 shutdown_connections 2024-03-22T07:00:07.317 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:06.698+0000 7f63a0551700 1 -- 172.21.15.81:0/816031298 wait complete. 2024-03-22T07:00:07.899 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:07 smithi081 ceph-mon[29245]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:07.899 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:07 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:07.899 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:07 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:00:08.413 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-22T07:00:08.413 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.704+0000 7f224da0f700 1 Processor -- start 2024-03-22T07:00:08.413 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.704+0000 7f224da0f700 1 -- start start 2024-03-22T07:00:08.413 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.704+0000 7f224da0f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f22481030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:08.413 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.704+0000 7f224da0f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248103620 con 0x7f2248102cd0 2024-03-22T07:00:08.413 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.704+0000 7f2246ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f22481030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:08.413 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.704+0000 7f2246ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f22481030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34560/0 (socket says 172.21.15.81:34560) 2024-03-22T07:00:08.413 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.704+0000 7f2246ffd700 1 -- 172.21.15.81:0/1475406139 learned_addr learned my addr 172.21.15.81:0/1475406139 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.704+0000 7f2246ffd700 1 -- 172.21.15.81:0/1475406139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2248103760 con 0x7f2248102cd0 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.704+0000 7f2246ffd700 1 --2- 172.21.15.81:0/1475406139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f22481030e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22380098e0 tx=0x7f2238009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6e95b90297b297 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f2245ffb700 1 -- 172.21.15.81:0/1475406139 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f223800b8f0 con 0x7f2248102cd0 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f2245ffb700 1 -- 172.21.15.81:0/1475406139 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2238011420 con 0x7f2248102cd0 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f2245ffb700 1 -- 172.21.15.81:0/1475406139 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2238010540 con 0x7f2248102cd0 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f224da0f700 1 -- 172.21.15.81:0/1475406139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 msgr2=0x7f22481030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f224da0f700 1 --2- 172.21.15.81:0/1475406139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f22481030e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22380098e0 tx=0x7f2238009250 comp rx=0 tx=0).stop 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f224da0f700 1 -- 172.21.15.81:0/1475406139 shutdown_connections 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f224da0f700 1 --2- 172.21.15.81:0/1475406139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f22481030e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f224da0f700 1 -- 172.21.15.81:0/1475406139 >> 172.21.15.81:0/1475406139 conn(0x7f22480fc4a0 msgr2=0x7f22480fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f224da0f700 1 -- 172.21.15.81:0/1475406139 shutdown_connections 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.705+0000 7f224da0f700 1 -- 172.21.15.81:0/1475406139 wait complete. 2024-03-22T07:00:08.414 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.706+0000 7f224da0f700 1 Processor -- start 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.706+0000 7f224da0f700 1 -- start start 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.706+0000 7f224da0f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f2248113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.706+0000 7f224da0f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248113d50 con 0x7f2248102cd0 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.706+0000 7f2246ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f2248113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.706+0000 7f2246ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f2248113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34570/0 (socket says 172.21.15.81:34570) 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.706+0000 7f2246ffd700 1 -- 172.21.15.81:0/3733159211 learned_addr learned my addr 172.21.15.81:0/3733159211 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.706+0000 7f2246ffd700 1 -- 172.21.15.81:0/3733159211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22380094f0 con 0x7f2248102cd0 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.707+0000 7f2246ffd700 1 --2- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f2248113810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2238010760 tx=0x7f2238009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.707+0000 7f222f7fe700 1 -- 172.21.15.81:0/3733159211 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2238010800 con 0x7f2248102cd0 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.707+0000 7f222f7fe700 1 -- 172.21.15.81:0/3733159211 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2238010df0 con 0x7f2248102cd0 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.707+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2248113f50 con 0x7f2248102cd0 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.707+0000 7f222f7fe700 1 -- 172.21.15.81:0/3733159211 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22380193c0 con 0x7f2248102cd0 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.707+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2248114ac0 con 0x7f2248102cd0 2024-03-22T07:00:08.415 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.707+0000 7f222f7fe700 1 -- 172.21.15.81:0/3733159211 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f2238019680 con 0x7f2248102cd0 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.708+0000 7f222f7fe700 1 --2- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f2230036790 0x7f2230038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.708+0000 7f222f7fe700 1 -- 172.21.15.81:0/3733159211 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f223804b5f0 con 0x7f2248102cd0 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.708+0000 7f22467fc700 1 --2- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f2230036790 0x7f2230038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.708+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224805f5f0 con 0x7f2248102cd0 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.708+0000 7f22467fc700 1 --2- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f2230036790 0x7f2230038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f223c009990 tx=0x7f223c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.713+0000 7f222f7fe700 1 -- 172.21.15.81:0/3733159211 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2238019980 con 0x7f2248102cd0 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.899+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f22480fd8c0 con 0x7f2230036790 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.899+0000 7f222f7fe700 1 -- 172.21.15.81:0/3733159211 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f22480fd8c0 con 0x7f2230036790 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f2230036790 msgr2=0x7f2230038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 --2- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f2230036790 0x7f2230038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f223c009990 tx=0x7f223c006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 msgr2=0x7f2248113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:08.416 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 --2- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f2248113810 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2238010760 tx=0x7f2238009250 comp rx=0 tx=0).stop 2024-03-22T07:00:08.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 shutdown_connections 2024-03-22T07:00:08.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 --2- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f2230036790 0x7f2230038c40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:08.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 --2- 172.21.15.81:0/3733159211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2248102cd0 0x7f2248113810 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:08.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 >> 172.21.15.81:0/3733159211 conn(0x7f22480fc4a0 msgr2=0x7f22480fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:08.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 shutdown_connections 2024-03-22T07:00:08.417 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:07.901+0000 7f224da0f700 1 -- 172.21.15.81:0/3733159211 wait complete. 2024-03-22T07:00:08.417 INFO:teuthology.orchestra.run.smithi081.stdout:Generating ssh key... 2024-03-22T07:00:08.784 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:08 smithi081 ceph-mon[29245]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:09.612 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.784+0000 7fa60f67e700 1 Processor -- start 2024-03-22T07:00:09.612 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.784+0000 7fa60f67e700 1 -- start start 2024-03-22T07:00:09.612 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.784+0000 7fa60f67e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:09.612 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.784+0000 7fa60f67e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa608103080 con 0x7fa608102730 2024-03-22T07:00:09.612 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.785+0000 7fa60d41a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:09.612 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.785+0000 7fa60d41a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34584/0 (socket says 172.21.15.81:34584) 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.785+0000 7fa60d41a700 1 -- 172.21.15.81:0/453986068 learned_addr learned my addr 172.21.15.81:0/453986068 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.785+0000 7fa60d41a700 1 -- 172.21.15.81:0/453986068 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6081031c0 con 0x7fa608102730 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.785+0000 7fa60d41a700 1 --2- 172.21.15.81:0/453986068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608102b40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa6040098e0 tx=0x7fa604009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229322664922ca3f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.785+0000 7fa5fffff700 1 -- 172.21.15.81:0/453986068 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa60400b860 con 0x7fa608102730 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.785+0000 7fa5fffff700 1 -- 172.21.15.81:0/453986068 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa60400be50 con 0x7fa608102730 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.785+0000 7fa5fffff700 1 -- 172.21.15.81:0/453986068 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa604010380 con 0x7fa608102730 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.786+0000 7fa60f67e700 1 -- 172.21.15.81:0/453986068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 msgr2=0x7fa608102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.786+0000 7fa60f67e700 1 --2- 172.21.15.81:0/453986068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608102b40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa6040098e0 tx=0x7fa604009250 comp rx=0 tx=0).stop 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.786+0000 7fa60f67e700 1 -- 172.21.15.81:0/453986068 shutdown_connections 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.786+0000 7fa60f67e700 1 --2- 172.21.15.81:0/453986068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608102b40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:09.613 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.786+0000 7fa60f67e700 1 -- 172.21.15.81:0/453986068 >> 172.21.15.81:0/453986068 conn(0x7fa6080fdf10 msgr2=0x7fa608100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.786+0000 7fa60f67e700 1 -- 172.21.15.81:0/453986068 shutdown_connections 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.786+0000 7fa60f67e700 1 -- 172.21.15.81:0/453986068 wait complete. 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60f67e700 1 Processor -- start 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60f67e700 1 -- start start 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60f67e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60f67e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa608079110 con 0x7fa608102730 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60d41a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60d41a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34596/0 (socket says 172.21.15.81:34596) 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60d41a700 1 -- 172.21.15.81:0/3494263086 learned_addr learned my addr 172.21.15.81:0/3494263086 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60d41a700 1 -- 172.21.15.81:0/3494263086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040094f0 con 0x7fa608102730 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60d41a700 1 --2- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608078bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b440 tx=0x7fa604009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:09.614 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa5fdffb700 1 -- 172.21.15.81:0/3494263086 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa604010760 con 0x7fa608102730 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa5fdffb700 1 -- 172.21.15.81:0/3494263086 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa604010d50 con 0x7fa608102730 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa608077180 con 0x7fa608102730 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.787+0000 7fa5fdffb700 1 -- 172.21.15.81:0/3494263086 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa60401adc0 con 0x7fa608102730 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.788+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa608077670 con 0x7fa608102730 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.788+0000 7fa5fdffb700 1 -- 172.21.15.81:0/3494263086 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa6040193c0 con 0x7fa608102730 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.788+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60804bed0 con 0x7fa608102730 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.788+0000 7fa5fdffb700 1 --2- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fa5f40367e0 0x7fa5f4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.789+0000 7fa5fdffb700 1 -- 172.21.15.81:0/3494263086 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa604049f80 con 0x7fa608102730 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.789+0000 7fa60cc19700 1 --2- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fa5f40367e0 0x7fa5f4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.789+0000 7fa60cc19700 1 --2- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fa5f40367e0 0x7fa5f4038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009990 tx=0x7fa5f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.793+0000 7fa5fdffb700 1 -- 172.21.15.81:0/3494263086 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa604023710 con 0x7fa608102730 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:08.978+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa608077d70 con 0x7fa5f40367e0 2024-03-22T07:00:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.058+0000 7fa5fdffb700 1 -- 172.21.15.81:0/3494263086 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa608077d70 con 0x7fa5f40367e0 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fa5f40367e0 msgr2=0x7fa5f4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 --2- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fa5f40367e0 0x7fa5f4038c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009990 tx=0x7fa5f8006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 msgr2=0x7fa608078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 --2- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608078bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b440 tx=0x7fa604009250 comp rx=0 tx=0).stop 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 shutdown_connections 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 --2- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fa5f40367e0 0x7fa5f4038c90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 --2- 172.21.15.81:0/3494263086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608102730 0x7fa608078bd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 >> 172.21.15.81:0/3494263086 conn(0x7fa6080fdf10 msgr2=0x7fa6080feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 shutdown_connections 2024-03-22T07:00:09.616 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.060+0000 7fa60f67e700 1 -- 172.21.15.81:0/3494263086 wait complete. 2024-03-22T07:00:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:10 smithi081 ceph-mon[29245]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:10 smithi081 ceph-mon[29245]: Generating ssh key... 2024-03-22T07:00:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:10 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:10 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:11.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:11 smithi081 ceph-mon[29245]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:11.437 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuo35KrZRPxEFbuUZrihiPb9iiptbdb46AKA+aJvSEWFAhyAfbbKMuZtWMbABmuTQHAIKR0FQeC2XjjKUJin9Ed0Ri6afJdP82iBUaNx79m4HFAVNp+8P/ICwzXo8xZg7d8HESsmn3LEmWYxadhuDo8oC0yp4fQ6GSI1EJybdNmeOoqXrBKU2zdn8VQ+l/HaXkraDFfnlBNzL4+SIApuUvIFNCTMA5UGfzm/QzF2+I7OAaMh6MKQ3mAlT2ZfeCoLlozAjoFfdYnh8LTCcjj9xen70Vdpt3Jk9dzw7/ZbQEyhGZrBpnQi/oM5KwsXGehE4BpfVteAI85OummuDqIClxOefKM/O8axxGAn0hCYYX6zrm6P5V7q0NF4eP7ySEKHiXTTcuN3n641hlTcYvCO2aKgpaW4MW8+QRkqmAjQgpeeM+TuSZ/ot0CGZWUVtFC+3TeJo7ZpueuonEEsgR+GpI7vq5LfhUKt7fJYlvJOSGomsp0a2OFX2xxIt2op15RD0= ceph-a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T07:00:11.437 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.990+0000 7f58276fb700 1 Processor -- start 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.990+0000 7f58276fb700 1 -- start start 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.990+0000 7f58276fb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f58201052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.990+0000 7f58276fb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5820105820 con 0x7f5820071360 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.991+0000 7f5825497700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f58201052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.991+0000 7f5825497700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f58201052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34600/0 (socket says 172.21.15.81:34600) 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.991+0000 7f5825497700 1 -- 172.21.15.81:0/2577177852 learned_addr learned my addr 172.21.15.81:0/2577177852 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.991+0000 7f5825497700 1 -- 172.21.15.81:0/2577177852 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5820105960 con 0x7f5820071360 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.991+0000 7f5825497700 1 --2- 172.21.15.81:0/2577177852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f58201052e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f58100098e0 tx=0x7f5810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2620963fb284b32 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.991+0000 7f5817fff700 1 -- 172.21.15.81:0/2577177852 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f581000b860 con 0x7f5820071360 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.991+0000 7f5817fff700 1 -- 172.21.15.81:0/2577177852 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f581000be50 con 0x7f5820071360 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.991+0000 7f5817fff700 1 -- 172.21.15.81:0/2577177852 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5810010380 con 0x7f5820071360 2024-03-22T07:00:11.438 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 -- 172.21.15.81:0/2577177852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 msgr2=0x7f58201052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 --2- 172.21.15.81:0/2577177852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f58201052e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f58100098e0 tx=0x7f5810009250 comp rx=0 tx=0).stop 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 -- 172.21.15.81:0/2577177852 shutdown_connections 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 --2- 172.21.15.81:0/2577177852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f58201052e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 -- 172.21.15.81:0/2577177852 >> 172.21.15.81:0/2577177852 conn(0x7f58200fcd50 msgr2=0x7f58200ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 -- 172.21.15.81:0/2577177852 shutdown_connections 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 -- 172.21.15.81:0/2577177852 wait complete. 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 Processor -- start 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 -- start start 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f5820111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f58276fb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5820111b40 con 0x7f5820071360 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.992+0000 7f5825497700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f5820111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:11.439 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.993+0000 7f5825497700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f5820111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34610/0 (socket says 172.21.15.81:34610) 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.993+0000 7f5825497700 1 -- 172.21.15.81:0/2863164547 learned_addr learned my addr 172.21.15.81:0/2863164547 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.993+0000 7f5825497700 1 -- 172.21.15.81:0/2863164547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58100094f0 con 0x7f5820071360 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.993+0000 7f5825497700 1 --2- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f5820111600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58100090b0 tx=0x7f5810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.993+0000 7f5815ffb700 1 -- 172.21.15.81:0/2863164547 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5810010760 con 0x7f5820071360 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.993+0000 7f5815ffb700 1 -- 172.21.15.81:0/2863164547 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5810010d50 con 0x7f5820071360 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.993+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5820111d40 con 0x7f5820071360 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.993+0000 7f5815ffb700 1 -- 172.21.15.81:0/2863164547 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f581001ace0 con 0x7f5820071360 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.993+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58201128b0 con 0x7f5820071360 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.994+0000 7f5815ffb700 1 -- 172.21.15.81:0/2863164547 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f58100108c0 con 0x7f5820071360 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.994+0000 7f5815ffb700 1 --2- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f580c036790 0x7f580c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.994+0000 7f5815ffb700 1 -- 172.21.15.81:0/2863164547 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f581004a1f0 con 0x7f5820071360 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.994+0000 7f5824c96700 1 --2- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f580c036790 0x7f580c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:11.440 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.994+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582005f5f0 con 0x7f5820071360 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.995+0000 7f5824c96700 1 --2- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f580c036790 0x7f580c038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f581c009990 tx=0x7f581c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:09.999+0000 7f5815ffb700 1 -- 172.21.15.81:0/2863164547 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f581002b940 con 0x7f5820071360 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.190+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f58200fe140 con 0x7f580c036790 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.190+0000 7f5815ffb700 1 -- 172.21.15.81:0/2863164547 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f58200fe140 con 0x7f580c036790 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.192+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f580c036790 msgr2=0x7f580c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.192+0000 7f58276fb700 1 --2- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f580c036790 0x7f580c038c40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f581c009990 tx=0x7f581c006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.192+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 msgr2=0x7f5820111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.192+0000 7f58276fb700 1 --2- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f5820111600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58100090b0 tx=0x7f5810009250 comp rx=0 tx=0).stop 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.192+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 shutdown_connections 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.192+0000 7f58276fb700 1 --2- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f580c036790 0x7f580c038c40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.192+0000 7f58276fb700 1 --2- 172.21.15.81:0/2863164547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5820071360 0x7f5820111600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.192+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 >> 172.21.15.81:0/2863164547 conn(0x7f58200fcd50 msgr2=0x7f58200fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:11.441 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.193+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 shutdown_connections 2024-03-22T07:00:11.442 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:10.193+0000 7f58276fb700 1 -- 172.21.15.81:0/2863164547 wait complete. 2024-03-22T07:00:11.442 INFO:teuthology.orchestra.run.smithi081.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-22T07:00:11.442 INFO:teuthology.orchestra.run.smithi081.stdout:Adding key to root@localhost authorized_keys... 2024-03-22T07:00:11.442 INFO:teuthology.orchestra.run.smithi081.stdout:Adding host smithi081... 2024-03-22T07:00:12.986 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:12 smithi081 ceph-mon[29245]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "addr": "172.21.15.81", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:13.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:13 smithi081 ceph-mon[29245]: Deploying cephadm binary to smithi081 2024-03-22T07:00:14.528 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout Added host 'smithi081' with addr '172.21.15.81' 2024-03-22T07:00:14.528 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.821+0000 7f1752d1d700 1 Processor -- start 2024-03-22T07:00:14.528 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.821+0000 7f1752d1d700 1 -- start start 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.821+0000 7f1752d1d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c0760d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.821+0000 7f1752d1d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c077fd0 con 0x7f174c077bc0 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.821+0000 7f1750ab9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c0760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.821+0000 7f1750ab9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c0760d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34624/0 (socket says 172.21.15.81:34624) 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.821+0000 7f1750ab9700 1 -- 172.21.15.81:0/1668614225 learned_addr learned my addr 172.21.15.81:0/1668614225 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.821+0000 7f1750ab9700 1 -- 172.21.15.81:0/1668614225 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c076610 con 0x7f174c077bc0 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.821+0000 7f1750ab9700 1 --2- 172.21.15.81:0/1668614225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c0760d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17400098e0 tx=0x7f1740009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b6285add4f47c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f174b7fe700 1 -- 172.21.15.81:0/1668614225 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f174000b860 con 0x7f174c077bc0 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f174b7fe700 1 -- 172.21.15.81:0/1668614225 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f174000be50 con 0x7f174c077bc0 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f174b7fe700 1 -- 172.21.15.81:0/1668614225 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1740010380 con 0x7f174c077bc0 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f1752d1d700 1 -- 172.21.15.81:0/1668614225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 msgr2=0x7f174c0760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:14.529 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f1752d1d700 1 --2- 172.21.15.81:0/1668614225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c0760d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f17400098e0 tx=0x7f1740009250 comp rx=0 tx=0).stop 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f1752d1d700 1 -- 172.21.15.81:0/1668614225 shutdown_connections 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f1752d1d700 1 --2- 172.21.15.81:0/1668614225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c0760d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f1752d1d700 1 -- 172.21.15.81:0/1668614225 >> 172.21.15.81:0/1668614225 conn(0x7f174c0fde70 msgr2=0x7f174c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f1752d1d700 1 -- 172.21.15.81:0/1668614225 shutdown_connections 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.822+0000 7f1752d1d700 1 -- 172.21.15.81:0/1668614225 wait complete. 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.823+0000 7f1752d1d700 1 Processor -- start 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.823+0000 7f1752d1d700 1 -- start start 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.823+0000 7f1752d1d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c1137d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.823+0000 7f1752d1d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c113d10 con 0x7f174c077bc0 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.823+0000 7f1750ab9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c1137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.823+0000 7f1750ab9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c1137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34636/0 (socket says 172.21.15.81:34636) 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.823+0000 7f1750ab9700 1 -- 172.21.15.81:0/3223708878 learned_addr learned my addr 172.21.15.81:0/3223708878 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.824+0000 7f1750ab9700 1 -- 172.21.15.81:0/3223708878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400094f0 con 0x7f174c077bc0 2024-03-22T07:00:14.530 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.824+0000 7f1750ab9700 1 --2- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c1137d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f174000b440 tx=0x7f1740009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.824+0000 7f17497fa700 1 -- 172.21.15.81:0/3223708878 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1740010760 con 0x7f174c077bc0 2024-03-22T07:00:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.824+0000 7f17497fa700 1 -- 172.21.15.81:0/3223708878 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1740010d50 con 0x7f174c077bc0 2024-03-22T07:00:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.824+0000 7f17497fa700 1 -- 172.21.15.81:0/3223708878 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f174001adc0 con 0x7f174c077bc0 2024-03-22T07:00:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.824+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c113f10 con 0x7f174c077bc0 2024-03-22T07:00:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.824+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c114a80 con 0x7f174c077bc0 2024-03-22T07:00:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.825+0000 7f17497fa700 1 -- 172.21.15.81:0/3223708878 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f17400108c0 con 0x7f174c077bc0 2024-03-22T07:00:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.825+0000 7f17497fa700 1 --2- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f173c0367e0 0x7f173c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.825+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174c05f5f0 con 0x7f174c077bc0 2024-03-22T07:00:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.825+0000 7f17497fa700 1 -- 172.21.15.81:0/3223708878 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1740049fe0 con 0x7f174c077bc0 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.825+0000 7f174bfff700 1 --2- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f173c0367e0 0x7f173c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.825+0000 7f174bfff700 1 --2- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f173c0367e0 0x7f173c038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1738009990 tx=0x7f1738006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:11.830+0000 7f17497fa700 1 -- 172.21.15.81:0/3223708878 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1740022d40 con 0x7f174c077bc0 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:12.019+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi081", "addr": "172.21.15.81", "target": ["mon-mgr", ""]}) v1 -- 0x7f174c0ff1c0 con 0x7f173c0367e0 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.010+0000 7f17497fa700 1 -- 172.21.15.81:0/3223708878 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f174c0ff1c0 con 0x7f173c0367e0 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f173c0367e0 msgr2=0x7f173c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 --2- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f173c0367e0 0x7f173c038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1738009990 tx=0x7f1738006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 msgr2=0x7f174c1137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 --2- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c1137d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f174000b440 tx=0x7f1740009250 comp rx=0 tx=0).stop 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 shutdown_connections 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 --2- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f173c0367e0 0x7f173c038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 --2- 172.21.15.81:0/3223708878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f174c077bc0 0x7f174c1137d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:14.532 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 >> 172.21.15.81:0/3223708878 conn(0x7f174c0fde70 msgr2=0x7f174c0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:14.533 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 shutdown_connections 2024-03-22T07:00:14.533 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.012+0000 7f1752d1d700 1 -- 172.21.15.81:0/3223708878 wait complete. 2024-03-22T07:00:14.533 INFO:teuthology.orchestra.run.smithi081.stdout:Deploying mon service with default placement... 2024-03-22T07:00:15.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:15 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:15.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:15 smithi081 ceph-mon[29245]: Added host smithi081 2024-03-22T07:00:15.159 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:15 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:00:15.929 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-22T07:00:15.929 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.934+0000 7f902470e700 1 Processor -- start 2024-03-22T07:00:15.929 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.934+0000 7f902470e700 1 -- start start 2024-03-22T07:00:15.929 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.934+0000 7f902470e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c1052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:15.929 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.934+0000 7f902470e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c105820 con 0x7f901c071360 2024-03-22T07:00:15.929 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.935+0000 7f90224aa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c1052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:15.929 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.935+0000 7f90224aa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c1052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41210/0 (socket says 172.21.15.81:41210) 2024-03-22T07:00:15.929 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.935+0000 7f90224aa700 1 -- 172.21.15.81:0/2802896301 learned_addr learned my addr 172.21.15.81:0/2802896301 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:15.929 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.935+0000 7f90224aa700 1 -- 172.21.15.81:0/2802896301 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901c105960 con 0x7f901c071360 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.935+0000 7f90224aa700 1 --2- 172.21.15.81:0/2802896301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c1052e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f900c0098e0 tx=0x7f900c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed61aa9f98a1b04 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.935+0000 7f90214a8700 1 -- 172.21.15.81:0/2802896301 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f900c00b860 con 0x7f901c071360 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.935+0000 7f90214a8700 1 -- 172.21.15.81:0/2802896301 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f900c00be50 con 0x7f901c071360 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.935+0000 7f90214a8700 1 -- 172.21.15.81:0/2802896301 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f900c010380 con 0x7f901c071360 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.936+0000 7f902470e700 1 -- 172.21.15.81:0/2802896301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 msgr2=0x7f901c1052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.936+0000 7f902470e700 1 --2- 172.21.15.81:0/2802896301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c1052e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f900c0098e0 tx=0x7f900c009250 comp rx=0 tx=0).stop 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.936+0000 7f902470e700 1 -- 172.21.15.81:0/2802896301 shutdown_connections 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.936+0000 7f902470e700 1 --2- 172.21.15.81:0/2802896301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c1052e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.936+0000 7f902470e700 1 -- 172.21.15.81:0/2802896301 >> 172.21.15.81:0/2802896301 conn(0x7f901c0fcd50 msgr2=0x7f901c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.936+0000 7f902470e700 1 -- 172.21.15.81:0/2802896301 shutdown_connections 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.936+0000 7f902470e700 1 -- 172.21.15.81:0/2802896301 wait complete. 2024-03-22T07:00:15.930 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.936+0000 7f902470e700 1 Processor -- start 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f902470e700 1 -- start start 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f902470e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f902470e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c111b40 con 0x7f901c071360 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f90224aa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f90224aa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41212/0 (socket says 172.21.15.81:41212) 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f90224aa700 1 -- 172.21.15.81:0/2855199491 learned_addr learned my addr 172.21.15.81:0/2855199491 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f90224aa700 1 -- 172.21.15.81:0/2855199491 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0094f0 con 0x7f901c071360 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f90224aa700 1 --2- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c111600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f900c011bd0 tx=0x7f900c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f9012ffd700 1 -- 172.21.15.81:0/2855199491 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f900c0107c0 con 0x7f901c071360 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f9012ffd700 1 -- 172.21.15.81:0/2855199491 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f900c010db0 con 0x7f901c071360 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f9012ffd700 1 -- 172.21.15.81:0/2855199491 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f900c01ad60 con 0x7f901c071360 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c111d40 con 0x7f901c071360 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.937+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c1128b0 con 0x7f901c071360 2024-03-22T07:00:15.931 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.938+0000 7f9012ffd700 1 -- 172.21.15.81:0/2855199491 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f900c010920 con 0x7f901c071360 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.938+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c05f5f0 con 0x7f901c071360 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.938+0000 7f9012ffd700 1 --2- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f90080367e0 0x7f9008038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.938+0000 7f9012ffd700 1 -- 172.21.15.81:0/2855199491 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f900c04ab60 con 0x7f901c071360 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.938+0000 7f9021ca9700 1 --2- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f90080367e0 0x7f9008038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.939+0000 7f9021ca9700 1 --2- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f90080367e0 0x7f9008038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9018009990 tx=0x7f9018006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:14.943+0000 7f9012ffd700 1 -- 172.21.15.81:0/2855199491 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f900c02d3e0 con 0x7f901c071360 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.158+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f901c0fe140 con 0x7f90080367e0 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.165+0000 7f9012ffd700 1 -- 172.21.15.81:0/2855199491 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f901c0fe140 con 0x7f90080367e0 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f90080367e0 msgr2=0x7f9008038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 --2- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f90080367e0 0x7f9008038c90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9018009990 tx=0x7f9018006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 msgr2=0x7f901c111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 --2- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c111600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f900c011bd0 tx=0x7f900c009250 comp rx=0 tx=0).stop 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 shutdown_connections 2024-03-22T07:00:15.932 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 --2- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f90080367e0 0x7f9008038c90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:15.933 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 --2- 172.21.15.81:0/2855199491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f901c071360 0x7f901c111600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:15.933 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 >> 172.21.15.81:0/2855199491 conn(0x7f901c0fcd50 msgr2=0x7f901c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:15.933 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 shutdown_connections 2024-03-22T07:00:15.933 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:15.167+0000 7f902470e700 1 -- 172.21.15.81:0/2855199491 wait complete. 2024-03-22T07:00:15.933 INFO:teuthology.orchestra.run.smithi081.stdout:Deploying mgr service with default placement... 2024-03-22T07:00:16.240 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:16 smithi081 ceph-mon[29245]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:16.241 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:16 smithi081 ceph-mon[29245]: Saving service mon spec with placement count:5 2024-03-22T07:00:16.241 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:16 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:17.104 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-22T07:00:17.104 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.335+0000 7fc71533c700 1 Processor -- start 2024-03-22T07:00:17.104 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.335+0000 7fc71533c700 1 -- start start 2024-03-22T07:00:17.104 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc71533c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710105490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:17.104 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc71533c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc710076ee0 con 0x7fc710076ad0 2024-03-22T07:00:17.104 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc70effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710105490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc70effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710105490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41224/0 (socket says 172.21.15.81:41224) 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc70effd700 1 -- 172.21.15.81:0/4129436254 learned_addr learned my addr 172.21.15.81:0/4129436254 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc70effd700 1 -- 172.21.15.81:0/4129436254 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7101059d0 con 0x7fc710076ad0 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc70effd700 1 --2- 172.21.15.81:0/4129436254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710105490 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc7040098e0 tx=0x7fc704009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da91a9d3b80ba84 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc70dffb700 1 -- 172.21.15.81:0/4129436254 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc70400b860 con 0x7fc710076ad0 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc70dffb700 1 -- 172.21.15.81:0/4129436254 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc70400be50 con 0x7fc710076ad0 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.336+0000 7fc70dffb700 1 -- 172.21.15.81:0/4129436254 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc704010380 con 0x7fc710076ad0 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.337+0000 7fc71533c700 1 -- 172.21.15.81:0/4129436254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 msgr2=0x7fc710105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.337+0000 7fc71533c700 1 --2- 172.21.15.81:0/4129436254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710105490 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc7040098e0 tx=0x7fc704009250 comp rx=0 tx=0).stop 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.337+0000 7fc71533c700 1 -- 172.21.15.81:0/4129436254 shutdown_connections 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.337+0000 7fc71533c700 1 --2- 172.21.15.81:0/4129436254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710105490 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.337+0000 7fc71533c700 1 -- 172.21.15.81:0/4129436254 >> 172.21.15.81:0/4129436254 conn(0x7fc7100fce40 msgr2=0x7fc7100ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.337+0000 7fc71533c700 1 -- 172.21.15.81:0/4129436254 shutdown_connections 2024-03-22T07:00:17.105 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.337+0000 7fc71533c700 1 -- 172.21.15.81:0/4129436254 wait complete. 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc71533c700 1 Processor -- start 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc71533c700 1 -- start start 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc71533c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc71533c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc710111b60 con 0x7fc710076ad0 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc70effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc70effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41238/0 (socket says 172.21.15.81:41238) 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc70effd700 1 -- 172.21.15.81:0/2145124412 learned_addr learned my addr 172.21.15.81:0/2145124412 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc70effd700 1 -- 172.21.15.81:0/2145124412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040094f0 con 0x7fc710076ad0 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc70effd700 1 --2- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710111620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc704006fd0 tx=0x7fc704009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc6f77fe700 1 -- 172.21.15.81:0/2145124412 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc704010760 con 0x7fc710076ad0 2024-03-22T07:00:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc710111d60 con 0x7fc710076ad0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc6f77fe700 1 -- 172.21.15.81:0/2145124412 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc704010d50 con 0x7fc710076ad0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc6f77fe700 1 -- 172.21.15.81:0/2145124412 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc70401adc0 con 0x7fc710076ad0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.338+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7101128d0 con 0x7fc710076ad0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.339+0000 7fc6f77fe700 1 -- 172.21.15.81:0/2145124412 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fc7040193c0 con 0x7fc710076ad0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.339+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71005f5f0 con 0x7fc710076ad0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.339+0000 7fc6f77fe700 1 --2- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fc6f8036790 0x7fc6f8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.340+0000 7fc6f77fe700 1 -- 172.21.15.81:0/2145124412 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc704049ec0 con 0x7fc710076ad0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.340+0000 7fc70e7fc700 1 --2- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fc6f8036790 0x7fc6f8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.340+0000 7fc70e7fc700 1 --2- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fc6f8036790 0x7fc6f8038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc700009990 tx=0x7fc700006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.344+0000 7fc6f77fe700 1 -- 172.21.15.81:0/2145124412 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc704023940 con 0x7fc710076ad0 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.542+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7100fe230 con 0x7fc6f8036790 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.549+0000 7fc6f77fe700 1 -- 172.21.15.81:0/2145124412 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc7100fe230 con 0x7fc6f8036790 2024-03-22T07:00:17.107 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fc6f8036790 msgr2=0x7fc6f8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:17.108 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 --2- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fc6f8036790 0x7fc6f8038c40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc700009990 tx=0x7fc700006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:17.108 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 msgr2=0x7fc710111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:17.108 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 --2- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710111620 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc704006fd0 tx=0x7fc704009250 comp rx=0 tx=0).stop 2024-03-22T07:00:17.108 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 shutdown_connections 2024-03-22T07:00:17.108 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 --2- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fc6f8036790 0x7fc6f8038c40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:17.108 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 --2- 172.21.15.81:0/2145124412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc710076ad0 0x7fc710111620 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:17.108 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 >> 172.21.15.81:0/2145124412 conn(0x7fc7100fce40 msgr2=0x7fc7100fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:17.108 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 shutdown_connections 2024-03-22T07:00:17.109 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:16.551+0000 7fc71533c700 1 -- 172.21.15.81:0/2145124412 wait complete. 2024-03-22T07:00:17.109 INFO:teuthology.orchestra.run.smithi081.stdout:Deploying crash service with default placement... 2024-03-22T07:00:17.690 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:17 smithi081 ceph-mon[29245]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:17.691 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:17 smithi081 ceph-mon[29245]: Saving service mgr spec with placement count:2 2024-03-22T07:00:17.691 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:17 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:18.757 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-22T07:00:18.757 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.879+0000 7f2494cf1700 1 Processor -- start 2024-03-22T07:00:18.757 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.879+0000 7f2494cf1700 1 -- start start 2024-03-22T07:00:18.757 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.879+0000 7f2494cf1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f24901019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.879+0000 7f2494cf1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490101f20 con 0x7f24901015d0 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.879+0000 7f248e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f24901019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.879+0000 7f248e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f24901019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41252/0 (socket says 172.21.15.81:41252) 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.880+0000 7f248e59c700 1 -- 172.21.15.81:0/2771288762 learned_addr learned my addr 172.21.15.81:0/2771288762 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.880+0000 7f248e59c700 1 -- 172.21.15.81:0/2771288762 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2490102060 con 0x7f24901015d0 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.880+0000 7f248e59c700 1 --2- 172.21.15.81:0/2771288762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f24901019e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f24800098e0 tx=0x7f2480009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58de2ea3e218c929 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.880+0000 7f248d59a700 1 -- 172.21.15.81:0/2771288762 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f248000b860 con 0x7f24901015d0 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.880+0000 7f248d59a700 1 -- 172.21.15.81:0/2771288762 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f248000be50 con 0x7f24901015d0 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.880+0000 7f248d59a700 1 -- 172.21.15.81:0/2771288762 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2480010380 con 0x7f24901015d0 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 -- 172.21.15.81:0/2771288762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 msgr2=0x7f24901019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 --2- 172.21.15.81:0/2771288762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f24901019e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f24800098e0 tx=0x7f2480009250 comp rx=0 tx=0).stop 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 -- 172.21.15.81:0/2771288762 shutdown_connections 2024-03-22T07:00:18.758 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 --2- 172.21.15.81:0/2771288762 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f24901019e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:18.759 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 -- 172.21.15.81:0/2771288762 >> 172.21.15.81:0/2771288762 conn(0x7f24900fcd50 msgr2=0x7f24900ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:18.759 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 -- 172.21.15.81:0/2771288762 shutdown_connections 2024-03-22T07:00:18.759 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 -- 172.21.15.81:0/2771288762 wait complete. 2024-03-22T07:00:18.759 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 Processor -- start 2024-03-22T07:00:18.759 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 -- start start 2024-03-22T07:00:18.759 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f249010f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:18.759 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.881+0000 7f2494cf1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249010f8c0 con 0x7f24901015d0 2024-03-22T07:00:18.759 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f248e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f249010f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:18.759 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f248e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f249010f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41266/0 (socket says 172.21.15.81:41266) 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f248e59c700 1 -- 172.21.15.81:0/1277379563 learned_addr learned my addr 172.21.15.81:0/1277379563 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f248e59c700 1 -- 172.21.15.81:0/1277379563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24800094f0 con 0x7f24901015d0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f248e59c700 1 --2- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f249010f380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f248000ba90 tx=0x7f2480009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f247effd700 1 -- 172.21.15.81:0/1277379563 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24800107c0 con 0x7f24901015d0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249010fac0 con 0x7f24901015d0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f247effd700 1 -- 172.21.15.81:0/1277379563 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2480010db0 con 0x7f24901015d0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f247effd700 1 -- 172.21.15.81:0/1277379563 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f248001add0 con 0x7f24901015d0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.882+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2490110630 con 0x7f24901015d0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.883+0000 7f247effd700 1 -- 172.21.15.81:0/1277379563 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f248002e440 con 0x7f24901015d0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.883+0000 7f247effd700 1 --2- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f24780389d0 0x7f247803ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.883+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249005f5f0 con 0x7f24901015d0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.883+0000 7f247effd700 1 -- 172.21.15.81:0/1277379563 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f248004a5b0 con 0x7f24901015d0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.883+0000 7f248dd9b700 1 --2- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f24780389d0 0x7f247803ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:18.760 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.884+0000 7f248dd9b700 1 --2- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f24780389d0 0x7f247803ae80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2484009990 tx=0x7f2484006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:17.888+0000 7f247effd700 1 -- 172.21.15.81:0/1277379563 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2480018940 con 0x7f24901015d0 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.083+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f24901109e0 con 0x7f24780389d0 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.088+0000 7f247effd700 1 -- 172.21.15.81:0/1277379563 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f24901109e0 con 0x7f24780389d0 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f24780389d0 msgr2=0x7f247803ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 --2- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f24780389d0 0x7f247803ae80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2484009990 tx=0x7f2484006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 msgr2=0x7f249010f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 --2- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f249010f380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f248000ba90 tx=0x7f2480009250 comp rx=0 tx=0).stop 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 shutdown_connections 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 --2- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f24780389d0 0x7f247803ae80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 --2- 172.21.15.81:0/1277379563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24901015d0 0x7f249010f380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 >> 172.21.15.81:0/1277379563 conn(0x7f24900fcd50 msgr2=0x7f2490105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 shutdown_connections 2024-03-22T07:00:18.761 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:18.090+0000 7f2494cf1700 1 -- 172.21.15.81:0/1277379563 wait complete. 2024-03-22T07:00:18.762 INFO:teuthology.orchestra.run.smithi081.stdout:Deploying prometheus service with default placement... 2024-03-22T07:00:19.127 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:19 smithi081 ceph-mon[29245]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:19.127 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:19 smithi081 ceph-mon[29245]: Saving service crash spec with placement * 2024-03-22T07:00:19.127 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:19 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:19.128 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:19 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:19.965 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-22T07:00:19.965 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.157+0000 7faba8952700 1 Processor -- start 2024-03-22T07:00:19.966 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.157+0000 7faba8952700 1 -- start start 2024-03-22T07:00:19.966 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.157+0000 7faba8952700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba4102b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:19.966 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.157+0000 7faba8952700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba4103060 con 0x7faba4102710 2024-03-22T07:00:19.966 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.158+0000 7faba259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba4102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:19.966 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.158+0000 7faba259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba4102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41270/0 (socket says 172.21.15.81:41270) 2024-03-22T07:00:19.966 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.158+0000 7faba259c700 1 -- 172.21.15.81:0/272317479 learned_addr learned my addr 172.21.15.81:0/272317479 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:19.966 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.158+0000 7faba259c700 1 -- 172.21.15.81:0/272317479 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba41031a0 con 0x7faba4102710 2024-03-22T07:00:19.966 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.158+0000 7faba259c700 1 --2- 172.21.15.81:0/272317479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba4102b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009e50 tx=0x7fab9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea6e1a201b5ef94 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:19.966 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.158+0000 7faba159a700 1 -- 172.21.15.81:0/272317479 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab9c0092e0 con 0x7faba4102710 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.158+0000 7faba159a700 1 -- 172.21.15.81:0/272317479 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab9c012430 con 0x7faba4102710 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.158+0000 7faba159a700 1 -- 172.21.15.81:0/272317479 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab9c0075a0 con 0x7faba4102710 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.159+0000 7faba8952700 1 -- 172.21.15.81:0/272317479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 msgr2=0x7faba4102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.159+0000 7faba8952700 1 --2- 172.21.15.81:0/272317479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba4102b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009e50 tx=0x7fab9c009840 comp rx=0 tx=0).stop 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.159+0000 7faba8952700 1 -- 172.21.15.81:0/272317479 shutdown_connections 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.159+0000 7faba8952700 1 --2- 172.21.15.81:0/272317479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba4102b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.159+0000 7faba8952700 1 -- 172.21.15.81:0/272317479 >> 172.21.15.81:0/272317479 conn(0x7faba40fde50 msgr2=0x7faba41002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.159+0000 7faba8952700 1 -- 172.21.15.81:0/272317479 shutdown_connections 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.159+0000 7faba8952700 1 -- 172.21.15.81:0/272317479 wait complete. 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.160+0000 7faba8952700 1 Processor -- start 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.160+0000 7faba8952700 1 -- start start 2024-03-22T07:00:19.967 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.160+0000 7faba8952700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba41115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.160+0000 7faba8952700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba4111b10 con 0x7faba4102710 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.160+0000 7faba259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba41115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.160+0000 7faba259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba41115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41278/0 (socket says 172.21.15.81:41278) 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.160+0000 7faba259c700 1 -- 172.21.15.81:0/1462667400 learned_addr learned my addr 172.21.15.81:0/1462667400 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.160+0000 7faba259c700 1 -- 172.21.15.81:0/1462667400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c009ae0 con 0x7faba4102710 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.160+0000 7faba259c700 1 --2- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba41115d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009fd0 tx=0x7fab9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.161+0000 7fab92ffd700 1 -- 172.21.15.81:0/1462667400 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab9c0092e0 con 0x7faba4102710 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.161+0000 7fab92ffd700 1 -- 172.21.15.81:0/1462667400 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab9c007ce0 con 0x7faba4102710 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.161+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba4111d10 con 0x7faba4102710 2024-03-22T07:00:19.968 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.161+0000 7fab92ffd700 1 -- 172.21.15.81:0/1462667400 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab9c025990 con 0x7faba4102710 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.161+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba4112880 con 0x7faba4102710 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.161+0000 7fab92ffd700 1 -- 172.21.15.81:0/1462667400 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fab9c019050 con 0x7faba4102710 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.162+0000 7fab92ffd700 1 --2- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fab8c0367e0 0x7fab8c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.162+0000 7fab92ffd700 1 -- 172.21.15.81:0/1462667400 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fab9c04b020 con 0x7faba4102710 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.162+0000 7faba1d9b700 1 --2- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fab8c0367e0 0x7fab8c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.162+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba405f5f0 con 0x7faba4102710 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.162+0000 7faba1d9b700 1 --2- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fab8c0367e0 0x7fab8c038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fab98009990 tx=0x7fab98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.167+0000 7fab92ffd700 1 -- 172.21.15.81:0/1462667400 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fab9c020020 con 0x7faba4102710 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.373+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7faba40ff1c0 con 0x7fab8c0367e0 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.378+0000 7fab92ffd700 1 -- 172.21.15.81:0/1462667400 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7faba40ff1c0 con 0x7fab8c0367e0 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fab8c0367e0 msgr2=0x7fab8c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:19.969 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 --2- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fab8c0367e0 0x7fab8c038c90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fab98009990 tx=0x7fab98006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:19.970 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 msgr2=0x7faba41115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:19.970 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 --2- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba41115d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009fd0 tx=0x7fab9c009840 comp rx=0 tx=0).stop 2024-03-22T07:00:19.970 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 shutdown_connections 2024-03-22T07:00:19.970 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 --2- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fab8c0367e0 0x7fab8c038c90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:19.970 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 --2- 172.21.15.81:0/1462667400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faba4102710 0x7faba41115d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:19.970 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 >> 172.21.15.81:0/1462667400 conn(0x7faba40fde50 msgr2=0x7faba40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:19.970 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 shutdown_connections 2024-03-22T07:00:19.970 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:19.380+0000 7faba8952700 1 -- 172.21.15.81:0/1462667400 wait complete. 2024-03-22T07:00:19.970 INFO:teuthology.orchestra.run.smithi081.stdout:Deploying grafana service with default placement... 2024-03-22T07:00:20.293 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:20 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:20.293 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:20 smithi081 ceph-mon[29245]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:20.293 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:20 smithi081 ceph-mon[29245]: Saving service prometheus spec with placement count:1 2024-03-22T07:00:20.293 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:20 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:21.330 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.390+0000 7f0eef15b700 1 Processor -- start 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.390+0000 7f0eef15b700 1 -- start start 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.390+0000 7f0eef15b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.390+0000 7f0eef15b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8076f00 con 0x7f0ee8076af0 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.391+0000 7f0eecef7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.391+0000 7f0eecef7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41290/0 (socket says 172.21.15.81:41290) 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.391+0000 7f0eecef7700 1 -- 172.21.15.81:0/1047261362 learned_addr learned my addr 172.21.15.81:0/1047261362 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.391+0000 7f0eecef7700 1 -- 172.21.15.81:0/1047261362 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee81059b0 con 0x7f0ee8076af0 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.391+0000 7f0eecef7700 1 --2- 172.21.15.81:0/1047261362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8105470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009e50 tx=0x7f0ee0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9bba2a154905ca server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.391+0000 7f0ee77fe700 1 -- 172.21.15.81:0/1047261362 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ee00092e0 con 0x7f0ee8076af0 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.391+0000 7f0ee77fe700 1 -- 172.21.15.81:0/1047261362 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ee0012430 con 0x7f0ee8076af0 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.391+0000 7f0ee77fe700 1 -- 172.21.15.81:0/1047261362 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ee00075a0 con 0x7f0ee8076af0 2024-03-22T07:00:21.331 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.392+0000 7f0eef15b700 1 -- 172.21.15.81:0/1047261362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 msgr2=0x7f0ee8105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.392+0000 7f0eef15b700 1 --2- 172.21.15.81:0/1047261362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8105470 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009e50 tx=0x7f0ee0009840 comp rx=0 tx=0).stop 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.392+0000 7f0eef15b700 1 -- 172.21.15.81:0/1047261362 shutdown_connections 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.392+0000 7f0eef15b700 1 --2- 172.21.15.81:0/1047261362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8105470 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.392+0000 7f0eef15b700 1 -- 172.21.15.81:0/1047261362 >> 172.21.15.81:0/1047261362 conn(0x7f0ee80fce40 msgr2=0x7f0ee80ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.392+0000 7f0eef15b700 1 -- 172.21.15.81:0/1047261362 shutdown_connections 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.392+0000 7f0eef15b700 1 -- 172.21.15.81:0/1047261362 wait complete. 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.392+0000 7f0eef15b700 1 Processor -- start 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.392+0000 7f0eef15b700 1 -- start start 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0eef15b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0eef15b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8103300 con 0x7f0ee8076af0 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0eecef7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0eecef7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8102dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41304/0 (socket says 172.21.15.81:41304) 2024-03-22T07:00:21.332 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0eecef7700 1 -- 172.21.15.81:0/2888813659 learned_addr learned my addr 172.21.15.81:0/2888813659 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0eecef7700 1 -- 172.21.15.81:0/2888813659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0009ae0 con 0x7f0ee8076af0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0eecef7700 1 --2- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8102dc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0012de0 tx=0x7f0ee0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0ee57fa700 1 -- 172.21.15.81:0/2888813659 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ee00092e0 con 0x7f0ee8076af0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0ee57fa700 1 -- 172.21.15.81:0/2888813659 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ee0007d30 con 0x7f0ee8076af0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0ee57fa700 1 -- 172.21.15.81:0/2888813659 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ee0025910 con 0x7f0ee8076af0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee8101370 con 0x7f0ee8076af0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.393+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee8101860 con 0x7f0ee8076af0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.394+0000 7f0ee57fa700 1 -- 172.21.15.81:0/2888813659 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f0ee0019050 con 0x7f0ee8076af0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.394+0000 7f0ee57fa700 1 --2- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f0ed4036790 0x7f0ed4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.394+0000 7f0ee57fa700 1 -- 172.21.15.81:0/2888813659 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f0ee0049fc0 con 0x7f0ee8076af0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.395+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee804bed0 con 0x7f0ee8076af0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.395+0000 7f0ee7fff700 1 --2- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f0ed4036790 0x7f0ed4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.395+0000 7f0ee7fff700 1 --2- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f0ed4036790 0x7f0ed4038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009990 tx=0x7f0ed0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:21.333 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.399+0000 7f0ee57fa700 1 -- 172.21.15.81:0/2888813659 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0ee0020020 con 0x7f0ee8076af0 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.630+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ee8101f70 con 0x7f0ed4036790 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.635+0000 7f0ee57fa700 1 -- 172.21.15.81:0/2888813659 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f0ee8101f70 con 0x7f0ed4036790 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f0ed4036790 msgr2=0x7f0ed4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 --2- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f0ed4036790 0x7f0ed4038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009990 tx=0x7f0ed0006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 msgr2=0x7f0ee8102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 --2- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8102dc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0012de0 tx=0x7f0ee0009840 comp rx=0 tx=0).stop 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 shutdown_connections 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 --2- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f0ed4036790 0x7f0ed4038c40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 --2- 172.21.15.81:0/2888813659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee8076af0 0x7f0ee8102dc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 >> 172.21.15.81:0/2888813659 conn(0x7f0ee80fce40 msgr2=0x7f0ee80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 shutdown_connections 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:20.637+0000 7f0eef15b700 1 -- 172.21.15.81:0/2888813659 wait complete. 2024-03-22T07:00:21.334 INFO:teuthology.orchestra.run.smithi081.stdout:Deploying node-exporter service with default placement... 2024-03-22T07:00:21.910 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:21 smithi081 ceph-mon[29245]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:21.910 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:21 smithi081 ceph-mon[29245]: Saving service grafana spec with placement count:1 2024-03-22T07:00:21.910 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:21 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:21.910 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:21 smithi081 ceph-mon[29245]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:00:22.732 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.722+0000 7fd743bae700 1 Processor -- start 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.722+0000 7fd743bae700 1 -- start start 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.722+0000 7fd743bae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.722+0000 7fd743bae700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c103080 con 0x7fd73c102730 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.722+0000 7fd74194a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.722+0000 7fd74194a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41312/0 (socket says 172.21.15.81:41312) 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.722+0000 7fd74194a700 1 -- 172.21.15.81:0/3814694162 learned_addr learned my addr 172.21.15.81:0/3814694162 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.722+0000 7fd74194a700 1 -- 172.21.15.81:0/3814694162 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c1031c0 con 0x7fd73c102730 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.722+0000 7fd74194a700 1 --2- 172.21.15.81:0/3814694162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c102b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd7380098e0 tx=0x7fd738009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162bc530ce8f6b8f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:22.733 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd740948700 1 -- 172.21.15.81:0/3814694162 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd73800b860 con 0x7fd73c102730 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd740948700 1 -- 172.21.15.81:0/3814694162 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd73800be90 con 0x7fd73c102730 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd740948700 1 -- 172.21.15.81:0/3814694162 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7380103c0 con 0x7fd73c102730 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd743bae700 1 -- 172.21.15.81:0/3814694162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 msgr2=0x7fd73c102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd743bae700 1 --2- 172.21.15.81:0/3814694162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c102b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd7380098e0 tx=0x7fd738009250 comp rx=0 tx=0).stop 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd743bae700 1 -- 172.21.15.81:0/3814694162 shutdown_connections 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd743bae700 1 --2- 172.21.15.81:0/3814694162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c102b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd743bae700 1 -- 172.21.15.81:0/3814694162 >> 172.21.15.81:0/3814694162 conn(0x7fd73c0fdf10 msgr2=0x7fd73c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd743bae700 1 -- 172.21.15.81:0/3814694162 shutdown_connections 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.723+0000 7fd743bae700 1 -- 172.21.15.81:0/3814694162 wait complete. 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd743bae700 1 Processor -- start 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd743bae700 1 -- start start 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd743bae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:22.734 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd743bae700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c111b70 con 0x7fd73c102730 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd74194a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd74194a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41322/0 (socket says 172.21.15.81:41322) 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd74194a700 1 -- 172.21.15.81:0/2762218910 learned_addr learned my addr 172.21.15.81:0/2762218910 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd74194a700 1 -- 172.21.15.81:0/2762218910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380094f0 con 0x7fd73c102730 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd74194a700 1 --2- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c111630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd738006fd0 tx=0x7fd738009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd72e7fc700 1 -- 172.21.15.81:0/2762218910 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd738010630 con 0x7fd73c102730 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.724+0000 7fd72e7fc700 1 -- 172.21.15.81:0/2762218910 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd738010c60 con 0x7fd73c102730 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.725+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c111d70 con 0x7fd73c102730 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.725+0000 7fd72e7fc700 1 -- 172.21.15.81:0/2762218910 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd73801ad40 con 0x7fd73c102730 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.725+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c1128e0 con 0x7fd73c102730 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.725+0000 7fd72e7fc700 1 -- 172.21.15.81:0/2762218910 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd7380193c0 con 0x7fd73c102730 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.725+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73c05f5f0 con 0x7fd73c102730 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.725+0000 7fd72e7fc700 1 --2- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd728036790 0x7fd728038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:22.735 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.726+0000 7fd72e7fc700 1 -- 172.21.15.81:0/2762218910 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd73804a080 con 0x7fd73c102730 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.726+0000 7fd741149700 1 --2- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd728036790 0x7fd728038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.726+0000 7fd741149700 1 --2- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd728036790 0x7fd728038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd730009990 tx=0x7fd730006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.730+0000 7fd72e7fc700 1 -- 172.21.15.81:0/2762218910 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd7380237d0 con 0x7fd73c102730 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.925+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fd73c0ff2d0 con 0x7fd728036790 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.930+0000 7fd72e7fc700 1 -- 172.21.15.81:0/2762218910 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fd73c0ff2d0 con 0x7fd728036790 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd728036790 msgr2=0x7fd728038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 --2- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd728036790 0x7fd728038c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd730009990 tx=0x7fd730006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 msgr2=0x7fd73c111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 --2- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c111630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd738006fd0 tx=0x7fd738009250 comp rx=0 tx=0).stop 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 shutdown_connections 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 --2- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd728036790 0x7fd728038c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 --2- 172.21.15.81:0/2762218910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd73c102730 0x7fd73c111630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:22.736 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 >> 172.21.15.81:0/2762218910 conn(0x7fd73c0fdf10 msgr2=0x7fd73c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:22.737 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 shutdown_connections 2024-03-22T07:00:22.737 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:21.932+0000 7fd743bae700 1 -- 172.21.15.81:0/2762218910 wait complete. 2024-03-22T07:00:22.737 INFO:teuthology.orchestra.run.smithi081.stdout:Deploying alertmanager service with default placement... 2024-03-22T07:00:23.026 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:22 smithi081 ceph-mon[29245]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:23.026 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:22 smithi081 ceph-mon[29245]: Saving service node-exporter spec with placement * 2024-03-22T07:00:23.026 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:22 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:23.906 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-22T07:00:23.906 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.128+0000 7fe228492700 1 Processor -- start 2024-03-22T07:00:23.906 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.128+0000 7fe228492700 1 -- start start 2024-03-22T07:00:23.906 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe228492700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe2201053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:23.906 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe228492700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2201058f0 con 0x7fe220076bb0 2024-03-22T07:00:23.906 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe22622e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe2201053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:23.906 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe22622e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe2201053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41338/0 (socket says 172.21.15.81:41338) 2024-03-22T07:00:23.906 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe22622e700 1 -- 172.21.15.81:0/2333762211 learned_addr learned my addr 172.21.15.81:0/2333762211 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe22622e700 1 -- 172.21.15.81:0/2333762211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220105a30 con 0x7fe220076bb0 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe22622e700 1 --2- 172.21.15.81:0/2333762211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe2201053b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2100098e0 tx=0x7fe210009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9891867311e01e48 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe22522c700 1 -- 172.21.15.81:0/2333762211 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe21000b860 con 0x7fe220076bb0 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe22522c700 1 -- 172.21.15.81:0/2333762211 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe21000be90 con 0x7fe220076bb0 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.129+0000 7fe22522c700 1 -- 172.21.15.81:0/2333762211 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2100103c0 con 0x7fe220076bb0 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.130+0000 7fe228492700 1 -- 172.21.15.81:0/2333762211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 msgr2=0x7fe2201053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.130+0000 7fe228492700 1 --2- 172.21.15.81:0/2333762211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe2201053b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe2100098e0 tx=0x7fe210009250 comp rx=0 tx=0).stop 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.130+0000 7fe228492700 1 -- 172.21.15.81:0/2333762211 shutdown_connections 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.130+0000 7fe228492700 1 --2- 172.21.15.81:0/2333762211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe2201053b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:23.907 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.130+0000 7fe228492700 1 -- 172.21.15.81:0/2333762211 >> 172.21.15.81:0/2333762211 conn(0x7fe2200fce40 msgr2=0x7fe2200ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.130+0000 7fe228492700 1 -- 172.21.15.81:0/2333762211 shutdown_connections 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.130+0000 7fe228492700 1 -- 172.21.15.81:0/2333762211 wait complete. 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe228492700 1 Processor -- start 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe228492700 1 -- start start 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe228492700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe220111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe228492700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe220111b50 con 0x7fe220076bb0 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe22622e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe220111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe22622e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe220111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41348/0 (socket says 172.21.15.81:41348) 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe22622e700 1 -- 172.21.15.81:0/3845657044 learned_addr learned my addr 172.21.15.81:0/3845657044 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe22622e700 1 -- 172.21.15.81:0/3845657044 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100094f0 con 0x7fe220076bb0 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe22622e700 1 --2- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe220111610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe2100094c0 tx=0x7fe210009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe216ffd700 1 -- 172.21.15.81:0/3845657044 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2100118f0 con 0x7fe220076bb0 2024-03-22T07:00:23.908 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe216ffd700 1 -- 172.21.15.81:0/3845657044 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe210010d60 con 0x7fe220076bb0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe220111d50 con 0x7fe220076bb0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe216ffd700 1 -- 172.21.15.81:0/3845657044 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe21001a430 con 0x7fe220076bb0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.131+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2201128c0 con 0x7fe220076bb0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.132+0000 7fe216ffd700 1 -- 172.21.15.81:0/3845657044 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fe21001a590 con 0x7fe220076bb0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.132+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22005f5f0 con 0x7fe220076bb0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.132+0000 7fe216ffd700 1 --2- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fe20c0367e0 0x7fe20c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.132+0000 7fe216ffd700 1 -- 172.21.15.81:0/3845657044 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe21004a5f0 con 0x7fe220076bb0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.133+0000 7fe225a2d700 1 --2- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fe20c0367e0 0x7fe20c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.133+0000 7fe225a2d700 1 --2- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fe20c0367e0 0x7fe20c038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009990 tx=0x7fe21c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.137+0000 7fe216ffd700 1 -- 172.21.15.81:0/3845657044 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe21001a840 con 0x7fe220076bb0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.344+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2200fe270 con 0x7fe20c0367e0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.349+0000 7fe216ffd700 1 -- 172.21.15.81:0/3845657044 <== mgr.14120 v2:172.21.15.81:6800/4051369418 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fe2200fe270 con 0x7fe20c0367e0 2024-03-22T07:00:23.909 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.351+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fe20c0367e0 msgr2=0x7fe20c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.351+0000 7fe228492700 1 --2- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fe20c0367e0 0x7fe20c038c90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009990 tx=0x7fe21c006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.351+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 msgr2=0x7fe220111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.351+0000 7fe228492700 1 --2- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe220111610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe2100094c0 tx=0x7fe210009250 comp rx=0 tx=0).stop 2024-03-22T07:00:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.352+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 shutdown_connections 2024-03-22T07:00:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.352+0000 7fe228492700 1 --2- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fe20c0367e0 0x7fe20c038c90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.352+0000 7fe228492700 1 --2- 172.21.15.81:0/3845657044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe220076bb0 0x7fe220111610 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.352+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 >> 172.21.15.81:0/3845657044 conn(0x7fe2200fce40 msgr2=0x7fe2200fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.352+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 shutdown_connections 2024-03-22T07:00:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:23.352+0000 7fe228492700 1 -- 172.21.15.81:0/3845657044 wait complete. 2024-03-22T07:00:24.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:24 smithi081 ceph-mon[29245]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:00:24.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:24 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:24.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:24 smithi081 ceph-mon[29245]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:24.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:24 smithi081 ceph-mon[29245]: Saving service alertmanager spec with placement count:1 2024-03-22T07:00:24.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:24 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:25.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.351+0000 7ffbf52db700 1 Processor -- start 2024-03-22T07:00:25.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.351+0000 7ffbf52db700 1 -- start start 2024-03-22T07:00:25.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.351+0000 7ffbf52db700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0107830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.352+0000 7ffbf52db700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf0103220 con 0x7ffbf0076b30 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.352+0000 7ffbeeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0107830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.352+0000 7ffbeeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0107830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41350/0 (socket says 172.21.15.81:41350) 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.352+0000 7ffbeeffd700 1 -- 172.21.15.81:0/308847702 learned_addr learned my addr 172.21.15.81:0/308847702 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.352+0000 7ffbeeffd700 1 -- 172.21.15.81:0/308847702 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbf0107d70 con 0x7ffbf0076b30 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.352+0000 7ffbeeffd700 1 --2- 172.21.15.81:0/308847702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0107830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00098e0 tx=0x7ffbe0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7211254bc0dee0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.352+0000 7ffbedffb700 1 -- 172.21.15.81:0/308847702 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbe000b860 con 0x7ffbf0076b30 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.352+0000 7ffbedffb700 1 -- 172.21.15.81:0/308847702 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbe000be90 con 0x7ffbf0076b30 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.352+0000 7ffbedffb700 1 -- 172.21.15.81:0/308847702 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbe00103c0 con 0x7ffbf0076b30 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.353+0000 7ffbf52db700 1 -- 172.21.15.81:0/308847702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 msgr2=0x7ffbf0107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.353+0000 7ffbf52db700 1 --2- 172.21.15.81:0/308847702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0107830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00098e0 tx=0x7ffbe0009250 comp rx=0 tx=0).stop 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.353+0000 7ffbf52db700 1 -- 172.21.15.81:0/308847702 shutdown_connections 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.353+0000 7ffbf52db700 1 --2- 172.21.15.81:0/308847702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0107830 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:25.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.353+0000 7ffbf52db700 1 -- 172.21.15.81:0/308847702 >> 172.21.15.81:0/308847702 conn(0x7ffbf00fce40 msgr2=0x7ffbf00ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.353+0000 7ffbf52db700 1 -- 172.21.15.81:0/308847702 shutdown_connections 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.353+0000 7ffbf52db700 1 -- 172.21.15.81:0/308847702 wait complete. 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.354+0000 7ffbf52db700 1 Processor -- start 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.354+0000 7ffbf52db700 1 -- start start 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.354+0000 7ffbf52db700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.354+0000 7ffbf52db700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf0113d40 con 0x7ffbf0076b30 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.354+0000 7ffbeeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.354+0000 7ffbeeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41354/0 (socket says 172.21.15.81:41354) 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.354+0000 7ffbeeffd700 1 -- 172.21.15.81:0/3098538992 learned_addr learned my addr 172.21.15.81:0/3098538992 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.354+0000 7ffbeeffd700 1 -- 172.21.15.81:0/3098538992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe00094f0 con 0x7ffbf0076b30 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.354+0000 7ffbeeffd700 1 --2- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0113800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00105b0 tx=0x7ffbe0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.355+0000 7ffbd77fe700 1 -- 172.21.15.81:0/3098538992 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbe0010840 con 0x7ffbf0076b30 2024-03-22T07:00:25.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.355+0000 7ffbd77fe700 1 -- 172.21.15.81:0/3098538992 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbe0010e70 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.355+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbf0113f40 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.355+0000 7ffbd77fe700 1 -- 172.21.15.81:0/3098538992 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbe001ae30 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.355+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbf0114ab0 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.355+0000 7ffbd77fe700 1 -- 172.21.15.81:0/3098538992 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7ffbe002d3f0 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.356+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbf005f5f0 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.356+0000 7ffbd77fe700 1 --2- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7ffbd8036790 0x7ffbd8038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.356+0000 7ffbd77fe700 1 -- 172.21.15.81:0/3098538992 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ffbe004a180 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.356+0000 7ffbee7fc700 1 --2- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7ffbd8036790 0x7ffbd8038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.356+0000 7ffbee7fc700 1 --2- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7ffbd8036790 0x7ffbd8038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009990 tx=0x7ffbe4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.360+0000 7ffbd77fe700 1 -- 172.21.15.81:0/3098538992 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ffbe002b8e0 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.554+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ffbf004bed0 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.559+0000 7ffbd77fe700 1 -- 172.21.15.81:0/3098538992 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ffbe002b2f0 con 0x7ffbf0076b30 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7ffbd8036790 msgr2=0x7ffbd8038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:25.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 --2- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7ffbd8036790 0x7ffbd8038c40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009990 tx=0x7ffbe4006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:25.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 msgr2=0x7ffbf0113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:25.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 --2- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0113800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00105b0 tx=0x7ffbe0009250 comp rx=0 tx=0).stop 2024-03-22T07:00:25.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 shutdown_connections 2024-03-22T07:00:25.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 --2- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7ffbd8036790 0x7ffbd8038c40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:25.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 --2- 172.21.15.81:0/3098538992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbf0076b30 0x7ffbf0113800 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:25.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 >> 172.21.15.81:0/3098538992 conn(0x7ffbf00fce40 msgr2=0x7ffbf00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:25.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 shutdown_connections 2024-03-22T07:00:25.132 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:24.562+0000 7ffbf52db700 1 -- 172.21.15.81:0/3098538992 wait complete. 2024-03-22T07:00:25.700 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:25 smithi081 ceph-mon[29245]: from='mgr.14120 172.21.15.81:0/473849636' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:25.700 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:25 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3098538992' entity='client.admin' 2024-03-22T07:00:26.573 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:26 smithi081 ceph-mon[29245]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:00:26.574 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:26 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2838709131' entity='client.admin' 2024-03-22T07:00:27.043 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.596+0000 7f85f1d96700 1 Processor -- start 2024-03-22T07:00:27.043 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.596+0000 7f85f1d96700 1 -- start start 2024-03-22T07:00:27.043 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85f1d96700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec1019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:27.043 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85f1d96700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec101f20 con 0x7f85ec1015d0 2024-03-22T07:00:27.043 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85eb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec1019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85eb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec1019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38530/0 (socket says 172.21.15.81:38530) 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85eb7fe700 1 -- 172.21.15.81:0/3140161795 learned_addr learned my addr 172.21.15.81:0/3140161795 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85eb7fe700 1 -- 172.21.15.81:0/3140161795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec102060 con 0x7f85ec1015d0 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85eb7fe700 1 --2- 172.21.15.81:0/3140161795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec1019e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00bb80 tx=0x7f85dc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9ad7af97227b61 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85ea7fc700 1 -- 172.21.15.81:0/3140161795 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc00dd00 con 0x7f85ec1015d0 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85ea7fc700 1 -- 172.21.15.81:0/3140161795 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85dc00de60 con 0x7f85ec1015d0 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.597+0000 7f85ea7fc700 1 -- 172.21.15.81:0/3140161795 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc018920 con 0x7f85ec1015d0 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.598+0000 7f85f1d96700 1 -- 172.21.15.81:0/3140161795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 msgr2=0x7f85ec1019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.598+0000 7f85f1d96700 1 --2- 172.21.15.81:0/3140161795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec1019e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00bb80 tx=0x7f85dc00b4f0 comp rx=0 tx=0).stop 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.598+0000 7f85f1d96700 1 -- 172.21.15.81:0/3140161795 shutdown_connections 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.598+0000 7f85f1d96700 1 --2- 172.21.15.81:0/3140161795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec1019e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.598+0000 7f85f1d96700 1 -- 172.21.15.81:0/3140161795 >> 172.21.15.81:0/3140161795 conn(0x7f85ec0fcd50 msgr2=0x7f85ec0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:27.044 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.598+0000 7f85f1d96700 1 -- 172.21.15.81:0/3140161795 shutdown_connections 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.598+0000 7f85f1d96700 1 -- 172.21.15.81:0/3140161795 wait complete. 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.599+0000 7f85f1d96700 1 Processor -- start 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.599+0000 7f85f1d96700 1 -- start start 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.599+0000 7f85f1d96700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec10f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.599+0000 7f85f1d96700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec10f960 con 0x7f85ec1015d0 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.599+0000 7f85eb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec10f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.599+0000 7f85eb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec10f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38542/0 (socket says 172.21.15.81:38542) 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.599+0000 7f85eb7fe700 1 -- 172.21.15.81:0/2838709131 learned_addr learned my addr 172.21.15.81:0/2838709131 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.600+0000 7f85eb7fe700 1 -- 172.21.15.81:0/2838709131 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc00b790 con 0x7f85ec1015d0 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.600+0000 7f85eb7fe700 1 --2- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec10f420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0106a0 tx=0x7f85dc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:27.045 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.600+0000 7f85d3fff700 1 -- 172.21.15.81:0/2838709131 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc010c30 con 0x7f85ec1015d0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.600+0000 7f85d3fff700 1 -- 172.21.15.81:0/2838709131 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85dc01d050 con 0x7f85ec1015d0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.600+0000 7f85d3fff700 1 -- 172.21.15.81:0/2838709131 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f85dc022590 con 0x7f85ec1015d0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.600+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec10fb60 con 0x7f85ec1015d0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.600+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec10ff30 con 0x7f85ec1015d0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.601+0000 7f85d3fff700 1 -- 172.21.15.81:0/2838709131 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f85dc023a40 con 0x7f85ec1015d0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.602+0000 7f85d3fff700 1 --2- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f85d40367e0 0x7f85d4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.602+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ec04bed0 con 0x7f85ec1015d0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.602+0000 7f85d3fff700 1 -- 172.21.15.81:0/2838709131 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f85dc014070 con 0x7f85ec1015d0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.602+0000 7f85eaffd700 1 --2- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f85d40367e0 0x7f85d4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.602+0000 7f85eaffd700 1 --2- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f85d40367e0 0x7f85d4038c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009990 tx=0x7f85e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.607+0000 7f85d3fff700 1 -- 172.21.15.81:0/2838709131 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f85dc0227c0 con 0x7f85ec1015d0 2024-03-22T07:00:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.801+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f85ec110920 con 0x7f85ec1015d0 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.807+0000 7f85d3fff700 1 -- 172.21.15.81:0/2838709131 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f85dc00fbc0 con 0x7f85ec1015d0 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.808+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f85d40367e0 msgr2=0x7f85d4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.808+0000 7f85f1d96700 1 --2- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f85d40367e0 0x7f85d4038c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009990 tx=0x7f85e0006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.808+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 msgr2=0x7f85ec10f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.808+0000 7f85f1d96700 1 --2- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec10f420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0106a0 tx=0x7f85dc00b4f0 comp rx=0 tx=0).stop 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.809+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 shutdown_connections 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.809+0000 7f85f1d96700 1 --2- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f85d40367e0 0x7f85d4038c90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.809+0000 7f85f1d96700 1 --2- 172.21.15.81:0/2838709131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85ec1015d0 0x7f85ec10f420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.809+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 >> 172.21.15.81:0/2838709131 conn(0x7f85ec0fcd50 msgr2=0x7f85ec105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:27.047 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.809+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 shutdown_connections 2024-03-22T07:00:27.048 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:25.809+0000 7f85f1d96700 1 -- 172.21.15.81:0/2838709131 wait complete. 2024-03-22T07:00:27.048 INFO:teuthology.orchestra.run.smithi081.stdout:Enabling the dashboard module... 2024-03-22T07:00:28.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:28 smithi081 ceph-mon[29245]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:00:28.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:28 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1349315744' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-22T07:00:29.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.481+0000 7f1db98b2700 1 Processor -- start 2024-03-22T07:00:29.122 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.481+0000 7f1db98b2700 1 -- start start 2024-03-22T07:00:29.127 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.481+0000 7f1db98b2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4077200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:29.127 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.481+0000 7f1db98b2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40790b0 con 0x7f1db4078ca0 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.481+0000 7f1db2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4077200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.481+0000 7f1db2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4077200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38554/0 (socket says 172.21.15.81:38554) 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.481+0000 7f1db2ffd700 1 -- 172.21.15.81:0/3865583403 learned_addr learned my addr 172.21.15.81:0/3865583403 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.481+0000 7f1db2ffd700 1 -- 172.21.15.81:0/3865583403 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db4077740 con 0x7f1db4078ca0 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.481+0000 7f1db2ffd700 1 --2- 172.21.15.81:0/3865583403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4077200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1da40098e0 tx=0x7f1da4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c78aa8fda2d668 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db1ffb700 1 -- 172.21.15.81:0/3865583403 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1da400b860 con 0x7f1db4078ca0 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db1ffb700 1 -- 172.21.15.81:0/3865583403 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1da400be90 con 0x7f1db4078ca0 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db1ffb700 1 -- 172.21.15.81:0/3865583403 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1da40103c0 con 0x7f1db4078ca0 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db98b2700 1 -- 172.21.15.81:0/3865583403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 msgr2=0x7f1db4077200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db98b2700 1 --2- 172.21.15.81:0/3865583403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4077200 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1da40098e0 tx=0x7f1da4009250 comp rx=0 tx=0).stop 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db98b2700 1 -- 172.21.15.81:0/3865583403 shutdown_connections 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db98b2700 1 --2- 172.21.15.81:0/3865583403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4077200 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:29.128 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db98b2700 1 -- 172.21.15.81:0/3865583403 >> 172.21.15.81:0/3865583403 conn(0x7f1db40fdea0 msgr2=0x7f1db41002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db98b2700 1 -- 172.21.15.81:0/3865583403 shutdown_connections 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.482+0000 7f1db98b2700 1 -- 172.21.15.81:0/3865583403 wait complete. 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.483+0000 7f1db98b2700 1 Processor -- start 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.483+0000 7f1db98b2700 1 -- start start 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.483+0000 7f1db98b2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.483+0000 7f1db98b2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4113d80 con 0x7f1db4078ca0 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.483+0000 7f1db2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.483+0000 7f1db2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38568/0 (socket says 172.21.15.81:38568) 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.483+0000 7f1db2ffd700 1 -- 172.21.15.81:0/1349315744 learned_addr learned my addr 172.21.15.81:0/1349315744 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.484+0000 7f1db2ffd700 1 -- 172.21.15.81:0/1349315744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da40094f0 con 0x7f1db4078ca0 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.484+0000 7f1db2ffd700 1 --2- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4113840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1da40105b0 tx=0x7f1da4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.484+0000 7f1d9b7fe700 1 -- 172.21.15.81:0/1349315744 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1da4010840 con 0x7f1db4078ca0 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.484+0000 7f1d9b7fe700 1 -- 172.21.15.81:0/1349315744 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1da4010e70 con 0x7f1db4078ca0 2024-03-22T07:00:29.129 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.484+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db4113f80 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.484+0000 7f1d9b7fe700 1 -- 172.21.15.81:0/1349315744 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1da401ae30 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.484+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db4114af0 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.485+0000 7f1d9b7fe700 1 -- 172.21.15.81:0/1349315744 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f1da402d3f0 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.485+0000 7f1d9b7fe700 1 --2- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f1d9c0367e0 0x7f1d9c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.485+0000 7f1d9b7fe700 1 -- 172.21.15.81:0/1349315744 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1da404b3e0 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.485+0000 7f1db27fc700 1 --2- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f1d9c0367e0 0x7f1d9c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.485+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db405f5f0 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.485+0000 7f1db27fc700 1 --2- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f1d9c0367e0 0x7f1d9c038c90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009990 tx=0x7f1da8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.490+0000 7f1d9b7fe700 1 -- 172.21.15.81:0/1349315744 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1da402b8e0 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:27.738+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f1db4114d30 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.563+0000 7f1d9b7fe700 1 -- 172.21.15.81:0/1349315744 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f1da401a430 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.565+0000 7f1d9b7fe700 1 -- 172.21.15.81:0/1349315744 <== mon.0 v2:172.21.15.81:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f1da402b8e0 con 0x7f1db4078ca0 2024-03-22T07:00:29.130 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.567+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f1d9c0367e0 msgr2=0x7f1d9c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.567+0000 7f1db98b2700 1 --2- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f1d9c0367e0 0x7f1d9c038c90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009990 tx=0x7f1da8006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.568+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 msgr2=0x7f1db4113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.568+0000 7f1db98b2700 1 --2- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4113840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1da40105b0 tx=0x7f1da4009250 comp rx=0 tx=0).stop 2024-03-22T07:00:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.568+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 shutdown_connections 2024-03-22T07:00:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.568+0000 7f1db98b2700 1 --2- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7f1d9c0367e0 0x7f1d9c038c90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.568+0000 7f1db98b2700 1 --2- 172.21.15.81:0/1349315744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1db4078ca0 0x7f1db4113840 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.568+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 >> 172.21.15.81:0/1349315744 conn(0x7f1db40fdea0 msgr2=0x7f1db40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.568+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 shutdown_connections 2024-03-22T07:00:29.131 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:28.568+0000 7f1db98b2700 1 -- 172.21.15.81:0/1349315744 wait complete. 2024-03-22T07:00:29.774 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:29 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1349315744' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-22T07:00:29.774 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:29 smithi081 ceph-mon[29245]: mgrmap e9: smithi081.bcxarw(active, since 27s) 2024-03-22T07:00:30.450 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:00:30.450 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-22T07:00:30.450 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "active_name": "smithi081.bcxarw", 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.567+0000 7fde52725700 1 Processor -- start 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.567+0000 7fde52725700 1 -- start start 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.567+0000 7fde52725700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.567+0000 7fde52725700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c103080 con 0x7fde4c102730 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.567+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.567+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38614/0 (socket says 172.21.15.81:38614) 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.567+0000 7fde4bfff700 1 -- 172.21.15.81:0/2355193977 learned_addr learned my addr 172.21.15.81:0/2355193977 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.567+0000 7fde4bfff700 1 -- 172.21.15.81:0/2355193977 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c1031c0 con 0x7fde4c102730 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.568+0000 7fde4bfff700 1 --2- 172.21.15.81:0/2355193977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c102b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0098e0 tx=0x7fde3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9351a95adf46e7f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:30.451 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.569+0000 7fde4b7fe700 1 -- 172.21.15.81:0/2355193977 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde3c00b860 con 0x7fde4c102730 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.569+0000 7fde4b7fe700 1 -- 172.21.15.81:0/2355193977 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde3c00be90 con 0x7fde4c102730 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.569+0000 7fde4b7fe700 1 -- 172.21.15.81:0/2355193977 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde3c0103c0 con 0x7fde4c102730 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.570+0000 7fde52725700 1 -- 172.21.15.81:0/2355193977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 msgr2=0x7fde4c102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.570+0000 7fde52725700 1 --2- 172.21.15.81:0/2355193977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c102b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0098e0 tx=0x7fde3c009250 comp rx=0 tx=0).stop 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.570+0000 7fde52725700 1 -- 172.21.15.81:0/2355193977 shutdown_connections 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.570+0000 7fde52725700 1 --2- 172.21.15.81:0/2355193977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c102b40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.570+0000 7fde52725700 1 -- 172.21.15.81:0/2355193977 >> 172.21.15.81:0/2355193977 conn(0x7fde4c0fdf10 msgr2=0x7fde4c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.570+0000 7fde52725700 1 -- 172.21.15.81:0/2355193977 shutdown_connections 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.570+0000 7fde52725700 1 -- 172.21.15.81:0/2355193977 wait complete. 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.570+0000 7fde52725700 1 Processor -- start 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.570+0000 7fde52725700 1 -- start start 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde52725700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde52725700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c111b70 con 0x7fde4c102730 2024-03-22T07:00:30.452 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38618/0 (socket says 172.21.15.81:38618) 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde4bfff700 1 -- 172.21.15.81:0/2816802838 learned_addr learned my addr 172.21.15.81:0/2816802838 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde4bfff700 1 -- 172.21.15.81:0/2816802838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0094f0 con 0x7fde4c102730 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde4bfff700 1 --2- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c111630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde3c006fd0 tx=0x7fde3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde497fa700 1 -- 172.21.15.81:0/2816802838 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde3c010630 con 0x7fde4c102730 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde497fa700 1 -- 172.21.15.81:0/2816802838 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde3c010c60 con 0x7fde4c102730 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde497fa700 1 -- 172.21.15.81:0/2816802838 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde3c01ad40 con 0x7fde4c102730 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c111d70 con 0x7fde4c102730 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.571+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c1128e0 con 0x7fde4c102730 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.572+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c05f5f0 con 0x7fde4c102730 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.573+0000 7fde497fa700 1 -- 172.21.15.81:0/2816802838 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fde3c01a5f0 con 0x7fde4c102730 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.573+0000 7fde497fa700 1 --2- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fde38036830 0x7fde38038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.573+0000 7fde497fa700 1 -- 172.21.15.81:0/2816802838 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fde3c04a470 con 0x7fde4c102730 2024-03-22T07:00:30.453 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.573+0000 7fde43fff700 1 -- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fde38036830 msgr2=0x7fde38038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.81:6800/4051369418 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.573+0000 7fde43fff700 1 --2- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fde38036830 0x7fde38038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.577+0000 7fde497fa700 1 -- 172.21.15.81:0/2816802838 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fde3c018940 con 0x7fde4c102730 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.774+0000 7fde43fff700 1 -- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fde38036830 msgr2=0x7fde38038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.81:6800/4051369418 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.774+0000 7fde43fff700 1 --2- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fde38036830 0x7fde38038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.825+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fde4c112c80 con 0x7fde4c102730 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.826+0000 7fde497fa700 1 -- 172.21.15.81:0/2816802838 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fde3c020030 con 0x7fde4c102730 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fde38036830 msgr2=0x7fde38038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 --2- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fde38036830 0x7fde38038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 msgr2=0x7fde4c111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 --2- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c111630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fde3c006fd0 tx=0x7fde3c009250 comp rx=0 tx=0).stop 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 shutdown_connections 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 --2- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fde38036830 0x7fde38038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:30.454 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 --2- 172.21.15.81:0/2816802838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde4c102730 0x7fde4c111630 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:30.455 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 >> 172.21.15.81:0/2816802838 conn(0x7fde4c0fdf10 msgr2=0x7fde4c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:30.455 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 shutdown_connections 2024-03-22T07:00:30.455 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:29.828+0000 7fde52725700 1 -- 172.21.15.81:0/2816802838 wait complete. 2024-03-22T07:00:30.455 INFO:teuthology.orchestra.run.smithi081.stdout:Waiting for the mgr to restart... 2024-03-22T07:00:30.455 INFO:teuthology.orchestra.run.smithi081.stdout:Waiting for mgr epoch 9... 2024-03-22T07:00:30.665 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:30 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2816802838' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: Active manager daemon smithi081.bcxarw restarted 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: Activating manager daemon smithi081.bcxarw 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: osdmap e3: 0 total, 0 up, 0 in 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: mgrmap e10: smithi081.bcxarw(active, starting, since 0.00322845s) 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:00:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: Manager daemon smithi081.bcxarw is now available 2024-03-22T07:00:35.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:00:35.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:35 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:00:36.376 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:36 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:00:36.376 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:36 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/trash_purge_schedule"}]: dispatch 2024-03-22T07:00:36.376 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:36 smithi081 ceph-mon[29245]: [22/Mar/2024:07:00:35] ENGINE Bus STARTING 2024-03-22T07:00:36.376 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:36 smithi081 ceph-mon[29245]: [22/Mar/2024:07:00:35] ENGINE Serving on https://172.21.15.81:7150 2024-03-22T07:00:36.376 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:36 smithi081 ceph-mon[29245]: [22/Mar/2024:07:00:35] ENGINE Bus STARTED 2024-03-22T07:00:36.376 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:36 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:36.376 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:36 smithi081 ceph-mon[29245]: mgrmap e11: smithi081.bcxarw(active, since 1.00529s) 2024-03-22T07:00:36.665 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:00:36.665 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.867+0000 7fd738b92700 1 Processor -- start 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.867+0000 7fd738b92700 1 -- start start 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.868+0000 7fd738b92700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.868+0000 7fd738b92700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd734103080 con 0x7fd734102730 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.868+0000 7fd73259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.868+0000 7fd73259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38632/0 (socket says 172.21.15.81:38632) 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.868+0000 7fd73259c700 1 -- 172.21.15.81:0/3443671404 learned_addr learned my addr 172.21.15.81:0/3443671404 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.868+0000 7fd73259c700 1 -- 172.21.15.81:0/3443671404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7341031c0 con 0x7fd734102730 2024-03-22T07:00:36.666 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.868+0000 7fd73259c700 1 --2- 172.21.15.81:0/3443671404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734102b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7240098e0 tx=0x7fd724009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20da6b5144426dc server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.868+0000 7fd73159a700 1 -- 172.21.15.81:0/3443671404 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd72400b860 con 0x7fd734102730 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.868+0000 7fd73159a700 1 -- 172.21.15.81:0/3443671404 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd72400be90 con 0x7fd734102730 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.869+0000 7fd73159a700 1 -- 172.21.15.81:0/3443671404 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7240103c0 con 0x7fd734102730 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.869+0000 7fd738b92700 1 -- 172.21.15.81:0/3443671404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 msgr2=0x7fd734102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.869+0000 7fd738b92700 1 --2- 172.21.15.81:0/3443671404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734102b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd7240098e0 tx=0x7fd724009250 comp rx=0 tx=0).stop 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.869+0000 7fd738b92700 1 -- 172.21.15.81:0/3443671404 shutdown_connections 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.869+0000 7fd738b92700 1 --2- 172.21.15.81:0/3443671404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734102b40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.869+0000 7fd738b92700 1 -- 172.21.15.81:0/3443671404 >> 172.21.15.81:0/3443671404 conn(0x7fd7340fdf10 msgr2=0x7fd734100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.869+0000 7fd738b92700 1 -- 172.21.15.81:0/3443671404 shutdown_connections 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.869+0000 7fd738b92700 1 -- 172.21.15.81:0/3443671404 wait complete. 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.870+0000 7fd738b92700 1 Processor -- start 2024-03-22T07:00:36.667 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.870+0000 7fd738b92700 1 -- start start 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.870+0000 7fd738b92700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.870+0000 7fd738b92700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd734111b70 con 0x7fd734102730 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.870+0000 7fd73259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.870+0000 7fd73259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38638/0 (socket says 172.21.15.81:38638) 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.870+0000 7fd73259c700 1 -- 172.21.15.81:0/1044592896 learned_addr learned my addr 172.21.15.81:0/1044592896 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.870+0000 7fd73259c700 1 -- 172.21.15.81:0/1044592896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240094f0 con 0x7fd734102730 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.870+0000 7fd73259c700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734111630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7240094c0 tx=0x7fd724009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.871+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd724010680 con 0x7fd734102730 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.871+0000 7fd738b92700 1 -- 172.21.15.81:0/1044592896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd734111d70 con 0x7fd734102730 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.871+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd724010cb0 con 0x7fd734102730 2024-03-22T07:00:36.668 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.871+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd72401ad40 con 0x7fd734102730 2024-03-22T07:00:36.669 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.871+0000 7fd738b92700 1 -- 172.21.15.81:0/1044592896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7341128e0 con 0x7fd734102730 2024-03-22T07:00:36.669 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.872+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fd724025400 con 0x7fd734102730 2024-03-22T07:00:36.669 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.872+0000 7fd722ffd700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 0x7fd71c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:36.669 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.872+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 --> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd71c0393a0 con 0x7fd71c0367e0 2024-03-22T07:00:36.669 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.872+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd72404a440 con 0x7fd734102730 2024-03-22T07:00:36.669 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.872+0000 7fd731d9b700 1 -- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 msgr2=0x7fd71c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.81:6800/4051369418 2024-03-22T07:00:36.669 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:30.872+0000 7fd731d9b700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 0x7fd71c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:00:36.669 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:31.073+0000 7fd731d9b700 1 -- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 msgr2=0x7fd71c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.81:6800/4051369418 2024-03-22T07:00:36.669 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:31.073+0000 7fd731d9b700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 0x7fd71c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:31.474+0000 7fd731d9b700 1 -- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 msgr2=0x7fd71c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.81:6800/4051369418 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:31.474+0000 7fd731d9b700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 0x7fd71c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:32.275+0000 7fd731d9b700 1 -- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 msgr2=0x7fd71c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.81:6800/4051369418 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:32.275+0000 7fd731d9b700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 0x7fd71c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:33.876+0000 7fd731d9b700 1 -- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 msgr2=0x7fd71c038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.81:6800/4051369418 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:33.876+0000 7fd731d9b700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 0x7fd71c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:35.071+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7fd7240257a0 con 0x7fd734102730 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:35.071+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 msgr2=0x7fd71c038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:35.071+0000 7fd722ffd700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 0x7fd71c038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.072+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fd724049a10 con 0x7fd734102730 2024-03-22T07:00:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.072+0000 7fd722ffd700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fd71c039fd0 0x7fd71c03c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.072+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 --> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd71c0393a0 con 0x7fd71c039fd0 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.073+0000 7fd731d9b700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fd71c039fd0 0x7fd71c03c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.073+0000 7fd731d9b700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fd71c039fd0 0x7fd71c03c3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd7280039d0 tx=0x7fd7280092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.074+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 <== mgr.14162 v2:172.21.15.81:6800/1870922013 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fd71c0393a0 con 0x7fd71c039fd0 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.079+0000 7fd738b92700 1 -- 172.21.15.81:0/1044592896 --> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd73405f5f0 con 0x7fd71c039fd0 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd722ffd700 1 -- 172.21.15.81:0/1044592896 <== mgr.14162 v2:172.21.15.81:6800/1870922013 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fd73405f5f0 con 0x7fd71c039fd0 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 -- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fd71c039fd0 msgr2=0x7fd71c03c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fd71c039fd0 0x7fd71c03c3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd7280039d0 tx=0x7fd7280092f0 comp rx=0 tx=0).stop 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 -- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 msgr2=0x7fd734111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734111630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7240094c0 tx=0x7fd724009250 comp rx=0 tx=0).stop 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 -- 172.21.15.81:0/1044592896 shutdown_connections 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd734102730 0x7fd734111630 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fd71c039fd0 0x7fd71c03c3b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 --2- 172.21.15.81:0/1044592896 >> [v2:172.21.15.81:6800/4051369418,v1:172.21.15.81:6801/4051369418] conn(0x7fd71c0367e0 0x7fd71c038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:36.671 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 -- 172.21.15.81:0/1044592896 >> 172.21.15.81:0/1044592896 conn(0x7fd7340fdf10 msgr2=0x7fd7340feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:36.672 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 -- 172.21.15.81:0/1044592896 shutdown_connections 2024-03-22T07:00:36.672 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:36.080+0000 7fd738b92700 1 -- 172.21.15.81:0/1044592896 wait complete. 2024-03-22T07:00:36.672 INFO:teuthology.orchestra.run.smithi081.stdout:mgr epoch 9 is available 2024-03-22T07:00:36.672 INFO:teuthology.orchestra.run.smithi081.stdout:Generating a dashboard self-signed certificate... 2024-03-22T07:00:37.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:37 smithi081 ceph-mon[29245]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-22T07:00:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:37 smithi081 ceph-mon[29245]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-22T07:00:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:37 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:37.983 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-22T07:00:37.983 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.085+0000 7fde941fa700 1 Processor -- start 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.085+0000 7fde941fa700 1 -- start start 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.085+0000 7fde941fa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.085+0000 7fde941fa700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c1058f0 con 0x7fde8c076b20 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.085+0000 7fde91f96700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.085+0000 7fde91f96700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53326/0 (socket says 172.21.15.81:53326) 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.085+0000 7fde91f96700 1 -- 172.21.15.81:0/3450304764 learned_addr learned my addr 172.21.15.81:0/3450304764 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.086+0000 7fde91f96700 1 -- 172.21.15.81:0/3450304764 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c105a30 con 0x7fde8c076b20 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.086+0000 7fde91f96700 1 --2- 172.21.15.81:0/3450304764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1053b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde880098e0 tx=0x7fde88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd328360c9e9809 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.086+0000 7fde90f94700 1 -- 172.21.15.81:0/3450304764 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde8800b860 con 0x7fde8c076b20 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.086+0000 7fde90f94700 1 -- 172.21.15.81:0/3450304764 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde8800be90 con 0x7fde8c076b20 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.086+0000 7fde90f94700 1 -- 172.21.15.81:0/3450304764 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde880103c0 con 0x7fde8c076b20 2024-03-22T07:00:37.984 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.087+0000 7fde941fa700 1 -- 172.21.15.81:0/3450304764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 msgr2=0x7fde8c1053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.087+0000 7fde941fa700 1 --2- 172.21.15.81:0/3450304764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1053b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde880098e0 tx=0x7fde88009250 comp rx=0 tx=0).stop 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.087+0000 7fde941fa700 1 -- 172.21.15.81:0/3450304764 shutdown_connections 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.087+0000 7fde941fa700 1 --2- 172.21.15.81:0/3450304764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1053b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.087+0000 7fde941fa700 1 -- 172.21.15.81:0/3450304764 >> 172.21.15.81:0/3450304764 conn(0x7fde8c0fcd80 msgr2=0x7fde8c0ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.087+0000 7fde941fa700 1 -- 172.21.15.81:0/3450304764 shutdown_connections 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.087+0000 7fde941fa700 1 -- 172.21.15.81:0/3450304764 wait complete. 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.088+0000 7fde941fa700 1 Processor -- start 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.088+0000 7fde941fa700 1 -- start start 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.088+0000 7fde941fa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.088+0000 7fde941fa700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c111b20 con 0x7fde8c076b20 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.088+0000 7fde91f96700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.088+0000 7fde91f96700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53330/0 (socket says 172.21.15.81:53330) 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.088+0000 7fde91f96700 1 -- 172.21.15.81:0/729384554 learned_addr learned my addr 172.21.15.81:0/729384554 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.088+0000 7fde91f96700 1 -- 172.21.15.81:0/729384554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880094f0 con 0x7fde8c076b20 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.089+0000 7fde91f96700 1 --2- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1115e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde880094c0 tx=0x7fde88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.089+0000 7fde827fc700 1 -- 172.21.15.81:0/729384554 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde88010890 con 0x7fde8c076b20 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.089+0000 7fde827fc700 1 -- 172.21.15.81:0/729384554 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde88010ec0 con 0x7fde8c076b20 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.089+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c111d20 con 0x7fde8c076b20 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.089+0000 7fde827fc700 1 -- 172.21.15.81:0/729384554 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde880243f0 con 0x7fde8c076b20 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.089+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c112890 con 0x7fde8c076b20 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.090+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8c05f5f0 con 0x7fde8c076b20 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.090+0000 7fde827fc700 1 -- 172.21.15.81:0/729384554 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7fde8802d3f0 con 0x7fde8c076b20 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.090+0000 7fde827fc700 1 --2- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde78036710 0x7fde78038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.090+0000 7fde827fc700 1 -- 172.21.15.81:0/729384554 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fde8804a190 con 0x7fde8c076b20 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.090+0000 7fde91795700 1 --2- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde78036710 0x7fde78038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.090+0000 7fde91795700 1 --2- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde78036710 0x7fde78038bc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009990 tx=0x7fde7c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:37.986 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.098+0000 7fde827fc700 1 -- 172.21.15.81:0/729384554 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fde8802b8e0 con 0x7fde8c076b20 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.324+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 --> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fde8c0fe120 con 0x7fde78036710 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.387+0000 7fde827fc700 1 -- 172.21.15.81:0/729384554 <== mgr.14162 v2:172.21.15.81:6800/1870922013 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fde8c0fe120 con 0x7fde78036710 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde78036710 msgr2=0x7fde78038bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 --2- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde78036710 0x7fde78038bc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009990 tx=0x7fde7c006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 msgr2=0x7fde8c1115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 --2- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1115e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde880094c0 tx=0x7fde88009250 comp rx=0 tx=0).stop 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 shutdown_connections 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 --2- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde78036710 0x7fde78038bc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 --2- 172.21.15.81:0/729384554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde8c076b20 0x7fde8c1115e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 >> 172.21.15.81:0/729384554 conn(0x7fde8c0fcd80 msgr2=0x7fde8c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 shutdown_connections 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:37.389+0000 7fde941fa700 1 -- 172.21.15.81:0/729384554 wait complete. 2024-03-22T07:00:37.987 INFO:teuthology.orchestra.run.smithi081.stdout:Creating initial admin user... 2024-03-22T07:00:38.646 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:38 smithi081 ceph-mon[29245]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:38.646 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:38 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:38.646 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:38 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:38.646 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:38 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:38.647 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:38 smithi081 ceph-mon[29245]: mgrmap e12: smithi081.bcxarw(active, since 2s) 2024-03-22T07:00:39.594 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$X2uJBydmBcqRNe5x3SAx0OOGXz/WwKX1x14E0c/CFKRT9fg0dtNHq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711090838, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.418+0000 7f63b1d3a700 1 Processor -- start 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.418+0000 7f63b1d3a700 1 -- start start 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.418+0000 7f63b1d3a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.418+0000 7f63b1d3a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac103080 con 0x7f63ac102730 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.418+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.418+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53336/0 (socket says 172.21.15.81:53336) 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.418+0000 7f63ab7fe700 1 -- 172.21.15.81:0/724001837 learned_addr learned my addr 172.21.15.81:0/724001837 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63ab7fe700 1 -- 172.21.15.81:0/724001837 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac1031c0 con 0x7f63ac102730 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63ab7fe700 1 --2- 172.21.15.81:0/724001837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac102b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f639c0098e0 tx=0x7f639c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80815c6dd7b30f82 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63aaffd700 1 -- 172.21.15.81:0/724001837 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f639c00b860 con 0x7f63ac102730 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63aaffd700 1 -- 172.21.15.81:0/724001837 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f639c00be90 con 0x7f63ac102730 2024-03-22T07:00:39.595 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63aaffd700 1 -- 172.21.15.81:0/724001837 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f639c0103c0 con 0x7f63ac102730 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63b1d3a700 1 -- 172.21.15.81:0/724001837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 msgr2=0x7f63ac102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63b1d3a700 1 --2- 172.21.15.81:0/724001837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac102b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f639c0098e0 tx=0x7f639c009250 comp rx=0 tx=0).stop 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63b1d3a700 1 -- 172.21.15.81:0/724001837 shutdown_connections 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63b1d3a700 1 --2- 172.21.15.81:0/724001837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac102b40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.419+0000 7f63b1d3a700 1 -- 172.21.15.81:0/724001837 >> 172.21.15.81:0/724001837 conn(0x7f63ac0fdf10 msgr2=0x7f63ac100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.420+0000 7f63b1d3a700 1 -- 172.21.15.81:0/724001837 shutdown_connections 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.420+0000 7f63b1d3a700 1 -- 172.21.15.81:0/724001837 wait complete. 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.420+0000 7f63b1d3a700 1 Processor -- start 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.420+0000 7f63b1d3a700 1 -- start start 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.420+0000 7f63b1d3a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.420+0000 7f63b1d3a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac111b50 con 0x7f63ac102730 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.420+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:39.596 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.420+0000 7f63ab7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53340/0 (socket says 172.21.15.81:53340) 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.420+0000 7f63ab7fe700 1 -- 172.21.15.81:0/1728660523 learned_addr learned my addr 172.21.15.81:0/1728660523 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.421+0000 7f63ab7fe700 1 -- 172.21.15.81:0/1728660523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c0094f0 con 0x7f63ac102730 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.421+0000 7f63ab7fe700 1 --2- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac111610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f639c0094c0 tx=0x7f639c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.421+0000 7f63a8ff9700 1 -- 172.21.15.81:0/1728660523 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f639c010680 con 0x7f63ac102730 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.421+0000 7f63a8ff9700 1 -- 172.21.15.81:0/1728660523 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f639c010cb0 con 0x7f63ac102730 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.421+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac111d50 con 0x7f63ac102730 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.421+0000 7f63a8ff9700 1 -- 172.21.15.81:0/1728660523 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f639c01ad40 con 0x7f63ac102730 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.421+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac1128c0 con 0x7f63ac102730 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.422+0000 7f63a8ff9700 1 -- 172.21.15.81:0/1728660523 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f639c025400 con 0x7f63ac102730 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.422+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac05f5f0 con 0x7f63ac102730 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.422+0000 7f63a8ff9700 1 --2- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63980367e0 0x7f6398038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.422+0000 7f63a8ff9700 1 -- 172.21.15.81:0/1728660523 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f639c04a3f0 con 0x7f63ac102730 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.422+0000 7f63a2ffd700 1 --2- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63980367e0 0x7f6398038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:39.597 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.422+0000 7f63a2ffd700 1 --2- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63980367e0 0x7f6398038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6394009990 tx=0x7f6394006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.427+0000 7f63a8ff9700 1 -- 172.21.15.81:0/1728660523 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f639c02c8f0 con 0x7f63ac102730 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.646+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 --> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f63ac05f410 con 0x7f63980367e0 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.870+0000 7f63a8ff9700 1 -- 172.21.15.81:0/1728660523 <== mgr.14162 v2:172.21.15.81:6800/1870922013 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f63ac05f410 con 0x7f63980367e0 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63980367e0 msgr2=0x7f6398038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 --2- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63980367e0 0x7f6398038c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6394009990 tx=0x7f6394006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 msgr2=0x7f63ac111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 --2- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac111610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f639c0094c0 tx=0x7f639c009250 comp rx=0 tx=0).stop 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 shutdown_connections 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 --2- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63980367e0 0x7f6398038c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 --2- 172.21.15.81:0/1728660523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63ac102730 0x7f63ac111610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 >> 172.21.15.81:0/1728660523 conn(0x7f63ac0fdf10 msgr2=0x7f63ac0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 shutdown_connections 2024-03-22T07:00:39.598 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:38.872+0000 7f63b1d3a700 1 -- 172.21.15.81:0/1728660523 wait complete. 2024-03-22T07:00:39.599 INFO:teuthology.orchestra.run.smithi081.stdout:Fetching dashboard port number... 2024-03-22T07:00:40.150 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:39 smithi081 ceph-mon[29245]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:40.150 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:39 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:40.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stdout 8443 2024-03-22T07:00:40.987 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.004+0000 7fc8b4b9e700 1 Processor -- start 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.004+0000 7fc8b4b9e700 1 -- start start 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.004+0000 7fc8b4b9e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0103140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.004+0000 7fc8b4b9e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0103680 con 0x7fc8b0102d30 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.004+0000 7fc8ae59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.004+0000 7fc8ae59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0103140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53356/0 (socket says 172.21.15.81:53356) 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.004+0000 7fc8ae59c700 1 -- 172.21.15.81:0/435566145 learned_addr learned my addr 172.21.15.81:0/435566145 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.004+0000 7fc8ae59c700 1 -- 172.21.15.81:0/435566145 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b01037c0 con 0x7fc8b0102d30 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.005+0000 7fc8ae59c700 1 --2- 172.21.15.81:0/435566145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0103140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00098e0 tx=0x7fc8a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79c6ccb3ee6b0eb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.005+0000 7fc8ad59a700 1 -- 172.21.15.81:0/435566145 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8a000b860 con 0x7fc8b0102d30 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.005+0000 7fc8ad59a700 1 -- 172.21.15.81:0/435566145 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8a000be90 con 0x7fc8b0102d30 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.005+0000 7fc8ad59a700 1 -- 172.21.15.81:0/435566145 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8a00103c0 con 0x7fc8b0102d30 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.005+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/435566145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 msgr2=0x7fc8b0103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.005+0000 7fc8b4b9e700 1 --2- 172.21.15.81:0/435566145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0103140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00098e0 tx=0x7fc8a0009250 comp rx=0 tx=0).stop 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/435566145 shutdown_connections 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8b4b9e700 1 --2- 172.21.15.81:0/435566145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0103140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/435566145 >> 172.21.15.81:0/435566145 conn(0x7fc8b00fc500 msgr2=0x7fc8b00fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/435566145 shutdown_connections 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/435566145 wait complete. 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8b4b9e700 1 Processor -- start 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8b4b9e700 1 -- start start 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8b4b9e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8b4b9e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0113cc0 con 0x7fc8b0102d30 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8ae59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8ae59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53364/0 (socket says 172.21.15.81:53364) 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.006+0000 7fc8ae59c700 1 -- 172.21.15.81:0/3985609907 learned_addr learned my addr 172.21.15.81:0/3985609907 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:40.989 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.007+0000 7fc8ae59c700 1 -- 172.21.15.81:0/3985609907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a00094f0 con 0x7fc8b0102d30 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.007+0000 7fc8ae59c700 1 --2- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0113780 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0011680 tx=0x7fc8a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.007+0000 7fc89effd700 1 -- 172.21.15.81:0/3985609907 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8a0010890 con 0x7fc8b0102d30 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.007+0000 7fc89effd700 1 -- 172.21.15.81:0/3985609907 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8a0010ec0 con 0x7fc8b0102d30 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.007+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b0113ec0 con 0x7fc8b0102d30 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.007+0000 7fc89effd700 1 -- 172.21.15.81:0/3985609907 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8a00243f0 con 0x7fc8b0102d30 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.007+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b0114a30 con 0x7fc8b0102d30 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.008+0000 7fc89effd700 1 -- 172.21.15.81:0/3985609907 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fc8a001a8c0 con 0x7fc8b0102d30 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.008+0000 7fc89effd700 1 --2- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fc8980367e0 0x7fc898038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.008+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b005f5f0 con 0x7fc8b0102d30 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.008+0000 7fc89effd700 1 -- 172.21.15.81:0/3985609907 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc8a004a3a0 con 0x7fc8b0102d30 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.008+0000 7fc8add9b700 1 --2- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fc8980367e0 0x7fc898038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:40.990 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.009+0000 7fc8add9b700 1 --2- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fc8980367e0 0x7fc898038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009990 tx=0x7fc8a4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.013+0000 7fc89effd700 1 -- 172.21.15.81:0/3985609907 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc8a002b8e0 con 0x7fc8b0102d30 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.207+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fc8b004bed0 con 0x7fc8b0102d30 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.208+0000 7fc89effd700 1 -- 172.21.15.81:0/3985609907 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fc8a002b2f0 con 0x7fc8b0102d30 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fc8980367e0 msgr2=0x7fc898038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 --2- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fc8980367e0 0x7fc898038c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009990 tx=0x7fc8a4006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 msgr2=0x7fc8b0113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 --2- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0113780 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0011680 tx=0x7fc8a0009250 comp rx=0 tx=0).stop 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 shutdown_connections 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 --2- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fc8980367e0 0x7fc898038c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 --2- 172.21.15.81:0/3985609907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8b0102d30 0x7fc8b0113780 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 >> 172.21.15.81:0/3985609907 conn(0x7fc8b00fc500 msgr2=0x7fc8b00fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 shutdown_connections 2024-03-22T07:00:40.991 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:40.210+0000 7fc8b4b9e700 1 -- 172.21.15.81:0/3985609907 wait complete. 2024-03-22T07:00:40.993 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T07:00:40.994 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout disabled 2024-03-22T07:00:41.001 INFO:teuthology.orchestra.run.smithi081.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T07:00:41.001 INFO:teuthology.orchestra.run.smithi081.stdout:systemctl: stdout inactive 2024-03-22T07:00:41.001 INFO:teuthology.orchestra.run.smithi081.stdout:firewalld.service is not enabled 2024-03-22T07:00:41.001 INFO:teuthology.orchestra.run.smithi081.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-22T07:00:41.003 INFO:teuthology.orchestra.run.smithi081.stdout:Ceph Dashboard is now available at: 2024-03-22T07:00:41.003 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:41.003 INFO:teuthology.orchestra.run.smithi081.stdout: URL: https://smithi081.front.sepia.ceph.com:8443/ 2024-03-22T07:00:41.003 INFO:teuthology.orchestra.run.smithi081.stdout: User: admin 2024-03-22T07:00:41.003 INFO:teuthology.orchestra.run.smithi081.stdout: Password: zdm69hfpt8 2024-03-22T07:00:41.003 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:41.004 INFO:teuthology.orchestra.run.smithi081.stdout:Saving cluster configuration to /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config directory 2024-03-22T07:00:41.004 INFO:teuthology.orchestra.run.smithi081.stdout:Enabling autotune for osd_memory_target 2024-03-22T07:00:41.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:40 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3985609907' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-22T07:00:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.423+0000 7f2e84243700 1 Processor -- start 2024-03-22T07:00:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.423+0000 7f2e84243700 1 -- start start 2024-03-22T07:00:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.423+0000 7f2e84243700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.423+0000 7f2e84243700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c1030c0 con 0x7f2e7c102770 2024-03-22T07:00:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.423+0000 7f2e81fdf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.423+0000 7f2e81fdf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53378/0 (socket says 172.21.15.81:53378) 2024-03-22T07:00:42.311 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.423+0000 7f2e81fdf700 1 -- 172.21.15.81:0/2985857890 learned_addr learned my addr 172.21.15.81:0/2985857890 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.424+0000 7f2e81fdf700 1 -- 172.21.15.81:0/2985857890 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c103200 con 0x7f2e7c102770 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.424+0000 7f2e81fdf700 1 --2- 172.21.15.81:0/2985857890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c102b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098e0 tx=0x7f2e78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa03c3cf12873d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.424+0000 7f2e80fdd700 1 -- 172.21.15.81:0/2985857890 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e7800b860 con 0x7f2e7c102770 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.424+0000 7f2e80fdd700 1 -- 172.21.15.81:0/2985857890 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e7800be90 con 0x7f2e7c102770 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.424+0000 7f2e80fdd700 1 -- 172.21.15.81:0/2985857890 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e780103c0 con 0x7f2e7c102770 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.424+0000 7f2e84243700 1 -- 172.21.15.81:0/2985857890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 msgr2=0x7f2e7c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.424+0000 7f2e84243700 1 --2- 172.21.15.81:0/2985857890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c102b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2e780098e0 tx=0x7f2e78009250 comp rx=0 tx=0).stop 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.425+0000 7f2e84243700 1 -- 172.21.15.81:0/2985857890 shutdown_connections 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.425+0000 7f2e84243700 1 --2- 172.21.15.81:0/2985857890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c102b80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.425+0000 7f2e84243700 1 -- 172.21.15.81:0/2985857890 >> 172.21.15.81:0/2985857890 conn(0x7f2e7c0fde90 msgr2=0x7f2e7c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.425+0000 7f2e84243700 1 -- 172.21.15.81:0/2985857890 shutdown_connections 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.425+0000 7f2e84243700 1 -- 172.21.15.81:0/2985857890 wait complete. 2024-03-22T07:00:42.312 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.425+0000 7f2e84243700 1 Processor -- start 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.425+0000 7f2e84243700 1 -- start start 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.425+0000 7f2e84243700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c1115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.425+0000 7f2e84243700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c111b10 con 0x7f2e7c102770 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e81fdf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c1115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e81fdf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c1115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53382/0 (socket says 172.21.15.81:53382) 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e81fdf700 1 -- 172.21.15.81:0/2940163015 learned_addr learned my addr 172.21.15.81:0/2940163015 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e81fdf700 1 -- 172.21.15.81:0/2940163015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e780094f0 con 0x7f2e7c102770 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e81fdf700 1 --2- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c1115d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e780090b0 tx=0x7f2e78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e727fc700 1 -- 172.21.15.81:0/2940163015 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e78010840 con 0x7f2e7c102770 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e727fc700 1 -- 172.21.15.81:0/2940163015 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e78010e70 con 0x7f2e7c102770 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7c111d10 con 0x7f2e7c102770 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e727fc700 1 -- 172.21.15.81:0/2940163015 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e78024e40 con 0x7f2e7c102770 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.426+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7c112880 con 0x7f2e7c102770 2024-03-22T07:00:42.313 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.427+0000 7f2e727fc700 1 -- 172.21.15.81:0/2940163015 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f2e78024430 con 0x7f2e7c102770 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.427+0000 7f2e727fc700 1 --2- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f2e68036790 0x7f2e68038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.427+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7c05f5f0 con 0x7f2e7c102770 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.427+0000 7f2e727fc700 1 -- 172.21.15.81:0/2940163015 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2e7804a8e0 con 0x7f2e7c102770 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.427+0000 7f2e817de700 1 --2- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f2e68036790 0x7f2e68038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.428+0000 7f2e817de700 1 --2- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f2e68036790 0x7f2e68038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009990 tx=0x7f2e6c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.432+0000 7f2e727fc700 1 -- 172.21.15.81:0/2940163015 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2e7802c910 con 0x7f2e7c102770 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.615+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f2e7c04bed0 con 0x7f2e7c102770 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.616+0000 7f2e727fc700 1 -- 172.21.15.81:0/2940163015 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2e7802c340 con 0x7f2e7c102770 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.617+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f2e68036790 msgr2=0x7f2e68038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.617+0000 7f2e84243700 1 --2- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f2e68036790 0x7f2e68038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009990 tx=0x7f2e6c006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.618+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 msgr2=0x7f2e7c1115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.618+0000 7f2e84243700 1 --2- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c1115d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e780090b0 tx=0x7f2e78009250 comp rx=0 tx=0).stop 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.618+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 shutdown_connections 2024-03-22T07:00:42.314 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.618+0000 7f2e84243700 1 --2- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f2e68036790 0x7f2e68038c40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:42.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.618+0000 7f2e84243700 1 --2- 172.21.15.81:0/2940163015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e7c102770 0x7f2e7c1115d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:42.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.618+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 >> 172.21.15.81:0/2940163015 conn(0x7f2e7c0fde90 msgr2=0x7f2e7c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:42.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.618+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 shutdown_connections 2024-03-22T07:00:42.315 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:41.618+0000 7f2e84243700 1 -- 172.21.15.81:0/2940163015 wait complete. 2024-03-22T07:00:43.011 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:42 smithi081 ceph-mon[29245]: mgrmap e13: smithi081.bcxarw(active, since 6s) 2024-03-22T07:00:43.576 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.713+0000 7f63dfc78700 1 Processor -- start 2024-03-22T07:00:43.576 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.713+0000 7f63dfc78700 1 -- start start 2024-03-22T07:00:43.576 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dfc78700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dfc78700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8103070 con 0x7f63d8102720 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dda14700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dda14700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53396/0 (socket says 172.21.15.81:53396) 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dda14700 1 -- 172.21.15.81:0/2769554811 learned_addr learned my addr 172.21.15.81:0/2769554811 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dda14700 1 -- 172.21.15.81:0/2769554811 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d81031b0 con 0x7f63d8102720 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dda14700 1 --2- 172.21.15.81:0/2769554811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8102b30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63d40098e0 tx=0x7f63d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c71db507c24fc31 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dca12700 1 -- 172.21.15.81:0/2769554811 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63d400b860 con 0x7f63d8102720 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dca12700 1 -- 172.21.15.81:0/2769554811 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63d400be90 con 0x7f63d8102720 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.714+0000 7f63dca12700 1 -- 172.21.15.81:0/2769554811 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63d40103c0 con 0x7f63d8102720 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.715+0000 7f63dfc78700 1 -- 172.21.15.81:0/2769554811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 msgr2=0x7f63d8102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.715+0000 7f63dfc78700 1 --2- 172.21.15.81:0/2769554811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8102b30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63d40098e0 tx=0x7f63d4009250 comp rx=0 tx=0).stop 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.715+0000 7f63dfc78700 1 -- 172.21.15.81:0/2769554811 shutdown_connections 2024-03-22T07:00:43.577 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.715+0000 7f63dfc78700 1 --2- 172.21.15.81:0/2769554811 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8102b30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.715+0000 7f63dfc78700 1 -- 172.21.15.81:0/2769554811 >> 172.21.15.81:0/2769554811 conn(0x7f63d80fde40 msgr2=0x7f63d81002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.715+0000 7f63dfc78700 1 -- 172.21.15.81:0/2769554811 shutdown_connections 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.715+0000 7f63dfc78700 1 -- 172.21.15.81:0/2769554811 wait complete. 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.715+0000 7f63dfc78700 1 Processor -- start 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dfc78700 1 -- start start 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dfc78700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dfc78700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8111ac0 con 0x7f63d8102720 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dda14700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dda14700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53408/0 (socket says 172.21.15.81:53408) 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dda14700 1 -- 172.21.15.81:0/2175877896 learned_addr learned my addr 172.21.15.81:0/2175877896 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dda14700 1 -- 172.21.15.81:0/2175877896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d40094f0 con 0x7f63d8102720 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dda14700 1 --2- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8111580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63d4011680 tx=0x7f63d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63ce7fc700 1 -- 172.21.15.81:0/2175877896 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63d4010890 con 0x7f63d8102720 2024-03-22T07:00:43.578 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63ce7fc700 1 -- 172.21.15.81:0/2175877896 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63d4010ec0 con 0x7f63d8102720 2024-03-22T07:00:43.579 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63ce7fc700 1 -- 172.21.15.81:0/2175877896 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63d40243f0 con 0x7f63d8102720 2024-03-22T07:00:43.579 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d8111cc0 con 0x7f63d8102720 2024-03-22T07:00:43.579 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.716+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d8112830 con 0x7f63d8102720 2024-03-22T07:00:43.579 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.717+0000 7f63ce7fc700 1 -- 172.21.15.81:0/2175877896 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f63d402d3f0 con 0x7f63d8102720 2024-03-22T07:00:43.579 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.717+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d805f5f0 con 0x7f63d8102720 2024-03-22T07:00:43.579 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.717+0000 7f63ce7fc700 1 --2- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63c40368d0 0x7f63c4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:43.579 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.717+0000 7f63ce7fc700 1 -- 172.21.15.81:0/2175877896 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f63d404a4d0 con 0x7f63d8102720 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.717+0000 7f63dd213700 1 --2- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63c40368d0 0x7f63c4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.718+0000 7f63dd213700 1 --2- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63c40368d0 0x7f63c4038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009990 tx=0x7f63c8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:42.722+0000 7f63ce7fc700 1 -- 172.21.15.81:0/2175877896 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f63d4018b00 con 0x7f63d8102720 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.011+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f63d804bed0 con 0x7f63d8102720 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.015+0000 7f63ce7fc700 1 -- 172.21.15.81:0/2175877896 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f63d4018b00 con 0x7f63d8102720 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.017+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63c40368d0 msgr2=0x7f63c4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.017+0000 7f63dfc78700 1 --2- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63c40368d0 0x7f63c4038d80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009990 tx=0x7f63c8006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.017+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 msgr2=0x7f63d8111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.017+0000 7f63dfc78700 1 --2- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8111580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63d4011680 tx=0x7f63d4009250 comp rx=0 tx=0).stop 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.017+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 shutdown_connections 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.017+0000 7f63dfc78700 1 --2- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f63c40368d0 0x7f63c4038d80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.017+0000 7f63dfc78700 1 --2- 172.21.15.81:0/2175877896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63d8102720 0x7f63d8111580 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:43.580 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.017+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 >> 172.21.15.81:0/2175877896 conn(0x7f63d80fde40 msgr2=0x7f63d80fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.018+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 shutdown_connections 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr 2024-03-22T07:00:43.018+0000 7f63dfc78700 1 -- 172.21.15.81:0/2175877896 wait complete. 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout:Or, if you are only running a single cluster on this host: 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout: ceph telemetry on 2024-03-22T07:00:43.581 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:43.582 INFO:teuthology.orchestra.run.smithi081.stdout:For more information see: 2024-03-22T07:00:43.582 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:43.582 INFO:teuthology.orchestra.run.smithi081.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-22T07:00:43.582 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:43.582 INFO:teuthology.orchestra.run.smithi081.stdout:Bootstrap complete. 2024-03-22T07:00:43.613 INFO:tasks.cephadm:Fetching config... 2024-03-22T07:00:43.613 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:00:43.613 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-22T07:00:43.670 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-22T07:00:43.670 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:00:43.670 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-22T07:00:43.725 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-22T07:00:43.725 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:00:43.725 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/keyring of=/dev/stdout 2024-03-22T07:00:43.792 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-22T07:00:43.792 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:00:43.792 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-22T07:00:43.848 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-22T07:00:43.848 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuo35KrZRPxEFbuUZrihiPb9iiptbdb46AKA+aJvSEWFAhyAfbbKMuZtWMbABmuTQHAIKR0FQeC2XjjKUJin9Ed0Ri6afJdP82iBUaNx79m4HFAVNp+8P/ICwzXo8xZg7d8HESsmn3LEmWYxadhuDo8oC0yp4fQ6GSI1EJybdNmeOoqXrBKU2zdn8VQ+l/HaXkraDFfnlBNzL4+SIApuUvIFNCTMA5UGfzm/QzF2+I7OAaMh6MKQ3mAlT2ZfeCoLlozAjoFfdYnh8LTCcjj9xen70Vdpt3Jk9dzw7/ZbQEyhGZrBpnQi/oM5KwsXGehE4BpfVteAI85OummuDqIClxOefKM/O8axxGAn0hCYYX6zrm6P5V7q0NF4eP7ySEKHiXTTcuN3n641hlTcYvCO2aKgpaW4MW8+QRkqmAjQgpeeM+TuSZ/ot0CGZWUVtFC+3TeJo7ZpueuonEEsgR+GpI7vq5LfhUKt7fJYlvJOSGomsp0a2OFX2xxIt2op15RD0= ceph-a42e8632-e819-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T07:00:43.932 INFO:teuthology.orchestra.run.smithi081.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuo35KrZRPxEFbuUZrihiPb9iiptbdb46AKA+aJvSEWFAhyAfbbKMuZtWMbABmuTQHAIKR0FQeC2XjjKUJin9Ed0Ri6afJdP82iBUaNx79m4HFAVNp+8P/ICwzXo8xZg7d8HESsmn3LEmWYxadhuDo8oC0yp4fQ6GSI1EJybdNmeOoqXrBKU2zdn8VQ+l/HaXkraDFfnlBNzL4+SIApuUvIFNCTMA5UGfzm/QzF2+I7OAaMh6MKQ3mAlT2ZfeCoLlozAjoFfdYnh8LTCcjj9xen70Vdpt3Jk9dzw7/ZbQEyhGZrBpnQi/oM5KwsXGehE4BpfVteAI85OummuDqIClxOefKM/O8axxGAn0hCYYX6zrm6P5V7q0NF4eP7ySEKHiXTTcuN3n641hlTcYvCO2aKgpaW4MW8+QRkqmAjQgpeeM+TuSZ/ot0CGZWUVtFC+3TeJo7ZpueuonEEsgR+GpI7vq5LfhUKt7fJYlvJOSGomsp0a2OFX2xxIt2op15RD0= ceph-a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T07:00:43.947 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuo35KrZRPxEFbuUZrihiPb9iiptbdb46AKA+aJvSEWFAhyAfbbKMuZtWMbABmuTQHAIKR0FQeC2XjjKUJin9Ed0Ri6afJdP82iBUaNx79m4HFAVNp+8P/ICwzXo8xZg7d8HESsmn3LEmWYxadhuDo8oC0yp4fQ6GSI1EJybdNmeOoqXrBKU2zdn8VQ+l/HaXkraDFfnlBNzL4+SIApuUvIFNCTMA5UGfzm/QzF2+I7OAaMh6MKQ3mAlT2ZfeCoLlozAjoFfdYnh8LTCcjj9xen70Vdpt3Jk9dzw7/ZbQEyhGZrBpnQi/oM5KwsXGehE4BpfVteAI85OummuDqIClxOefKM/O8axxGAn0hCYYX6zrm6P5V7q0NF4eP7ySEKHiXTTcuN3n641hlTcYvCO2aKgpaW4MW8+QRkqmAjQgpeeM+TuSZ/ot0CGZWUVtFC+3TeJo7ZpueuonEEsgR+GpI7vq5LfhUKt7fJYlvJOSGomsp0a2OFX2xxIt2op15RD0= ceph-a42e8632-e819-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T07:00:43.997 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCuo35KrZRPxEFbuUZrihiPb9iiptbdb46AKA+aJvSEWFAhyAfbbKMuZtWMbABmuTQHAIKR0FQeC2XjjKUJin9Ed0Ri6afJdP82iBUaNx79m4HFAVNp+8P/ICwzXo8xZg7d8HESsmn3LEmWYxadhuDo8oC0yp4fQ6GSI1EJybdNmeOoqXrBKU2zdn8VQ+l/HaXkraDFfnlBNzL4+SIApuUvIFNCTMA5UGfzm/QzF2+I7OAaMh6MKQ3mAlT2ZfeCoLlozAjoFfdYnh8LTCcjj9xen70Vdpt3Jk9dzw7/ZbQEyhGZrBpnQi/oM5KwsXGehE4BpfVteAI85OummuDqIClxOefKM/O8axxGAn0hCYYX6zrm6P5V7q0NF4eP7ySEKHiXTTcuN3n641hlTcYvCO2aKgpaW4MW8+QRkqmAjQgpeeM+TuSZ/ot0CGZWUVtFC+3TeJo7ZpueuonEEsgR+GpI7vq5LfhUKt7fJYlvJOSGomsp0a2OFX2xxIt2op15RD0= ceph-a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T07:00:44.013 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-22T07:00:44.272 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:00:44.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:44 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2175877896' entity='client.admin' 2024-03-22T07:00:46.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.358+0000 7fee11c83700 1 -- 172.21.15.81:0/916768614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 msgr2=0x7fee0c0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:46.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.358+0000 7fee11c83700 1 --2- 172.21.15.81:0/916768614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 0x7fee0c0fce70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fee000048c0 tx=0x7fee00009e30 comp rx=0 tx=0).stop 2024-03-22T07:00:46.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.358+0000 7fee11c83700 1 -- 172.21.15.81:0/916768614 shutdown_connections 2024-03-22T07:00:46.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.358+0000 7fee11c83700 1 --2- 172.21.15.81:0/916768614 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 0x7fee0c0fce70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:46.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.358+0000 7fee11c83700 1 -- 172.21.15.81:0/916768614 >> 172.21.15.81:0/916768614 conn(0x7fee0c0f81c0 msgr2=0x7fee0c0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:46.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.358+0000 7fee11c83700 1 -- 172.21.15.81:0/916768614 shutdown_connections 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.358+0000 7fee11c83700 1 -- 172.21.15.81:0/916768614 wait complete. 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.358+0000 7fee11c83700 1 Processor -- start 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.359+0000 7fee11c83700 1 -- start start 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.359+0000 7fee11c83700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 0x7fee0c10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.359+0000 7fee11c83700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c10d5f0 con 0x7fee0c0fca60 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.359+0000 7fee0b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 0x7fee0c10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.359+0000 7fee0b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 0x7fee0c10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40330/0 (socket says 172.21.15.81:40330) 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.359+0000 7fee0b7fe700 1 -- 172.21.15.81:0/438427533 learned_addr learned my addr 172.21.15.81:0/438427533 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.359+0000 7fee0b7fe700 1 -- 172.21.15.81:0/438427533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00005040 con 0x7fee0c0fca60 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.359+0000 7fee0b7fe700 1 --2- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 0x7fee0c10d0b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee000096b0 tx=0x7fee00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.360+0000 7fedf3fff700 1 -- 172.21.15.81:0/438427533 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee0000bed0 con 0x7fee0c0fca60 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.360+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c10d7f0 con 0x7fee0c0fca60 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.360+0000 7fedf3fff700 1 -- 172.21.15.81:0/438427533 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee0000b5c0 con 0x7fee0c0fca60 2024-03-22T07:00:46.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.360+0000 7fedf3fff700 1 -- 172.21.15.81:0/438427533 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee0001c5b0 con 0x7fee0c0fca60 2024-03-22T07:00:46.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.360+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c10e360 con 0x7fee0c0fca60 2024-03-22T07:00:46.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.360+0000 7fedf3fff700 1 -- 172.21.15.81:0/438427533 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fee0001c710 con 0x7fee0c0fca60 2024-03-22T07:00:46.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.361+0000 7fedf3fff700 1 --2- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fedf40368d0 0x7fedf4038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:46.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.361+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c102f70 con 0x7fee0c0fca60 2024-03-22T07:00:46.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.361+0000 7fedf3fff700 1 -- 172.21.15.81:0/438427533 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fee0004aa00 con 0x7fee0c0fca60 2024-03-22T07:00:46.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.361+0000 7fee0affd700 1 --2- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fedf40368d0 0x7fedf4038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:46.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.361+0000 7fee0affd700 1 --2- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fedf40368d0 0x7fedf4038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009990 tx=0x7fedfc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:46.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.366+0000 7fedf3fff700 1 -- 172.21.15.81:0/438427533 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fee00025400 con 0x7fee0c0fca60 2024-03-22T07:00:46.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.559+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fee0c05f5f0 con 0x7fee0c0fca60 2024-03-22T07:00:46.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.562+0000 7fedf3fff700 1 -- 172.21.15.81:0/438427533 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fee00011370 con 0x7fee0c0fca60 2024-03-22T07:00:46.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.564+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fedf40368d0 msgr2=0x7fedf4038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:46.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.564+0000 7fee11c83700 1 --2- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fedf40368d0 0x7fedf4038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009990 tx=0x7fedfc006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:46.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.564+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 msgr2=0x7fee0c10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:46.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.564+0000 7fee11c83700 1 --2- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 0x7fee0c10d0b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee000096b0 tx=0x7fee00009e30 comp rx=0 tx=0).stop 2024-03-22T07:00:46.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.565+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 shutdown_connections 2024-03-22T07:00:46.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.565+0000 7fee11c83700 1 --2- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fedf40368d0 0x7fedf4038d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:46.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.565+0000 7fee11c83700 1 --2- 172.21.15.81:0/438427533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee0c0fca60 0x7fee0c10d0b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:46.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.565+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 >> 172.21.15.81:0/438427533 conn(0x7fee0c0f81c0 msgr2=0x7fee0c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:46.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.565+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 shutdown_connections 2024-03-22T07:00:46.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:46.565+0000 7fee11c83700 1 -- 172.21.15.81:0/438427533 wait complete. 2024-03-22T07:00:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:46 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:46 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:46 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:00:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:46 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:46 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi081", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:00:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:46 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi081", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-22T07:00:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:46 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:00:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:46 smithi081 ceph-mon[29245]: Deploying daemon crash.smithi081 on smithi081 2024-03-22T07:00:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:46 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/438427533' entity='client.admin' 2024-03-22T07:00:47.380 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-22T07:00:47.381 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-22T07:00:48.006 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:00:49.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 -- 172.21.15.81:0/1933726058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 msgr2=0x7ff0340fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 --2- 172.21.15.81:0/1933726058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 0x7ff0340fce70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0300048c0 tx=0x7ff030009e30 comp rx=0 tx=0).stop 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 -- 172.21.15.81:0/1933726058 shutdown_connections 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 --2- 172.21.15.81:0/1933726058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 0x7ff0340fce70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 -- 172.21.15.81:0/1933726058 >> 172.21.15.81:0/1933726058 conn(0x7ff0340f81c0 msgr2=0x7ff0340fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 -- 172.21.15.81:0/1933726058 shutdown_connections 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 -- 172.21.15.81:0/1933726058 wait complete. 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 Processor -- start 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 -- start start 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 0x7ff03410d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03c629700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03410d5f0 con 0x7ff0340fca60 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03a3c5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 0x7ff03410d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03a3c5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 0x7ff03410d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40354/0 (socket says 172.21.15.81:40354) 2024-03-22T07:00:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.593+0000 7ff03a3c5700 1 -- 172.21.15.81:0/2950304920 learned_addr learned my addr 172.21.15.81:0/2950304920 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.594+0000 7ff03a3c5700 1 -- 172.21.15.81:0/2950304920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff030005040 con 0x7ff0340fca60 2024-03-22T07:00:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.594+0000 7ff03a3c5700 1 --2- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 0x7ff03410d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff030009680 tx=0x7ff030009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.594+0000 7ff02affd700 1 -- 172.21.15.81:0/2950304920 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff03000b5c0 con 0x7ff0340fca60 2024-03-22T07:00:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.594+0000 7ff02affd700 1 -- 172.21.15.81:0/2950304920 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff03000be70 con 0x7ff0340fca60 2024-03-22T07:00:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.594+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03410d7f0 con 0x7ff0340fca60 2024-03-22T07:00:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.594+0000 7ff02affd700 1 -- 172.21.15.81:0/2950304920 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0300135a0 con 0x7ff0340fca60 2024-03-22T07:00:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.594+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03410e360 con 0x7ff0340fca60 2024-03-22T07:00:49.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.595+0000 7ff02affd700 1 -- 172.21.15.81:0/2950304920 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7ff030013810 con 0x7ff0340fca60 2024-03-22T07:00:49.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.595+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff034102f70 con 0x7ff0340fca60 2024-03-22T07:00:49.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.595+0000 7ff02affd700 1 --2- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7ff0200368d0 0x7ff020038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:49.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.595+0000 7ff02affd700 1 -- 172.21.15.81:0/2950304920 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff030049330 con 0x7ff0340fca60 2024-03-22T07:00:49.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.595+0000 7ff039bc4700 1 --2- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7ff0200368d0 0x7ff020038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:49.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.596+0000 7ff039bc4700 1 --2- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7ff0200368d0 0x7ff020038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff024009990 tx=0x7ff024006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:49.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.600+0000 7ff02affd700 1 -- 172.21.15.81:0/2950304920 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff030011960 con 0x7ff0340fca60 2024-03-22T07:00:49.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.791+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 --> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ff03405f690 con 0x7ff0200368d0 2024-03-22T07:00:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.794+0000 7ff02affd700 1 -- 172.21.15.81:0/2950304920 <== mgr.14162 v2:172.21.15.81:6800/1870922013 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff03405f690 con 0x7ff0200368d0 2024-03-22T07:00:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7ff0200368d0 msgr2=0x7ff020038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 --2- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7ff0200368d0 0x7ff020038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff024009990 tx=0x7ff024006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 msgr2=0x7ff03410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 --2- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 0x7ff03410d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff030009680 tx=0x7ff030009e30 comp rx=0 tx=0).stop 2024-03-22T07:00:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 shutdown_connections 2024-03-22T07:00:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 --2- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7ff0200368d0 0x7ff020038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 --2- 172.21.15.81:0/2950304920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0340fca60 0x7ff03410d0b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 >> 172.21.15.81:0/2950304920 conn(0x7ff0340f81c0 msgr2=0x7ff034103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 shutdown_connections 2024-03-22T07:00:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:49.797+0000 7ff03c629700 1 -- 172.21.15.81:0/2950304920 wait complete. 2024-03-22T07:00:50.517 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-03-22T07:00:50.518 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:00:50.518 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-03-22T07:00:50.538 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:00:50.538 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:00:50.595 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-03-22T07:00:50.596 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch host add smithi149 2024-03-22T07:00:50.942 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:50 smithi081 ceph-mon[29245]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:50.943 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:50 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:51.040 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.542+0000 7f940ed4e700 1 -- 172.21.15.81:0/3750890952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 msgr2=0x7f94080fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.542+0000 7f940ed4e700 1 --2- 172.21.15.81:0/3750890952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 0x7f94080fce70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9400009c60 tx=0x7f9400004d10 comp rx=0 tx=0).stop 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.542+0000 7f940ed4e700 1 -- 172.21.15.81:0/3750890952 shutdown_connections 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.542+0000 7f940ed4e700 1 --2- 172.21.15.81:0/3750890952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 0x7f94080fce70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.542+0000 7f940ed4e700 1 -- 172.21.15.81:0/3750890952 >> 172.21.15.81:0/3750890952 conn(0x7f94080f81c0 msgr2=0x7f94080fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.542+0000 7f940ed4e700 1 -- 172.21.15.81:0/3750890952 shutdown_connections 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.542+0000 7f940ed4e700 1 -- 172.21.15.81:0/3750890952 wait complete. 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.542+0000 7f940ed4e700 1 Processor -- start 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.543+0000 7f940ed4e700 1 -- start start 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.543+0000 7f940ed4e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 0x7f940810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.543+0000 7f940ed4e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940810d5f0 con 0x7f94080fca60 2024-03-22T07:00:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.543+0000 7f940caea700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 0x7f940810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.543+0000 7f940caea700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 0x7f940810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40364/0 (socket says 172.21.15.81:40364) 2024-03-22T07:00:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.543+0000 7f940caea700 1 -- 172.21.15.81:0/32550831 learned_addr learned my addr 172.21.15.81:0/32550831 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.543+0000 7f940caea700 1 -- 172.21.15.81:0/32550831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9400009940 con 0x7f94080fca60 2024-03-22T07:00:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.543+0000 7f940caea700 1 --2- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 0x7f940810d0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9400004e60 tx=0x7f9400004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.543+0000 7f94057fa700 1 -- 172.21.15.81:0/32550831 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9400012410 con 0x7f94080fca60 2024-03-22T07:00:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.544+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940810d7f0 con 0x7f94080fca60 2024-03-22T07:00:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.544+0000 7f94057fa700 1 -- 172.21.15.81:0/32550831 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9400012a40 con 0x7f94080fca60 2024-03-22T07:00:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.544+0000 7f94057fa700 1 -- 172.21.15.81:0/32550831 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f940001ba40 con 0x7f94080fca60 2024-03-22T07:00:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.544+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940810e360 con 0x7f94080fca60 2024-03-22T07:00:52.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.544+0000 7f94057fa700 1 -- 172.21.15.81:0/32550831 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f9400019040 con 0x7f94080fca60 2024-03-22T07:00:52.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.545+0000 7f94057fa700 1 --2- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f93f4036880 0x7f93f4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:52.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.545+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9408102f70 con 0x7f94080fca60 2024-03-22T07:00:52.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.545+0000 7f94057fa700 1 -- 172.21.15.81:0/32550831 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9400049be0 con 0x7f94080fca60 2024-03-22T07:00:52.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.545+0000 7f9407fff700 1 --2- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f93f4036880 0x7f93f4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:52.546 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.545+0000 7f9407fff700 1 --2- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f93f4036880 0x7f93f4038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009990 tx=0x7f93f8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:52.549 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.549+0000 7f94057fa700 1 -- 172.21.15.81:0/32550831 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f940001b3f0 con 0x7f94080fca60 2024-03-22T07:00:52.741 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:52.741+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 --> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7f940810e5a0 con 0x7f93f4036880 2024-03-22T07:00:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:54 smithi081 ceph-mon[29245]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:00:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:54 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:54 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:54 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:54 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:54 smithi081 ceph-mon[29245]: Deploying daemon node-exporter.smithi081 on smithi081 2024-03-22T07:00:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:54 smithi081 ceph-mon[29245]: Deploying cephadm binary to smithi149 2024-03-22T07:00:54.920 INFO:teuthology.orchestra.run.smithi081.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-03-22T07:00:54.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.919+0000 7f94057fa700 1 -- 172.21.15.81:0/32550831 <== mgr.14162 v2:172.21.15.81:6800/1870922013 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f940810e5a0 con 0x7f93f4036880 2024-03-22T07:00:54.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f93f4036880 msgr2=0x7f93f4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 --2- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f93f4036880 0x7f93f4038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009990 tx=0x7f93f8006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 msgr2=0x7f940810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 --2- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 0x7f940810d0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9400004e60 tx=0x7f9400004d10 comp rx=0 tx=0).stop 2024-03-22T07:00:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 shutdown_connections 2024-03-22T07:00:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 --2- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f93f4036880 0x7f93f4038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 --2- 172.21.15.81:0/32550831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94080fca60 0x7f940810d0b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 >> 172.21.15.81:0/32550831 conn(0x7f94080f81c0 msgr2=0x7f9408103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 shutdown_connections 2024-03-22T07:00:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:54.921+0000 7f940ed4e700 1 -- 172.21.15.81:0/32550831 wait complete. 2024-03-22T07:00:56.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:55 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:56.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:55 smithi081 ceph-mon[29245]: Added host smithi149 2024-03-22T07:00:56.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:55 smithi081 ceph-mon[29245]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:00:56.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:55 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:56.141 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-22T07:00:56.823 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:00:58.030 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:57 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:58.030 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:57 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:58.030 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:57 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:58.031 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:57 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:00:58.031 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:57 smithi081 ceph-mon[29245]: Deploying daemon alertmanager.smithi081 on smithi081 2024-03-22T07:00:58.031 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:57 smithi081 ceph-mon[29245]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:00:58.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.330+0000 7f6f00ab6700 1 -- 172.21.15.81:0/3536033047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 msgr2=0x7f6efc0fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:58.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.330+0000 7f6f00ab6700 1 --2- 172.21.15.81:0/3536033047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 0x7f6efc0fbd70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0048c0 tx=0x7f6eec009e30 comp rx=0 tx=0).stop 2024-03-22T07:00:58.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.330+0000 7f6f00ab6700 1 -- 172.21.15.81:0/3536033047 shutdown_connections 2024-03-22T07:00:58.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.330+0000 7f6f00ab6700 1 --2- 172.21.15.81:0/3536033047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 0x7f6efc0fbd70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.330+0000 7f6f00ab6700 1 -- 172.21.15.81:0/3536033047 >> 172.21.15.81:0/3536033047 conn(0x7f6efc0f70c0 msgr2=0x7f6efc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.330+0000 7f6f00ab6700 1 -- 172.21.15.81:0/3536033047 shutdown_connections 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.331+0000 7f6f00ab6700 1 -- 172.21.15.81:0/3536033047 wait complete. 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.331+0000 7f6f00ab6700 1 Processor -- start 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.331+0000 7f6f00ab6700 1 -- start start 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.331+0000 7f6f00ab6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 0x7f6efc066840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.331+0000 7f6f00ab6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc10fdd0 con 0x7f6efc0fb960 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.331+0000 7f6efa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 0x7f6efc066840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.331+0000 7f6efa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 0x7f6efc066840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49286/0 (socket says 172.21.15.81:49286) 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.331+0000 7f6efa59c700 1 -- 172.21.15.81:0/412844197 learned_addr learned my addr 172.21.15.81:0/412844197 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.332+0000 7f6efa59c700 1 -- 172.21.15.81:0/412844197 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec005040 con 0x7f6efc0fb960 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.332+0000 7f6efa59c700 1 --2- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 0x7f6efc066840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0096b0 tx=0x7f6eec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.332+0000 7f6eeaffd700 1 -- 172.21.15.81:0/412844197 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6eec00bed0 con 0x7f6efc0fb960 2024-03-22T07:00:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.332+0000 7f6eeaffd700 1 -- 172.21.15.81:0/412844197 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eec00b5c0 con 0x7f6efc0fb960 2024-03-22T07:00:58.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.332+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc10ff10 con 0x7f6efc0fb960 2024-03-22T07:00:58.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.332+0000 7f6eeaffd700 1 -- 172.21.15.81:0/412844197 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6eec01c5b0 con 0x7f6efc0fb960 2024-03-22T07:00:58.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.332+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc110400 con 0x7f6efc0fb960 2024-03-22T07:00:58.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.333+0000 7f6eeaffd700 1 -- 172.21.15.81:0/412844197 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f6eec01c820 con 0x7f6efc0fb960 2024-03-22T07:00:58.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.333+0000 7f6eeaffd700 1 --2- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6ee4036880 0x7f6ee4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:00:58.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.333+0000 7f6eeaffd700 1 -- 172.21.15.81:0/412844197 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6eec0499f0 con 0x7f6efc0fb960 2024-03-22T07:00:58.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.333+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc105920 con 0x7f6efc0fb960 2024-03-22T07:00:58.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.333+0000 7f6ef9d9b700 1 --2- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6ee4036880 0x7f6ee4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:00:58.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.333+0000 7f6ef9d9b700 1 --2- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6ee4036880 0x7f6ee4038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009990 tx=0x7f6ef0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:00:58.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.338+0000 7f6eeaffd700 1 -- 172.21.15.81:0/412844197 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6eec023300 con 0x7f6efc0fb960 2024-03-22T07:00:58.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.537+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 --> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6efc1017b0 con 0x7f6ee4036880 2024-03-22T07:00:58.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.538+0000 7f6eeaffd700 1 -- 172.21.15.81:0/412844197 <== mgr.14162 v2:172.21.15.81:6800/1870922013 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f6efc1017b0 con 0x7f6ee4036880 2024-03-22T07:00:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:00:58.538 INFO:teuthology.orchestra.run.smithi081.stdout:[{"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-03-22T07:00:58.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6ee4036880 msgr2=0x7f6ee4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:58.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 --2- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6ee4036880 0x7f6ee4038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009990 tx=0x7f6ef0006e40 comp rx=0 tx=0).stop 2024-03-22T07:00:58.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 msgr2=0x7f6efc066840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:00:58.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 --2- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 0x7f6efc066840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0096b0 tx=0x7f6eec009e30 comp rx=0 tx=0).stop 2024-03-22T07:00:58.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 shutdown_connections 2024-03-22T07:00:58.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 --2- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6ee4036880 0x7f6ee4038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:58.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 --2- 172.21.15.81:0/412844197 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6efc0fb960 0x7f6efc066840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:00:58.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 >> 172.21.15.81:0/412844197 conn(0x7f6efc0f70c0 msgr2=0x7f6efc102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:00:58.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 shutdown_connections 2024-03-22T07:00:58.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:00:58.540+0000 7f6f00ab6700 1 -- 172.21.15.81:0/412844197 wait complete. 2024-03-22T07:00:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:58 smithi081 ceph-mon[29245]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:00:59.440 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-22T07:00:59.440 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-22T07:00:59.701 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:01:00.165 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:00:59 smithi081 ceph-mon[29245]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:01.147 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:01 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:01.829 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.827+0000 7fe1166dd700 1 -- 172.21.15.81:0/2706737335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 msgr2=0x7fe1100fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.827+0000 7fe1166dd700 1 --2- 172.21.15.81:0/2706737335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 0x7fe1100fce50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe1000048c0 tx=0x7fe100009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.827+0000 7fe1166dd700 1 -- 172.21.15.81:0/2706737335 shutdown_connections 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.827+0000 7fe1166dd700 1 --2- 172.21.15.81:0/2706737335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 0x7fe1100fce50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.827+0000 7fe1166dd700 1 -- 172.21.15.81:0/2706737335 >> 172.21.15.81:0/2706737335 conn(0x7fe1100f81c0 msgr2=0x7fe1100fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.827+0000 7fe1166dd700 1 -- 172.21.15.81:0/2706737335 shutdown_connections 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.827+0000 7fe1166dd700 1 -- 172.21.15.81:0/2706737335 wait complete. 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.827+0000 7fe1166dd700 1 Processor -- start 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.828+0000 7fe1166dd700 1 -- start start 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.828+0000 7fe1166dd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 0x7fe110071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.828+0000 7fe1166dd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100722a0 con 0x7fe1100fca40 2024-03-22T07:01:01.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.828+0000 7fe10ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 0x7fe110071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:01.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.828+0000 7fe10ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 0x7fe110071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49302/0 (socket says 172.21.15.81:49302) 2024-03-22T07:01:01.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.828+0000 7fe10ffff700 1 -- 172.21.15.81:0/1810725191 learned_addr learned my addr 172.21.15.81:0/1810725191 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:01:01.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.828+0000 7fe10ffff700 1 -- 172.21.15.81:0/1810725191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe100005040 con 0x7fe1100fca40 2024-03-22T07:01:01.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.828+0000 7fe10ffff700 1 --2- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 0x7fe110071d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1000096b0 tx=0x7fe100009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:01.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.828+0000 7fe10cff9700 1 -- 172.21.15.81:0/1810725191 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe10000bed0 con 0x7fe1100fca40 2024-03-22T07:01:01.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.829+0000 7fe10cff9700 1 -- 172.21.15.81:0/1810725191 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe10000b5c0 con 0x7fe1100fca40 2024-03-22T07:01:01.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.829+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe110070370 con 0x7fe1100fca40 2024-03-22T07:01:01.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.829+0000 7fe10cff9700 1 -- 172.21.15.81:0/1810725191 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe10001c5b0 con 0x7fe1100fca40 2024-03-22T07:01:01.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.829+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe110070860 con 0x7fe1100fca40 2024-03-22T07:01:01.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.830+0000 7fe10cff9700 1 -- 172.21.15.81:0/1810725191 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe10001c820 con 0x7fe1100fca40 2024-03-22T07:01:01.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.830+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11005f5f0 con 0x7fe1100fca40 2024-03-22T07:01:01.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.830+0000 7fe10cff9700 1 --2- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe0f8036880 0x7fe0f8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:01.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.830+0000 7fe10cff9700 1 -- 172.21.15.81:0/1810725191 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe100049a40 con 0x7fe1100fca40 2024-03-22T07:01:01.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.830+0000 7fe10f7fe700 1 --2- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe0f8036880 0x7fe0f8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:01.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.831+0000 7fe10f7fe700 1 --2- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe0f8036880 0x7fe0f8038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe104009990 tx=0x7fe104006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:01.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:01.835+0000 7fe10cff9700 1 -- 172.21.15.81:0/1810725191 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe100023300 con 0x7fe1100fca40 2024-03-22T07:01:02.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.043+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe11004bed0 con 0x7fe1100fca40 2024-03-22T07:01:02.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.132+0000 7fe10cff9700 1 -- 172.21.15.81:0/1810725191 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe100011b60 con 0x7fe1100fca40 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.134+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe0f8036880 msgr2=0x7fe0f8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.134+0000 7fe1166dd700 1 --2- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe0f8036880 0x7fe0f8038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe104009990 tx=0x7fe104006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.134+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 msgr2=0x7fe110071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.134+0000 7fe1166dd700 1 --2- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 0x7fe110071d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1000096b0 tx=0x7fe100009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.135+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 shutdown_connections 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.135+0000 7fe1166dd700 1 --2- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe0f8036880 0x7fe0f8038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.135+0000 7fe1166dd700 1 --2- 172.21.15.81:0/1810725191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1100fca40 0x7fe110071d60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.135+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 >> 172.21.15.81:0/1810725191 conn(0x7fe1100f81c0 msgr2=0x7fe110103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.135+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 shutdown_connections 2024-03-22T07:01:02.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:01:02.135+0000 7fe1166dd700 1 -- 172.21.15.81:0/1810725191 wait complete. 2024-03-22T07:01:02.136 INFO:teuthology.orchestra.run.smithi081.stderr:adjusted tunables profile to default 2024-03-22T07:01:02.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:02 smithi081 ceph-mon[29245]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:02.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:02 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1810725191' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-22T07:01:03.366 INFO:tasks.cephadm:Adding mon.smithi081 on smithi081 2024-03-22T07:01:03.367 INFO:tasks.cephadm:Adding mon.smithi149 on smithi149 2024-03-22T07:01:03.367 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch apply mon '2;smithi081:172.21.15.81=smithi081;smithi149:172.21.15.149=smithi149' 2024-03-22T07:01:03.448 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:03 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1810725191' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-22T07:01:03.448 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:03 smithi081 ceph-mon[29245]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T07:01:03.618 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:03.666 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:04.455 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:04 smithi081 ceph-mon[29245]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:05.079 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.077+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1770537283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 msgr2=0x7fe6c80fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.077+0000 7fe6ceb87700 1 --2- 172.21.15.149:0/1770537283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 0x7fe6c80fbcf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40048c0 tx=0x7fe6c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.077+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1770537283 shutdown_connections 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.077+0000 7fe6ceb87700 1 --2- 172.21.15.149:0/1770537283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 0x7fe6c80fbcf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.077+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1770537283 >> 172.21.15.149:0/1770537283 conn(0x7fe6c80f7080 msgr2=0x7fe6c80f9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.077+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1770537283 shutdown_connections 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.077+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1770537283 wait complete. 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.078+0000 7fe6ceb87700 1 Processor -- start 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.078+0000 7fe6ceb87700 1 -- start start 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.078+0000 7fe6ceb87700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 0x7fe6c8066880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.078+0000 7fe6ceb87700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c810fd30 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6cc923700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 0x7fe6c8066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:05.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6cc923700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 0x7fe6c8066880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:34862/0 (socket says 172.21.15.149:34862) 2024-03-22T07:01:05.081 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6cc923700 1 -- 172.21.15.149:0/1815598412 learned_addr learned my addr 172.21.15.149:0/1815598412 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:05.081 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6cc923700 1 -- 172.21.15.149:0/1815598412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c4005040 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.081 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6cc923700 1 --2- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 0x7fe6c8066880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009680 tx=0x7fe6c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:05.081 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6bd7fa700 1 -- 172.21.15.149:0/1815598412 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6c400b5c0 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.081 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6bd7fa700 1 -- 172.21.15.149:0/1815598412 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6c400be70 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.081 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c810fe70 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.081 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6bd7fa700 1 -- 172.21.15.149:0/1815598412 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6c40135a0 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.081 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.079+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c8110360 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.082 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.080+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c81058a0 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.082 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.080+0000 7fe6bd7fa700 1 -- 172.21.15.149:0/1815598412 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe6c4013860 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.082 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.080+0000 7fe6bd7fa700 1 --2- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe6b0036880 0x7fe6b0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:05.082 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.080+0000 7fe6bd7fa700 1 -- 172.21.15.149:0/1815598412 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe6c40494b0 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.082 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.081+0000 7fe6bffff700 1 --2- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe6b0036880 0x7fe6b0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:05.082 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.081+0000 7fe6bffff700 1 --2- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe6b0036880 0x7fe6b0038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009990 tx=0x7fe6b8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:05.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.086+0000 7fe6bd7fa700 1 -- 172.21.15.149:0/1815598412 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe6c4011960 con 0x7fe6c80fb8e0 2024-03-22T07:01:05.278 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.278+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 --> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi081:172.21.15.81=smithi081;smithi149:172.21.15.149=smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6c805e400 con 0x7fe6b0036880 2024-03-22T07:01:05.285 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.286+0000 7fe6bd7fa700 1 -- 172.21.15.149:0/1815598412 <== mgr.14162 v2:172.21.15.81:6800/1870922013 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe6c805e400 con 0x7fe6b0036880 2024-03-22T07:01:05.285 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mon update... 2024-03-22T07:01:05.287 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe6b0036880 msgr2=0x7fe6b0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:05.287 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 --2- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe6b0036880 0x7fe6b0038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009990 tx=0x7fe6b8006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:05.287 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 msgr2=0x7fe6c8066880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:05.287 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 --2- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 0x7fe6c8066880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009680 tx=0x7fe6c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:05.287 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 shutdown_connections 2024-03-22T07:01:05.287 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 --2- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe6b0036880 0x7fe6b0038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:05.287 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 --2- 172.21.15.149:0/1815598412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6c80fb8e0 0x7fe6c8066880 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:05.288 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 >> 172.21.15.149:0/1815598412 conn(0x7fe6c80f7080 msgr2=0x7fe6c8102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:05.288 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 shutdown_connections 2024-03-22T07:01:05.288 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:05.287+0000 7fe6ceb87700 1 -- 172.21.15.149:0/1815598412 wait complete. 2024-03-22T07:01:05.941 DEBUG:teuthology.orchestra.run.smithi149:mon.smithi149> sudo journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi149.service 2024-03-22T07:01:05.944 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:05.944 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:06.326 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:06.374 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:06.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:06 smithi081 ceph-mon[29245]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:06.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:06 smithi081 ceph-mon[29245]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi081:172.21.15.81=smithi081;smithi149:172.21.15.149=smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:01:06.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:06 smithi081 ceph-mon[29245]: Saving service mon spec with placement smithi081:172.21.15.81=smithi081;smithi149:172.21.15.149=smithi149;count:2 2024-03-22T07:01:06.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:06 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:07.831 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.831+0000 7fde7209e700 1 -- 172.21.15.149:0/1454555821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 msgr2=0x7fde6c0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:07.831 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.831+0000 7fde7209e700 1 --2- 172.21.15.149:0/1454555821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 0x7fde6c0fd460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fde64009c60 tx=0x7fde64004d10 comp rx=0 tx=0).stop 2024-03-22T07:01:07.831 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.831+0000 7fde7209e700 1 -- 172.21.15.149:0/1454555821 shutdown_connections 2024-03-22T07:01:07.831 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.831+0000 7fde7209e700 1 --2- 172.21.15.149:0/1454555821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 0x7fde6c0fd460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:07.831 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.831+0000 7fde7209e700 1 -- 172.21.15.149:0/1454555821 >> 172.21.15.149:0/1454555821 conn(0x7fde6c0f70d0 msgr2=0x7fde6c0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:07.832 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.831+0000 7fde7209e700 1 -- 172.21.15.149:0/1454555821 shutdown_connections 2024-03-22T07:01:07.832 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.831+0000 7fde7209e700 1 -- 172.21.15.149:0/1454555821 wait complete. 2024-03-22T07:01:07.832 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.832+0000 7fde7209e700 1 Processor -- start 2024-03-22T07:01:07.832 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.832+0000 7fde7209e700 1 -- start start 2024-03-22T07:01:07.832 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.832+0000 7fde7209e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 0x7fde6c065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:07.832 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.832+0000 7fde7209e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6c10dbd0 con 0x7fde6c0fd050 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.832+0000 7fde6b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 0x7fde6c065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.832+0000 7fde6b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 0x7fde6c065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:34882/0 (socket says 172.21.15.149:34882) 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.832+0000 7fde6b7fe700 1 -- 172.21.15.149:0/2622749389 learned_addr learned my addr 172.21.15.149:0/2622749389 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.833+0000 7fde6b7fe700 1 -- 172.21.15.149:0/2622749389 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde64009940 con 0x7fde6c0fd050 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.833+0000 7fde6b7fe700 1 --2- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 0x7fde6c065bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fde64009500 tx=0x7fde64004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.833+0000 7fde63fff700 1 -- 172.21.15.149:0/2622749389 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde64007c80 con 0x7fde6c0fd050 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.833+0000 7fde63fff700 1 -- 172.21.15.149:0/2622749389 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde6401d910 con 0x7fde6c0fd050 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.833+0000 7fde63fff700 1 -- 172.21.15.149:0/2622749389 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde640129d0 con 0x7fde6c0fd050 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.833+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6c1121e0 con 0x7fde6c0fd050 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.833+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6c112610 con 0x7fde6c0fd050 2024-03-22T07:01:07.834 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.834+0000 7fde63fff700 1 -- 172.21.15.149:0/2622749389 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fde64019040 con 0x7fde6c0fd050 2024-03-22T07:01:07.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.834+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6c105920 con 0x7fde6c0fd050 2024-03-22T07:01:07.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.834+0000 7fde63fff700 1 --2- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde58036880 0x7fde58038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:07.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.834+0000 7fde63fff700 1 -- 172.21.15.149:0/2622749389 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fde6404a3e0 con 0x7fde6c0fd050 2024-03-22T07:01:07.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.834+0000 7fde6affd700 1 --2- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde58036880 0x7fde58038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:07.835 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.835+0000 7fde6affd700 1 --2- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde58036880 0x7fde58038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde54009990 tx=0x7fde54006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:07.838 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:07.839+0000 7fde63fff700 1 -- 172.21.15.149:0/2622749389 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fde6402eb40 con 0x7fde6c0fd050 2024-03-22T07:01:08.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.096+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fde6c05f5f0 con 0x7fde6c0fd050 2024-03-22T07:01:08.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.097+0000 7fde63fff700 1 -- 172.21.15.149:0/2622749389 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fde64017020 con 0x7fde6c0fd050 2024-03-22T07:01:08.096 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:08.096 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde58036880 msgr2=0x7fde58038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 --2- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde58036880 0x7fde58038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde54009990 tx=0x7fde54006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 msgr2=0x7fde6c065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 --2- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 0x7fde6c065bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fde64009500 tx=0x7fde64004d10 comp rx=0 tx=0).stop 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 shutdown_connections 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 --2- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fde58036880 0x7fde58038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 --2- 172.21.15.149:0/2622749389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde6c0fd050 0x7fde6c065bd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 >> 172.21.15.149:0/2622749389 conn(0x7fde6c0f70d0 msgr2=0x7fde6c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 shutdown_connections 2024-03-22T07:01:08.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:08.099+0000 7fde7209e700 1 -- 172.21.15.149:0/2622749389 wait complete. 2024-03-22T07:01:08.099 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:07 smithi081 ceph-mon[29245]: Deploying daemon grafana.smithi081 on smithi081 2024-03-22T07:01:09.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:09 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/2622749389' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:09.671 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:09.672 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:09.927 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:09.971 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:10.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:10 smithi081 ceph-mon[29245]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:11.317 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.315+0000 7f6942895700 1 -- 172.21.15.149:0/909434379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 msgr2=0x7f693c0fc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:11.317 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.315+0000 7f6942895700 1 --2- 172.21.15.149:0/909434379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 0x7f693c0fc1f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f692c0048c0 tx=0x7f692c009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:11.317 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.315+0000 7f6942895700 1 -- 172.21.15.149:0/909434379 shutdown_connections 2024-03-22T07:01:11.317 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.315+0000 7f6942895700 1 --2- 172.21.15.149:0/909434379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 0x7f693c0fc1f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:11.317 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.315+0000 7f6942895700 1 -- 172.21.15.149:0/909434379 >> 172.21.15.149:0/909434379 conn(0x7f693c0f6850 msgr2=0x7f693c0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:11.317 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.315+0000 7f6942895700 1 -- 172.21.15.149:0/909434379 shutdown_connections 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.315+0000 7f6942895700 1 -- 172.21.15.149:0/909434379 wait complete. 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.315+0000 7f6942895700 1 Processor -- start 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.316+0000 7f6942895700 1 -- start start 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.316+0000 7f6942895700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 0x7f693c1136c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.316+0000 7f6942895700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693c113c00 con 0x7f693c0fbde0 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.316+0000 7f693bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 0x7f693c1136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.316+0000 7f693bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 0x7f693c1136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:34906/0 (socket says 172.21.15.149:34906) 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.316+0000 7f693bfff700 1 -- 172.21.15.149:0/2537269873 learned_addr learned my addr 172.21.15.149:0/2537269873 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.316+0000 7f693bfff700 1 -- 172.21.15.149:0/2537269873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c005040 con 0x7f693c0fbde0 2024-03-22T07:01:11.318 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.316+0000 7f693bfff700 1 --2- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 0x7f693c1136c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f692c0096b0 tx=0x7f692c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:11.319 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.316+0000 7f6938ff9700 1 -- 172.21.15.149:0/2537269873 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f692c00bed0 con 0x7f693c0fbde0 2024-03-22T07:01:11.319 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.317+0000 7f6938ff9700 1 -- 172.21.15.149:0/2537269873 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f692c00b5c0 con 0x7f693c0fbde0 2024-03-22T07:01:11.319 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.317+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693c113e00 con 0x7f693c0fbde0 2024-03-22T07:01:11.319 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.317+0000 7f6938ff9700 1 -- 172.21.15.149:0/2537269873 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f692c01c5b0 con 0x7f693c0fbde0 2024-03-22T07:01:11.319 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.317+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693c114970 con 0x7f693c0fbde0 2024-03-22T07:01:11.320 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.317+0000 7f6938ff9700 1 -- 172.21.15.149:0/2537269873 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f692c01c820 con 0x7f693c0fbde0 2024-03-22T07:01:11.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.317+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693c100d50 con 0x7f693c0fbde0 2024-03-22T07:01:11.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.318+0000 7f6938ff9700 1 --2- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6924036880 0x7f6924038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:11.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.318+0000 7f6938ff9700 1 -- 172.21.15.149:0/2537269873 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f692c049c80 con 0x7f693c0fbde0 2024-03-22T07:01:11.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.318+0000 7f693b7fe700 1 --2- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6924036880 0x7f6924038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:11.321 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.318+0000 7f693b7fe700 1 --2- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6924036880 0x7f6924038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6930009990 tx=0x7f6930006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:11.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.322+0000 7f6938ff9700 1 -- 172.21.15.149:0/2537269873 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f692c023300 con 0x7f693c0fbde0 2024-03-22T07:01:11.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:11 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:11.583 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.582+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f693c115380 con 0x7f693c0fbde0 2024-03-22T07:01:11.583 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.583+0000 7f6938ff9700 1 -- 172.21.15.149:0/2537269873 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f692c011b60 con 0x7f693c0fbde0 2024-03-22T07:01:11.584 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:11.584 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:11.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6924036880 msgr2=0x7f6924038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:11.585 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 --2- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6924036880 0x7f6924038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6930009990 tx=0x7f6930006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:11.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 msgr2=0x7f693c1136c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:11.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 --2- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 0x7f693c1136c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f692c0096b0 tx=0x7f692c009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:11.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 shutdown_connections 2024-03-22T07:01:11.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 --2- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f6924036880 0x7f6924038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:11.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 --2- 172.21.15.149:0/2537269873 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f693c0fbde0 0x7f693c1136c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:11.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 >> 172.21.15.149:0/2537269873 conn(0x7f693c0f6850 msgr2=0x7f693c10aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:11.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 shutdown_connections 2024-03-22T07:01:11.586 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:11.585+0000 7f6942895700 1 -- 172.21.15.149:0/2537269873 wait complete. 2024-03-22T07:01:11.587 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:12.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:12 smithi081 ceph-mon[29245]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:12.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:12 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/2537269873' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:13.067 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:13.067 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:13.320 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:13.367 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:14.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:14 smithi081 ceph-mon[29245]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.799+0000 7fccec5eb700 1 -- 172.21.15.149:0/57933401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 msgr2=0x7fcce40fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.799+0000 7fccec5eb700 1 --2- 172.21.15.149:0/57933401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 0x7fcce40fd450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009c60 tx=0x7fccdc004d10 comp rx=0 tx=0).stop 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.799+0000 7fccec5eb700 1 -- 172.21.15.149:0/57933401 shutdown_connections 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.799+0000 7fccec5eb700 1 --2- 172.21.15.149:0/57933401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 0x7fcce40fd450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.799+0000 7fccec5eb700 1 -- 172.21.15.149:0/57933401 >> 172.21.15.149:0/57933401 conn(0x7fcce40f70c0 msgr2=0x7fcce40f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.799+0000 7fccec5eb700 1 -- 172.21.15.149:0/57933401 shutdown_connections 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.799+0000 7fccec5eb700 1 -- 172.21.15.149:0/57933401 wait complete. 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.799+0000 7fccec5eb700 1 Processor -- start 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.799+0000 7fccec5eb700 1 -- start start 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fccec5eb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 0x7fcce410bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:14.800 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fccec5eb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce410c4c0 con 0x7fcce40fd040 2024-03-22T07:01:14.802 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fccea387700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 0x7fcce410bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:14.802 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fccea387700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 0x7fcce410bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:45170/0 (socket says 172.21.15.149:45170) 2024-03-22T07:01:14.802 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fccea387700 1 -- 172.21.15.149:0/704398612 learned_addr learned my addr 172.21.15.149:0/704398612 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:14.802 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fccea387700 1 -- 172.21.15.149:0/704398612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc009940 con 0x7fcce40fd040 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fccea387700 1 --2- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 0x7fcce410bf80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009500 tx=0x7fccdc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fcce2ffd700 1 -- 172.21.15.149:0/704398612 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccdc007c80 con 0x7fcce40fd040 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce410c6c0 con 0x7fcce40fd040 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fcce2ffd700 1 -- 172.21.15.149:0/704398612 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccdc01d910 con 0x7fcce40fd040 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fcce2ffd700 1 -- 172.21.15.149:0/704398612 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccdc0129d0 con 0x7fcce40fd040 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.800+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce410d230 con 0x7fcce40fd040 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.801+0000 7fcce2ffd700 1 -- 172.21.15.149:0/704398612 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fccdc019040 con 0x7fcce40fd040 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.801+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce4101e60 con 0x7fcce40fd040 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.801+0000 7fcce2ffd700 1 --2- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fcccc036880 0x7fcccc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.802+0000 7fcce2ffd700 1 -- 172.21.15.149:0/704398612 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fccdc04a3e0 con 0x7fcce40fd040 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.802+0000 7fcce9b86700 1 --2- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fcccc036880 0x7fcccc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:14.803 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.802+0000 7fcce9b86700 1 --2- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fcccc036880 0x7fcccc038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009990 tx=0x7fccd8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:14.807 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:14.807+0000 7fcce2ffd700 1 -- 172.21.15.149:0/704398612 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fccdc02eb40 con 0x7fcce40fd040 2024-03-22T07:01:15.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.072+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcce410db90 con 0x7fcce40fd040 2024-03-22T07:01:15.072 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.072+0000 7fcce2ffd700 1 -- 172.21.15.149:0/704398612 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fccdc017020 con 0x7fcce40fd040 2024-03-22T07:01:15.073 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:15.073 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:15.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fcccc036880 msgr2=0x7fcccc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:15.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 --2- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fcccc036880 0x7fcccc038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009990 tx=0x7fccd8006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:15.075 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 msgr2=0x7fcce410bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:15.076 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 --2- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 0x7fcce410bf80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009500 tx=0x7fccdc004d10 comp rx=0 tx=0).stop 2024-03-22T07:01:15.076 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 shutdown_connections 2024-03-22T07:01:15.076 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 --2- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fcccc036880 0x7fcccc038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:15.076 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 --2- 172.21.15.149:0/704398612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce40fd040 0x7fcce410bf80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:15.076 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 >> 172.21.15.149:0/704398612 conn(0x7fcce40f70c0 msgr2=0x7fcce4102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:15.076 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 shutdown_connections 2024-03-22T07:01:15.076 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:15.075+0000 7fccec5eb700 1 -- 172.21.15.149:0/704398612 wait complete. 2024-03-22T07:01:15.076 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:15.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:15 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/704398612' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:16.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:16 smithi081 ceph-mon[29245]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:16.681 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:16.682 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:16.927 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:16.974 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:18.454 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.452+0000 7f3dcedd2700 1 -- 172.21.15.149:0/1002013233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 msgr2=0x7f3dc80fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:18.454 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.452+0000 7f3dcedd2700 1 --2- 172.21.15.149:0/1002013233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 0x7f3dc80fce70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0048c0 tx=0x7f3dbc009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:18.454 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.452+0000 7f3dcedd2700 1 -- 172.21.15.149:0/1002013233 shutdown_connections 2024-03-22T07:01:18.454 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.452+0000 7f3dcedd2700 1 --2- 172.21.15.149:0/1002013233 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 0x7f3dc80fce70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:18.454 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.452+0000 7f3dcedd2700 1 -- 172.21.15.149:0/1002013233 >> 172.21.15.149:0/1002013233 conn(0x7f3dc80f81c0 msgr2=0x7f3dc80fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:18.454 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.452+0000 7f3dcedd2700 1 -- 172.21.15.149:0/1002013233 shutdown_connections 2024-03-22T07:01:18.454 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.452+0000 7f3dcedd2700 1 -- 172.21.15.149:0/1002013233 wait complete. 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.452+0000 7f3dcedd2700 1 Processor -- start 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.452+0000 7f3dcedd2700 1 -- start start 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.453+0000 7f3dcedd2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 0x7f3dc810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.453+0000 7f3dcedd2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc810d5d0 con 0x7f3dc80fca60 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.453+0000 7f3dccb6e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 0x7f3dc810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.453+0000 7f3dccb6e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 0x7f3dc810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:45188/0 (socket says 172.21.15.149:45188) 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.453+0000 7f3dccb6e700 1 -- 172.21.15.149:0/2595445913 learned_addr learned my addr 172.21.15.149:0/2595445913 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.453+0000 7f3dccb6e700 1 -- 172.21.15.149:0/2595445913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc005040 con 0x7f3dc80fca60 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.454+0000 7f3dccb6e700 1 --2- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 0x7f3dc810d090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0096b0 tx=0x7f3dbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.454+0000 7f3dc57fa700 1 -- 172.21.15.149:0/2595445913 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dbc00bed0 con 0x7f3dc80fca60 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.454+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc810d7d0 con 0x7f3dc80fca60 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.454+0000 7f3dc57fa700 1 -- 172.21.15.149:0/2595445913 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3dbc00b5c0 con 0x7f3dc80fca60 2024-03-22T07:01:18.455 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.454+0000 7f3dc57fa700 1 -- 172.21.15.149:0/2595445913 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3dbc01c5b0 con 0x7f3dc80fca60 2024-03-22T07:01:18.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.454+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc810e340 con 0x7f3dc80fca60 2024-03-22T07:01:18.456 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.455+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc8102f70 con 0x7f3dc80fca60 2024-03-22T07:01:18.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.455+0000 7f3dc57fa700 1 -- 172.21.15.149:0/2595445913 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f3dbc013420 con 0x7f3dc80fca60 2024-03-22T07:01:18.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.455+0000 7f3dc57fa700 1 --2- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f3db8036880 0x7f3db8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:18.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.455+0000 7f3dc57fa700 1 -- 172.21.15.149:0/2595445913 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3dbc049a90 con 0x7f3dc80fca60 2024-03-22T07:01:18.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.455+0000 7f3dc7fff700 1 --2- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f3db8036880 0x7f3db8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:18.457 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.455+0000 7f3dc7fff700 1 --2- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f3db8036880 0x7f3db8038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009990 tx=0x7f3db4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:18.460 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.459+0000 7f3dc57fa700 1 -- 172.21.15.149:0/2595445913 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3dbc013a80 con 0x7f3dc80fca60 2024-03-22T07:01:18.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:18 smithi081 ceph-mon[29245]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:18.724 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.723+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3dc810ed50 con 0x7f3dc80fca60 2024-03-22T07:01:18.724 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.724+0000 7f3dc57fa700 1 -- 172.21.15.149:0/2595445913 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3dbc011370 con 0x7f3dc80fca60 2024-03-22T07:01:18.724 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:18.725 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f3db8036880 msgr2=0x7f3db8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 --2- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f3db8036880 0x7f3db8038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3db4009990 tx=0x7f3db4006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 msgr2=0x7f3dc810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 --2- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 0x7f3dc810d090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0096b0 tx=0x7f3dbc009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 shutdown_connections 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 --2- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f3db8036880 0x7f3db8038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 --2- 172.21.15.149:0/2595445913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3dc80fca60 0x7f3dc810d090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 >> 172.21.15.149:0/2595445913 conn(0x7f3dc80f81c0 msgr2=0x7f3dc8103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 shutdown_connections 2024-03-22T07:01:18.727 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:18.726+0000 7f3dcedd2700 1 -- 172.21.15.149:0/2595445913 wait complete. 2024-03-22T07:01:18.728 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:19.411 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:19 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/2595445913' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:20.224 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:20.225 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:20.275 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:20 smithi081 ceph-mon[29245]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:20.475 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:20.522 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:21.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.882+0000 7f24368c2700 1 -- 172.21.15.149:0/3306323682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 msgr2=0x7f24300fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:21.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.882+0000 7f24368c2700 1 --2- 172.21.15.149:0/3306323682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 0x7f24300fe4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f24200048c0 tx=0x7f2420009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:21.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.882+0000 7f24368c2700 1 -- 172.21.15.149:0/3306323682 shutdown_connections 2024-03-22T07:01:21.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.882+0000 7f24368c2700 1 --2- 172.21.15.149:0/3306323682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 0x7f24300fe4b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:21.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.882+0000 7f24368c2700 1 -- 172.21.15.149:0/3306323682 >> 172.21.15.149:0/3306323682 conn(0x7f24300f6820 msgr2=0x7f24300f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:21.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.882+0000 7f24368c2700 1 -- 172.21.15.149:0/3306323682 shutdown_connections 2024-03-22T07:01:21.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.882+0000 7f24368c2700 1 -- 172.21.15.149:0/3306323682 wait complete. 2024-03-22T07:01:21.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.883+0000 7f24368c2700 1 Processor -- start 2024-03-22T07:01:21.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.883+0000 7f24368c2700 1 -- start start 2024-03-22T07:01:21.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.883+0000 7f24368c2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 0x7f243010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:21.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.883+0000 7f24368c2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243010d5f0 con 0x7f24300fe0a0 2024-03-22T07:01:21.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.883+0000 7f242ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 0x7f243010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:21.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.883+0000 7f242ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 0x7f243010d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:41672/0 (socket says 172.21.15.149:41672) 2024-03-22T07:01:21.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.883+0000 7f242ffff700 1 -- 172.21.15.149:0/945331050 learned_addr learned my addr 172.21.15.149:0/945331050 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:21.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.884+0000 7f242ffff700 1 -- 172.21.15.149:0/945331050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2420005040 con 0x7f24300fe0a0 2024-03-22T07:01:21.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.884+0000 7f242ffff700 1 --2- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 0x7f243010d0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24200096b0 tx=0x7f2420009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:21.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.884+0000 7f242cff9700 1 -- 172.21.15.149:0/945331050 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f242000bed0 con 0x7f24300fe0a0 2024-03-22T07:01:21.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.884+0000 7f242cff9700 1 -- 172.21.15.149:0/945331050 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f242000b5c0 con 0x7f24300fe0a0 2024-03-22T07:01:21.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.884+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243010d7f0 con 0x7f24300fe0a0 2024-03-22T07:01:21.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.884+0000 7f242cff9700 1 -- 172.21.15.149:0/945331050 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f242001c5b0 con 0x7f24300fe0a0 2024-03-22T07:01:21.886 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.884+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243010e360 con 0x7f24300fe0a0 2024-03-22T07:01:21.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.885+0000 7f242cff9700 1 -- 172.21.15.149:0/945331050 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f242001c710 con 0x7f24300fe0a0 2024-03-22T07:01:21.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.885+0000 7f242cff9700 1 --2- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f24180368d0 0x7f2418038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:21.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.885+0000 7f242cff9700 1 -- 172.21.15.149:0/945331050 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2420049c00 con 0x7f24300fe0a0 2024-03-22T07:01:21.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.885+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2430102f70 con 0x7f24300fe0a0 2024-03-22T07:01:21.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.886+0000 7f242f7fe700 1 --2- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f24180368d0 0x7f2418038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:21.887 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.886+0000 7f242f7fe700 1 --2- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f24180368d0 0x7f2418038d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2424009990 tx=0x7f2424006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:21.904 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:21.904+0000 7f242cff9700 1 -- 172.21.15.149:0/945331050 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2420025400 con 0x7f24300fe0a0 2024-03-22T07:01:22.164 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.163+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f243005f5f0 con 0x7f24300fe0a0 2024-03-22T07:01:22.164 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.164+0000 7f242cff9700 1 -- 172.21.15.149:0/945331050 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2420011370 con 0x7f24300fe0a0 2024-03-22T07:01:22.164 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:22.164 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:22.166 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f24180368d0 msgr2=0x7f2418038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:22.166 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 --2- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f24180368d0 0x7f2418038d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2424009990 tx=0x7f2424006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:22.166 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 msgr2=0x7f243010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:22.166 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 --2- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 0x7f243010d0b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24200096b0 tx=0x7f2420009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:22.166 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 shutdown_connections 2024-03-22T07:01:22.166 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 --2- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f24180368d0 0x7f2418038d80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:22.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 --2- 172.21.15.149:0/945331050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24300fe0a0 0x7f243010d0b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:22.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 >> 172.21.15.149:0/945331050 conn(0x7f24300f6820 msgr2=0x7f2430103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:22.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 shutdown_connections 2024-03-22T07:01:22.167 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:22.166+0000 7f24368c2700 1 -- 172.21.15.149:0/945331050 wait complete. 2024-03-22T07:01:22.167 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:22.267 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:22 smithi081 ceph-mon[29245]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:22.267 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:22 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:22.267 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:22 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:22.268 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:22 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:22.268 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:22 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:22.268 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:22 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:22.268 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:22 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:23.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:23 smithi081 ceph-mon[29245]: Deploying daemon prometheus.smithi081 on smithi081 2024-03-22T07:01:23.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:23 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/945331050' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:23.829 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:23.830 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:24.080 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:24.127 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:24.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:24 smithi081 ceph-mon[29245]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.542+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2138764513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 msgr2=0x7fe8a40fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.542+0000 7fe8a9e2d700 1 --2- 172.21.15.149:0/2138764513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 0x7fe8a40fce70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8940048c0 tx=0x7fe894009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.542+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2138764513 shutdown_connections 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.542+0000 7fe8a9e2d700 1 --2- 172.21.15.149:0/2138764513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 0x7fe8a40fce70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.542+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2138764513 >> 172.21.15.149:0/2138764513 conn(0x7fe8a40f81c0 msgr2=0x7fe8a40fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.542+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2138764513 shutdown_connections 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.542+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2138764513 wait complete. 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.543+0000 7fe8a9e2d700 1 Processor -- start 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.543+0000 7fe8a9e2d700 1 -- start start 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.543+0000 7fe8a9e2d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 0x7fe8a410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.543+0000 7fe8a9e2d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a410d5d0 con 0x7fe8a40fca60 2024-03-22T07:01:25.544 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.543+0000 7fe8a37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 0x7fe8a410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:25.545 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.543+0000 7fe8a37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 0x7fe8a410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:41694/0 (socket says 172.21.15.149:41694) 2024-03-22T07:01:25.545 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.543+0000 7fe8a37fe700 1 -- 172.21.15.149:0/2589336733 learned_addr learned my addr 172.21.15.149:0/2589336733 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:25.545 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.544+0000 7fe8a37fe700 1 -- 172.21.15.149:0/2589336733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe894005040 con 0x7fe8a40fca60 2024-03-22T07:01:25.545 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.544+0000 7fe8a37fe700 1 --2- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 0x7fe8a410d090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe8940096b0 tx=0x7fe894009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:25.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.544+0000 7fe88bfff700 1 -- 172.21.15.149:0/2589336733 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe89400bed0 con 0x7fe8a40fca60 2024-03-22T07:01:25.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.544+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a410d7d0 con 0x7fe8a40fca60 2024-03-22T07:01:25.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.544+0000 7fe88bfff700 1 -- 172.21.15.149:0/2589336733 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe89400b5c0 con 0x7fe8a40fca60 2024-03-22T07:01:25.546 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.544+0000 7fe88bfff700 1 -- 172.21.15.149:0/2589336733 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe89401c5b0 con 0x7fe8a40fca60 2024-03-22T07:01:25.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.544+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a410e340 con 0x7fe8a40fca60 2024-03-22T07:01:25.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.545+0000 7fe88bfff700 1 -- 172.21.15.149:0/2589336733 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe89401c820 con 0x7fe8a40fca60 2024-03-22T07:01:25.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.545+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a4102f70 con 0x7fe8a40fca60 2024-03-22T07:01:25.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.545+0000 7fe88bfff700 1 --2- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe88c036880 0x7fe88c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:25.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.545+0000 7fe88bfff700 1 -- 172.21.15.149:0/2589336733 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe894049a70 con 0x7fe8a40fca60 2024-03-22T07:01:25.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.545+0000 7fe8a2ffd700 1 --2- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe88c036880 0x7fe88c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:25.547 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.546+0000 7fe8a2ffd700 1 --2- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe88c036880 0x7fe88c038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe898009990 tx=0x7fe898006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:25.550 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.550+0000 7fe88bfff700 1 -- 172.21.15.149:0/2589336733 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe894023300 con 0x7fe8a40fca60 2024-03-22T07:01:25.814 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.813+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe8a410ed50 con 0x7fe8a40fca60 2024-03-22T07:01:25.815 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.814+0000 7fe88bfff700 1 -- 172.21.15.149:0/2589336733 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe894049360 con 0x7fe8a40fca60 2024-03-22T07:01:25.815 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:25.815 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:25.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe88c036880 msgr2=0x7fe88c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:25.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 --2- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe88c036880 0x7fe88c038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe898009990 tx=0x7fe898006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:25.817 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 msgr2=0x7fe8a410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:25.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 --2- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 0x7fe8a410d090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe8940096b0 tx=0x7fe894009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:25.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 shutdown_connections 2024-03-22T07:01:25.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 --2- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7fe88c036880 0x7fe88c038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:25.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 --2- 172.21.15.149:0/2589336733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8a40fca60 0x7fe8a410d090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:25.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 >> 172.21.15.149:0/2589336733 conn(0x7fe8a40f81c0 msgr2=0x7fe8a4103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:25.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 shutdown_connections 2024-03-22T07:01:25.818 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:25.816+0000 7fe8a9e2d700 1 -- 172.21.15.149:0/2589336733 wait complete. 2024-03-22T07:01:25.818 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:26.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:26 smithi081 ceph-mon[29245]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:26 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:26 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/2589336733' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:27.242 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:27.242 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:27.495 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:27.542 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:28.386 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:28 smithi081 ceph-mon[29245]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:28.989 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.987+0000 7f8b0453e700 1 -- 172.21.15.149:0/407632060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 msgr2=0x7f8afc0fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:28.989 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.987+0000 7f8b0453e700 1 --2- 172.21.15.149:0/407632060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 0x7f8afc0fd460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8af40048c0 tx=0x7f8af4009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:28.989 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.987+0000 7f8b0453e700 1 -- 172.21.15.149:0/407632060 shutdown_connections 2024-03-22T07:01:28.989 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.987+0000 7f8b0453e700 1 --2- 172.21.15.149:0/407632060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 0x7f8afc0fd460 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:28.989 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.987+0000 7f8b0453e700 1 -- 172.21.15.149:0/407632060 >> 172.21.15.149:0/407632060 conn(0x7f8afc0f70d0 msgr2=0x7f8afc0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.987+0000 7f8b0453e700 1 -- 172.21.15.149:0/407632060 shutdown_connections 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.987+0000 7f8b0453e700 1 -- 172.21.15.149:0/407632060 wait complete. 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.987+0000 7f8b0453e700 1 Processor -- start 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.988+0000 7f8b0453e700 1 -- start start 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.988+0000 7f8b0453e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 0x7f8afc065bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.988+0000 7f8b0453e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc10fde0 con 0x7f8afc0fd050 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.988+0000 7f8b022da700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 0x7f8afc065bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.988+0000 7f8b022da700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 0x7f8afc065bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:41708/0 (socket says 172.21.15.149:41708) 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.988+0000 7f8b022da700 1 -- 172.21.15.149:0/341791863 learned_addr learned my addr 172.21.15.149:0/341791863 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.988+0000 7f8b022da700 1 -- 172.21.15.149:0/341791863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4005040 con 0x7f8afc0fd050 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.989+0000 7f8b022da700 1 --2- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 0x7f8afc065bd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8af40096b0 tx=0x7f8af4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.989+0000 7f8af2ffd700 1 -- 172.21.15.149:0/341791863 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8af400bed0 con 0x7f8afc0fd050 2024-03-22T07:01:28.990 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.989+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc10ff20 con 0x7f8afc0fd050 2024-03-22T07:01:28.991 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.989+0000 7f8af2ffd700 1 -- 172.21.15.149:0/341791863 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8af400b5c0 con 0x7f8afc0fd050 2024-03-22T07:01:28.991 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.989+0000 7f8af2ffd700 1 -- 172.21.15.149:0/341791863 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8af401c5b0 con 0x7f8afc0fd050 2024-03-22T07:01:28.991 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.989+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc110410 con 0x7f8afc0fd050 2024-03-22T07:01:28.992 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.990+0000 7f8af2ffd700 1 -- 172.21.15.149:0/341791863 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f8af4013420 con 0x7f8afc0fd050 2024-03-22T07:01:28.992 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.990+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc105920 con 0x7f8afc0fd050 2024-03-22T07:01:28.992 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.990+0000 7f8af2ffd700 1 --2- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f8ae8036880 0x7f8ae8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:28.992 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.990+0000 7f8af2ffd700 1 -- 172.21.15.149:0/341791863 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8af4049a10 con 0x7f8afc0fd050 2024-03-22T07:01:28.992 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.990+0000 7f8b01ad9700 1 --2- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f8ae8036880 0x7f8ae8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:28.992 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.991+0000 7f8b01ad9700 1 --2- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f8ae8036880 0x7f8ae8038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009990 tx=0x7f8aec006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:28.995 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:28.995+0000 7f8af2ffd700 1 -- 172.21.15.149:0/341791863 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8af4013a80 con 0x7f8afc0fd050 2024-03-22T07:01:29.255 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.254+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8afc05f5f0 con 0x7f8afc0fd050 2024-03-22T07:01:29.255 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.255+0000 7f8af2ffd700 1 -- 172.21.15.149:0/341791863 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f8af4011370 con 0x7f8afc0fd050 2024-03-22T07:01:29.255 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:29.255 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:29.257 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f8ae8036880 msgr2=0x7f8ae8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 --2- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f8ae8036880 0x7f8ae8038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009990 tx=0x7f8aec006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 msgr2=0x7f8afc065bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 --2- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 0x7f8afc065bd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8af40096b0 tx=0x7f8af4009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 shutdown_connections 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 --2- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f8ae8036880 0x7f8ae8038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 --2- 172.21.15.149:0/341791863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8afc0fd050 0x7f8afc065bd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 >> 172.21.15.149:0/341791863 conn(0x7f8afc0f70d0 msgr2=0x7f8afc102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 shutdown_connections 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:29.257+0000 7f8b0453e700 1 -- 172.21.15.149:0/341791863 wait complete. 2024-03-22T07:01:29.258 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:30 smithi081 ceph-mon[29245]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:30 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/341791863' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:30.886 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:30.887 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:31.139 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:31.188 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:32.551 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.547+0000 7f82e4b7f700 1 -- 172.21.15.149:0/934341368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 msgr2=0x7f82e00fe4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:32.551 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.547+0000 7f82e4b7f700 1 --2- 172.21.15.149:0/934341368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 0x7f82e00fe4b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f82d00048c0 tx=0x7f82d0009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.547+0000 7f82e4b7f700 1 -- 172.21.15.149:0/934341368 shutdown_connections 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.547+0000 7f82e4b7f700 1 --2- 172.21.15.149:0/934341368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 0x7f82e00fe4b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.547+0000 7f82e4b7f700 1 -- 172.21.15.149:0/934341368 >> 172.21.15.149:0/934341368 conn(0x7f82e00f6840 msgr2=0x7f82e00f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.547+0000 7f82e4b7f700 1 -- 172.21.15.149:0/934341368 shutdown_connections 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.547+0000 7f82e4b7f700 1 -- 172.21.15.149:0/934341368 wait complete. 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.548+0000 7f82e4b7f700 1 Processor -- start 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.548+0000 7f82e4b7f700 1 -- start start 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.548+0000 7f82e4b7f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 0x7f82e010d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.548+0000 7f82e4b7f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e010d620 con 0x7f82e00fe0a0 2024-03-22T07:01:32.552 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.548+0000 7f82de59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 0x7f82e010d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:32.553 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.548+0000 7f82de59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 0x7f82e010d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:38062/0 (socket says 172.21.15.149:38062) 2024-03-22T07:01:32.553 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.548+0000 7f82de59c700 1 -- 172.21.15.149:0/1829086743 learned_addr learned my addr 172.21.15.149:0/1829086743 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:32.553 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.549+0000 7f82de59c700 1 -- 172.21.15.149:0/1829086743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d0005040 con 0x7f82e00fe0a0 2024-03-22T07:01:32.553 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.549+0000 7f82de59c700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 0x7f82e010d0e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f82d00096b0 tx=0x7f82d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:32.553 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.549+0000 7f82ceffd700 1 -- 172.21.15.149:0/1829086743 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82d000bed0 con 0x7f82e00fe0a0 2024-03-22T07:01:32.553 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.549+0000 7f82e4b7f700 1 -- 172.21.15.149:0/1829086743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e010d820 con 0x7f82e00fe0a0 2024-03-22T07:01:32.554 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.549+0000 7f82ceffd700 1 -- 172.21.15.149:0/1829086743 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82d000b5c0 con 0x7f82e00fe0a0 2024-03-22T07:01:32.554 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.549+0000 7f82ceffd700 1 -- 172.21.15.149:0/1829086743 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82d001c5b0 con 0x7f82e00fe0a0 2024-03-22T07:01:32.554 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.549+0000 7f82e4b7f700 1 -- 172.21.15.149:0/1829086743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e010e390 con 0x7f82e00fe0a0 2024-03-22T07:01:32.554 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.550+0000 7f82e4b7f700 1 -- 172.21.15.149:0/1829086743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e0102f70 con 0x7f82e00fe0a0 2024-03-22T07:01:32.554 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.550+0000 7f82ceffd700 1 -- 172.21.15.149:0/1829086743 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f82d0013a80 con 0x7f82e00fe0a0 2024-03-22T07:01:32.554 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.550+0000 7f82ceffd700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 0x7f82c8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:32.555 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.550+0000 7f82ceffd700 1 -- 172.21.15.149:0/1829086743 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f82d00499d0 con 0x7f82e00fe0a0 2024-03-22T07:01:32.555 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.550+0000 7f82ddd9b700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 0x7f82c8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:32.555 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.551+0000 7f82ddd9b700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 0x7f82c8038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009990 tx=0x7f82d4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:32.555 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.555+0000 7f82ceffd700 1 -- 172.21.15.149:0/1829086743 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f82d0011550 con 0x7f82e00fe0a0 2024-03-22T07:01:32.632 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.631+0000 7f82ceffd700 1 -- 172.21.15.149:0/1829086743 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f82d0025400 con 0x7f82e00fe0a0 2024-03-22T07:01:32.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.655+0000 7f82ddd9b700 1 -- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 msgr2=0x7f82c8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-22T07:01:32.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.655+0000 7f82ddd9b700 1 -- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 msgr2=0x7f82c8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:01:32.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.655+0000 7f82ddd9b700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 0x7f82c8038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009990 tx=0x7f82d4006e40 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:01:32.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.655+0000 7f82ddd9b700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 0x7f82c8038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009990 tx=0x7f82d4006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:32.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.658+0000 7f82ceffd700 1 -- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 msgr2=0x7f82c8038d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-22T07:01:32.659 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.658+0000 7f82ceffd700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 0x7f82c8038d30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:32.827 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.826+0000 7f82e4b7f700 1 -- 172.21.15.149:0/1829086743 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f82c4005160 con 0x7f82e00fe0a0 2024-03-22T07:01:32.827 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.827+0000 7f82ceffd700 1 -- 172.21.15.149:0/1829086743 <== mon.0 v2:172.21.15.81:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f82d0011940 con 0x7f82e00fe0a0 2024-03-22T07:01:32.828 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:32.828 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:32.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.829+0000 7f82e4b7f700 1 -- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 msgr2=0x7f82e010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:32.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.829+0000 7f82e4b7f700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 0x7f82e010d0e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f82d00096b0 tx=0x7f82d0009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:32.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.829+0000 7f82e4b7f700 1 -- 172.21.15.149:0/1829086743 shutdown_connections 2024-03-22T07:01:32.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.829+0000 7f82e4b7f700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f82c8036880 0x7f82c8038d30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:32.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.829+0000 7f82e4b7f700 1 --2- 172.21.15.149:0/1829086743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82e00fe0a0 0x7f82e010d0e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:32.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.829+0000 7f82e4b7f700 1 -- 172.21.15.149:0/1829086743 >> 172.21.15.149:0/1829086743 conn(0x7f82e00f6840 msgr2=0x7f82e00f74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:32.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.829+0000 7f82e4b7f700 1 -- 172.21.15.149:0/1829086743 shutdown_connections 2024-03-22T07:01:32.830 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:32.829+0000 7f82e4b7f700 1 -- 172.21.15.149:0/1829086743 wait complete. 2024-03-22T07:01:32.831 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:33.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:32 smithi081 ceph-mon[29245]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:01:33.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:32 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:33.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:32 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:33.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:32 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:33.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:32 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-22T07:01:33.928 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:33 smithi081 ceph-mon[29245]: from='mgr.14162 172.21.15.81:0/678946040' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-22T07:01:33.929 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:33 smithi081 ceph-mon[29245]: mgrmap e14: smithi081.bcxarw(active, since 57s) 2024-03-22T07:01:33.929 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:33 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/1829086743' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:34.381 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:34.381 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:34.631 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:34.678 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.110+0000 7f9e66674700 1 -- 172.21.15.149:0/4247858006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 msgr2=0x7f9e600fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.110+0000 7f9e66674700 1 --2- 172.21.15.149:0/4247858006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 0x7f9e600fce50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9e540048c0 tx=0x7f9e54009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.110+0000 7f9e66674700 1 -- 172.21.15.149:0/4247858006 shutdown_connections 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.110+0000 7f9e66674700 1 --2- 172.21.15.149:0/4247858006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 0x7f9e600fce50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.110+0000 7f9e66674700 1 -- 172.21.15.149:0/4247858006 >> 172.21.15.149:0/4247858006 conn(0x7f9e600f81c0 msgr2=0x7f9e600fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.110+0000 7f9e66674700 1 -- 172.21.15.149:0/4247858006 shutdown_connections 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.110+0000 7f9e66674700 1 -- 172.21.15.149:0/4247858006 wait complete. 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.111+0000 7f9e66674700 1 Processor -- start 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.111+0000 7f9e66674700 1 -- start start 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.111+0000 7f9e66674700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 0x7f9e6010d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:36.112 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.111+0000 7f9e66674700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6010d5d0 con 0x7f9e600fca40 2024-03-22T07:01:36.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.111+0000 7f9e5ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 0x7f9e6010d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:36.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.111+0000 7f9e5ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 0x7f9e6010d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:38090/0 (socket says 172.21.15.149:38090) 2024-03-22T07:01:36.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.111+0000 7f9e5ffff700 1 -- 172.21.15.149:0/2566495337 learned_addr learned my addr 172.21.15.149:0/2566495337 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:36.113 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.112+0000 7f9e5ffff700 1 -- 172.21.15.149:0/2566495337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e54005040 con 0x7f9e600fca40 2024-03-22T07:01:36.114 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.112+0000 7f9e5ffff700 1 --2- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 0x7f9e6010d090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e540096b0 tx=0x7f9e54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:36.114 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.112+0000 7f9e5cff9700 1 -- 172.21.15.149:0/2566495337 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e5400bed0 con 0x7f9e600fca40 2024-03-22T07:01:36.114 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.112+0000 7f9e5cff9700 1 -- 172.21.15.149:0/2566495337 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e5400b5c0 con 0x7f9e600fca40 2024-03-22T07:01:36.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.112+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6010d7d0 con 0x7f9e600fca40 2024-03-22T07:01:36.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.112+0000 7f9e5cff9700 1 -- 172.21.15.149:0/2566495337 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e5401c5b0 con 0x7f9e600fca40 2024-03-22T07:01:36.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.112+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6010e340 con 0x7f9e600fca40 2024-03-22T07:01:36.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.113+0000 7f9e5cff9700 1 -- 172.21.15.149:0/2566495337 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f9e54013420 con 0x7f9e600fca40 2024-03-22T07:01:36.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.113+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e60102f50 con 0x7f9e600fca40 2024-03-22T07:01:36.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.113+0000 7f9e5cff9700 1 --2- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f9e48036920 0x7f9e48038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:36.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.113+0000 7f9e5cff9700 1 -- 172.21.15.149:0/2566495337 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9e54049ae0 con 0x7f9e600fca40 2024-03-22T07:01:36.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.113+0000 7f9e5f7fe700 1 -- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f9e48036920 msgr2=0x7f9e48038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.81:6800/1870922013 2024-03-22T07:01:36.115 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.113+0000 7f9e5f7fe700 1 --2- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f9e48036920 0x7f9e48038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:01:36.118 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.118+0000 7f9e5cff9700 1 -- 172.21.15.149:0/2566495337 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9e54013a80 con 0x7f9e600fca40 2024-03-22T07:01:36.314 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.314+0000 7f9e5f7fe700 1 -- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f9e48036920 msgr2=0x7f9e48038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.81:6800/1870922013 2024-03-22T07:01:36.314 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.314+0000 7f9e5f7fe700 1 --2- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f9e48036920 0x7f9e48038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T07:01:36.377 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.377+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9e6010ed50 con 0x7f9e600fca40 2024-03-22T07:01:36.378 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.378+0000 7f9e5cff9700 1 -- 172.21.15.149:0/2566495337 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9e54011370 con 0x7f9e600fca40 2024-03-22T07:01:36.378 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:36.378 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:36.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f9e48036920 msgr2=0x7f9e48038dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:01:36.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 --2- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f9e48036920 0x7f9e48038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:36.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 msgr2=0x7f9e6010d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:36.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 --2- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 0x7f9e6010d090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9e540096b0 tx=0x7f9e54009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:36.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 shutdown_connections 2024-03-22T07:01:36.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 --2- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:6800/1870922013,v1:172.21.15.81:6801/1870922013] conn(0x7f9e48036920 0x7f9e48038dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:36.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 --2- 172.21.15.149:0/2566495337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e600fca40 0x7f9e6010d090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:36.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 >> 172.21.15.149:0/2566495337 conn(0x7f9e600f81c0 msgr2=0x7f9e60103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:36.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 shutdown_connections 2024-03-22T07:01:36.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:36.380+0000 7f9e66674700 1 -- 172.21.15.149:0/2566495337 wait complete. 2024-03-22T07:01:36.381 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:36.725 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:36 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/2566495337' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:38.011 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:38.012 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:38.264 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:38.312 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:39.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: Active manager daemon smithi081.bcxarw restarted 2024-03-22T07:01:39.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: Activating manager daemon smithi081.bcxarw 2024-03-22T07:01:39.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T07:01:39.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: mgrmap e15: smithi081.bcxarw(active, starting, since 0.0031908s) 2024-03-22T07:01:39.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:01:39.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:01:39.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:01:39.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:01:39.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:01:39.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: Manager daemon smithi081.bcxarw is now available 2024-03-22T07:01:39.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:01:39.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:01:39.853 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.853+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/1792267001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 msgr2=0x7f5c780fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:39.853 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.853+0000 7f5c7d3a0700 1 --2- 172.21.15.149:0/1792267001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 0x7f5c780fbd70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0048c0 tx=0x7f5c6c009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:39.854 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.853+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/1792267001 shutdown_connections 2024-03-22T07:01:39.854 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.853+0000 7f5c7d3a0700 1 --2- 172.21.15.149:0/1792267001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 0x7f5c780fbd70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:39.854 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.853+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/1792267001 >> 172.21.15.149:0/1792267001 conn(0x7f5c780f70c0 msgr2=0x7f5c780f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:39.854 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.853+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/1792267001 shutdown_connections 2024-03-22T07:01:39.854 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.853+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/1792267001 wait complete. 2024-03-22T07:01:39.855 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.853+0000 7f5c7d3a0700 1 Processor -- start 2024-03-22T07:01:39.855 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.854+0000 7f5c7d3a0700 1 -- start start 2024-03-22T07:01:39.855 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.854+0000 7f5c7d3a0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 0x7f5c780664c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:39.855 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.854+0000 7f5c7d3a0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7810fd50 con 0x7f5c780fb960 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.854+0000 7f5c76ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 0x7f5c780664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.854+0000 7f5c76ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 0x7f5c780664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:38108/0 (socket says 172.21.15.149:38108) 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.854+0000 7f5c76ffd700 1 -- 172.21.15.149:0/2087578163 learned_addr learned my addr 172.21.15.149:0/2087578163 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.854+0000 7f5c76ffd700 1 -- 172.21.15.149:0/2087578163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c005040 con 0x7f5c780fb960 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.854+0000 7f5c76ffd700 1 --2- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 0x7f5c780664c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0096b0 tx=0x7f5c6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.855+0000 7f5c5f7fe700 1 -- 172.21.15.149:0/2087578163 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c6c00bed0 con 0x7f5c780fb960 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.855+0000 7f5c5f7fe700 1 -- 172.21.15.149:0/2087578163 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c6c00b5c0 con 0x7f5c780fb960 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.855+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7810fef0 con 0x7f5c780fb960 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.855+0000 7f5c5f7fe700 1 -- 172.21.15.149:0/2087578163 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c6c01c5b0 con 0x7f5c780fb960 2024-03-22T07:01:39.856 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.855+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c78110380 con 0x7f5c780fb960 2024-03-22T07:01:39.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.856+0000 7f5c5f7fe700 1 -- 172.21.15.149:0/2087578163 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 15) v1 ==== 43162+0+0 (secure 0 0 0) 0x7f5c6c01c820 con 0x7f5c780fb960 2024-03-22T07:01:39.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.856+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c78105920 con 0x7f5c780fb960 2024-03-22T07:01:39.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.856+0000 7f5c5f7fe700 1 -- 172.21.15.149:0/2087578163 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5c6c0496b0 con 0x7f5c780fb960 2024-03-22T07:01:39.864 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:39.864+0000 7f5c5f7fe700 1 -- 172.21.15.149:0/2087578163 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5c6c011dd0 con 0x7f5c780fb960 2024-03-22T07:01:40.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.054+0000 7f5c5f7fe700 1 -- 172.21.15.149:0/2087578163 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mgrmap(e 16) v1 ==== 43289+0+0 (secure 0 0 0) 0x7f5c6c004d30 con 0x7f5c780fb960 2024-03-22T07:01:40.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.054+0000 7f5c5f7fe700 1 --2- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5c60037970 0x7f5c60039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:40.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.054+0000 7f5c767fc700 1 --2- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5c60037970 0x7f5c60039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:40.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.055+0000 7f5c767fc700 1 --2- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5c60037970 0x7f5c60039d50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009990 tx=0x7f5c68006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:40.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.140+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5c7805f5f0 con 0x7f5c780fb960 2024-03-22T07:01:40.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.140+0000 7f5c5f7fe700 1 -- 172.21.15.149:0/2087578163 <== mon.0 v2:172.21.15.81:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5c6c011720 con 0x7f5c780fb960 2024-03-22T07:01:40.141 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:40.141 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:40.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5c60037970 msgr2=0x7f5c60039d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:40.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 --2- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5c60037970 0x7f5c60039d50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009990 tx=0x7f5c68006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:40.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 msgr2=0x7f5c780664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:40.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 --2- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 0x7f5c780664c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0096b0 tx=0x7f5c6c009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:40.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 shutdown_connections 2024-03-22T07:01:40.143 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 --2- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5c60037970 0x7f5c60039d50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:40.144 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 --2- 172.21.15.149:0/2087578163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c780fb960 0x7f5c780664c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:40.144 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 >> 172.21.15.149:0/2087578163 conn(0x7f5c780f70c0 msgr2=0x7f5c78102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:40.144 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 shutdown_connections 2024-03-22T07:01:40.144 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:40.143+0000 7f5c7d3a0700 1 -- 172.21.15.149:0/2087578163 wait complete. 2024-03-22T07:01:40.144 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:01:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:01:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/trash_purge_schedule"}]: dispatch 2024-03-22T07:01:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:40 smithi081 ceph-mon[29245]: mgrmap e16: smithi081.bcxarw(active, since 1.0073s) 2024-03-22T07:01:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:41.455 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:41 smithi081 ceph-mon[29245]: [22/Mar/2024:07:01:39] ENGINE Bus STARTING 2024-03-22T07:01:41.455 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:41 smithi081 ceph-mon[29245]: [22/Mar/2024:07:01:40] ENGINE Serving on https://172.21.15.81:7150 2024-03-22T07:01:41.455 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:41 smithi081 ceph-mon[29245]: [22/Mar/2024:07:01:40] ENGINE Bus STARTED 2024-03-22T07:01:41.455 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:41 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/2087578163' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:41.455 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:41 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:41.725 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:41.726 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:42.102 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:42.493 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:01:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:42 smithi081 ceph-mon[29245]: mgrmap e17: smithi081.bcxarw(active, since 2s) 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:01:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:01:44.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.881+0000 7f440435f700 1 -- 172.21.15.149:0/859801797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 msgr2=0x7f43fc0fde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:44.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.881+0000 7f440435f700 1 --2- 172.21.15.149:0/859801797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 0x7f43fc0fde30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f43f80048c0 tx=0x7f43f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:44.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.881+0000 7f440435f700 1 -- 172.21.15.149:0/859801797 shutdown_connections 2024-03-22T07:01:44.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.881+0000 7f440435f700 1 --2- 172.21.15.149:0/859801797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 0x7f43fc0fde30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:44.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.881+0000 7f440435f700 1 -- 172.21.15.149:0/859801797 >> 172.21.15.149:0/859801797 conn(0x7f43fc0f7510 msgr2=0x7f43fc0f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:44.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.881+0000 7f440435f700 1 -- 172.21.15.149:0/859801797 shutdown_connections 2024-03-22T07:01:44.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.881+0000 7f440435f700 1 -- 172.21.15.149:0/859801797 wait complete. 2024-03-22T07:01:44.882 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.882+0000 7f440435f700 1 Processor -- start 2024-03-22T07:01:44.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.882+0000 7f440435f700 1 -- start start 2024-03-22T07:01:44.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.882+0000 7f440435f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 0x7f43fc113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:44.883 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.882+0000 7f440435f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc1136d0 con 0x7f43fc0fda60 2024-03-22T07:01:44.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.882+0000 7f44020fb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 0x7f43fc113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:44.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.882+0000 7f44020fb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 0x7f43fc113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:34124/0 (socket says 172.21.15.149:34124) 2024-03-22T07:01:44.884 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.882+0000 7f44020fb700 1 -- 172.21.15.149:0/574621628 learned_addr learned my addr 172.21.15.149:0/574621628 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.883+0000 7f44020fb700 1 -- 172.21.15.149:0/574621628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8005040 con 0x7f43fc0fda60 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.883+0000 7f44020fb700 1 --2- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 0x7f43fc113190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f43f800b7f0 tx=0x7f43f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.883+0000 7f43f2ffd700 1 -- 172.21.15.149:0/574621628 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43f8009370 con 0x7f43fc0fda60 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.883+0000 7f43f2ffd700 1 -- 172.21.15.149:0/574621628 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43f8013420 con 0x7f43fc0fda60 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.883+0000 7f43f2ffd700 1 -- 172.21.15.149:0/574621628 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43f801c590 con 0x7f43fc0fda60 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.883+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc1138d0 con 0x7f43fc0fda60 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.883+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc113d40 con 0x7f43fc0fda60 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.884+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc0665c0 con 0x7f43fc0fda60 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.884+0000 7f43f2ffd700 1 -- 172.21.15.149:0/574621628 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f43f8026440 con 0x7f43fc0fda60 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.884+0000 7f43f2ffd700 1 --2- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f43e8036970 0x7f43e8038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.884+0000 7f43f2ffd700 1 -- 172.21.15.149:0/574621628 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f43f80493f0 con 0x7f43fc0fda60 2024-03-22T07:01:44.885 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.884+0000 7f44018fa700 1 --2- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f43e8036970 0x7f43e8038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:44.886 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.885+0000 7f44018fa700 1 --2- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f43e8036970 0x7f43e8038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009990 tx=0x7f43ec006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:44.889 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:44.889+0000 7f43f2ffd700 1 -- 172.21.15.149:0/574621628 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f43f8011780 con 0x7f43fc0fda60 2024-03-22T07:01:45.169 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.168+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f43fc05f5f0 con 0x7f43fc0fda60 2024-03-22T07:01:45.169 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.169+0000 7f43f2ffd700 1 -- 172.21.15.149:0/574621628 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f43f8021030 con 0x7f43fc0fda60 2024-03-22T07:01:45.169 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:45.169 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:45.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f43e8036970 msgr2=0x7f43e8038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:45.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 --2- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f43e8036970 0x7f43e8038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f43ec009990 tx=0x7f43ec006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:45.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 msgr2=0x7f43fc113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:45.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 --2- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 0x7f43fc113190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f43f800b7f0 tx=0x7f43f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:45.172 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 shutdown_connections 2024-03-22T07:01:45.172 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 --2- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f43e8036970 0x7f43e8038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:45.172 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 --2- 172.21.15.149:0/574621628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43fc0fda60 0x7f43fc113190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:45.172 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 >> 172.21.15.149:0/574621628 conn(0x7f43fc0f7510 msgr2=0x7f43fc103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:45.172 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 shutdown_connections 2024-03-22T07:01:45.172 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:45.171+0000 7f440435f700 1 -- 172.21.15.149:0/574621628 wait complete. 2024-03-22T07:01:45.172 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: Updating smithi081:/etc/ceph/ceph.conf 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:45.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:45.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:45.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:45.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:01:45.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-22T07:01:45.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:45 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:01:46.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:46 smithi081 ceph-mon[29245]: Deploying daemon crash.smithi149 on smithi149 2024-03-22T07:01:46.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:46 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/574621628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:47.244 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:47.245 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:47.513 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:01:49.235 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.234+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/83836342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 msgr2=0x7f9f24102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.234+0000 7f9f2a6dd700 1 --2- 172.21.15.149:0/83836342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 0x7f9f24102ec0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9f140048c0 tx=0x7f9f14009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.234+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/83836342 shutdown_connections 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.234+0000 7f9f2a6dd700 1 --2- 172.21.15.149:0/83836342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 0x7f9f24102ec0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.234+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/83836342 >> 172.21.15.149:0/83836342 conn(0x7f9f240f6c30 msgr2=0x7f9f240f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.234+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/83836342 shutdown_connections 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.234+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/83836342 wait complete. 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.234+0000 7f9f2a6dd700 1 Processor -- start 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.235+0000 7f9f2a6dd700 1 -- start start 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.235+0000 7f9f2a6dd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 0x7f9f24113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.235+0000 7f9f2a6dd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24113640 con 0x7f9f24102af0 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.235+0000 7f9f23fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 0x7f9f24113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:49.236 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.235+0000 7f9f23fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 0x7f9f24113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:34158/0 (socket says 172.21.15.149:34158) 2024-03-22T07:01:49.237 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.235+0000 7f9f23fff700 1 -- 172.21.15.149:0/3645036426 learned_addr learned my addr 172.21.15.149:0/3645036426 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:49.237 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.236+0000 7f9f23fff700 1 -- 172.21.15.149:0/3645036426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14005040 con 0x7f9f24102af0 2024-03-22T07:01:49.238 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.236+0000 7f9f23fff700 1 --2- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 0x7f9f24113100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:49.238 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.236+0000 7f9f20ff9700 1 -- 172.21.15.149:0/3645036426 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f14009370 con 0x7f9f24102af0 2024-03-22T07:01:49.238 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.236+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f241138a0 con 0x7f9f24102af0 2024-03-22T07:01:49.238 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.236+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f24113d10 con 0x7f9f24102af0 2024-03-22T07:01:49.238 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.236+0000 7f9f20ff9700 1 -- 172.21.15.149:0/3645036426 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f14013420 con 0x7f9f24102af0 2024-03-22T07:01:49.238 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.236+0000 7f9f20ff9700 1 -- 172.21.15.149:0/3645036426 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9f14025560 con 0x7f9f24102af0 2024-03-22T07:01:49.238 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.237+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f240fe1e0 con 0x7f9f24102af0 2024-03-22T07:01:49.238 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.237+0000 7f9f20ff9700 1 -- 172.21.15.149:0/3645036426 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f9f14013a50 con 0x7f9f24102af0 2024-03-22T07:01:49.239 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.237+0000 7f9f20ff9700 1 --2- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9f0c036970 0x7f9f0c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:49.239 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.237+0000 7f9f20ff9700 1 -- 172.21.15.149:0/3645036426 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9f14049b40 con 0x7f9f24102af0 2024-03-22T07:01:49.239 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.238+0000 7f9f237fe700 1 --2- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9f0c036970 0x7f9f0c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:49.239 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.238+0000 7f9f237fe700 1 --2- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9f0c036970 0x7f9f0c038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f18006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:49.242 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.242+0000 7f9f20ff9700 1 -- 172.21.15.149:0/3645036426 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9f140094d0 con 0x7f9f24102af0 2024-03-22T07:01:49.534 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.533+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9f2405f5f0 con 0x7f9f24102af0 2024-03-22T07:01:49.534 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.534+0000 7f9f20ff9700 1 -- 172.21.15.149:0/3645036426 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f9f14023300 con 0x7f9f24102af0 2024-03-22T07:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:49.535 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.537+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9f0c036970 msgr2=0x7f9f0c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.537+0000 7f9f2a6dd700 1 --2- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9f0c036970 0x7f9f0c038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9f18009990 tx=0x7f9f18006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.537+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 msgr2=0x7f9f24113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.537+0000 7f9f2a6dd700 1 --2- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 0x7f9f24113100 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000c00 tx=0x7f9f14009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.537+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 shutdown_connections 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.537+0000 7f9f2a6dd700 1 --2- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9f0c036970 0x7f9f0c038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.538+0000 7f9f2a6dd700 1 --2- 172.21.15.149:0/3645036426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f24102af0 0x7f9f24113100 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.538+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 >> 172.21.15.149:0/3645036426 conn(0x7f9f240f6c30 msgr2=0x7f9f240f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.538+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 shutdown_connections 2024-03-22T07:01:49.538 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:49.538+0000 7f9f2a6dd700 1 -- 172.21.15.149:0/3645036426 wait complete. 2024-03-22T07:01:49.539 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:50.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:50.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:50.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:50.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:50.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:50 smithi081 ceph-mon[29245]: Deploying daemon node-exporter.smithi149 on smithi149 2024-03-22T07:01:50.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:50 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/3645036426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:51.156 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:51.156 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:51.409 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:01:53.897 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.835+0000 7f085bd84700 1 -- 172.21.15.149:0/3891764880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 msgr2=0x7f0854102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.835+0000 7f085bd84700 1 --2- 172.21.15.149:0/3891764880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 0x7f0854102fa0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0850009c60 tx=0x7f0850004d10 comp rx=0 tx=0).stop 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.835+0000 7f085bd84700 1 -- 172.21.15.149:0/3891764880 shutdown_connections 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.835+0000 7f085bd84700 1 --2- 172.21.15.149:0/3891764880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 0x7f0854102fa0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.835+0000 7f085bd84700 1 -- 172.21.15.149:0/3891764880 >> 172.21.15.149:0/3891764880 conn(0x7f08540f85e0 msgr2=0x7f08540fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.836+0000 7f085bd84700 1 -- 172.21.15.149:0/3891764880 shutdown_connections 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.836+0000 7f085bd84700 1 -- 172.21.15.149:0/3891764880 wait complete. 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.836+0000 7f085bd84700 1 Processor -- start 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.836+0000 7f085bd84700 1 -- start start 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.836+0000 7f085bd84700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 0x7f0854113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.836+0000 7f085bd84700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0854113770 con 0x7f0854102bd0 2024-03-22T07:01:53.898 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f0859b20700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 0x7f0854113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f0859b20700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 0x7f0854113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:54860/0 (socket says 172.21.15.149:54860) 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f0859b20700 1 -- 172.21.15.149:0/4278755853 learned_addr learned my addr 172.21.15.149:0/4278755853 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f0859b20700 1 -- 172.21.15.149:0/4278755853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0850009940 con 0x7f0854102bd0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f0859b20700 1 --2- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 0x7f0854113230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f08500096d0 tx=0x7f0850004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f084a7fc700 1 -- 172.21.15.149:0/4278755853 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0850007c00 con 0x7f0854102bd0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0854113970 con 0x7f0854102bd0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f084a7fc700 1 -- 172.21.15.149:0/4278755853 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f085001d910 con 0x7f0854102bd0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f084a7fc700 1 -- 172.21.15.149:0/4278755853 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0850012900 con 0x7f0854102bd0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.837+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0854113e60 con 0x7f0854102bd0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.838+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085405f5f0 con 0x7f0854102bd0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.838+0000 7f084a7fc700 1 -- 172.21.15.149:0/4278755853 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f0850019040 con 0x7f0854102bd0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.838+0000 7f084a7fc700 1 --2- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f0840036970 0x7f0840038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.839+0000 7f084a7fc700 1 -- 172.21.15.149:0/4278755853 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f085004a380 con 0x7f0854102bd0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.839+0000 7f085931f700 1 --2- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f0840036970 0x7f0840038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:53.899 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.839+0000 7f085931f700 1 --2- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f0840036970 0x7f0840038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0844009990 tx=0x7f0844006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:53.900 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:53.843+0000 7f084a7fc700 1 -- 172.21.15.149:0/4278755853 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f085002e9a0 con 0x7f0854102bd0 2024-03-22T07:01:54.106 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.105+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f085410e880 con 0x7f0854102bd0 2024-03-22T07:01:54.106 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.106+0000 7f084a7fc700 1 -- 172.21.15.149:0/4278755853 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f0850017090 con 0x7f0854102bd0 2024-03-22T07:01:54.107 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:54.107 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:54.109 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.108+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f0840036970 msgr2=0x7f0840038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:54.109 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.108+0000 7f085bd84700 1 --2- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f0840036970 0x7f0840038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0844009990 tx=0x7f0844006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:54.109 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.108+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 msgr2=0x7f0854113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:54.109 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.108+0000 7f085bd84700 1 --2- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 0x7f0854113230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f08500096d0 tx=0x7f0850004d10 comp rx=0 tx=0).stop 2024-03-22T07:01:54.109 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.108+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 shutdown_connections 2024-03-22T07:01:54.110 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.108+0000 7f085bd84700 1 --2- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f0840036970 0x7f0840038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:54.110 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.108+0000 7f085bd84700 1 --2- 172.21.15.149:0/4278755853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0854102bd0 0x7f0854113230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:54.110 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.108+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 >> 172.21.15.149:0/4278755853 conn(0x7f08540f85e0 msgr2=0x7f08540f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:54.110 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.109+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 shutdown_connections 2024-03-22T07:01:54.110 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:54.109+0000 7f085bd84700 1 -- 172.21.15.149:0/4278755853 wait complete. 2024-03-22T07:01:54.110 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:55.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:55 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:55.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:55 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/4278755853' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:01:55.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:55 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:01:55.872 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:01:55.872 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:01:56.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:56.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:56.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:56.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:56.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:01:56.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T07:01:56.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:01:56.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:01:56.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:56 smithi081 ceph-mon[29245]: Deploying daemon mgr.smithi149.sgmtoa on smithi149 2024-03-22T07:01:57.056 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:01:58.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.956+0000 7fed41f56700 1 -- 172.21.15.149:0/1812057523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 msgr2=0x7fed3c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:58.957 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.956+0000 7fed41f56700 1 --2- 172.21.15.149:0/1812057523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 0x7fed3c101d90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0048c0 tx=0x7fed2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:58.958 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.956+0000 7fed41f56700 1 -- 172.21.15.149:0/1812057523 shutdown_connections 2024-03-22T07:01:58.958 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.956+0000 7fed41f56700 1 --2- 172.21.15.149:0/1812057523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 0x7fed3c101d90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:58.958 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.956+0000 7fed41f56700 1 -- 172.21.15.149:0/1812057523 >> 172.21.15.149:0/1812057523 conn(0x7fed3c0f74e0 msgr2=0x7fed3c0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:58.958 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.956+0000 7fed41f56700 1 -- 172.21.15.149:0/1812057523 shutdown_connections 2024-03-22T07:01:58.958 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.956+0000 7fed41f56700 1 -- 172.21.15.149:0/1812057523 wait complete. 2024-03-22T07:01:58.958 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.957+0000 7fed41f56700 1 Processor -- start 2024-03-22T07:01:58.958 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.957+0000 7fed41f56700 1 -- start start 2024-03-22T07:01:58.958 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.957+0000 7fed41f56700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 0x7fed3c112120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:58.958 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.957+0000 7fed41f56700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c112660 con 0x7fed3c1019c0 2024-03-22T07:01:58.960 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.958+0000 7fed3b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 0x7fed3c112120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:58.960 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.958+0000 7fed3b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 0x7fed3c112120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:54884/0 (socket says 172.21.15.149:54884) 2024-03-22T07:01:58.960 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.958+0000 7fed3b7fe700 1 -- 172.21.15.149:0/2508838583 learned_addr learned my addr 172.21.15.149:0/2508838583 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:01:58.960 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.958+0000 7fed3b7fe700 1 -- 172.21.15.149:0/2508838583 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c005040 con 0x7fed3c1019c0 2024-03-22T07:01:58.960 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.958+0000 7fed3b7fe700 1 --2- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 0x7fed3c112120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00b7f0 tx=0x7fed2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:58.960 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.958+0000 7fed23fff700 1 -- 172.21.15.149:0/2508838583 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed2c009370 con 0x7fed3c1019c0 2024-03-22T07:01:58.960 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.959+0000 7fed23fff700 1 -- 172.21.15.149:0/2508838583 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed2c013420 con 0x7fed3c1019c0 2024-03-22T07:01:58.961 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.959+0000 7fed23fff700 1 -- 172.21.15.149:0/2508838583 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fed2c01c590 con 0x7fed3c1019c0 2024-03-22T07:01:58.961 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.959+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c112860 con 0x7fed3c1019c0 2024-03-22T07:01:58.961 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.959+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c112cd0 con 0x7fed3c1019c0 2024-03-22T07:01:58.961 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.960+0000 7fed23fff700 1 -- 172.21.15.149:0/2508838583 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fed2c026440 con 0x7fed3c1019c0 2024-03-22T07:01:58.961 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.960+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c0fd0c0 con 0x7fed3c1019c0 2024-03-22T07:01:58.961 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.960+0000 7fed23fff700 1 --2- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fed24038bd0 0x7fed2403b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:01:58.961 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.960+0000 7fed23fff700 1 -- 172.21.15.149:0/2508838583 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fed2c0493a0 con 0x7fed3c1019c0 2024-03-22T07:01:58.961 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.960+0000 7fed3affd700 1 --2- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fed24038bd0 0x7fed2403b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:01:58.961 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.961+0000 7fed3affd700 1 --2- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fed24038bd0 0x7fed2403b080 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fed30009990 tx=0x7fed30006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:01:58.966 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:58.966+0000 7fed23fff700 1 -- 172.21.15.149:0/2508838583 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fed2c011780 con 0x7fed3c1019c0 2024-03-22T07:01:59.246 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.246+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fed3c05f5f0 con 0x7fed3c1019c0 2024-03-22T07:01:59.247 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.246+0000 7fed23fff700 1 -- 172.21.15.149:0/2508838583 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fed2c021030 con 0x7fed3c1019c0 2024-03-22T07:01:59.247 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:01:59.247 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.248+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fed24038bd0 msgr2=0x7fed2403b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.248+0000 7fed41f56700 1 --2- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fed24038bd0 0x7fed2403b080 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fed30009990 tx=0x7fed30006e40 comp rx=0 tx=0).stop 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.249+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 msgr2=0x7fed3c112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.249+0000 7fed41f56700 1 --2- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 0x7fed3c112120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00b7f0 tx=0x7fed2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.249+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 shutdown_connections 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.249+0000 7fed41f56700 1 --2- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fed24038bd0 0x7fed2403b080 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.249+0000 7fed41f56700 1 --2- 172.21.15.149:0/2508838583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed3c1019c0 0x7fed3c112120 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.249+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 >> 172.21.15.149:0/2508838583 conn(0x7fed3c0f74e0 msgr2=0x7fed3c0f8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.249+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 shutdown_connections 2024-03-22T07:01:59.249 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:01:59.249+0000 7fed41f56700 1 -- 172.21.15.149:0/2508838583 wait complete. 2024-03-22T07:01:59.250 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:01:59.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:59.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:59.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:59.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:59.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:01:59.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:01:59.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:59 smithi081 ceph-mon[29245]: Deploying daemon mon.smithi149 on smithi149 2024-03-22T07:01:59.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:01:59.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:01:59 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/2508838583' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:02:01.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:00 smithi081 ceph-mon[29245]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:01.174 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:02:01.174 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:02:01.659 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi149/config 2024-03-22T07:02:03.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:02 smithi081 ceph-mon[29245]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:03.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.150+0000 7fb97ae37700 1 -- 172.21.15.149:0/3570484288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 msgr2=0x7fb974101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:03.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.150+0000 7fb97ae37700 1 --2- 172.21.15.149:0/3570484288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 0x7fb974101600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb964009c60 tx=0x7fb964004d10 comp rx=0 tx=0).stop 2024-03-22T07:02:03.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.150+0000 7fb97ae37700 1 -- 172.21.15.149:0/3570484288 shutdown_connections 2024-03-22T07:02:03.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.150+0000 7fb97ae37700 1 --2- 172.21.15.149:0/3570484288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 0x7fb974101600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:03.151 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.150+0000 7fb97ae37700 1 -- 172.21.15.149:0/3570484288 >> 172.21.15.149:0/3570484288 conn(0x7fb974072070 msgr2=0x7fb974072470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:02:03.152 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.150+0000 7fb97ae37700 1 -- 172.21.15.149:0/3570484288 shutdown_connections 2024-03-22T07:02:03.152 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.150+0000 7fb97ae37700 1 -- 172.21.15.149:0/3570484288 wait complete. 2024-03-22T07:02:03.152 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.151+0000 7fb97ae37700 1 Processor -- start 2024-03-22T07:02:03.152 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.151+0000 7fb97ae37700 1 -- start start 2024-03-22T07:02:03.153 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.151+0000 7fb97ae37700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 0x7fb974112140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.151+0000 7fb97ae37700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb974112680 con 0x7fb974101230 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.152+0000 7fb978bd3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 0x7fb974112140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.152+0000 7fb978bd3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 0x7fb974112140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.149:42964/0 (socket says 172.21.15.149:42964) 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.152+0000 7fb978bd3700 1 -- 172.21.15.149:0/1057558337 learned_addr learned my addr 172.21.15.149:0/1057558337 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.152+0000 7fb978bd3700 1 -- 172.21.15.149:0/1057558337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964009940 con 0x7fb974101230 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.152+0000 7fb978bd3700 1 --2- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 0x7fb974112140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb96400b740 tx=0x7fb964004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.152+0000 7fb9717fa700 1 -- 172.21.15.149:0/1057558337 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb964007b30 con 0x7fb974101230 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.152+0000 7fb9717fa700 1 -- 172.21.15.149:0/1057558337 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb96401d910 con 0x7fb974101230 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.152+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb974112880 con 0x7fb974101230 2024-03-22T07:02:03.154 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.153+0000 7fb9717fa700 1 -- 172.21.15.149:0/1057558337 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9640128b0 con 0x7fb974101230 2024-03-22T07:02:03.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.153+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb974112d70 con 0x7fb974101230 2024-03-22T07:02:03.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.153+0000 7fb9717fa700 1 -- 172.21.15.149:0/1057558337 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fb964019040 con 0x7fb974101230 2024-03-22T07:02:03.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.154+0000 7fb9717fa700 1 --2- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb960036970 0x7fb960038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:03.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.154+0000 7fb9717fa700 1 -- 172.21.15.149:0/1057558337 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fb96401da80 con 0x7fb974101230 2024-03-22T07:02:03.155 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.154+0000 7fb973fff700 1 --2- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb960036970 0x7fb960038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:03.156 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.154+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9740fc980 con 0x7fb974101230 2024-03-22T07:02:03.156 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.154+0000 7fb973fff700 1 --2- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb960036970 0x7fb960038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009990 tx=0x7fb95c006e40 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:03.159 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.159+0000 7fb9717fa700 1 -- 172.21.15.149:0/1057558337 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb964021080 con 0x7fb974101230 2024-03-22T07:02:03.461 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.461+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb97404bf60 con 0x7fb974101230 2024-03-22T07:02:03.462 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.462+0000 7fb9717fa700 1 -- 172.21.15.149:0/1057558337 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb964017020 con 0x7fb974101230 2024-03-22T07:02:03.462 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:02:03.462 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T06:59:32.666823Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:02:03.465 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb960036970 msgr2=0x7fb960038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:03.465 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 --2- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb960036970 0x7fb960038e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009990 tx=0x7fb95c006e40 comp rx=0 tx=0).stop 2024-03-22T07:02:03.465 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 msgr2=0x7fb974112140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:03.465 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 --2- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 0x7fb974112140 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb96400b740 tx=0x7fb964004d10 comp rx=0 tx=0).stop 2024-03-22T07:02:03.465 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 shutdown_connections 2024-03-22T07:02:03.465 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 --2- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb960036970 0x7fb960038e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:03.465 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 --2- 172.21.15.149:0/1057558337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb974101230 0x7fb974112140 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:03.465 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 >> 172.21.15.149:0/1057558337 conn(0x7fb974072070 msgr2=0x7fb974101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:02:03.466 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 shutdown_connections 2024-03-22T07:02:03.466 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:03.464+0000 7fb97ae37700 1 -- 172.21.15.149:0/1057558337 wait complete. 2024-03-22T07:02:03.466 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2024-03-22T07:02:04.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:03 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/1057558337' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:02:05.358 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:02:05.359 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:02:05.611 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi149/config 2024-03-22T07:02:09.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:09.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:02:09.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: mon.smithi081 calling monitor election 2024-03-22T07:02:09.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:02:09.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:02:09.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/3248567565' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/crt"}]: dispatch 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: mon.smithi149 calling monitor election 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: mon.smithi081 is new leader, mons smithi081,smithi149 in quorum (ranks 0,1) 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: monmap e2: 2 mons at {smithi081=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0],smithi149=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: fsmap 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: mgrmap e17: smithi081.bcxarw(active, since 29s) 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: overall HEALTH_OK 2024-03-22T07:02:09.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: Standby manager daemon smithi149.sgmtoa started 2024-03-22T07:02:09.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/3248567565' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:02:09.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:09.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/3248567565' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/key"}]: dispatch 2024-03-22T07:02:09.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/3248567565' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:02:09.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:08 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:08 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:10.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:10.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: mgrmap e18: smithi081.bcxarw(active, since 29s), standbys: smithi149.sgmtoa 2024-03-22T07:02:10.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi149.sgmtoa", "id": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:02:10.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:10.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:10.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:02:10.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:10.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:10.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:02:10.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:02:10.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.079+0000 7ff44e954700 1 -- 172.21.15.149:0/2087050684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448102bb0 msgr2=0x7ff4300055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:10.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.079+0000 7ff44e954700 1 --2- 172.21.15.149:0/2087050684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448102bb0 0x7ff4300055d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff43c007ea0 tx=0x7ff43c00b040 comp rx=0 tx=0).stop 2024-03-22T07:02:10.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.079+0000 7ff44e954700 1 -- 172.21.15.149:0/2087050684 shutdown_connections 2024-03-22T07:02:10.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.079+0000 7ff44e954700 1 --2- 172.21.15.149:0/2087050684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448102bb0 0x7ff4300055d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:10.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.079+0000 7ff44e954700 1 -- 172.21.15.149:0/2087050684 >> 172.21.15.149:0/2087050684 conn(0x7ff448072030 msgr2=0x7ff448072430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:02:10.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.079+0000 7ff44e954700 1 -- 172.21.15.149:0/2087050684 shutdown_connections 2024-03-22T07:02:10.080 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.079+0000 7ff44e954700 1 -- 172.21.15.149:0/2087050684 wait complete. 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: mgrmap e18: smithi081.bcxarw(active, since 29s), standbys: smithi149.sgmtoa 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi149.sgmtoa", "id": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:02:10.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:02:10.084 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff44e954700 1 Processor -- start 2024-03-22T07:02:10.084 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff44e954700 1 -- start start 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff44e954700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 0x7ff448113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff44e954700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448113680 0x7ff44810d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff44e954700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff448113c90 con 0x7ff448113680 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff44e954700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff448113e00 con 0x7ff448102bb0 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff447fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 0x7ff448113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff447fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 0x7ff448113140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:50840/0 (socket says 172.21.15.149:50840) 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff447fff700 1 -- 172.21.15.149:0/3555280413 learned_addr learned my addr 172.21.15.149:0/3555280413 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff447fff700 1 -- 172.21.15.149:0/3555280413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 msgr2=0x7ff448113140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff447fff700 1 -- 172.21.15.149:0/3555280413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 msgr2=0x7ff448113140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff447fff700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 0x7ff448113140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.080+0000 7ff447fff700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 0x7ff448113140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff4477fe700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448113680 0x7ff44810d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff4477fe700 1 -- 172.21.15.149:0/3555280413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 msgr2=0x7ff448113140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:02:10.085 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff4477fe700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 0x7ff448113140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff4477fe700 1 -- 172.21.15.149:0/3555280413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c005040 con 0x7ff448113680 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff4477fe700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448113680 0x7ff44810d0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff43800b790 tx=0x7ff4380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff444ff9700 1 -- 172.21.15.149:0/3555280413 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff438011810 con 0x7ff448113680 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff444ff9700 1 -- 172.21.15.149:0/3555280413 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff438011e40 con 0x7ff448113680 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44810d740 con 0x7ff448113680 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff444ff9700 1 -- 172.21.15.149:0/3555280413 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff43800f2e0 con 0x7ff448113680 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.081+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44810dd00 con 0x7ff448113680 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.082+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff448107300 con 0x7ff448113680 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.083+0000 7ff444ff9700 1 -- 172.21.15.149:0/3555280413 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7ff438010380 con 0x7ff448113680 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.083+0000 7ff444ff9700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ff434069360 0x7ff43406b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.083+0000 7ff444ff9700 1 -- 172.21.15.149:0/3555280413 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff4380863c0 con 0x7ff448113680 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.083+0000 7ff447fff700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ff434069360 0x7ff43406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:10.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.084+0000 7ff447fff700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ff434069360 0x7ff43406b810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009090 tx=0x7ff43c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:10.088 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.088+0000 7ff444ff9700 1 -- 172.21.15.149:0/3555280413 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff4380536f0 con 0x7ff448113680 2024-03-22T07:02:10.354 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.354+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff44810e7d0 con 0x7ff448113680 2024-03-22T07:02:10.355 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.354+0000 7ff444ff9700 1 -- 172.21.15.149:0/3555280413 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7ff438015090 con 0x7ff448113680 2024-03-22T07:02:10.355 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-22T07:02:10.355 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","modified":"2024-03-22T07:02:03.630371Z","created":"2024-03-22T06:59:32.666823Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi081","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi149","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.356+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ff434069360 msgr2=0x7ff43406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.356+0000 7ff44e954700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ff434069360 0x7ff43406b810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff43c009090 tx=0x7ff43c00b040 comp rx=0 tx=0).stop 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.356+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448113680 msgr2=0x7ff44810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.356+0000 7ff44e954700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448113680 0x7ff44810d0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff43800b790 tx=0x7ff4380091a0 comp rx=0 tx=0).stop 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.357+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 shutdown_connections 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.357+0000 7ff44e954700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ff434069360 0x7ff43406b810 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.357+0000 7ff44e954700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff448102bb0 0x7ff448113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.357+0000 7ff44e954700 1 --2- 172.21.15.149:0/3555280413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff448113680 0x7ff44810d0e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.357+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 >> 172.21.15.149:0/3555280413 conn(0x7ff448072030 msgr2=0x7ff4480fa560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.357+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 shutdown_connections 2024-03-22T07:02:10.357 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:02:10.357+0000 7ff44e954700 1 -- 172.21.15.149:0/3555280413 wait complete. 2024-03-22T07:02:10.358 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2024-03-22T07:02:11.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:10 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/3555280413' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:02:11.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:10 smithi149 ceph-mon[36068]: from='client.? 172.21.15.149:0/3555280413' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:02:11.136 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-22T07:02:11.137 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-22T07:02:11.422 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:02:12.470 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:12.470 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.470 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.470 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:12.470 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:02:12.470 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: Updating smithi081:/etc/ceph/ceph.conf 2024-03-22T07:02:12.470 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:02:12.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:12.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:12.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: Updating smithi081:/etc/ceph/ceph.conf 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:12.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:02:12.604 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:02:12.604 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:13.526 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:13 smithi081 ceph-mon[29245]: Reconfiguring mon.smithi081 (unknown last config time)... 2024-03-22T07:02:13.526 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:13 smithi081 ceph-mon[29245]: Reconfiguring daemon mon.smithi081 on smithi081 2024-03-22T07:02:13.527 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:13 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:13.527 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:13 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:13.527 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:13 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi081.bcxarw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:02:13.527 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:13 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:02:13.527 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:13 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:13.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:13 smithi149 ceph-mon[36068]: Reconfiguring mon.smithi081 (unknown last config time)... 2024-03-22T07:02:13.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:13 smithi149 ceph-mon[36068]: Reconfiguring daemon mon.smithi081 on smithi081 2024-03-22T07:02:13.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:13 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:13.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:13 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:13.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:13 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi081.bcxarw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:02:13.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:13 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:02:13.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:13 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:14.687 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:14 smithi081 ceph-mon[29245]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:14.687 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:14 smithi081 ceph-mon[29245]: Reconfiguring mgr.smithi081.bcxarw (unknown last config time)... 2024-03-22T07:02:14.687 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:14 smithi081 ceph-mon[29245]: Reconfiguring daemon mgr.smithi081.bcxarw on smithi081 2024-03-22T07:02:14.688 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:14 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:14.688 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:14 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:14.688 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:14 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi081", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:02:14.688 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:14 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.932+0000 7f6d8b885700 1 -- 172.21.15.81:0/3142262358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d841037b0 msgr2=0x7f6d840fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.932+0000 7f6d8b885700 1 --2- 172.21.15.81:0/3142262358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d841037b0 0x7f6d840fce20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d80004940 tx=0x7f6d80009e30 comp rx=0 tx=0).stop 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.932+0000 7f6d8b885700 1 -- 172.21.15.81:0/3142262358 shutdown_connections 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.932+0000 7f6d8b885700 1 --2- 172.21.15.81:0/3142262358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d841037b0 0x7f6d840fce20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.932+0000 7f6d8b885700 1 --2- 172.21.15.81:0/3142262358 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d84102e10 0x7f6d841031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.932+0000 7f6d8b885700 1 -- 172.21.15.81:0/3142262358 >> 172.21.15.81:0/3142262358 conn(0x7f6d840f89a0 msgr2=0x7f6d840fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.932+0000 7f6d8b885700 1 -- 172.21.15.81:0/3142262358 shutdown_connections 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.932+0000 7f6d8b885700 1 -- 172.21.15.81:0/3142262358 wait complete. 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d8b885700 1 Processor -- start 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d8b885700 1 -- start start 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d8b885700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d84102e10 0x7f6d84115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d8b885700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d841037b0 0x7f6d84116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d8b885700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84116970 con 0x7f6d84102e10 2024-03-22T07:02:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d8b885700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8410fd90 con 0x7f6d841037b0 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d89621700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d84102e10 0x7f6d84115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d89621700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d84102e10 0x7f6d84115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58208/0 (socket says 172.21.15.81:58208) 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d89621700 1 -- 172.21.15.81:0/319282696 learned_addr learned my addr 172.21.15.81:0/319282696 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.933+0000 7f6d88e20700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d841037b0 0x7f6d84116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d89621700 1 -- 172.21.15.81:0/319282696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d841037b0 msgr2=0x7f6d84116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d89621700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d841037b0 0x7f6d84116320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d89621700 1 -- 172.21.15.81:0/319282696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d80005040 con 0x7f6d84102e10 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d88e20700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d841037b0 0x7f6d84116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d89621700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d84102e10 0x7f6d84115de0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d74002a50 tx=0x7f6d740046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d79ffb700 1 -- 172.21.15.81:0/319282696 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d74007cc0 con 0x7f6d84102e10 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d79ffb700 1 -- 172.21.15.81:0/319282696 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d7400f040 con 0x7f6d84102e10 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d79ffb700 1 -- 172.21.15.81:0/319282696 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7400db70 con 0x7f6d84102e10 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d84110070 con 0x7f6d84102e10 2024-03-22T07:02:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.934+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d84110630 con 0x7f6d84102e10 2024-03-22T07:02:14.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.935+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d841018d0 con 0x7f6d84102e10 2024-03-22T07:02:14.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.935+0000 7f6d79ffb700 1 -- 172.21.15.81:0/319282696 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f6d7400dd10 con 0x7f6d84102e10 2024-03-22T07:02:14.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.936+0000 7f6d79ffb700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f6d70069210 0x7f6d7006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:14.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.936+0000 7f6d79ffb700 1 -- 172.21.15.81:0/319282696 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f6d740856e0 con 0x7f6d84102e10 2024-03-22T07:02:14.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.936+0000 7f6d88e20700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f6d70069210 0x7f6d7006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:14.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.936+0000 7f6d88e20700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f6d70069210 0x7f6d7006b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000b3b0 tx=0x7f6d80009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:14.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:14.940+0000 7f6d79ffb700 1 -- 172.21.15.81:0/319282696 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6d74056580 con 0x7f6d84102e10 2024-03-22T07:02:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:14 smithi149 ceph-mon[36068]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:14 smithi149 ceph-mon[36068]: Reconfiguring mgr.smithi081.bcxarw (unknown last config time)... 2024-03-22T07:02:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:14 smithi149 ceph-mon[36068]: Reconfiguring daemon mgr.smithi081.bcxarw on smithi081 2024-03-22T07:02:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:14 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:14 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:14 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi081", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:02:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:14 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:15.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.131+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6d840fe560 con 0x7f6d84102e10 2024-03-22T07:02:15.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.132+0000 7f6d79ffb700 1 -- 172.21.15.81:0/319282696 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f6d74017090 con 0x7f6d84102e10 2024-03-22T07:02:15.132 INFO:teuthology.orchestra.run.smithi081.stdout:# minimal ceph.conf for a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T07:02:15.132 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2024-03-22T07:02:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T07:02:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: mon_host = [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-03-22T07:02:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.133+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f6d70069210 msgr2=0x7f6d7006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.133+0000 7f6d8b885700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f6d70069210 0x7f6d7006b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000b3b0 tx=0x7f6d80009e30 comp rx=0 tx=0).stop 2024-03-22T07:02:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.133+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d84102e10 msgr2=0x7f6d84115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.133+0000 7f6d8b885700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d84102e10 0x7f6d84115de0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d74002a50 tx=0x7f6d740046b0 comp rx=0 tx=0).stop 2024-03-22T07:02:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.134+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 shutdown_connections 2024-03-22T07:02:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.134+0000 7f6d8b885700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f6d70069210 0x7f6d7006b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.134+0000 7f6d8b885700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d84102e10 0x7f6d84115de0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:15.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.134+0000 7f6d8b885700 1 --2- 172.21.15.81:0/319282696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d841037b0 0x7f6d84116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:15.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.134+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 >> 172.21.15.81:0/319282696 conn(0x7f6d840f89a0 msgr2=0x7f6d84102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:02:15.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.134+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 shutdown_connections 2024-03-22T07:02:15.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:15.134+0000 7f6d8b885700 1 -- 172.21.15.81:0/319282696 wait complete. 2024-03-22T07:02:15.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:15 smithi081 ceph-mon[29245]: Reconfiguring crash.smithi081 (monmap changed)... 2024-03-22T07:02:15.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:15 smithi081 ceph-mon[29245]: Reconfiguring daemon crash.smithi081 on smithi081 2024-03-22T07:02:15.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:15 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/319282696' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:15.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:15 smithi149 ceph-mon[36068]: Reconfiguring crash.smithi081 (monmap changed)... 2024-03-22T07:02:15.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:15 smithi149 ceph-mon[36068]: Reconfiguring daemon crash.smithi081 on smithi081 2024-03-22T07:02:15.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:15 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/319282696' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:16.672 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-22T07:02:16.672 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:02:16.672 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T07:02:16.700 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:02:16.700 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:02:16.768 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:02:16.768 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T07:02:16.799 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:02:16.799 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:02:16.867 INFO:tasks.cephadm:Deploying OSDs... 2024-03-22T07:02:16.867 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:02:16.867 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T07:02:16.883 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T07:02:16.883 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-03-22T07:02:16.883 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:16 smithi081 ceph-mon[29245]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:16.883 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:16 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:16.883 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:16 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T07:02:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:02:16.938 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-22T07:02:16.938 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:02:16.939 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:02:16.939 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-22 07:01:30.979370452 +0000 2024-03-22T07:02:16.939 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-22 06:56:07.272747501 +0000 2024-03-22T07:02:16.939 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-22 06:56:07.272747501 +0000 2024-03-22T07:02:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-22 06:56:07.272747501 +0000 2024-03-22T07:02:16.939 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T07:02:17.004 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-22T07:02:17.004 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-22T07:02:17.004 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000241258 s, 2.1 MB/s 2024-03-22T07:02:17.005 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T07:02:17.061 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-03-22T07:02:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:16 smithi149 ceph-mon[36068]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:16 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:16 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:17.118 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T07:02:17.118 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:02:17.119 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-22T07:02:17.119 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:02:17.119 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:02:17.119 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-22 07:01:30.979370452 +0000 2024-03-22T07:02:17.119 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-22 06:56:07.617740828 +0000 2024-03-22T07:02:17.120 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-22 06:56:07.617740828 +0000 2024-03-22T07:02:17.120 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-22 06:56:07.617740828 +0000 2024-03-22T07:02:17.120 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T07:02:17.185 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-22T07:02:17.185 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-22T07:02:17.185 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000199043 s, 2.6 MB/s 2024-03-22T07:02:17.186 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T07:02:17.243 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-03-22T07:02:17.298 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T07:02:17.298 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:02:17.298 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-22T07:02:17.299 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:02:17.299 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:02:17.299 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-22 07:01:30.980370433 +0000 2024-03-22T07:02:17.299 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-22 06:56:07.954734310 +0000 2024-03-22T07:02:17.299 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-22 06:56:07.954734310 +0000 2024-03-22T07:02:17.299 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-22 06:56:07.954734310 +0000 2024-03-22T07:02:17.300 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T07:02:17.364 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-22T07:02:17.364 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-22T07:02:17.364 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000172867 s, 3.0 MB/s 2024-03-22T07:02:17.365 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T07:02:17.421 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-03-22T07:02:17.477 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T07:02:17.477 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:02:17.477 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-22T07:02:17.477 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:02:17.478 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:02:17.478 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-03-22 07:01:30.980370433 +0000 2024-03-22T07:02:17.478 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-22 06:56:08.331727018 +0000 2024-03-22T07:02:17.478 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-22 06:56:08.331727018 +0000 2024-03-22T07:02:17.478 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-22 06:56:08.331727018 +0000 2024-03-22T07:02:17.479 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T07:02:17.544 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-03-22T07:02:17.544 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-03-22T07:02:17.544 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.0001777 s, 2.9 MB/s 2024-03-22T07:02:17.545 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T07:02:17.601 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:02:17.601 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T07:02:17.620 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T07:02:17.620 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-03-22T07:02:17.678 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T07:02:17.679 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:02:17.679 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-22T07:02:17.679 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:02:17.679 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:02:17.679 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-22 07:02:01.874916543 +0000 2024-03-22T07:02:17.679 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-22 06:56:03.913814909 +0000 2024-03-22T07:02:17.680 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-22 06:56:03.913814909 +0000 2024-03-22T07:02:17.680 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-22 06:56:03.913814909 +0000 2024-03-22T07:02:17.680 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T07:02:17.747 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-22T07:02:17.748 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-22T07:02:17.748 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000234989 s, 2.2 MB/s 2024-03-22T07:02:17.749 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T07:02:17.807 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-03-22T07:02:17.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:17 smithi081 ceph-mon[29245]: Reconfiguring alertmanager.smithi081 (dependencies changed)... 2024-03-22T07:02:17.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:17 smithi081 ceph-mon[29245]: Reconfiguring daemon alertmanager.smithi081 on smithi081 2024-03-22T07:02:17.863 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T07:02:17.863 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:02:17.863 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-22T07:02:17.863 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:02:17.864 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:02:17.864 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-22 07:02:01.874916543 +0000 2024-03-22T07:02:17.864 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-22 06:56:04.257807926 +0000 2024-03-22T07:02:17.864 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-22 06:56:04.257807926 +0000 2024-03-22T07:02:17.864 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-22 06:56:04.257807926 +0000 2024-03-22T07:02:17.864 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T07:02:17.930 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:17 smithi149 ceph-mon[36068]: Reconfiguring alertmanager.smithi081 (dependencies changed)... 2024-03-22T07:02:17.930 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:17 smithi149 ceph-mon[36068]: Reconfiguring daemon alertmanager.smithi081 on smithi081 2024-03-22T07:02:17.932 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-22T07:02:17.932 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-22T07:02:17.932 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000230695 s, 2.2 MB/s 2024-03-22T07:02:17.934 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T07:02:17.992 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-03-22T07:02:18.048 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T07:02:18.049 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:02:18.049 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-22T07:02:18.049 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:02:18.049 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:02:18.049 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-22 07:02:01.874916543 +0000 2024-03-22T07:02:18.049 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-22 06:56:04.609800781 +0000 2024-03-22T07:02:18.049 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-22 06:56:04.609800781 +0000 2024-03-22T07:02:18.049 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-22 06:56:04.609800781 +0000 2024-03-22T07:02:18.049 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T07:02:18.117 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-22T07:02:18.118 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-22T07:02:18.118 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000293812 s, 1.7 MB/s 2024-03-22T07:02:18.119 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T07:02:18.176 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-03-22T07:02:18.233 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T07:02:18.234 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:02:18.234 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-22T07:02:18.234 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:02:18.234 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:02:18.234 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-22 07:02:01.875916522 +0000 2024-03-22T07:02:18.234 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-22 06:56:04.963793595 +0000 2024-03-22T07:02:18.234 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-22 06:56:04.963793595 +0000 2024-03-22T07:02:18.234 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-22 06:56:04.963793595 +0000 2024-03-22T07:02:18.234 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T07:02:18.302 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-22T07:02:18.302 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-22T07:02:18.302 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000229102 s, 2.2 MB/s 2024-03-22T07:02:18.303 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T07:02:18.361 INFO:tasks.cephadm:Deploying osd.0 on smithi081 with /dev/vg_nvme/lv_4... 2024-03-22T07:02:18.361 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T07:02:18.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:18 smithi081 ceph-mon[29245]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:19.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:18 smithi149 ceph-mon[36068]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:19.259 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:02:20.777 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:20 smithi081 ceph-mon[29245]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:21.014 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T07:02:21.014 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T07:02:21.014 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-03-22T07:02:21.014 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-03-22T07:02:21.015 INFO:teuthology.orchestra.run.smithi081.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115585 s, 907 MB/s 2024-03-22T07:02:21.015 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-03-22T07:02:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:20 smithi149 ceph-mon[36068]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:21.746 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi081:vg_nvme/lv_4 2024-03-22T07:02:22.131 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:02:23.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:22 smithi081 ceph-mon[29245]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:23.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:22 smithi149 ceph-mon[36068]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:23.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.566+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4118963852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 msgr2=0x7fd0480fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:23.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.566+0000 7fd04cbe0700 1 --2- 172.21.15.81:0/4118963852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 0x7fd0480fc0e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd038004940 tx=0x7fd038009e30 comp rx=0 tx=0).stop 2024-03-22T07:02:23.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.566+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4118963852 shutdown_connections 2024-03-22T07:02:23.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.566+0000 7fd04cbe0700 1 --2- 172.21.15.81:0/4118963852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0480fcf80 0x7fd0480fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:23.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.566+0000 7fd04cbe0700 1 --2- 172.21.15.81:0/4118963852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 0x7fd0480fc0e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.566+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4118963852 >> 172.21.15.81:0/4118963852 conn(0x7fd0480f7430 msgr2=0x7fd0480f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.566+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4118963852 shutdown_connections 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.566+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4118963852 wait complete. 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04cbe0700 1 Processor -- start 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04cbe0700 1 -- start start 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04cbe0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 0x7fd04810c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04cbe0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0480fcf80 0x7fd04810cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04cbe0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04810d3a0 con 0x7fd0480fbcd0 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04cbe0700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04810d510 con 0x7fd0480fcf80 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 0x7fd04810c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 0x7fd04810c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58228/0 (socket says 172.21.15.81:58228) 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04659c700 1 -- 172.21.15.81:0/4223076472 learned_addr learned my addr 172.21.15.81:0/4223076472 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd045d9b700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0480fcf80 0x7fd04810cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:23.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04659c700 1 -- 172.21.15.81:0/4223076472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0480fcf80 msgr2=0x7fd04810cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04659c700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0480fcf80 0x7fd04810cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04659c700 1 -- 172.21.15.81:0/4223076472 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd038005040 con 0x7fd0480fbcd0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd045d9b700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0480fcf80 0x7fd04810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.567+0000 7fd04659c700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 0x7fd04810c860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b3b0 tx=0x7fd038009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.568+0000 7fd036ffd700 1 -- 172.21.15.81:0/4223076472 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd03800e040 con 0x7fd0480fbcd0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.568+0000 7fd036ffd700 1 -- 172.21.15.81:0/4223076472 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd03800bcc0 con 0x7fd0480fbcd0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.568+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd048111f10 con 0x7fd0480fbcd0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.568+0000 7fd036ffd700 1 -- 172.21.15.81:0/4223076472 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd03801c630 con 0x7fd0480fbcd0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.568+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd048112440 con 0x7fd0480fbcd0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.569+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04804c710 con 0x7fd0480fbcd0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.569+0000 7fd036ffd700 1 -- 172.21.15.81:0/4223076472 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fd038013980 con 0x7fd0480fbcd0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.569+0000 7fd036ffd700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd030069210 0x7fd03006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.569+0000 7fd036ffd700 1 -- 172.21.15.81:0/4223076472 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd038086ea0 con 0x7fd0480fbcd0 2024-03-22T07:02:23.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.569+0000 7fd045d9b700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd030069210 0x7fd03006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:23.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.570+0000 7fd045d9b700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd030069210 0x7fd03006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd03c007fd0 tx=0x7fd03c009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:23.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.574+0000 7fd036ffd700 1 -- 172.21.15.81:0/4223076472 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd038057cc0 con 0x7fd0480fbcd0 2024-03-22T07:02:23.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:23.783+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0481012e0 con 0x7fd030069210 2024-03-22T07:02:24.745 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:24 smithi081 ceph-mon[29245]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:24.745 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:02:24.745 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:02:24.745 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:24.745 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:02:24.745 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:24.745 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:24 smithi149 ceph-mon[36068]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:02:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:02:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:02:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:25.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:25 smithi081 ceph-mon[29245]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:02:25.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:25 smithi081 ceph-mon[29245]: Reconfiguring grafana.smithi081 (dependencies changed)... 2024-03-22T07:02:25.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:25 smithi081 ceph-mon[29245]: Reconfiguring daemon grafana.smithi081 on smithi081 2024-03-22T07:02:26.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:25 smithi149 ceph-mon[36068]: from='client.14246 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:02:26.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:25 smithi149 ceph-mon[36068]: Reconfiguring grafana.smithi081 (dependencies changed)... 2024-03-22T07:02:26.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:25 smithi149 ceph-mon[36068]: Reconfiguring daemon grafana.smithi081 on smithi081 2024-03-22T07:02:26.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:26 smithi081 ceph-mon[29245]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:27.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:26 smithi149 ceph-mon[36068]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:28.962 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:28 smithi081 ceph-mon[29245]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:28 smithi149 ceph-mon[36068]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:30.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:29 smithi081 ceph-mon[29245]: Reconfiguring prometheus.smithi081 (dependencies changed)... 2024-03-22T07:02:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:29 smithi081 ceph-mon[29245]: Reconfiguring daemon prometheus.smithi081 on smithi081 2024-03-22T07:02:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:29 smithi081 ceph-mon[29245]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:29 smithi149 ceph-mon[36068]: Reconfiguring prometheus.smithi081 (dependencies changed)... 2024-03-22T07:02:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:29 smithi149 ceph-mon[36068]: Reconfiguring daemon prometheus.smithi081 on smithi081 2024-03-22T07:02:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:29 smithi149 ceph-mon[36068]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:31 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/973401379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69addaec-0295-4219-9e22-5ebf831a65d8"}]: dispatch 2024-03-22T07:02:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:31 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/973401379' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69addaec-0295-4219-9e22-5ebf831a65d8"}]': finished 2024-03-22T07:02:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:31 smithi081 ceph-mon[29245]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T07:02:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:31 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:31 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/956019743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:02:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:31 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/973401379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "69addaec-0295-4219-9e22-5ebf831a65d8"}]: dispatch 2024-03-22T07:02:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:31 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/973401379' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "69addaec-0295-4219-9e22-5ebf831a65d8"}]': finished 2024-03-22T07:02:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:31 smithi149 ceph-mon[36068]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T07:02:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:31 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:31 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/956019743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:02:32.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:32 smithi081 ceph-mon[29245]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:32 smithi149 ceph-mon[36068]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:34.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:34 smithi081 ceph-mon[29245]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:34.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:34 smithi149 ceph-mon[36068]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:36.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:36 smithi081 ceph-mon[29245]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:36.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:36 smithi149 ceph-mon[36068]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:38.421 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:38 smithi081 ceph-mon[29245]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:38 smithi149 ceph-mon[36068]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:39.421 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:02:39.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:02:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:40 smithi081 ceph-mon[29245]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:02:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:40.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:40 smithi149 ceph-mon[36068]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:40.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:40 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:40.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:40 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:40.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:40 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:02:40.435 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:40 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:41.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:41 smithi081 ceph-mon[29245]: Reconfiguring crash.smithi149 (monmap changed)... 2024-03-22T07:02:41.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:41 smithi081 ceph-mon[29245]: Reconfiguring daemon crash.smithi149 on smithi149 2024-03-22T07:02:41.541 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:41 smithi149 ceph-mon[36068]: Reconfiguring crash.smithi149 (monmap changed)... 2024-03-22T07:02:41.541 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:41 smithi149 ceph-mon[36068]: Reconfiguring daemon crash.smithi149 on smithi149 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: Reconfiguring mgr.smithi149.sgmtoa (monmap changed)... 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: Reconfiguring daemon mgr.smithi149.sgmtoa on smithi149 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:02:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:42.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:42 smithi081 ceph-mon[29245]: Deploying daemon osd.0 on smithi081 2024-03-22T07:02:42.406 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: Reconfiguring mgr.smithi149.sgmtoa (monmap changed)... 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: Reconfiguring daemon mgr.smithi149.sgmtoa on smithi149 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:42.407 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:42 smithi149 ceph-mon[36068]: Deploying daemon osd.0 on smithi081 2024-03-22T07:02:43.718 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:43 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:43.719 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:43 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:43.719 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:43 smithi149 ceph-mon[36068]: Reconfiguring mon.smithi149 (monmap changed)... 2024-03-22T07:02:43.719 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:43 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:02:43.719 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:43 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:02:43.719 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:43 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:43.719 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:43 smithi149 ceph-mon[36068]: Reconfiguring daemon mon.smithi149 on smithi149 2024-03-22T07:02:43.774 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:43 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:43.774 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:43 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:43.774 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:43 smithi081 ceph-mon[29245]: Reconfiguring mon.smithi149 (monmap changed)... 2024-03-22T07:02:43.774 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:43 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:02:43.774 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:43 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:02:43.774 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:43 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:43.774 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:43 smithi081 ceph-mon[29245]: Reconfiguring daemon mon.smithi149 on smithi149 2024-03-22T07:02:44.734 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi081.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi081.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi081.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.735 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:02:44.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:44.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:02:44.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi081.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T07:02:44.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:02:44.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi081.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T07:02:44.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:02:44.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi081.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T07:02:44.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:44.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:02:45.639 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:45 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:02:45.639 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:45 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi081.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T07:02:45.639 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:45 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:02:45.639 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:45 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi081.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T07:02:45.639 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:45 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:02:45.639 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:45 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi081.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T07:02:45.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:45 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:02:45.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:45 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi081.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T07:02:45.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:45 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:02:45.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:45 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi081.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T07:02:45.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:45 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:02:45.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:45 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi081.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T07:02:46.657 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:46 smithi081 ceph-mon[29245]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:46.657 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:46 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:46.657 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:46 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:46.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:46 smithi149 ceph-mon[36068]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:46.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:46 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:46.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:46 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:48 smithi081 ceph-mon[29245]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:48 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:48 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:48.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:48 smithi149 ceph-mon[36068]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:48.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:48 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:48.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:48 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:50.648 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:50 smithi081 ceph-mon[29245]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:50.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:50 smithi149 ceph-mon[36068]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:51.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.467+0000 7fd036ffd700 1 -- 172.21.15.81:0/4223076472 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd0481012e0 con 0x7fd030069210 2024-03-22T07:02:51.468 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 0 on host 'smithi081' 2024-03-22T07:02:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.469+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd030069210 msgr2=0x7fd03006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.469+0000 7fd04cbe0700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd030069210 0x7fd03006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd03c007fd0 tx=0x7fd03c009040 comp rx=0 tx=0).stop 2024-03-22T07:02:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.470+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 msgr2=0x7fd04810c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.470+0000 7fd04cbe0700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 0x7fd04810c860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b3b0 tx=0x7fd038009e30 comp rx=0 tx=0).stop 2024-03-22T07:02:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.470+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 shutdown_connections 2024-03-22T07:02:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.470+0000 7fd04cbe0700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd030069210 0x7fd03006b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.470+0000 7fd04cbe0700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0480fbcd0 0x7fd04810c860 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.470+0000 7fd04cbe0700 1 --2- 172.21.15.81:0/4223076472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0480fcf80 0x7fd04810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.470+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 >> 172.21.15.81:0/4223076472 conn(0x7fd0480f7430 msgr2=0x7fd0480f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:02:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.470+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 shutdown_connections 2024-03-22T07:02:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:51.470+0000 7fd04cbe0700 1 -- 172.21.15.81:0/4223076472 wait complete. 2024-03-22T07:02:52.062 DEBUG:teuthology.orchestra.run.smithi081:osd.0> sudo journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.0.service 2024-03-22T07:02:52.065 INFO:tasks.cephadm:Deploying osd.1 on smithi081 with /dev/vg_nvme/lv_3... 2024-03-22T07:02:52.065 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T07:02:52.380 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:02:52.775 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:52 smithi081 ceph-mon[29245]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:52.775 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:52.775 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:52.776 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:52.776 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:52.776 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:52 smithi081 ceph-mon[29245]: from='osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T07:02:52.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:52 smithi149 ceph-mon[36068]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:52.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:52.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:52.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:52.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:52.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:52 smithi149 ceph-mon[36068]: from='osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T07:02:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:53 smithi081 ceph-mon[29245]: from='osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T07:02:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:53 smithi081 ceph-mon[29245]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T07:02:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:53 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:53 smithi081 ceph-mon[29245]: from='osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:02:53.819 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:02:53 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[47932]: 2024-03-22T07:02:53.560+0000 7f364b7a5700 -1 osd.0 0 waiting for initial osdmap 2024-03-22T07:02:53.820 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:02:53 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[47932]: 2024-03-22T07:02:53.563+0000 7f3644d9d700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:02:53.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:53 smithi149 ceph-mon[36068]: from='osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T07:02:53.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:53 smithi149 ceph-mon[36068]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T07:02:53.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:53 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:53.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:53 smithi149 ceph-mon[36068]: from='osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:02:54.779 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:54 smithi081 ceph-mon[29245]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:54.779 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:54 smithi081 ceph-mon[29245]: from='osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-03-22T07:02:54.779 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:54 smithi081 ceph-mon[29245]: osdmap e8: 1 total, 0 up, 1 in 2024-03-22T07:02:54.779 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:54.779 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:54.779 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:02:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:54 smithi149 ceph-mon[36068]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:54 smithi149 ceph-mon[36068]: from='osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-03-22T07:02:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:54 smithi149 ceph-mon[36068]: osdmap e8: 1 total, 0 up, 1 in 2024-03-22T07:02:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:02:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T07:02:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T07:02:54.963 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-03-22T07:02:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-03-22T07:02:54.963 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0153934 s, 681 MB/s 2024-03-22T07:02:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-03-22T07:02:55.628 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:55 smithi081 ceph-mon[29245]: purged_snaps scrub starts 2024-03-22T07:02:55.628 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:55 smithi081 ceph-mon[29245]: purged_snaps scrub ok 2024-03-22T07:02:55.629 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:55 smithi081 ceph-mon[29245]: osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] boot 2024-03-22T07:02:55.629 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:55 smithi081 ceph-mon[29245]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T07:02:55.629 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:55 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:55.630 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi081:vg_nvme/lv_3 2024-03-22T07:02:55.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:55 smithi149 ceph-mon[36068]: purged_snaps scrub starts 2024-03-22T07:02:55.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:55 smithi149 ceph-mon[36068]: purged_snaps scrub ok 2024-03-22T07:02:55.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:55 smithi149 ceph-mon[36068]: osd.0 [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] boot 2024-03-22T07:02:55.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:55 smithi149 ceph-mon[36068]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T07:02:55.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:55 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:02:56.282 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:02:56.655 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:56 smithi081 ceph-mon[29245]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:56.655 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:56 smithi081 ceph-mon[29245]: osdmap e10: 1 total, 1 up, 1 in 2024-03-22T07:02:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:56 smithi149 ceph-mon[36068]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:02:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:56 smithi149 ceph-mon[36068]: osdmap e10: 1 total, 1 up, 1 in 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.458+0000 7f547298b700 1 -- 172.21.15.81:0/1725556206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 msgr2=0x7f546c0faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.458+0000 7f547298b700 1 --2- 172.21.15.81:0/1725556206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 0x7f546c0faf50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5460004940 tx=0x7f5460009e30 comp rx=0 tx=0).stop 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.458+0000 7f547298b700 1 -- 172.21.15.81:0/1725556206 shutdown_connections 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.458+0000 7f547298b700 1 --2- 172.21.15.81:0/1725556206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 0x7f546c0faf50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 --2- 172.21.15.81:0/1725556206 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f546c0fe4c0 0x7f546c0fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 -- 172.21.15.81:0/1725556206 >> 172.21.15.81:0/1725556206 conn(0x7f546c0f6ba0 msgr2=0x7f546c0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 -- 172.21.15.81:0/1725556206 shutdown_connections 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 -- 172.21.15.81:0/1725556206 wait complete. 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 Processor -- start 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 -- start start 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f546c0fe4c0 0x7f546c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:58.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 0x7f546c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c10e540 con 0x7f546c0fee10 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.459+0000 7f547298b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c10ed90 con 0x7f546c0fe4c0 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f546b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 0x7f546c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f546b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 0x7f546c10df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55158/0 (socket says 172.21.15.81:55158) 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f546b7fe700 1 -- 172.21.15.81:0/1575078683 learned_addr learned my addr 172.21.15.81:0/1575078683 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f546bfff700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f546c0fe4c0 0x7f546c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f546b7fe700 1 -- 172.21.15.81:0/1575078683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f546c0fe4c0 msgr2=0x7f546c10da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f546b7fe700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f546c0fe4c0 0x7f546c10da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f546b7fe700 1 -- 172.21.15.81:0/1575078683 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460005040 con 0x7f546c0fee10 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f546b7fe700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 0x7f546c10df40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f546000b3b0 tx=0x7f5460009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f5468ff9700 1 -- 172.21.15.81:0/1575078683 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546000e040 con 0x7f546c0fee10 2024-03-22T07:02:58.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f5468ff9700 1 -- 172.21.15.81:0/1575078683 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f546000bbf0 con 0x7f546c0fee10 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546c10f010 con 0x7f546c0fee10 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f5468ff9700 1 -- 172.21.15.81:0/1575078683 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546001c630 con 0x7f546c0fee10 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.460+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546c10f550 con 0x7f546c0fee10 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.461+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546c05f5f0 con 0x7f546c0fee10 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.462+0000 7f5468ff9700 1 -- 172.21.15.81:0/1575078683 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f5460013420 con 0x7f546c0fee10 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.462+0000 7f5468ff9700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5454069210 0x7f545406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.462+0000 7f5468ff9700 1 -- 172.21.15.81:0/1575078683 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f5460086ef0 con 0x7f546c0fee10 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.462+0000 7f546bfff700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5454069210 0x7f545406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.463+0000 7f546bfff700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5454069210 0x7f545406b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f545c004f40 tx=0x7f545c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:02:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.467+0000 7f5468ff9700 1 -- 172.21.15.81:0/1575078683 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5460057bd0 con 0x7f546c0fee10 2024-03-22T07:02:58.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:02:58.683+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f546c10f8d0 con 0x7f5454069210 2024-03-22T07:02:58.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:58 smithi081 ceph-mon[29245]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:02:58.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:58 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:58.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:58 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:58.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:58 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:02:58.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:58 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:58.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:58 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:58.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:58 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:02:58.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:58 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:58.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:58 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:02:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:58 smithi149 ceph-mon[36068]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:02:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:58 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:58 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:58 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:02:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:58 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:58 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:58 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:02:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:58 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:02:58.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:58 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:02:59.701 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:59 smithi081 ceph-mon[29245]: Detected new or changed devices on smithi081 2024-03-22T07:02:59.701 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:59 smithi081 ceph-mon[29245]: Adjusting osd_memory_target on smithi081 to 12864M 2024-03-22T07:02:59.701 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:59 smithi081 ceph-mon[29245]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:02:59.701 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:02:59.701 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:02:59.701 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:02:59 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:02:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:59 smithi149 ceph-mon[36068]: Detected new or changed devices on smithi081 2024-03-22T07:02:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:59 smithi149 ceph-mon[36068]: Adjusting osd_memory_target on smithi081 to 12864M 2024-03-22T07:02:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:59 smithi149 ceph-mon[36068]: from='client.14264 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:02:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:59 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:02:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:59 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:02:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:02:59 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:00.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:00 smithi081 ceph-mon[29245]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:00.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:00 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:00.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:00 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:00.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:00 smithi149 ceph-mon[36068]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:00.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:00 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:00.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:00 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:01.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:01 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1674068507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd572358-1fd5-4e96-8caf-3d00d6e0608f"}]: dispatch 2024-03-22T07:03:01.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:01 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1674068507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd572358-1fd5-4e96-8caf-3d00d6e0608f"}]': finished 2024-03-22T07:03:01.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:01 smithi149 ceph-mon[36068]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T07:03:01.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:01 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:01.870 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:01 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1674068507' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd572358-1fd5-4e96-8caf-3d00d6e0608f"}]: dispatch 2024-03-22T07:03:01.870 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:01 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1674068507' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd572358-1fd5-4e96-8caf-3d00d6e0608f"}]': finished 2024-03-22T07:03:01.870 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:01 smithi081 ceph-mon[29245]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T07:03:01.870 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:01 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:02.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:02 smithi149 ceph-mon[36068]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:02.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:02 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3758665055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:03:03.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:02 smithi081 ceph-mon[29245]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:03.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:02 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3758665055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:03:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:04 smithi149 ceph-mon[36068]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:05.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:04 smithi081 ceph-mon[29245]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:06.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:06 smithi081 ceph-mon[29245]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:06.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:06 smithi149 ceph-mon[36068]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:08.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:08 smithi081 ceph-mon[29245]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:08.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:08 smithi149 ceph-mon[36068]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:09.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:03:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:03:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:03:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:03:10.778 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:10 smithi081 ceph-mon[29245]: Deploying daemon osd.1 on smithi081 2024-03-22T07:03:10.779 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:10 smithi081 ceph-mon[29245]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:10.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:10 smithi149 ceph-mon[36068]: Deploying daemon osd.1 on smithi081 2024-03-22T07:03:10.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:10 smithi149 ceph-mon[36068]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:12 smithi081 ceph-mon[29245]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:13.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:12 smithi149 ceph-mon[36068]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:14.724 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:14 smithi081 ceph-mon[29245]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:14.724 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:14 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:14.724 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:14 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:14.724 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:14 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:03:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:14 smithi149 ceph-mon[36068]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:14 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:14 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:14 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:03:16.996 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:16 smithi081 ceph-mon[29245]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:16.996 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:16 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:16.996 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:16 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:16 smithi149 ceph-mon[36068]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:16 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:16 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:17.299 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 1 on host 'smithi081' 2024-03-22T07:03:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.298+0000 7f5468ff9700 1 -- 172.21.15.81:0/1575078683 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f546c10f8d0 con 0x7f5454069210 2024-03-22T07:03:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.301+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5454069210 msgr2=0x7f545406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:03:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.301+0000 7f547298b700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5454069210 0x7f545406b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f545c004f40 tx=0x7f545c00b040 comp rx=0 tx=0).stop 2024-03-22T07:03:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.301+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 msgr2=0x7f546c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:03:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.301+0000 7f547298b700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 0x7f546c10df40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f546000b3b0 tx=0x7f5460009e30 comp rx=0 tx=0).stop 2024-03-22T07:03:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.301+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 shutdown_connections 2024-03-22T07:03:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.301+0000 7f547298b700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5454069210 0x7f545406b6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.301+0000 7f547298b700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f546c0fe4c0 0x7f546c10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.301+0000 7f547298b700 1 --2- 172.21.15.81:0/1575078683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f546c0fee10 0x7f546c10df40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.301+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 >> 172.21.15.81:0/1575078683 conn(0x7f546c0f6ba0 msgr2=0x7f546c0f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:03:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.302+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 shutdown_connections 2024-03-22T07:03:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:17.302+0000 7f547298b700 1 -- 172.21.15.81:0/1575078683 wait complete. 2024-03-22T07:03:17.967 DEBUG:teuthology.orchestra.run.smithi081:osd.1> sudo journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.1.service 2024-03-22T07:03:17.970 INFO:tasks.cephadm:Deploying osd.2 on smithi081 with /dev/vg_nvme/lv_2... 2024-03-22T07:03:17.970 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T07:03:18.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:18 smithi081 ceph-mon[29245]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:18.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:18 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:18.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:18 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:18.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:18 smithi081 ceph-mon[29245]: from='osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T07:03:18.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:18 smithi149 ceph-mon[36068]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:18.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:18 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:18.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:18 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:18.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:18 smithi149 ceph-mon[36068]: from='osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T07:03:18.731 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:03:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:19 smithi081 ceph-mon[29245]: from='osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T07:03:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:19 smithi081 ceph-mon[29245]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T07:03:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:19 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:19 smithi081 ceph-mon[29245]: from='osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:03:20.069 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:19 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[55116]: 2024-03-22T07:03:19.620+0000 7f2ee2cba700 -1 osd.1 0 waiting for initial osdmap 2024-03-22T07:03:20.069 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:19 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[55116]: 2024-03-22T07:03:19.625+0000 7f2edbab1700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:03:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:19 smithi149 ceph-mon[36068]: from='osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T07:03:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:19 smithi149 ceph-mon[36068]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T07:03:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:19 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:19 smithi149 ceph-mon[36068]: from='osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:03:20.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:20 smithi081 ceph-mon[29245]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:20.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:20 smithi081 ceph-mon[29245]: from='osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-03-22T07:03:20.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:20 smithi081 ceph-mon[29245]: osdmap e13: 2 total, 1 up, 2 in 2024-03-22T07:03:20.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:20 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:20.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:20 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:20 smithi149 ceph-mon[36068]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:20 smithi149 ceph-mon[36068]: from='osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-03-22T07:03:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:20 smithi149 ceph-mon[36068]: osdmap e13: 2 total, 1 up, 2 in 2024-03-22T07:03:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:20 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:20 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:21.628 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:21 smithi081 ceph-mon[29245]: purged_snaps scrub starts 2024-03-22T07:03:21.628 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:21 smithi081 ceph-mon[29245]: purged_snaps scrub ok 2024-03-22T07:03:21.628 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:21 smithi081 ceph-mon[29245]: osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] boot 2024-03-22T07:03:21.628 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:21 smithi081 ceph-mon[29245]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T07:03:21.628 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:21 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:21.793 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T07:03:21.793 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T07:03:21.793 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-03-22T07:03:21.794 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-03-22T07:03:21.794 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127107 s, 825 MB/s 2024-03-22T07:03:21.794 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-03-22T07:03:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:21 smithi149 ceph-mon[36068]: purged_snaps scrub starts 2024-03-22T07:03:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:21 smithi149 ceph-mon[36068]: purged_snaps scrub ok 2024-03-22T07:03:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:21 smithi149 ceph-mon[36068]: osd.1 [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] boot 2024-03-22T07:03:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:21 smithi149 ceph-mon[36068]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T07:03:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:21 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:03:22.960 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:22 smithi081 ceph-mon[29245]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:22.960 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:22 smithi081 ceph-mon[29245]: osdmap e15: 2 total, 2 up, 2 in 2024-03-22T07:03:22.986 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi081:vg_nvme/lv_2 2024-03-22T07:03:23.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:22 smithi149 ceph-mon[36068]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:03:23.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:22 smithi149 ceph-mon[36068]: osdmap e15: 2 total, 2 up, 2 in 2024-03-22T07:03:23.254 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:03:24.284 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:24 smithi081 ceph-mon[29245]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:24.284 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:24.284 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:24.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:24 smithi149 ceph-mon[36068]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:24.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:24.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 -- 172.21.15.81:0/4115637083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fe0d0 msgr2=0x7f5b680fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:03:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 --2- 172.21.15.81:0/4115637083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fe0d0 0x7f5b680fe540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b54004940 tx=0x7f5b54009e30 comp rx=0 tx=0).stop 2024-03-22T07:03:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 -- 172.21.15.81:0/4115637083 shutdown_connections 2024-03-22T07:03:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 --2- 172.21.15.81:0/4115637083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fe0d0 0x7f5b680fe540 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 --2- 172.21.15.81:0/4115637083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b680fcee0 0x7f5b680fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 -- 172.21.15.81:0/4115637083 >> 172.21.15.81:0/4115637083 conn(0x7f5b680f8560 msgr2=0x7f5b680fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:03:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 -- 172.21.15.81:0/4115637083 shutdown_connections 2024-03-22T07:03:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 -- 172.21.15.81:0/4115637083 wait complete. 2024-03-22T07:03:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 Processor -- start 2024-03-22T07:03:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.962+0000 7f5b6f193700 1 -- start start 2024-03-22T07:03:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6f193700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fcee0 0x7f5b6810dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:03:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6f193700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b680fe0d0 0x7f5b6810e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:03:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6f193700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6810e600 con 0x7f5b680fcee0 2024-03-22T07:03:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6f193700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6810ede0 con 0x7f5b680fe0d0 2024-03-22T07:03:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6cf2f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fcee0 0x7f5b6810dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:03:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6cf2f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fcee0 0x7f5b6810dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36686/0 (socket says 172.21.15.81:36686) 2024-03-22T07:03:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6cf2f700 1 -- 172.21.15.81:0/151994264 learned_addr learned my addr 172.21.15.81:0/151994264 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:03:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b67fff700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b680fe0d0 0x7f5b6810e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:03:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6cf2f700 1 -- 172.21.15.81:0/151994264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b680fe0d0 msgr2=0x7f5b6810e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6cf2f700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b680fe0d0 0x7f5b6810e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6cf2f700 1 -- 172.21.15.81:0/151994264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b54005040 con 0x7f5b680fcee0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6cf2f700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fcee0 0x7f5b6810dac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c8b0 tx=0x7f5b5c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b657fa700 1 -- 172.21.15.81:0/151994264 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b5c00a760 con 0x7f5b680fcee0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b657fa700 1 -- 172.21.15.81:0/151994264 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b5c00ad90 con 0x7f5b680fcee0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6810f0c0 con 0x7f5b680fcee0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.963+0000 7f5b657fa700 1 -- 172.21.15.81:0/151994264 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b5c0113a0 con 0x7f5b680fcee0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.964+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6810f600 con 0x7f5b680fcee0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.965+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6805f5f0 con 0x7f5b680fcee0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.965+0000 7f5b657fa700 1 -- 172.21.15.81:0/151994264 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f5b5c011500 con 0x7f5b680fcee0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.965+0000 7f5b657fa700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5b58069210 0x7f5b5806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.965+0000 7f5b657fa700 1 -- 172.21.15.81:0/151994264 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f5b5c0863f0 con 0x7f5b680fcee0 2024-03-22T07:03:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.965+0000 7f5b67fff700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5b58069210 0x7f5b5806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:03:24.968 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.966+0000 7f5b67fff700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5b58069210 0x7f5b5806b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400b3b0 tx=0x7f5b54009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:03:24.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:24.969+0000 7f5b657fa700 1 -- 172.21.15.81:0/151994264 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5b5c056f20 con 0x7f5b680fcee0 2024-03-22T07:03:25.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:25 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:03:25.171 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:25.171+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b6810fa40 con 0x7f5b58069210 2024-03-22T07:03:25.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:25 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:03:26.388 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:26 smithi081 ceph-mon[29245]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:26.388 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:26 smithi081 ceph-mon[29245]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:03:26.388 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:03:26.388 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:03:26.388 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:26 smithi149 ceph-mon[36068]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:26 smithi149 ceph-mon[36068]: from='client.14282 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:03:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:03:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:03:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:28.525 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:28 smithi081 ceph-mon[29245]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:28.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:28 smithi149 ceph-mon[36068]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:29.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: Detected new or changed devices on smithi081 2024-03-22T07:03:29.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:29.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:29.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:29.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:29.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: Adjusting osd_memory_target on smithi081 to 6432M 2024-03-22T07:03:29.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:29.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:29.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:29.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:29.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:03:29.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: Detected new or changed devices on smithi081 2024-03-22T07:03:29.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:29.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:29.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:29.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:29.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: Adjusting osd_memory_target on smithi081 to 6432M 2024-03-22T07:03:29.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:29.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:29.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:29.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:29.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:03:30.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:30 smithi081 ceph-mon[29245]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:30.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:30 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3725935393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba6f9b7e-712d-4f4b-8b33-6e051daf2875"}]: dispatch 2024-03-22T07:03:30.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:30 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3725935393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba6f9b7e-712d-4f4b-8b33-6e051daf2875"}]': finished 2024-03-22T07:03:30.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:30 smithi081 ceph-mon[29245]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T07:03:30.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:30 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:30.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:30 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3110702418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:03:30.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:30 smithi149 ceph-mon[36068]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:30.931 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:30 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3725935393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba6f9b7e-712d-4f4b-8b33-6e051daf2875"}]: dispatch 2024-03-22T07:03:30.931 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:30 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3725935393' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba6f9b7e-712d-4f4b-8b33-6e051daf2875"}]': finished 2024-03-22T07:03:30.932 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:30 smithi149 ceph-mon[36068]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T07:03:30.933 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:30 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:30.933 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:30 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3110702418' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:03:32.456 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:32 smithi081 ceph-mon[29245]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:32.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:32 smithi149 ceph-mon[36068]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:34.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:33 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:34.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:33 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:34.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:33 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:34.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:33 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:34.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:33 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:34.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:33 smithi081 ceph-mon[29245]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:33 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:33 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:33 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:33 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:33 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:33 smithi149 ceph-mon[36068]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:36.524 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:36 smithi081 ceph-mon[29245]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:36.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:36 smithi149 ceph-mon[36068]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:37.311 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:37 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:03:37.311 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:37 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:37.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:37 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:03:37.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:37 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:38.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:38 smithi081 ceph-mon[29245]: Deploying daemon osd.2 on smithi081 2024-03-22T07:03:38.238 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:38 smithi081 ceph-mon[29245]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:38 smithi149 ceph-mon[36068]: Deploying daemon osd.2 on smithi081 2024-03-22T07:03:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:38 smithi149 ceph-mon[36068]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:39.529 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:03:39.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:03:40.468 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:40 smithi081 ceph-mon[29245]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:40.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:40 smithi149 ceph-mon[36068]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:42.476 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:42 smithi081 ceph-mon[29245]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:42.477 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:42.477 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:42.477 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:03:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:42 smithi149 ceph-mon[36068]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:03:44.421 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:44 smithi081 ceph-mon[29245]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:44.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:44 smithi149 ceph-mon[36068]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:45.126 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.125+0000 7f5b657fa700 1 -- 172.21.15.81:0/151994264 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5b6810fa40 con 0x7f5b58069210 2024-03-22T07:03:45.126 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 2 on host 'smithi081' 2024-03-22T07:03:45.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5b58069210 msgr2=0x7f5b5806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5b58069210 0x7f5b5806b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400b3b0 tx=0x7f5b54009e30 comp rx=0 tx=0).stop 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fcee0 msgr2=0x7f5b6810dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fcee0 0x7f5b6810dac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c8b0 tx=0x7f5b5c004470 comp rx=0 tx=0).stop 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 shutdown_connections 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5b58069210 0x7f5b5806b6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5b680fcee0 0x7f5b6810dac0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 --2- 172.21.15.81:0/151994264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b680fe0d0 0x7f5b6810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 >> 172.21.15.81:0/151994264 conn(0x7f5b680f8560 msgr2=0x7f5b680f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 shutdown_connections 2024-03-22T07:03:45.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:03:45.127+0000 7f5b6f193700 1 -- 172.21.15.81:0/151994264 wait complete. 2024-03-22T07:03:45.832 DEBUG:teuthology.orchestra.run.smithi081:osd.2> sudo journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.2.service 2024-03-22T07:03:45.835 INFO:tasks.cephadm:Deploying osd.3 on smithi149 with /dev/vg_nvme/lv_4... 2024-03-22T07:03:45.835 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T07:03:46.044 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:45 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[62641]: 2024-03-22T07:03:45.964+0000 7faa8bc9f540 -1 osd.2 0 log_to_monitors true 2024-03-22T07:03:46.097 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi149/config 2024-03-22T07:03:46.115 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:46 smithi149 ceph-mon[36068]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:46.115 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:46 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:46.116 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:46 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:46.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:46 smithi081 ceph-mon[29245]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:46.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:46 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:46.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:46 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:46.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:46 smithi081 ceph-mon[29245]: from='osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T07:03:46.392 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:46 smithi149 ceph-mon[36068]: from='osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T07:03:47.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:47 smithi149 ceph-mon[36068]: from='osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T07:03:47.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:47 smithi149 ceph-mon[36068]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T07:03:47.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:47 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:47.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:47 smithi149 ceph-mon[36068]: from='osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:03:47.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:47 smithi081 ceph-mon[29245]: from='osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T07:03:47.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:47 smithi081 ceph-mon[29245]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T07:03:47.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:47 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:47.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:47 smithi081 ceph-mon[29245]: from='osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:03:47.488 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:47 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[62641]: 2024-03-22T07:03:47.151+0000 7faa82471700 -1 osd.2 0 waiting for initial osdmap 2024-03-22T07:03:47.488 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:47 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[62641]: 2024-03-22T07:03:47.156+0000 7faa7f270700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:03:47.934 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T07:03:47.934 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T07:03:47.934 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-22T07:03:47.934 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-22T07:03:47.934 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114606 s, 915 MB/s 2024-03-22T07:03:47.934 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-22T07:03:48.294 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:48 smithi149 ceph-mon[36068]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:48.295 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:48 smithi149 ceph-mon[36068]: from='osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-03-22T07:03:48.295 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:48 smithi149 ceph-mon[36068]: osdmap e18: 3 total, 2 up, 3 in 2024-03-22T07:03:48.295 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:48 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:48.295 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:48 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:48.428 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:48 smithi081 ceph-mon[29245]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:48.429 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:48 smithi081 ceph-mon[29245]: from='osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-03-22T07:03:48.429 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:48 smithi081 ceph-mon[29245]: osdmap e18: 3 total, 2 up, 3 in 2024-03-22T07:03:48.429 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:48 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:48.429 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:48 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:48.554 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-03-22T07:03:48.810 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi149/config 2024-03-22T07:03:49.558 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:49 smithi081 ceph-mon[29245]: purged_snaps scrub starts 2024-03-22T07:03:49.558 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:49 smithi081 ceph-mon[29245]: purged_snaps scrub ok 2024-03-22T07:03:49.558 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:49 smithi081 ceph-mon[29245]: osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] boot 2024-03-22T07:03:49.558 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:49 smithi081 ceph-mon[29245]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T07:03:49.558 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:49 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:49.558 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:49 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:49.558 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:49 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:49.558 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:49 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-22T07:03:49.576 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:49 smithi149 ceph-mon[36068]: purged_snaps scrub starts 2024-03-22T07:03:49.576 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:49 smithi149 ceph-mon[36068]: purged_snaps scrub ok 2024-03-22T07:03:49.576 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:49 smithi149 ceph-mon[36068]: osd.2 [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] boot 2024-03-22T07:03:49.576 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:49 smithi149 ceph-mon[36068]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T07:03:49.576 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:49 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:03:49.576 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:49 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:49.577 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:49 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:49.577 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:49 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-22T07:03:50.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/851841946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea00fff00 msgr2=0x7f1ea0102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:03:50.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 --2- 172.21.15.149:0/851841946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea00fff00 0x7f1ea0102320 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1e94004940 tx=0x7f1e94009e30 comp rx=0 tx=0).stop 2024-03-22T07:03:50.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/851841946 shutdown_connections 2024-03-22T07:03:50.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 --2- 172.21.15.149:0/851841946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea00fff00 0x7f1ea0102320 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:50.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 --2- 172.21.15.149:0/851841946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea0066f70 0x7f1ea00ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:50.372 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/851841946 >> 172.21.15.149:0/851841946 conn(0x7f1ea00f7430 msgr2=0x7f1ea00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/851841946 shutdown_connections 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/851841946 wait complete. 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 Processor -- start 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 -- start start 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea0066f70 0x7f1ea010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea00fff00 0x7f1ea010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea010e490 con 0x7f1ea00fff00 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.370+0000 7f1ea4c5a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea010e600 con 0x7f1ea0066f70 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e9e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea0066f70 0x7f1ea010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:03:50.373 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e9e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea0066f70 0x7f1ea010d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:49742/0 (socket says 172.21.15.149:49742) 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e9e59c700 1 -- 172.21.15.149:0/1048185890 learned_addr learned my addr 172.21.15.149:0/1048185890 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e9dd9b700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea00fff00 0x7f1ea010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e9e59c700 1 -- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea00fff00 msgr2=0x7f1ea010de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e9e59c700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea00fff00 0x7f1ea010de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e9e59c700 1 -- 172.21.15.149:0/1048185890 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e94005040 con 0x7f1ea0066f70 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e9e59c700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea0066f70 0x7f1ea010d950 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1e90002a50 tx=0x7f1e90004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e8effd700 1 -- 172.21.15.149:0/1048185890 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e90007cc0 con 0x7f1ea0066f70 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e8effd700 1 -- 172.21.15.149:0/1048185890 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e9000f040 con 0x7f1ea0066f70 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea0113060 con 0x7f1ea0066f70 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1e8effd700 1 -- 172.21.15.149:0/1048185890 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e9000d9c0 con 0x7f1ea0066f70 2024-03-22T07:03:50.374 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.371+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea0113620 con 0x7f1ea0066f70 2024-03-22T07:03:50.375 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.372+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea004c710 con 0x7f1ea0066f70 2024-03-22T07:03:50.376 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.373+0000 7f1e8effd700 1 -- 172.21.15.149:0/1048185890 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f1e90009730 con 0x7f1ea0066f70 2024-03-22T07:03:50.376 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.373+0000 7f1e8effd700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e88069210 0x7f1e8806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:03:50.376 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.373+0000 7f1e8effd700 1 -- 172.21.15.149:0/1048185890 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f1e90086560 con 0x7f1ea0066f70 2024-03-22T07:03:50.376 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.373+0000 7f1e9dd9b700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e88069210 0x7f1e8806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:03:50.376 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.374+0000 7f1e9dd9b700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e88069210 0x7f1e8806b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400b3b0 tx=0x7f1e94009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:03:50.382 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.382+0000 7f1e8effd700 1 -- 172.21.15.149:0/1048185890 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1e90056c40 con 0x7f1ea0066f70 2024-03-22T07:03:50.430 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:50 smithi081 ceph-mon[29245]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:50.430 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-22T07:03:50.431 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:50 smithi081 ceph-mon[29245]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T07:03:50.431 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T07:03:50.571 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:50.571+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ea00f8f70 con 0x7f1e88069210 2024-03-22T07:03:50.572 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:50 smithi149 ceph-mon[36068]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:03:50.572 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:50 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-22T07:03:50.572 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:50 smithi149 ceph-mon[36068]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T07:03:50.572 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:50 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T07:03:51.553 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:51 smithi081 ceph-mon[29245]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:03:51.553 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:51 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:03:51.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:51 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:03:51.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:51 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:51.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:51 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T07:03:51.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:51 smithi081 ceph-mon[29245]: osdmap e21: 3 total, 3 up, 3 in 2024-03-22T07:03:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:51 smithi149 ceph-mon[36068]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:03:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:51 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:03:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:51 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:03:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:51 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:51 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T07:03:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:51 smithi149 ceph-mon[36068]: osdmap e21: 3 total, 3 up, 3 in 2024-03-22T07:03:51.818 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:03:51 smithi081 sudo[66938]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:03:51.819 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:03:51 smithi081 sudo[66938]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:51.819 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:03:51 smithi081 sudo[66938]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:51.819 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:03:51 smithi081 sudo[66938]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:52.237 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:03:51 smithi081 sudo[66978]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:03:52.238 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:03:51 smithi081 sudo[66978]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:52.238 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:03:51 smithi081 sudo[66978]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:52.238 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:03:51 smithi081 sudo[66978]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:52.380 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:03:52.379+0000 7f1e8effd700 1 -- 172.21.15.149:0/1048185890 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1e90053580 con 0x7f1ea0066f70 2024-03-22T07:03:52.511 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67049]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:03:52.511 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67049]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:52.511 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67049]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:52.512 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67049]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: osdmap e22: 3 total, 3 up, 3 in 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:52.512 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:52.513 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.513 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.513 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:03:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:03:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: osdmap e22: 3 total, 3 up, 3 in 2024-03-22T07:03:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:03:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:03:52.766 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67135]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:03:52.766 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67135]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:52.766 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67135]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:52.766 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67135]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:53.026 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67157]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:03:53.026 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67157]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:53.026 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67157]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:53.026 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:52 smithi081 sudo[67157]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:53.288 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67162]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:03:53.288 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67162]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:53.288 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67162]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:53.288 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67162]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:53.544 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67249]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:03:53.544 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67249]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:53.544 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67249]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:53.544 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67249]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:53.808 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:53 smithi149 ceph-mon[36068]: Detected new or changed devices on smithi081 2024-03-22T07:03:53.808 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:53 smithi149 ceph-mon[36068]: Adjusting osd_memory_target on smithi081 to 4288M 2024-03-22T07:03:53.808 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:53 smithi149 ceph-mon[36068]: mgrmap e19: smithi081.bcxarw(active, since 2m), standbys: smithi149.sgmtoa 2024-03-22T07:03:53.808 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:53 smithi149 sudo[39556]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:03:53.808 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:53 smithi149 sudo[39556]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:53.808 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:53 smithi149 sudo[39556]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:53.808 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:53 smithi149 sudo[39556]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:53.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67273]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:03:53.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67273]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:53.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67273]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 sudo[67273]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 ceph-mon[29245]: Detected new or changed devices on smithi081 2024-03-22T07:03:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 ceph-mon[29245]: Adjusting osd_memory_target on smithi081 to 4288M 2024-03-22T07:03:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:53 smithi081 ceph-mon[29245]: mgrmap e19: smithi081.bcxarw(active, since 2m), standbys: smithi149.sgmtoa 2024-03-22T07:03:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 sudo[39559]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:03:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 sudo[39559]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:03:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 sudo[39559]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:03:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 sudo[39559]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:03:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T07:03:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/3835728916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5adfd7b-e190-45b4-b8ca-da7c45d21d93"}]: dispatch 2024-03-22T07:03:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:03:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T07:03:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:03:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:03:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5adfd7b-e190-45b4-b8ca-da7c45d21d93"}]: dispatch 2024-03-22T07:03:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T07:03:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:03:54.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5adfd7b-e190-45b4-b8ca-da7c45d21d93"}]': finished 2024-03-22T07:03:54.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: osdmap e23: 4 total, 3 up, 4 in 2024-03-22T07:03:54.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:03:54.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:03:54.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T07:03:54.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:54.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:03:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T07:03:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='client.? 172.21.15.149:0/3835728916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5adfd7b-e190-45b4-b8ca-da7c45d21d93"}]: dispatch 2024-03-22T07:03:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:03:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T07:03:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:03:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:03:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e5adfd7b-e190-45b4-b8ca-da7c45d21d93"}]: dispatch 2024-03-22T07:03:54.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T07:03:54.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:03:54.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e5adfd7b-e190-45b4-b8ca-da7c45d21d93"}]': finished 2024-03-22T07:03:54.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: osdmap e23: 4 total, 3 up, 4 in 2024-03-22T07:03:54.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:03:54.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:03:54.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T07:03:54.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:54.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:03:55.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:55 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/868351960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:03:55.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:55 smithi149 ceph-mon[36068]: from='client.? 172.21.15.149:0/868351960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:03:56.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:56 smithi081 ceph-mon[29245]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:03:56.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:56.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:56.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:56.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:56.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:56 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:56 smithi149 ceph-mon[36068]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:03:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:56 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:56 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:56 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:03:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:56 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:03:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:56 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:03:58.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:03:58 smithi081 ceph-mon[29245]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.1 KiB/s rd, 249 KiB/s wr, 17 op/s 2024-03-22T07:03:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:03:58 smithi149 ceph-mon[36068]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.1 KiB/s rd, 249 KiB/s wr, 17 op/s 2024-03-22T07:04:00.799 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:00 smithi149 ceph-mon[36068]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.7 KiB/s rd, 180 KiB/s wr, 12 op/s 2024-03-22T07:04:00.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:00 smithi081 ceph-mon[29245]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.7 KiB/s rd, 180 KiB/s wr, 12 op/s 2024-03-22T07:04:01.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:01 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:04:01.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:01 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:01.844 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:01 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:04:01.844 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:01 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:02.775 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:02 smithi149 ceph-mon[36068]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 141 KiB/s wr, 9 op/s 2024-03-22T07:04:02.776 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:02 smithi149 ceph-mon[36068]: Deploying daemon osd.3 on smithi149 2024-03-22T07:04:03.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:02 smithi081 ceph-mon[29245]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 141 KiB/s wr, 9 op/s 2024-03-22T07:04:03.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:02 smithi081 ceph-mon[29245]: Deploying daemon osd.3 on smithi149 2024-03-22T07:04:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:04 smithi149 ceph-mon[36068]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-22T07:04:05.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:04 smithi081 ceph-mon[29245]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-22T07:04:06.688 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:06 smithi149 ceph-mon[36068]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 2.3 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-03-22T07:04:06.688 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:06.688 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:06.688 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:07.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:06 smithi081 ceph-mon[29245]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 2.3 KiB/s rd, 113 KiB/s wr, 7 op/s 2024-03-22T07:04:07.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:07.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:07.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:08.765 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:08 smithi149 ceph-mon[36068]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-22T07:04:09.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:08 smithi081 ceph-mon[29245]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-22T07:04:09.669 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:04:09.742 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.742+0000 7f1e8effd700 1 -- 172.21.15.149:0/1048185890 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1ea00f8f70 con 0x7f1e88069210 2024-03-22T07:04:09.742 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 3 on host 'smithi149' 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e88069210 msgr2=0x7f1e8806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e88069210 0x7f1e8806b6c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400b3b0 tx=0x7f1e94009e30 comp rx=0 tx=0).stop 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea0066f70 msgr2=0x7f1ea010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea0066f70 0x7f1ea010d950 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1e90002a50 tx=0x7f1e90004600 comp rx=0 tx=0).stop 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 shutdown_connections 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e88069210 0x7f1e8806b6c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea0066f70 0x7f1ea010d950 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 --2- 172.21.15.149:0/1048185890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea00fff00 0x7f1ea010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 >> 172.21.15.149:0/1048185890 conn(0x7f1ea00f7430 msgr2=0x7f1ea00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 shutdown_connections 2024-03-22T07:04:09.745 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:09.744+0000 7f1ea4c5a700 1 -- 172.21.15.149:0/1048185890 wait complete. 2024-03-22T07:04:10.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:04:10.353 DEBUG:teuthology.orchestra.run.smithi149:osd.3> sudo journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.3.service 2024-03-22T07:04:10.356 INFO:tasks.cephadm:Deploying osd.4 on smithi149 with /dev/vg_nvme/lv_3... 2024-03-22T07:04:10.356 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T07:04:10.658 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi149/config 2024-03-22T07:04:10.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:10 smithi149 ceph-mon[36068]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:04:10.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:10 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:10.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:10 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:10.991 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:10 smithi149 ceph-mon[36068]: from='osd.3 [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:04:10.992 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:10 smithi149 ceph-mon[36068]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:04:10.992 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:10 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:10.992 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:10 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:10 smithi081 ceph-mon[29245]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:04:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:10 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:10 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:10 smithi081 ceph-mon[29245]: from='osd.3 [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:04:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:10 smithi081 ceph-mon[29245]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:04:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:10 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:10 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:11.939 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:11 smithi149 ceph-mon[36068]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T07:04:11.939 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:11 smithi149 ceph-mon[36068]: osdmap e24: 4 total, 3 up, 4 in 2024-03-22T07:04:11.939 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:11 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:04:11.939 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:11 smithi149 ceph-mon[36068]: from='osd.3 [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:04:11.939 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:11 smithi149 ceph-mon[36068]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:04:11.939 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:04:11 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[41639]: 2024-03-22T07:04:11.716+0000 7fc32b910700 -1 osd.3 0 waiting for initial osdmap 2024-03-22T07:04:11.939 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:04:11 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[41639]: 2024-03-22T07:04:11.722+0000 7fc32670b700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:04:12.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:11 smithi081 ceph-mon[29245]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T07:04:12.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:11 smithi081 ceph-mon[29245]: osdmap e24: 4 total, 3 up, 4 in 2024-03-22T07:04:12.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:11 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:04:12.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:11 smithi081 ceph-mon[29245]: from='osd.3 [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:04:12.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:11 smithi081 ceph-mon[29245]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:04:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:12 smithi149 ceph-mon[36068]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:04:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:12 smithi149 ceph-mon[36068]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-22T07:04:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:12 smithi149 ceph-mon[36068]: osdmap e25: 4 total, 3 up, 4 in 2024-03-22T07:04:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:04:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:12 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:04:13.013 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T07:04:13.013 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T07:04:13.014 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-22T07:04:13.014 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-22T07:04:13.014 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123823 s, 847 MB/s 2024-03-22T07:04:13.014 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-22T07:04:13.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:12 smithi081 ceph-mon[29245]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:04:13.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:12 smithi081 ceph-mon[29245]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-22T07:04:13.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:12 smithi081 ceph-mon[29245]: osdmap e25: 4 total, 3 up, 4 in 2024-03-22T07:04:13.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:04:13.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:12 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:04:13.699 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-03-22T07:04:13.933 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:13 smithi149 ceph-mon[36068]: purged_snaps scrub starts 2024-03-22T07:04:13.934 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:13 smithi149 ceph-mon[36068]: purged_snaps scrub ok 2024-03-22T07:04:13.934 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:13 smithi149 ceph-mon[36068]: osd.3 [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] boot 2024-03-22T07:04:13.934 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:13 smithi149 ceph-mon[36068]: osdmap e26: 4 total, 4 up, 4 in 2024-03-22T07:04:13.934 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:13 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:04:13.934 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:13 smithi149 ceph-mon[36068]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:13.934 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:13 smithi149 ceph-mon[36068]: osdmap e27: 4 total, 4 up, 4 in 2024-03-22T07:04:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:13 smithi081 ceph-mon[29245]: purged_snaps scrub starts 2024-03-22T07:04:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:13 smithi081 ceph-mon[29245]: purged_snaps scrub ok 2024-03-22T07:04:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:13 smithi081 ceph-mon[29245]: osd.3 [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] boot 2024-03-22T07:04:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:13 smithi081 ceph-mon[29245]: osdmap e26: 4 total, 4 up, 4 in 2024-03-22T07:04:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:13 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:04:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:13 smithi081 ceph-mon[29245]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:13 smithi081 ceph-mon[29245]: osdmap e27: 4 total, 4 up, 4 in 2024-03-22T07:04:14.176 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi149/config 2024-03-22T07:04:16.030 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: osdmap e28: 4 total, 4 up, 4 in 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: osdmap e29: 4 total, 4 up, 4 in 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: Detected new or changed devices on smithi149 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: Adjusting osd_memory_target on smithi149 to 4664M 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:04:16.031 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.032 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:16.032 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:16.032 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:04:16.032 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:15 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: osdmap e28: 4 total, 4 up, 4 in 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: osdmap e29: 4 total, 4 up, 4 in 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: Detected new or changed devices on smithi149 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: Adjusting osd_memory_target on smithi149 to 4664M 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:04:16.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:16.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:16.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:04:16.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:15 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:16.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.322+0000 7f507d781700 1 -- 172.21.15.149:0/3701872840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 msgr2=0x7f50780fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:16.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.322+0000 7f507d781700 1 --2- 172.21.15.149:0/3701872840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 0x7f50780fd3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f506c004940 tx=0x7f506c009e30 comp rx=0 tx=0).stop 2024-03-22T07:04:16.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.322+0000 7f507d781700 1 -- 172.21.15.149:0/3701872840 shutdown_connections 2024-03-22T07:04:16.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.322+0000 7f507d781700 1 --2- 172.21.15.149:0/3701872840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 0x7f50780fd3f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:16.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.322+0000 7f507d781700 1 --2- 172.21.15.149:0/3701872840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fbcd0 0x7f50780fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:16.323 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.322+0000 7f507d781700 1 -- 172.21.15.149:0/3701872840 >> 172.21.15.149:0/3701872840 conn(0x7f50780f7430 msgr2=0x7f50780f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:04:16.324 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.322+0000 7f507d781700 1 -- 172.21.15.149:0/3701872840 shutdown_connections 2024-03-22T07:04:16.324 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.322+0000 7f507d781700 1 -- 172.21.15.149:0/3701872840 wait complete. 2024-03-22T07:04:16.324 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f507d781700 1 Processor -- start 2024-03-22T07:04:16.324 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f507d781700 1 -- start start 2024-03-22T07:04:16.324 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f507d781700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fbcd0 0x7f5078066c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:04:16.324 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f507d781700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 0x7f5078114ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:04:16.324 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f507d781700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507810e4c0 con 0x7f50780fbcd0 2024-03-22T07:04:16.324 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f507d781700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078115020 con 0x7f50780fcf80 2024-03-22T07:04:16.325 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f50767fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 0x7f5078114ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:04:16.325 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f50767fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 0x7f5078114ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:42882/0 (socket says 172.21.15.149:42882) 2024-03-22T07:04:16.325 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f50767fc700 1 -- 172.21.15.149:0/4136008029 learned_addr learned my addr 172.21.15.149:0/4136008029 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f5076ffd700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fbcd0 0x7f5078066c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f50767fc700 1 -- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fbcd0 msgr2=0x7f5078066c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f50767fc700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fbcd0 0x7f5078066c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.323+0000 7f50767fc700 1 -- 172.21.15.149:0/4136008029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c005040 con 0x7f50780fcf80 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.324+0000 7f50767fc700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 0x7f5078114ae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f506c009150 tx=0x7f506c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.324+0000 7f505f7fe700 1 -- 172.21.15.149:0/4136008029 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f506c00e040 con 0x7f50780fcf80 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.324+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50781151c0 con 0x7f50780fcf80 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.324+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5078115730 con 0x7f50780fcf80 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.324+0000 7f505f7fe700 1 -- 172.21.15.149:0/4136008029 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f506c00bcc0 con 0x7f50780fcf80 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.324+0000 7f505f7fe700 1 -- 172.21.15.149:0/4136008029 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f506c01c630 con 0x7f50780fcf80 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.325+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507804c710 con 0x7f50780fcf80 2024-03-22T07:04:16.326 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.325+0000 7f505f7fe700 1 -- 172.21.15.149:0/4136008029 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f506c026440 con 0x7f50780fcf80 2024-03-22T07:04:16.327 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.326+0000 7f505f7fe700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:04:16.327 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.326+0000 7f505f7fe700 1 -- 172.21.15.149:0/4136008029 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f506c087860 con 0x7f50780fcf80 2024-03-22T07:04:16.327 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.326+0000 7f5076ffd700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:04:16.327 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.326+0000 7f5076ffd700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5068009fd0 tx=0x7f5068009f60 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:04:16.330 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.330+0000 7f505f7fe700 1 -- 172.21.15.149:0/4136008029 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f506c057cc0 con 0x7f50780fcf80 2024-03-22T07:04:16.535 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:16.534+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5078101260 con 0x7f50600692a0 2024-03-22T07:04:17.210 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:17 smithi149 ceph-mon[36068]: osdmap e30: 4 total, 4 up, 4 in 2024-03-22T07:04:17.211 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:17 smithi149 ceph-mon[36068]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:04:17.211 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:17 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:04:17.211 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:17 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:04:17.211 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:17 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:17.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:17 smithi081 ceph-mon[29245]: osdmap e30: 4 total, 4 up, 4 in 2024-03-22T07:04:17.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:17 smithi081 ceph-mon[29245]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:04:17.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:17 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:04:17.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:17 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:04:17.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:17 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:18.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:18 smithi081 ceph-mon[29245]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T07:04:18.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:18 smithi149 ceph-mon[36068]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T07:04:19.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:19 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/1985215425' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a85bb681-b621-4fbf-b77a-cbd090a41a79"}]: dispatch 2024-03-22T07:04:19.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:19 smithi081 ceph-mon[29245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a85bb681-b621-4fbf-b77a-cbd090a41a79"}]: dispatch 2024-03-22T07:04:19.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:19 smithi081 ceph-mon[29245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a85bb681-b621-4fbf-b77a-cbd090a41a79"}]': finished 2024-03-22T07:04:19.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:19 smithi081 ceph-mon[29245]: osdmap e31: 5 total, 4 up, 5 in 2024-03-22T07:04:19.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:19 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:19.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:19 smithi149 ceph-mon[36068]: from='client.? 172.21.15.149:0/1985215425' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a85bb681-b621-4fbf-b77a-cbd090a41a79"}]: dispatch 2024-03-22T07:04:19.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:19 smithi149 ceph-mon[36068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a85bb681-b621-4fbf-b77a-cbd090a41a79"}]: dispatch 2024-03-22T07:04:19.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:19 smithi149 ceph-mon[36068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a85bb681-b621-4fbf-b77a-cbd090a41a79"}]': finished 2024-03-22T07:04:19.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:19 smithi149 ceph-mon[36068]: osdmap e31: 5 total, 4 up, 5 in 2024-03-22T07:04:19.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:19 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:20.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:20 smithi081 ceph-mon[29245]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-03-22T07:04:20.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:20 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/4091964241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:04:20.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:20 smithi149 ceph-mon[36068]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-03-22T07:04:20.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:20 smithi149 ceph-mon[36068]: from='client.? 172.21.15.149:0/4091964241' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:04:22.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:22 smithi081 ceph-mon[29245]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-22T07:04:22.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:22 smithi149 ceph-mon[36068]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-22T07:04:24.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:24 smithi081 ceph-mon[29245]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-22T07:04:24.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:04:24.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:24 smithi149 ceph-mon[36068]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-22T07:04:24.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:04:26.512 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:26 smithi149 ceph-mon[36068]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-22T07:04:26.513 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:04:26.513 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:26.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:26 smithi081 ceph-mon[29245]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-22T07:04:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:04:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:27.416 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:27 smithi149 ceph-mon[36068]: Deploying daemon osd.4 on smithi149 2024-03-22T07:04:27.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:27 smithi081 ceph-mon[29245]: Deploying daemon osd.4 on smithi149 2024-03-22T07:04:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:28 smithi149 ceph-mon[36068]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:28.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:28 smithi081 ceph-mon[29245]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:30.314 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:30 smithi149 ceph-mon[36068]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:30.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:30 smithi081 ceph-mon[29245]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:32.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:32 smithi149 ceph-mon[36068]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:32.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:32 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:32.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:32 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:32.283 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:32 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:32.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:32 smithi081 ceph-mon[29245]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:32.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:32 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:32.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:32 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:32.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:32 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:34 smithi081 ceph-mon[29245]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:34.593 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:34 smithi149 ceph-mon[36068]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:34.650 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.649+0000 7f505f7fe700 1 -- 172.21.15.149:0/4136008029 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5078101260 con 0x7f50600692a0 2024-03-22T07:04:34.650 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 4 on host 'smithi149' 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 msgr2=0x7f506006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5068009fd0 tx=0x7f5068009f60 comp rx=0 tx=0).stop 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 msgr2=0x7f5078114ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 0x7f5078114ae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f506c009150 tx=0x7f506c009e30 comp rx=0 tx=0).stop 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 shutdown_connections 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fbcd0 0x7f5078066c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 --2- 172.21.15.149:0/4136008029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fcf80 0x7f5078114ae0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 >> 172.21.15.149:0/4136008029 conn(0x7f50780f7430 msgr2=0x7f50780f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 shutdown_connections 2024-03-22T07:04:34.653 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:34.652+0000 7f507d781700 1 -- 172.21.15.149:0/4136008029 wait complete. 2024-03-22T07:04:35.715 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:35 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:35.715 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:35 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:35.716 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:35 smithi149 ceph-mon[36068]: from='osd.4 [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:04:35.716 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:35 smithi149 ceph-mon[36068]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:04:35.739 DEBUG:teuthology.orchestra.run.smithi149:osd.4> sudo journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.4.service 2024-03-22T07:04:35.741 INFO:tasks.cephadm:Deploying osd.5 on smithi149 with /dev/vg_nvme/lv_2... 2024-03-22T07:04:35.742 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T07:04:36.044 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi149/config 2024-03-22T07:04:36.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:35 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:36.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:35 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:36.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:35 smithi081 ceph-mon[29245]: from='osd.4 [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:04:36.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:35 smithi081 ceph-mon[29245]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:04:36.830 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-mon[36068]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:36.830 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-mon[36068]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T07:04:36.830 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-mon[36068]: osdmap e32: 5 total, 4 up, 5 in 2024-03-22T07:04:36.830 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:36.830 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-mon[36068]: from='osd.4 [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:04:36.830 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-mon[36068]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:04:36.830 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:36.830 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:36.831 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[48471]: 2024-03-22T07:04:36.651+0000 7fc603eb1700 -1 osd.4 0 waiting for initial osdmap 2024-03-22T07:04:36.831 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:04:36 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[48471]: 2024-03-22T07:04:36.658+0000 7fc5fe4ab700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:04:37.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:36 smithi081 ceph-mon[29245]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:36 smithi081 ceph-mon[29245]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T07:04:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:36 smithi081 ceph-mon[29245]: osdmap e32: 5 total, 4 up, 5 in 2024-03-22T07:04:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:36 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:36 smithi081 ceph-mon[29245]: from='osd.4 [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:04:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:36 smithi081 ceph-mon[29245]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:04:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:36 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:36 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:37.946 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:37 smithi149 ceph-mon[36068]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-22T07:04:37.946 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:37 smithi149 ceph-mon[36068]: osdmap e33: 5 total, 4 up, 5 in 2024-03-22T07:04:37.947 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:37 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:37.947 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:37 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:37.947 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:37 smithi149 ceph-mon[36068]: osd.4 [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] boot 2024-03-22T07:04:37.947 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:37 smithi149 ceph-mon[36068]: osdmap e34: 5 total, 5 up, 5 in 2024-03-22T07:04:37.947 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:37 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:37 smithi081 ceph-mon[29245]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-22T07:04:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:37 smithi081 ceph-mon[29245]: osdmap e33: 5 total, 4 up, 5 in 2024-03-22T07:04:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:37 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:37 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:37 smithi081 ceph-mon[29245]: osd.4 [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] boot 2024-03-22T07:04:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:37 smithi081 ceph-mon[29245]: osdmap e34: 5 total, 5 up, 5 in 2024-03-22T07:04:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:37 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:04:38.248 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T07:04:38.248 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T07:04:38.248 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-22T07:04:38.248 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-22T07:04:38.248 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125398 s, 836 MB/s 2024-03-22T07:04:38.248 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-22T07:04:38.827 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:38 smithi149 ceph-mon[36068]: purged_snaps scrub starts 2024-03-22T07:04:38.827 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:38 smithi149 ceph-mon[36068]: purged_snaps scrub ok 2024-03-22T07:04:38.827 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:38 smithi149 ceph-mon[36068]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:38.852 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-03-22T07:04:39.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:38 smithi081 ceph-mon[29245]: purged_snaps scrub starts 2024-03-22T07:04:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:38 smithi081 ceph-mon[29245]: purged_snaps scrub ok 2024-03-22T07:04:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:38 smithi081 ceph-mon[29245]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:04:39.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:39 smithi149 ceph-mon[36068]: osdmap e35: 5 total, 5 up, 5 in 2024-03-22T07:04:39.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:04:39.872 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi149/config 2024-03-22T07:04:40.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:39 smithi081 ceph-mon[29245]: osdmap e35: 5 total, 5 up, 5 in 2024-03-22T07:04:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:04:41.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:40 smithi081 ceph-mon[29245]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:41.081 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:40 smithi149 ceph-mon[36068]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:42.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.030+0000 7fca12226700 1 -- 172.21.15.149:0/540269334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fed80 msgr2=0x7fca0c0faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:42.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.030+0000 7fca12226700 1 --2- 172.21.15.149:0/540269334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fed80 0x7fca0c0faec0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fca00004940 tx=0x7fca00009e30 comp rx=0 tx=0).stop 2024-03-22T07:04:42.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.030+0000 7fca12226700 1 -- 172.21.15.149:0/540269334 shutdown_connections 2024-03-22T07:04:42.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.030+0000 7fca12226700 1 --2- 172.21.15.149:0/540269334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fed80 0x7fca0c0faec0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:42.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.030+0000 7fca12226700 1 --2- 172.21.15.149:0/540269334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca0c0fe430 0x7fca0c0fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:42.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.030+0000 7fca12226700 1 -- 172.21.15.149:0/540269334 >> 172.21.15.149:0/540269334 conn(0x7fca0c0f6bd0 msgr2=0x7fca0c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:04:42.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.030+0000 7fca12226700 1 -- 172.21.15.149:0/540269334 shutdown_connections 2024-03-22T07:04:42.031 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.030+0000 7fca12226700 1 -- 172.21.15.149:0/540269334 wait complete. 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.031+0000 7fca12226700 1 Processor -- start 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.031+0000 7fca12226700 1 -- start start 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.031+0000 7fca12226700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fe430 0x7fca0c0fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.031+0000 7fca12226700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca0c0fed80 0x7fca0c0fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.031+0000 7fca12226700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c0fe040 con 0x7fca0c0fed80 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.031+0000 7fca12226700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c0fc630 con 0x7fca0c0fe430 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.031+0000 7fca0b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fe430 0x7fca0c0fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.031+0000 7fca0b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fe430 0x7fca0c0fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:51000/0 (socket says 172.21.15.149:51000) 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.031+0000 7fca0b7fe700 1 -- 172.21.15.149:0/3555232389 learned_addr learned my addr 172.21.15.149:0/3555232389 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fca0affd700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca0c0fed80 0x7fca0c0fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fca0b7fe700 1 -- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca0c0fed80 msgr2=0x7fca0c0fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:42.033 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fca0b7fe700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca0c0fed80 0x7fca0c0fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:42.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fca0b7fe700 1 -- 172.21.15.149:0/3555232389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca00005040 con 0x7fca0c0fe430 2024-03-22T07:04:42.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fca0b7fe700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fe430 0x7fca0c0fda40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc002a80 tx=0x7fc9fc0046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:04:42.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fc9f3fff700 1 -- 172.21.15.149:0/3555232389 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9fc007cc0 con 0x7fca0c0fe430 2024-03-22T07:04:42.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fc9f3fff700 1 -- 172.21.15.149:0/3555232389 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9fc00f040 con 0x7fca0c0fe430 2024-03-22T07:04:42.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c0fc8e0 con 0x7fca0c0fe430 2024-03-22T07:04:42.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fc9f3fff700 1 -- 172.21.15.149:0/3555232389 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9fc0096f0 con 0x7fca0c0fe430 2024-03-22T07:04:42.034 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.032+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c0fcea0 con 0x7fca0c0fe430 2024-03-22T07:04:42.035 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.033+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c04c710 con 0x7fca0c0fe430 2024-03-22T07:04:42.035 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.034+0000 7fc9f3fff700 1 -- 172.21.15.149:0/3555232389 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc9fc00d690 con 0x7fca0c0fe430 2024-03-22T07:04:42.035 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.034+0000 7fc9f3fff700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc9f40692a0 0x7fc9f406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:04:42.035 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.034+0000 7fc9f3fff700 1 -- 172.21.15.149:0/3555232389 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7fc9fc086420 con 0x7fca0c0fe430 2024-03-22T07:04:42.035 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.034+0000 7fca0affd700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc9f40692a0 0x7fc9f406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:04:42.035 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.035+0000 7fca0affd700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc9f40692a0 0x7fc9f406b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca00000c00 tx=0x7fca00009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:04:42.038 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.038+0000 7fc9f3fff700 1 -- 172.21.15.149:0/3555232389 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc9fc04d190 con 0x7fca0c0fe430 2024-03-22T07:04:42.237 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:42.237+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fca0c102480 con 0x7fc9f40692a0 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: Detected new or changed devices on smithi149 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: Adjusting osd_memory_target on smithi149 to 2332M 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:04:42.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:42.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:42.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:42.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:04:42.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: Detected new or changed devices on smithi149 2024-03-22T07:04:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:04:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:04:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: Adjusting osd_memory_target on smithi149 to 2332M 2024-03-22T07:04:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:04:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:04:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:43.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:43 smithi149 ceph-mon[36068]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:04:43.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:43 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:04:43.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:43 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:04:43.278 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:43 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:43.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:43 smithi081 ceph-mon[29245]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:04:43.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:43 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:04:43.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:43 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:04:43.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:43 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:44 smithi081 ceph-mon[29245]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:44.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:44 smithi149 ceph-mon[36068]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:45 smithi149 ceph-mon[36068]: from='client.? 172.21.15.149:0/2853677497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e83552d-4e6c-4426-9b7c-99d9ad5c45ff"}]: dispatch 2024-03-22T07:04:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:45 smithi149 ceph-mon[36068]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e83552d-4e6c-4426-9b7c-99d9ad5c45ff"}]: dispatch 2024-03-22T07:04:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:45 smithi149 ceph-mon[36068]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e83552d-4e6c-4426-9b7c-99d9ad5c45ff"}]': finished 2024-03-22T07:04:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:45 smithi149 ceph-mon[36068]: osdmap e36: 6 total, 5 up, 6 in 2024-03-22T07:04:45.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:45 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:04:45.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:45 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/2853677497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e83552d-4e6c-4426-9b7c-99d9ad5c45ff"}]: dispatch 2024-03-22T07:04:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:45 smithi081 ceph-mon[29245]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e83552d-4e6c-4426-9b7c-99d9ad5c45ff"}]: dispatch 2024-03-22T07:04:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:45 smithi081 ceph-mon[29245]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e83552d-4e6c-4426-9b7c-99d9ad5c45ff"}]': finished 2024-03-22T07:04:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:45 smithi081 ceph-mon[29245]: osdmap e36: 6 total, 5 up, 6 in 2024-03-22T07:04:45.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:45 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:04:46.538 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:46 smithi149 ceph-mon[36068]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:46.539 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:46 smithi149 ceph-mon[36068]: from='client.? 172.21.15.149:0/2296033915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:04:46.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:46 smithi081 ceph-mon[29245]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:46.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:46 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/2296033915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:04:48.525 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:48 smithi149 ceph-mon[36068]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:48.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:48 smithi081 ceph-mon[29245]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:50.466 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:50 smithi149 ceph-mon[36068]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:50.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:50 smithi081 ceph-mon[29245]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:52.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:52 smithi149 ceph-mon[36068]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:52.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:04:52.451 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:52 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:52.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:52 smithi081 ceph-mon[29245]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:52.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:04:52.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:52 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:04:53.460 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:53 smithi149 ceph-mon[36068]: Deploying daemon osd.5 on smithi149 2024-03-22T07:04:53.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:53 smithi081 ceph-mon[29245]: Deploying daemon osd.5 on smithi149 2024-03-22T07:04:54.419 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:54 smithi149 ceph-mon[36068]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:54.419 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:04:54.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:54 smithi081 ceph-mon[29245]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:04:56.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:56 smithi149 ceph-mon[36068]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:56.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:56 smithi081 ceph-mon[29245]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:58.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:57 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:58.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:57 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:58.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:57 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:58.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:04:57 smithi081 ceph-mon[29245]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:58.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:57 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:58.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:57 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:04:58.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:57 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:04:58.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:04:57 smithi149 ceph-mon[36068]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:04:59.855 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.854+0000 7fc9f3fff700 1 -- 172.21.15.149:0/3555232389 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fca0c102480 con 0x7fc9f40692a0 2024-03-22T07:04:59.855 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 5 on host 'smithi149' 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.856+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc9f40692a0 msgr2=0x7fc9f406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.856+0000 7fca12226700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc9f40692a0 0x7fc9f406b750 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca00000c00 tx=0x7fca00009e30 comp rx=0 tx=0).stop 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.856+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fe430 msgr2=0x7fca0c0fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.856+0000 7fca12226700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fe430 0x7fca0c0fda40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc002a80 tx=0x7fc9fc0046b0 comp rx=0 tx=0).stop 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.856+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 shutdown_connections 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.856+0000 7fca12226700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc9f40692a0 0x7fc9f406b750 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.856+0000 7fca12226700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca0c0fe430 0x7fca0c0fda40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.856+0000 7fca12226700 1 --2- 172.21.15.149:0/3555232389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca0c0fed80 0x7fca0c0fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.857+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 >> 172.21.15.149:0/3555232389 conn(0x7fca0c0f6bd0 msgr2=0x7fca0c0f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.857+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 shutdown_connections 2024-03-22T07:04:59.857 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:04:59.857+0000 7fca12226700 1 -- 172.21.15.149:0/3555232389 wait complete. 2024-03-22T07:05:00.442 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:00 smithi149 ceph-mon[36068]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:05:00.442 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:00 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:00.442 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:00 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:00.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:00 smithi081 ceph-mon[29245]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:05:00.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:00 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:00.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:00 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:00.759 DEBUG:teuthology.orchestra.run.smithi149:osd.5> sudo journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.5.service 2024-03-22T07:05:00.762 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-22T07:05:00.762 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-22T07:05:01.004 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:05:00 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[55376]: 2024-03-22T07:05:00.826+0000 7fe92e6f1540 -1 osd.5 0 log_to_monitors true 2024-03-22T07:05:01.032 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:01 smithi149 ceph-mon[36068]: from='osd.5 [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:05:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:01 smithi149 ceph-mon[36068]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:05:01.456 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:01 smithi081 ceph-mon[29245]: from='osd.5 [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:05:01.456 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:01 smithi081 ceph-mon[29245]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:05:02.473 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:02 smithi081 ceph-mon[29245]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:05:02.473 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:02 smithi081 ceph-mon[29245]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T07:05:02.474 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:02 smithi081 ceph-mon[29245]: osdmap e37: 6 total, 5 up, 6 in 2024-03-22T07:05:02.474 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:02 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:05:02.474 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:02 smithi081 ceph-mon[29245]: from='osd.5 [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:05:02.474 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:02 smithi081 ceph-mon[29245]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:05:02.474 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:02 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:02.474 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:02 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:02.505 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[55376]: 2024-03-22T07:05:02.158+0000 7fe9266c6700 -1 osd.5 0 waiting for initial osdmap 2024-03-22T07:05:02.506 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[55376]: 2024-03-22T07:05:02.168+0000 7fe9214c1700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:05:02.506 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-mon[36068]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:05:02.506 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-mon[36068]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T07:05:02.506 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-mon[36068]: osdmap e37: 6 total, 5 up, 6 in 2024-03-22T07:05:02.506 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:05:02.506 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-mon[36068]: from='osd.5 [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:05:02.506 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-mon[36068]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:05:02.506 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:02.506 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:02 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:02.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.660+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2451704427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fcf90 msgr2=0x7fb1dc0fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:02.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.660+0000 7fb1e2c40700 1 --2- 172.21.15.81:0/2451704427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fcf90 0x7fb1dc0fd400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc004940 tx=0x7fb1cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:02.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.660+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2451704427 shutdown_connections 2024-03-22T07:05:02.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.660+0000 7fb1e2c40700 1 --2- 172.21.15.81:0/2451704427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fcf90 0x7fb1dc0fd400 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:02.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.660+0000 7fb1e2c40700 1 --2- 172.21.15.81:0/2451704427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1dc0fbce0 0x7fb1dc0fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:02.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.660+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2451704427 >> 172.21.15.81:0/2451704427 conn(0x7fb1dc0f7440 msgr2=0x7fb1dc0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:02.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.660+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2451704427 shutdown_connections 2024-03-22T07:05:02.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.660+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2451704427 wait complete. 2024-03-22T07:05:02.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e2c40700 1 Processor -- start 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e2c40700 1 -- start start 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e2c40700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fbce0 0x7fb1dc10c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e2c40700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1dc0fcf90 0x7fb1dc10ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e2c40700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc10d470 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e2c40700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc10dcc0 con 0x7fb1dc0fcf90 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e09dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fbce0 0x7fb1dc10c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e09dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fbce0 0x7fb1dc10c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40142/0 (socket says 172.21.15.81:40142) 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e09dc700 1 -- 172.21.15.81:0/2627464192 learned_addr learned my addr 172.21.15.81:0/2627464192 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1dbfff700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1dc0fcf90 0x7fb1dc10ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e09dc700 1 -- 172.21.15.81:0/2627464192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1dc0fcf90 msgr2=0x7fb1dc10ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e09dc700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1dc0fcf90 0x7fb1dc10ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.661+0000 7fb1e09dc700 1 -- 172.21.15.81:0/2627464192 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc005040 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.662+0000 7fb1e09dc700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fbce0 0x7fb1dc10c930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4001d80 tx=0x7fb1d4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:02.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.662+0000 7fb1d97fa700 1 -- 172.21.15.81:0/2627464192 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d4007d10 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.662+0000 7fb1d97fa700 1 -- 172.21.15.81:0/2627464192 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1d4001e80 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.662+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1dc10dfa0 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.662+0000 7fb1d97fa700 1 -- 172.21.15.81:0/2627464192 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d4014640 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.662+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc10e4e0 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.663+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc05f5f0 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.663+0000 7fb1d97fa700 1 -- 172.21.15.81:0/2627464192 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb1d400e960 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.664+0000 7fb1d97fa700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb1c80692a0 0x7fb1c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:02.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.664+0000 7fb1dbfff700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb1c80692a0 0x7fb1c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:02.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.664+0000 7fb1d97fa700 1 -- 172.21.15.81:0/2627464192 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7fb1d4086ab0 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.664+0000 7fb1dbfff700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb1c80692a0 0x7fb1c806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00b970 tx=0x7fb1cc009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:02.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.668+0000 7fb1d97fa700 1 -- 172.21.15.81:0/2627464192 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb1d404e190 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.855+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb1dc04bed0 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.856+0000 7fb1d97fa700 1 -- 172.21.15.81:0/2627464192 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7fb1d4056cf0 con 0x7fb1dc0fbce0 2024-03-22T07:05:02.856 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:05:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb1c80692a0 msgr2=0x7fb1c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb1c80692a0 0x7fb1c806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00b970 tx=0x7fb1cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fbce0 msgr2=0x7fb1dc10c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fbce0 0x7fb1dc10c930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4001d80 tx=0x7fb1d4009ec0 comp rx=0 tx=0).stop 2024-03-22T07:05:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 shutdown_connections 2024-03-22T07:05:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb1c80692a0 0x7fb1c806b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1dc0fbce0 0x7fb1dc10c930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 --2- 172.21.15.81:0/2627464192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1dc0fcf90 0x7fb1dc10ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:02.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 >> 172.21.15.81:0/2627464192 conn(0x7fb1dc0f7440 msgr2=0x7fb1dc0f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:02.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 shutdown_connections 2024-03-22T07:05:02.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:02.858+0000 7fb1e2c40700 1 -- 172.21.15.81:0/2627464192 wait complete. 2024-03-22T07:05:03.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:03 smithi081 ceph-mon[29245]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-22T07:05:03.417 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:03 smithi081 ceph-mon[29245]: osdmap e38: 6 total, 5 up, 6 in 2024-03-22T07:05:03.417 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:03 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:05:03.417 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:03 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:05:03.417 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:03 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2627464192' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T07:05:03.440 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1711091077,"num_in_osds":6,"osd_in_since":1711091084,"num_remapped_pgs":0} 2024-03-22T07:05:03.516 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:03 smithi149 ceph-mon[36068]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-22T07:05:03.516 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:03 smithi149 ceph-mon[36068]: osdmap e38: 6 total, 5 up, 6 in 2024-03-22T07:05:03.516 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:03 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:05:03.516 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:03 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:05:03.516 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:03 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2627464192' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T07:05:04.441 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-22T07:05:04.467 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:04 smithi081 ceph-mon[29245]: purged_snaps scrub starts 2024-03-22T07:05:04.468 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:04 smithi081 ceph-mon[29245]: purged_snaps scrub ok 2024-03-22T07:05:04.468 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:04 smithi081 ceph-mon[29245]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:05:04.468 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:04 smithi081 ceph-mon[29245]: osd.5 [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] boot 2024-03-22T07:05:04.468 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:04 smithi081 ceph-mon[29245]: osdmap e39: 6 total, 6 up, 6 in 2024-03-22T07:05:04.468 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:05:04.540 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:04 smithi149 ceph-mon[36068]: purged_snaps scrub starts 2024-03-22T07:05:04.540 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:04 smithi149 ceph-mon[36068]: purged_snaps scrub ok 2024-03-22T07:05:04.540 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:04 smithi149 ceph-mon[36068]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:05:04.540 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:04 smithi149 ceph-mon[36068]: osd.5 [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] boot 2024-03-22T07:05:04.540 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:04 smithi149 ceph-mon[36068]: osdmap e39: 6 total, 6 up, 6 in 2024-03-22T07:05:04.540 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:05:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:05 smithi081 ceph-mon[29245]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T07:05:05.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:05 smithi149 ceph-mon[36068]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T07:05:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 -- 172.21.15.81:0/152458251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 msgr2=0x7fe64c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 --2- 172.21.15.81:0/152458251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 0x7fe64c0fd280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe63c004940 tx=0x7fe63c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 -- 172.21.15.81:0/152458251 shutdown_connections 2024-03-22T07:05:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 --2- 172.21.15.81:0/152458251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64c0fdfe0 0x7fe64c0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 --2- 172.21.15.81:0/152458251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 0x7fe64c0fd280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 -- 172.21.15.81:0/152458251 >> 172.21.15.81:0/152458251 conn(0x7fe64c0f8550 msgr2=0x7fe64c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 -- 172.21.15.81:0/152458251 shutdown_connections 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 -- 172.21.15.81:0/152458251 wait complete. 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 Processor -- start 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 -- start start 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 0x7fe64c10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64c0fdfe0 0x7fe64c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c10e510 con 0x7fe64c0fce70 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.356+0000 7fe6513d9700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64c112eb0 con 0x7fe64c0fdfe0 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe64affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 0x7fe64c10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe64affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 0x7fe64c10d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55994/0 (socket says 172.21.15.81:55994) 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe64affd700 1 -- 172.21.15.81:0/145381106 learned_addr learned my addr 172.21.15.81:0/145381106 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe6427fc700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64c0fdfe0 0x7fe64c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe64affd700 1 -- 172.21.15.81:0/145381106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64c0fdfe0 msgr2=0x7fe64c10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe64affd700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64c0fdfe0 0x7fe64c10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe64affd700 1 -- 172.21.15.81:0/145381106 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c005040 con 0x7fe64c0fce70 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe6427fc700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64c0fdfe0 0x7fe64c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:05:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe64affd700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 0x7fe64c10d940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00b3b0 tx=0x7fe63c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:06.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe643fff700 1 -- 172.21.15.81:0/145381106 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe63c00e040 con 0x7fe64c0fce70 2024-03-22T07:05:06.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe643fff700 1 -- 172.21.15.81:0/145381106 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe63c00bcc0 con 0x7fe64c0fce70 2024-03-22T07:05:06.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe643fff700 1 -- 172.21.15.81:0/145381106 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe63c01c630 con 0x7fe64c0fce70 2024-03-22T07:05:06.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64c1130b0 con 0x7fe64c0fce70 2024-03-22T07:05:06.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.357+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64c113620 con 0x7fe64c0fce70 2024-03-22T07:05:06.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.358+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64c04c710 con 0x7fe64c0fce70 2024-03-22T07:05:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.359+0000 7fe643fff700 1 -- 172.21.15.81:0/145381106 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe63c013980 con 0x7fe64c0fce70 2024-03-22T07:05:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.359+0000 7fe643fff700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6380692a0 0x7fe63806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.359+0000 7fe643fff700 1 -- 172.21.15.81:0/145381106 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe63c087c70 con 0x7fe64c0fce70 2024-03-22T07:05:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.359+0000 7fe6427fc700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6380692a0 0x7fe63806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.359+0000 7fe6427fc700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6380692a0 0x7fe63806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe634009b10 tx=0x7fe634008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:06.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.364+0000 7fe643fff700 1 -- 172.21.15.81:0/145381106 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe63c057cf0 con 0x7fe64c0fce70 2024-03-22T07:05:06.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.548+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fe64c113a20 con 0x7fe64c0fce70 2024-03-22T07:05:06.548 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:06.548 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: Detected new or changed devices on smithi149 2024-03-22T07:05:06.548 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: Adjusting osd_memory_target on smithi149 to 1554M 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:05:06.549 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:06 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:06.550 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.548+0000 7fe643fff700 1 -- 172.21.15.81:0/145381106 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7fe63c057880 con 0x7fe64c0fce70 2024-03-22T07:05:06.550 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6380692a0 msgr2=0x7fe63806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6380692a0 0x7fe63806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe634009b10 tx=0x7fe634008040 comp rx=0 tx=0).stop 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 msgr2=0x7fe64c10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 0x7fe64c10d940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00b3b0 tx=0x7fe63c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 shutdown_connections 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6380692a0 0x7fe63806b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe64c0fce70 0x7fe64c10d940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 --2- 172.21.15.81:0/145381106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe64c0fdfe0 0x7fe64c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 >> 172.21.15.81:0/145381106 conn(0x7fe64c0f8550 msgr2=0x7fe64c0f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 shutdown_connections 2024-03-22T07:05:06.551 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:06.550+0000 7fe6513d9700 1 -- 172.21.15.81:0/145381106 wait complete. 2024-03-22T07:05:06.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:06.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: Detected new or changed devices on smithi149 2024-03-22T07:05:06.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:06.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:06.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: Adjusting osd_memory_target on smithi149 to 1554M 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:05:06.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:06 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:05:07.296 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1711091103,"num_in_osds":6,"osd_in_since":1711091084,"num_remapped_pgs":0} 2024-03-22T07:05:07.297 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T07:05:07.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:07 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/145381106' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T07:05:07.593 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:07.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:07 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/145381106' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T07:05:08.555 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:08 smithi081 ceph-mon[29245]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:08.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:08 smithi149 ceph-mon[36068]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 -- 172.21.15.81:0/3528600509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 msgr2=0x7f3c40103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 --2- 172.21.15.81:0/3528600509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 0x7f3c40103320 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3c30004940 tx=0x7f3c30009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 -- 172.21.15.81:0/3528600509 shutdown_connections 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 --2- 172.21.15.81:0/3528600509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c400fb4d0 0x7f3c400fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 --2- 172.21.15.81:0/3528600509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 0x7f3c40103320 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 -- 172.21.15.81:0/3528600509 >> 172.21.15.81:0/3528600509 conn(0x7f3c400714c0 msgr2=0x7f3c400718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 -- 172.21.15.81:0/3528600509 shutdown_connections 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 -- 172.21.15.81:0/3528600509 wait complete. 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 Processor -- start 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.013+0000 7f3c465d2700 1 -- start start 2024-03-22T07:05:09.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c465d2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c400fb4d0 0x7f3c400fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c465d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 0x7f3c400fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c465d2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400fe640 con 0x7f3c40102f50 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c465d2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400fd130 con 0x7f3c400fb4d0 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 0x7f3c400fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 0x7f3c400fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56012/0 (socket says 172.21.15.81:56012) 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3f7fe700 1 -- 172.21.15.81:0/930374486 learned_addr learned my addr 172.21.15.81:0/930374486 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3ffff700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c400fb4d0 0x7f3c400fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3f7fe700 1 -- 172.21.15.81:0/930374486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c400fb4d0 msgr2=0x7f3c400fc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3f7fe700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c400fb4d0 0x7f3c400fc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3f7fe700 1 -- 172.21.15.81:0/930374486 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c30005040 con 0x7f3c40102f50 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3f7fe700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 0x7f3c400fcbf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400b790 tx=0x7f3c34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3cff9700 1 -- 172.21.15.81:0/930374486 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c3400f6f0 con 0x7f3c40102f50 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3cff9700 1 -- 172.21.15.81:0/930374486 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c3400fd20 con 0x7f3c40102f50 2024-03-22T07:05:09.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c3cff9700 1 -- 172.21.15.81:0/930374486 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c340103a0 con 0x7f3c40102f50 2024-03-22T07:05:09.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c400fd410 con 0x7f3c40102f50 2024-03-22T07:05:09.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.014+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4011a220 con 0x7f3c40102f50 2024-03-22T07:05:09.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.016+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c40100200 con 0x7f3c40102f50 2024-03-22T07:05:09.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.016+0000 7f3c3cff9700 1 -- 172.21.15.81:0/930374486 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3c34011400 con 0x7f3c40102f50 2024-03-22T07:05:09.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.016+0000 7f3c3cff9700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3c280692a0 0x7f3c2806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:09.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.016+0000 7f3c3cff9700 1 -- 172.21.15.81:0/930374486 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3c34086b50 con 0x7f3c40102f50 2024-03-22T07:05:09.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.016+0000 7f3c3ffff700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3c280692a0 0x7f3c2806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:09.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.017+0000 7f3c3ffff700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3c280692a0 0x7f3c2806b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c30004890 tx=0x7f3c30009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:09.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.021+0000 7f3c3cff9700 1 -- 172.21.15.81:0/930374486 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3c34053260 con 0x7f3c40102f50 2024-03-22T07:05:09.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.213+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c400fe780 con 0x7f3c40102f50 2024-03-22T07:05:09.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.213+0000 7f3c3cff9700 1 -- 172.21.15.81:0/930374486 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11070 (secure 0 0 0) 0x7f3c34056880 con 0x7f3c40102f50 2024-03-22T07:05:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:05:09.214 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":40,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","created":"2024-03-22T06:59:35.415160+0000","modified":"2024-03-22T07:05:04.161841+0000","last_up_change":"2024-03-22T07:05:03.156764+0000","last_in_change":"2024-03-22T07:04:44.637120+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:03:49.371549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"69addaec-0295-4219-9e22-5ebf831a65d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3909132761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3909132761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6809","nonce":3909132761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3909132761}]},"public_addr":"172.21.15.81:6803/3909132761","cluster_addr":"172.21.15.81:6805/3909132761","heartbeat_back_addr":"172.21.15.81:6809/3909132761","heartbeat_front_addr":"172.21.15.81:6807/3909132761","state":["exists","up"]},{"osd":1,"uuid":"cd572358-1fd5-4e96-8caf-3d00d6e0608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6811","nonce":3867177766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6813","nonce":3867177766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6817","nonce":3867177766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6815","nonce":3867177766}]},"public_addr":"172.21.15.81:6811/3867177766","cluster_addr":"172.21.15.81:6813/3867177766","heartbeat_back_addr":"172.21.15.81:6817/3867177766","heartbeat_front_addr":"172.21.15.81:6815/3867177766","state":["exists","up"]},{"osd":2,"uuid":"ba6f9b7e-712d-4f4b-8b33-6e051daf2875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6819","nonce":181541459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6821","nonce":181541459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6825","nonce":181541459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6823","nonce":181541459}]},"public_addr":"172.21.15.81:6819/181541459","cluster_addr":"172.21.15.81:6821/181541459","heartbeat_back_addr":"172.21.15.81:6825/181541459","heartbeat_front_addr":"172.21.15.81:6823/181541459","state":["exists","up"]},{"osd":3,"uuid":"e5adfd7b-e190-45b4-b8ca-da7c45d21d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6801","nonce":868260375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6803","nonce":868260375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6807","nonce":868260375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6805","nonce":868260375}]},"public_addr":"172.21.15.149:6801/868260375","cluster_addr":"172.21.15.149:6803/868260375","heartbeat_back_addr":"172.21.15.149:6807/868260375","heartbeat_front_addr":"172.21.15.149:6805/868260375","state":["exists","up"]},{"osd":4,"uuid":"a85bb681-b621-4fbf-b77a-cbd090a41a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6809","nonce":1702132597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6811","nonce":1702132597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6815","nonce":1702132597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6813","nonce":1702132597}]},"public_addr":"172.21.15.149:6809/1702132597","cluster_addr":"172.21.15.149:6811/1702132597","heartbeat_back_addr":"172.21.15.149:6815/1702132597","heartbeat_front_addr":"172.21.15.149:6813/1702132597","state":["exists","up"]},{"osd":5,"uuid":"0e83552d-4e6c-4426-9b7c-99d9ad5c45ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6817","nonce":3036205309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6819","nonce":3036205309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6823","nonce":3036205309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6821","nonce":3036205309}]},"public_addr":"172.21.15.149:6817/3036205309","cluster_addr":"172.21.15.149:6819/3036205309","heartbeat_back_addr":"172.21.15.149:6823/3036205309","heartbeat_front_addr":"172.21.15.149:6821/3036205309","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:02:52.797148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:18.980557+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:46.975511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:10.893766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:36.048181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:05:01.848799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:0/988274144":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/1939667879":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/770908409":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/3756300008":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6801/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6800/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/473242716":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6800/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6800/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2438362157":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2249539268":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6801/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6801/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2472307158":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/1030883466":"2024-03-23T07:00:35.069960+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.215+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3c280692a0 msgr2=0x7f3c2806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.215+0000 7f3c465d2700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3c280692a0 0x7f3c2806b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c30004890 tx=0x7f3c30009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.215+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 msgr2=0x7f3c400fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.215+0000 7f3c465d2700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 0x7f3c400fcbf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400b790 tx=0x7f3c34009250 comp rx=0 tx=0).stop 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.215+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 shutdown_connections 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.215+0000 7f3c465d2700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3c280692a0 0x7f3c2806b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.215+0000 7f3c465d2700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c400fb4d0 0x7f3c400fc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.215+0000 7f3c465d2700 1 --2- 172.21.15.81:0/930374486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c40102f50 0x7f3c400fcbf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.215+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 >> 172.21.15.81:0/930374486 conn(0x7f3c400714c0 msgr2=0x7f3c400fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.216+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 shutdown_connections 2024-03-22T07:05:09.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:09.216+0000 7f3c465d2700 1 -- 172.21.15.81:0/930374486 wait complete. 2024-03-22T07:05:09.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:05:09.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:09 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/930374486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:05:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:05:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:09 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/930374486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:05:10.003 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:03:49.371549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-22T07:05:10.004 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-22T07:05:10.260 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:10.665 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:10 smithi081 ceph-mon[29245]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:10.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:10 smithi149 ceph-mon[36068]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.863+0000 7f5336847700 1 -- 172.21.15.81:0/2864175689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5330101d00 msgr2=0x7f53301020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.863+0000 7f5336847700 1 --2- 172.21.15.81:0/2864175689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5330101d00 0x7f53301020d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5320004940 tx=0x7f5320009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.863+0000 7f5336847700 1 -- 172.21.15.81:0/2864175689 shutdown_connections 2024-03-22T07:05:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.863+0000 7f5336847700 1 --2- 172.21.15.81:0/2864175689 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53301026a0 0x7f53300fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.863+0000 7f5336847700 1 --2- 172.21.15.81:0/2864175689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5330101d00 0x7f53301020d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.863+0000 7f5336847700 1 -- 172.21.15.81:0/2864175689 >> 172.21.15.81:0/2864175689 conn(0x7f53300f7850 msgr2=0x7f53300f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.863+0000 7f5336847700 1 -- 172.21.15.81:0/2864175689 shutdown_connections 2024-03-22T07:05:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.863+0000 7f5336847700 1 -- 172.21.15.81:0/2864175689 wait complete. 2024-03-22T07:05:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f5336847700 1 Processor -- start 2024-03-22T07:05:11.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f5336847700 1 -- start start 2024-03-22T07:05:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f5336847700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5330101d00 0x7f53301128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f5336847700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53301026a0 0x7f5330112df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f5336847700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5330113440 con 0x7f53301026a0 2024-03-22T07:05:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f5336847700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533010ca40 con 0x7f5330101d00 2024-03-22T07:05:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f532f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53301026a0 0x7f5330112df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f532f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53301026a0 0x7f5330112df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56028/0 (socket says 172.21.15.81:56028) 2024-03-22T07:05:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f532f7fe700 1 -- 172.21.15.81:0/1619144836 learned_addr learned my addr 172.21.15.81:0/1619144836 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f532ffff700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5330101d00 0x7f53301128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f532f7fe700 1 -- 172.21.15.81:0/1619144836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5330101d00 msgr2=0x7f53301128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f532f7fe700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5330101d00 0x7f53301128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.864+0000 7f532f7fe700 1 -- 172.21.15.81:0/1619144836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5320005040 con 0x7f53301026a0 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.865+0000 7f532f7fe700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53301026a0 0x7f5330112df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f532400fce0 tx=0x7f5324004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.865+0000 7f532cff9700 1 -- 172.21.15.81:0/1619144836 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f532400fe90 con 0x7f53301026a0 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.865+0000 7f532cff9700 1 -- 172.21.15.81:0/1619144836 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53240090d0 con 0x7f53301026a0 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.865+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533010ccc0 con 0x7f53301026a0 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.865+0000 7f532cff9700 1 -- 172.21.15.81:0/1619144836 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53240126a0 con 0x7f53301026a0 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.865+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533010d280 con 0x7f53301026a0 2024-03-22T07:05:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.866+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53300fd450 con 0x7f53301026a0 2024-03-22T07:05:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.866+0000 7f532cff9700 1 -- 172.21.15.81:0/1619144836 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5324009240 con 0x7f53301026a0 2024-03-22T07:05:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.867+0000 7f532cff9700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f53180692a0 0x7f531806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.867+0000 7f532ffff700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f53180692a0 0x7f531806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.867+0000 7f532cff9700 1 -- 172.21.15.81:0/1619144836 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f53240895d0 con 0x7f53301026a0 2024-03-22T07:05:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.867+0000 7f532ffff700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f53180692a0 0x7f531806b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5320004f90 tx=0x7f5320009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:11.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:11.871+0000 7f532cff9700 1 -- 172.21.15.81:0/1619144836 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5324059650 con 0x7f53301026a0 2024-03-22T07:05:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.057+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5330108e00 con 0x7f53301026a0 2024-03-22T07:05:12.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.057+0000 7f532cff9700 1 -- 172.21.15.81:0/1619144836 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53240591e0 con 0x7f53301026a0 2024-03-22T07:05:12.058 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f53180692a0 msgr2=0x7f531806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f53180692a0 0x7f531806b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5320004f90 tx=0x7f5320009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53301026a0 msgr2=0x7f5330112df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53301026a0 0x7f5330112df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f532400fce0 tx=0x7f5324004600 comp rx=0 tx=0).stop 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 shutdown_connections 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f53180692a0 0x7f531806b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5330101d00 0x7f53301128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 --2- 172.21.15.81:0/1619144836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53301026a0 0x7f5330112df0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 >> 172.21.15.81:0/1619144836 conn(0x7f53300f7850 msgr2=0x7f53300f87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.059+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 shutdown_connections 2024-03-22T07:05:12.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:12.060+0000 7f5336847700 1 -- 172.21.15.81:0/1619144836 wait complete. 2024-03-22T07:05:12.574 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:12 smithi081 ceph-mon[29245]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:12.574 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:12 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1619144836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:05:12.692 INFO:tasks.cephadm:Setting up client nodes... 2024-03-22T07:05:12.693 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-22T07:05:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:12 smithi149 ceph-mon[36068]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:12 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1619144836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:05:12.948 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.475+0000 7f2a25112700 1 -- 172.21.15.81:0/1684134592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200feda0 msgr2=0x7f2a200faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.475+0000 7f2a25112700 1 --2- 172.21.15.81:0/1684134592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200feda0 0x7f2a200faee0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a08004940 tx=0x7f2a08009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.476+0000 7f2a25112700 1 -- 172.21.15.81:0/1684134592 shutdown_connections 2024-03-22T07:05:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.476+0000 7f2a25112700 1 --2- 172.21.15.81:0/1684134592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200feda0 0x7f2a200faee0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.476+0000 7f2a25112700 1 --2- 172.21.15.81:0/1684134592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a200fe450 0x7f2a200fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.476+0000 7f2a25112700 1 -- 172.21.15.81:0/1684134592 >> 172.21.15.81:0/1684134592 conn(0x7f2a200f6c10 msgr2=0x7f2a200f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.476+0000 7f2a25112700 1 -- 172.21.15.81:0/1684134592 shutdown_connections 2024-03-22T07:05:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.476+0000 7f2a25112700 1 -- 172.21.15.81:0/1684134592 wait complete. 2024-03-22T07:05:14.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.476+0000 7f2a25112700 1 Processor -- start 2024-03-22T07:05:14.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.476+0000 7f2a25112700 1 -- start start 2024-03-22T07:05:14.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a25112700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200fe450 0x7f2a2010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:14.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a25112700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a200feda0 0x7f2a2010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a25112700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2010e510 con 0x7f2a200fe450 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a25112700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20112eb0 con 0x7f2a200feda0 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a1ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200fe450 0x7f2a2010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a1ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200fe450 0x7f2a2010d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56042/0 (socket says 172.21.15.81:56042) 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a1ed9d700 1 -- 172.21.15.81:0/1066762911 learned_addr learned my addr 172.21.15.81:0/1066762911 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a1659c700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a200feda0 0x7f2a2010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a1ed9d700 1 -- 172.21.15.81:0/1066762911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a200feda0 msgr2=0x7f2a2010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a1ed9d700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a200feda0 0x7f2a2010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a1ed9d700 1 -- 172.21.15.81:0/1066762911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a08005040 con 0x7f2a200fe450 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a1659c700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a200feda0 0x7f2a2010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a1ed9d700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200fe450 0x7f2a2010d940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b790 tx=0x7f2a10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a17fff700 1 -- 172.21.15.81:0/1066762911 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1000f770 con 0x7f2a200fe450 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a20113110 con 0x7f2a200fe450 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a20113680 con 0x7f2a200fe450 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a17fff700 1 -- 172.21.15.81:0/1066762911 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a1000fda0 con 0x7f2a200fe450 2024-03-22T07:05:14.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.477+0000 7f2a17fff700 1 -- 172.21.15.81:0/1066762911 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1000d310 con 0x7f2a200fe450 2024-03-22T07:05:14.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.478+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a000052e0 con 0x7f2a200fe450 2024-03-22T07:05:14.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.479+0000 7f2a17fff700 1 -- 172.21.15.81:0/1066762911 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2a1000f8d0 con 0x7f2a200fe450 2024-03-22T07:05:14.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.479+0000 7f2a17fff700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2a0c0692a0 0x7f2a0c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:14.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.479+0000 7f2a1659c700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2a0c0692a0 0x7f2a0c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:14.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.479+0000 7f2a17fff700 1 -- 172.21.15.81:0/1066762911 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2a10086da0 con 0x7f2a200fe450 2024-03-22T07:05:14.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.480+0000 7f2a1659c700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2a0c0692a0 0x7f2a0c06b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800b3b0 tx=0x7f2a08009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:14.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.484+0000 7f2a17fff700 1 -- 172.21.15.81:0/1066762911 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2a10056c10 con 0x7f2a200fe450 2024-03-22T07:05:14.708 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:14 smithi081 ceph-mon[29245]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:14.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.749+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2a00005590 con 0x7f2a200fe450 2024-03-22T07:05:14.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.753+0000 7f2a17fff700 1 -- 172.21.15.81:0/1066762911 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2a10056a30 con 0x7f2a200fe450 2024-03-22T07:05:14.754 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2024-03-22T07:05:14.754 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCqLf1lenrELBAAFB/yqs73MBrqHNJ1wM9gVA== 2024-03-22T07:05:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.755+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2a0c0692a0 msgr2=0x7f2a0c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.755+0000 7f2a25112700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2a0c0692a0 0x7f2a0c06b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800b3b0 tx=0x7f2a08009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.755+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200fe450 msgr2=0x7f2a2010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.755+0000 7f2a25112700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200fe450 0x7f2a2010d940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b790 tx=0x7f2a10009250 comp rx=0 tx=0).stop 2024-03-22T07:05:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.756+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 shutdown_connections 2024-03-22T07:05:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.756+0000 7f2a25112700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2a0c0692a0 0x7f2a0c06b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:14.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.756+0000 7f2a25112700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a200fe450 0x7f2a2010d940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.756+0000 7f2a25112700 1 --2- 172.21.15.81:0/1066762911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a200feda0 0x7f2a2010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.756+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 >> 172.21.15.81:0/1066762911 conn(0x7f2a200f6c10 msgr2=0x7f2a200f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.756+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 shutdown_connections 2024-03-22T07:05:14.757 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:14.756+0000 7f2a25112700 1 -- 172.21.15.81:0/1066762911 wait complete. 2024-03-22T07:05:14.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:14 smithi149 ceph-mon[36068]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:15.486 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:05:15.486 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-22T07:05:15.486 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-22T07:05:15.531 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-22T07:05:15.796 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi149/config 2024-03-22T07:05:15.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:15 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1066762911' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:05:15.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:15 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1066762911' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T07:05:15.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:15 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1066762911' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:05:15.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:15 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1066762911' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T07:05:16.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:16 smithi149 ceph-mon[36068]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:16.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:16 smithi081 ceph-mon[29245]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:17.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 -- 172.21.15.149:0/1015098333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 msgr2=0x7efd48101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:17.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 --2- 172.21.15.149:0/1015098333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 0x7efd48101e40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efd38004970 tx=0x7efd38009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:17.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 -- 172.21.15.149:0/1015098333 shutdown_connections 2024-03-22T07:05:17.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 --2- 172.21.15.149:0/1015098333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd48102380 0x7efd48104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:17.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 --2- 172.21.15.149:0/1015098333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 0x7efd48101e40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:17.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 -- 172.21.15.149:0/1015098333 >> 172.21.15.149:0/1015098333 conn(0x7efd480f7430 msgr2=0x7efd480f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:17.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 -- 172.21.15.149:0/1015098333 shutdown_connections 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 -- 172.21.15.149:0/1015098333 wait complete. 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 Processor -- start 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.299+0000 7efd4ee1c700 1 -- start start 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4ee1c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 0x7efd4810fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4ee1c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd48102380 0x7efd48110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4ee1c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd48110600 con 0x7efd48102380 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4ee1c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd48110770 con 0x7efd48066f10 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4cbb8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 0x7efd4810fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4cbb8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 0x7efd4810fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:41078/0 (socket says 172.21.15.149:41078) 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4cbb8700 1 -- 172.21.15.149:0/355748349 learned_addr learned my addr 172.21.15.149:0/355748349 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd47fff700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd48102380 0x7efd48110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4cbb8700 1 -- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd48102380 msgr2=0x7efd48110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:17.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4cbb8700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd48102380 0x7efd48110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:17.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4cbb8700 1 -- 172.21.15.149:0/355748349 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd38005040 con 0x7efd48066f10 2024-03-22T07:05:17.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd4cbb8700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 0x7efd4810fac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efd38000c00 tx=0x7efd38009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:17.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd457fa700 1 -- 172.21.15.149:0/355748349 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd3800e040 con 0x7efd48066f10 2024-03-22T07:05:17.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.300+0000 7efd457fa700 1 -- 172.21.15.149:0/355748349 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd3800bc90 con 0x7efd48066f10 2024-03-22T07:05:17.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.301+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd48115120 con 0x7efd48066f10 2024-03-22T07:05:17.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.301+0000 7efd457fa700 1 -- 172.21.15.149:0/355748349 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd3801c610 con 0x7efd48066f10 2024-03-22T07:05:17.303 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.301+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd481156e0 con 0x7efd48066f10 2024-03-22T07:05:17.304 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.302+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4804c710 con 0x7efd48066f10 2024-03-22T07:05:17.304 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.302+0000 7efd457fa700 1 -- 172.21.15.149:0/355748349 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7efd38013a80 con 0x7efd48066f10 2024-03-22T07:05:17.304 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.303+0000 7efd457fa700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efd300692a0 0x7efd3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:17.304 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.303+0000 7efd457fa700 1 -- 172.21.15.149:0/355748349 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efd380882b0 con 0x7efd48066f10 2024-03-22T07:05:17.304 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.303+0000 7efd47fff700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efd300692a0 0x7efd3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:17.304 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.303+0000 7efd47fff700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efd300692a0 0x7efd3006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd3c004990 tx=0x7efd3c009ee0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:17.307 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.307+0000 7efd457fa700 1 -- 172.21.15.149:0/355748349 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efd380583b0 con 0x7efd48066f10 2024-03-22T07:05:17.568 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.567+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7efd48107a10 con 0x7efd48066f10 2024-03-22T07:05:17.572 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.572+0000 7efd457fa700 1 -- 172.21.15.149:0/355748349 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7efd38004de0 con 0x7efd48066f10 2024-03-22T07:05:17.572 INFO:teuthology.orchestra.run.smithi149.stdout:[client.1] 2024-03-22T07:05:17.573 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQCtLf1lBjP2IRAACelI8tHd8qwhd0x0ciUhDw== 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efd300692a0 msgr2=0x7efd3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efd300692a0 0x7efd3006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd3c004990 tx=0x7efd3c009ee0 comp rx=0 tx=0).stop 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 msgr2=0x7efd4810fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 0x7efd4810fac0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efd38000c00 tx=0x7efd38009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 shutdown_connections 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efd300692a0 0x7efd3006b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd48066f10 0x7efd4810fac0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 --2- 172.21.15.149:0/355748349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd48102380 0x7efd48110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 >> 172.21.15.149:0/355748349 conn(0x7efd480f7430 msgr2=0x7efd480f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 shutdown_connections 2024-03-22T07:05:17.575 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-22T07:05:17.574+0000 7efd4ee1c700 1 -- 172.21.15.149:0/355748349 wait complete. 2024-03-22T07:05:18.262 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:05:18.262 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-22T07:05:18.263 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-22T07:05:18.306 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-22T07:05:18.306 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-22T07:05:18.306 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-22T07:05:18.519 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:18 smithi149 ceph-mon[36068]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:18.519 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:18 smithi149 ceph-mon[36068]: from='client.? 172.21.15.149:0/355748349' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:05:18.519 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:18 smithi149 ceph-mon[36068]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:05:18.519 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:18 smithi149 ceph-mon[36068]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T07:05:18.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:18 smithi081 ceph-mon[29245]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:18.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:18 smithi081 ceph-mon[29245]: from='client.? 172.21.15.149:0/355748349' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:05:18.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:18 smithi081 ceph-mon[29245]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:05:18.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:18 smithi081 ceph-mon[29245]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T07:05:18.573 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:20.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.188+0000 7fb681618700 1 -- 172.21.15.81:0/1670375889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c102de0 msgr2=0x7fb67c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:20.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.188+0000 7fb681618700 1 --2- 172.21.15.81:0/1670375889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c102de0 0x7fb67c1031b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb66c004940 tx=0x7fb66c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:20.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.188+0000 7fb681618700 1 -- 172.21.15.81:0/1670375889 shutdown_connections 2024-03-22T07:05:20.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.188+0000 7fb681618700 1 --2- 172.21.15.81:0/1670375889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb67c103780 0x7fb67c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:20.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.188+0000 7fb681618700 1 --2- 172.21.15.81:0/1670375889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c102de0 0x7fb67c1031b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:20.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.188+0000 7fb681618700 1 -- 172.21.15.81:0/1670375889 >> 172.21.15.81:0/1670375889 conn(0x7fb67c0f8970 msgr2=0x7fb67c0fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:20.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.188+0000 7fb681618700 1 -- 172.21.15.81:0/1670375889 shutdown_connections 2024-03-22T07:05:20.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.188+0000 7fb681618700 1 -- 172.21.15.81:0/1670375889 wait complete. 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb681618700 1 Processor -- start 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb681618700 1 -- start start 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb681618700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb67c102de0 0x7fb67c113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb681618700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c103780 0x7fb67c1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb681618700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c114720 con 0x7fb67c103780 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb681618700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c10db30 con 0x7fb67c102de0 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb67a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c103780 0x7fb67c1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb67a7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c103780 0x7fb67c1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45708/0 (socket says 172.21.15.81:45708) 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb67a7fc700 1 -- 172.21.15.81:0/989088515 learned_addr learned my addr 172.21.15.81:0/989088515 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb67a7fc700 1 -- 172.21.15.81:0/989088515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb67c102de0 msgr2=0x7fb67c113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb67a7fc700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb67c102de0 0x7fb67c113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.189+0000 7fb67a7fc700 1 -- 172.21.15.81:0/989088515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c005040 con 0x7fb67c103780 2024-03-22T07:05:20.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.190+0000 7fb67a7fc700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c103780 0x7fb67c1140d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb670002a50 tx=0x7fb670004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:20.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.190+0000 7fb6637fe700 1 -- 172.21.15.81:0/989088515 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb670007cc0 con 0x7fb67c103780 2024-03-22T07:05:20.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.190+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c10de10 con 0x7fb67c103780 2024-03-22T07:05:20.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.190+0000 7fb6637fe700 1 -- 172.21.15.81:0/989088515 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb67000f040 con 0x7fb67c103780 2024-03-22T07:05:20.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.190+0000 7fb6637fe700 1 -- 172.21.15.81:0/989088515 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb67000daf0 con 0x7fb67c103780 2024-03-22T07:05:20.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.190+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c10e3d0 con 0x7fb67c103780 2024-03-22T07:05:20.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.191+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c1000f0 con 0x7fb67c103780 2024-03-22T07:05:20.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.191+0000 7fb6637fe700 1 -- 172.21.15.81:0/989088515 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb670009810 con 0x7fb67c103780 2024-03-22T07:05:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.192+0000 7fb6637fe700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb6640692a0 0x7fb66406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.192+0000 7fb67affd700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb6640692a0 0x7fb66406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.192+0000 7fb6637fe700 1 -- 172.21.15.81:0/989088515 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb670086bf0 con 0x7fb67c103780 2024-03-22T07:05:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.192+0000 7fb67affd700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb6640692a0 0x7fb66406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb66c004f90 tx=0x7fb66c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.196+0000 7fb6637fe700 1 -- 172.21.15.81:0/989088515 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb6700531e0 con 0x7fb67c103780 2024-03-22T07:05:20.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.434+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb67c10f1e0 con 0x7fb67c103780 2024-03-22T07:05:20.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.437+0000 7fb6637fe700 1 -- 172.21.15.81:0/989088515 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167166 (secure 0 0 0) 0x7fb670056800 con 0x7fb67c103780 2024-03-22T07:05:20.440 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:05:20.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.440+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb6640692a0 msgr2=0x7fb66406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:20.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.440+0000 7fb681618700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb6640692a0 0x7fb66406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb66c004f90 tx=0x7fb66c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:20.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.440+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c103780 msgr2=0x7fb67c1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:20.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.440+0000 7fb681618700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c103780 0x7fb67c1140d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb670002a50 tx=0x7fb670004600 comp rx=0 tx=0).stop 2024-03-22T07:05:20.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.440+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 shutdown_connections 2024-03-22T07:05:20.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.440+0000 7fb681618700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fb6640692a0 0x7fb66406b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:20.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.440+0000 7fb681618700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb67c102de0 0x7fb67c113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:20.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.440+0000 7fb681618700 1 --2- 172.21.15.81:0/989088515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb67c103780 0x7fb67c1140d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:20.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.441+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 >> 172.21.15.81:0/989088515 conn(0x7fb67c0f8970 msgr2=0x7fb67c1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:20.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.441+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 shutdown_connections 2024-03-22T07:05:20.442 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:20.441+0000 7fb681618700 1 -- 172.21.15.81:0/989088515 wait complete. 2024-03-22T07:05:20.737 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:20 smithi081 ceph-mon[29245]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:20.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:20 smithi149 ceph-mon[36068]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:21.102 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":19,"active_gid":14212,"active_name":"smithi081.bcxarw","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":742540357},{"type":"v1","addr":"172.21.15.81:6801","nonce":742540357}]},"active_addr":"172.21.15.81:6801/742540357","active_change":"2024-03-22T07:01:39.049963+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14230,"name":"smithi149.sgmtoa","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.81:8443/","prometheus":"http://172.21.15.81:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":3192935300}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":4180230166}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":2809233037}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":1980154994}]}]} 2024-03-22T07:05:21.106 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-22T07:05:21.106 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-22T07:05:21.106 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T07:05:21.370 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:21.674 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:21 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/989088515' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T07:05:21.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:21 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/989088515' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T07:05:22.726 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:22 smithi081 ceph-mon[29245]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:22.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:22 smithi149 ceph-mon[36068]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:22.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.936+0000 7fe026000700 1 -- 172.21.15.81:0/3261167243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0201026c0 msgr2=0x7fe0200fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:22.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.936+0000 7fe026000700 1 --2- 172.21.15.81:0/3261167243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0201026c0 0x7fe0200fbd30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe008004940 tx=0x7fe008009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:22.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.936+0000 7fe026000700 1 -- 172.21.15.81:0/3261167243 shutdown_connections 2024-03-22T07:05:22.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.936+0000 7fe026000700 1 --2- 172.21.15.81:0/3261167243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe0201026c0 0x7fe0200fbd30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:22.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.936+0000 7fe026000700 1 --2- 172.21.15.81:0/3261167243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe020101d20 0x7fe0201020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:22.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.936+0000 7fe026000700 1 -- 172.21.15.81:0/3261167243 >> 172.21.15.81:0/3261167243 conn(0x7fe0200f7870 msgr2=0x7fe0200f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:22.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.936+0000 7fe026000700 1 -- 172.21.15.81:0/3261167243 shutdown_connections 2024-03-22T07:05:22.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.936+0000 7fe026000700 1 -- 172.21.15.81:0/3261167243 wait complete. 2024-03-22T07:05:22.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.936+0000 7fe026000700 1 Processor -- start 2024-03-22T07:05:22.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe026000700 1 -- start start 2024-03-22T07:05:22.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe026000700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe020101d20 0x7fe02010eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:22.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe026000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0201026c0 0x7fe02010f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe026000700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02010f5e0 con 0x7fe020101d20 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe026000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02010f720 con 0x7fe0201026c0 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe01f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe020101d20 0x7fe02010eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe01f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe020101d20 0x7fe02010eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45728/0 (socket says 172.21.15.81:45728) 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe01f7fe700 1 -- 172.21.15.81:0/3962689916 learned_addr learned my addr 172.21.15.81:0/3962689916 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe016ffd700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0201026c0 0x7fe02010f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe01f7fe700 1 -- 172.21.15.81:0/3962689916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0201026c0 msgr2=0x7fe02010f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe01f7fe700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0201026c0 0x7fe02010f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe01f7fe700 1 -- 172.21.15.81:0/3962689916 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008005040 con 0x7fe020101d20 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.937+0000 7fe01f7fe700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe020101d20 0x7fe02010eb60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c8c0 tx=0x7fe0100044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.938+0000 7fe01cff9700 1 -- 172.21.15.81:0/3962689916 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe01000a980 con 0x7fe020101d20 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.938+0000 7fe01cff9700 1 -- 172.21.15.81:0/3962689916 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe01000ce60 con 0x7fe020101d20 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.938+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe020117bd0 con 0x7fe020101d20 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.938+0000 7fe01cff9700 1 -- 172.21.15.81:0/3962689916 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0100115e0 con 0x7fe020101d20 2024-03-22T07:05:22.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.938+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe020117fa0 con 0x7fe020101d20 2024-03-22T07:05:22.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.939+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0200fd470 con 0x7fe020101d20 2024-03-22T07:05:22.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.939+0000 7fe01cff9700 1 -- 172.21.15.81:0/3962689916 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe010012400 con 0x7fe020101d20 2024-03-22T07:05:22.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.939+0000 7fe01cff9700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe00c0692a0 0x7fe00c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:22.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.939+0000 7fe01cff9700 1 -- 172.21.15.81:0/3962689916 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe010086f50 con 0x7fe020101d20 2024-03-22T07:05:22.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.940+0000 7fe016ffd700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe00c0692a0 0x7fe00c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:22.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.940+0000 7fe016ffd700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe00c0692a0 0x7fe00c06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe020066a80 tx=0x7fe008009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:22.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:22.944+0000 7fe01cff9700 1 -- 172.21.15.81:0/3962689916 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe0100535c0 con 0x7fe020101d20 2024-03-22T07:05:23.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.131+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe02004c710 con 0x7fe020101d20 2024-03-22T07:05:23.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.132+0000 7fe01cff9700 1 -- 172.21.15.81:0/3962689916 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11070 (secure 0 0 0) 0x7fe0100126b0 con 0x7fe020101d20 2024-03-22T07:05:23.132 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:05:23.133 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":40,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","created":"2024-03-22T06:59:35.415160+0000","modified":"2024-03-22T07:05:04.161841+0000","last_up_change":"2024-03-22T07:05:03.156764+0000","last_in_change":"2024-03-22T07:04:44.637120+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:03:49.371549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"69addaec-0295-4219-9e22-5ebf831a65d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3909132761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3909132761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6809","nonce":3909132761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3909132761}]},"public_addr":"172.21.15.81:6803/3909132761","cluster_addr":"172.21.15.81:6805/3909132761","heartbeat_back_addr":"172.21.15.81:6809/3909132761","heartbeat_front_addr":"172.21.15.81:6807/3909132761","state":["exists","up"]},{"osd":1,"uuid":"cd572358-1fd5-4e96-8caf-3d00d6e0608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6811","nonce":3867177766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6813","nonce":3867177766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6817","nonce":3867177766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6815","nonce":3867177766}]},"public_addr":"172.21.15.81:6811/3867177766","cluster_addr":"172.21.15.81:6813/3867177766","heartbeat_back_addr":"172.21.15.81:6817/3867177766","heartbeat_front_addr":"172.21.15.81:6815/3867177766","state":["exists","up"]},{"osd":2,"uuid":"ba6f9b7e-712d-4f4b-8b33-6e051daf2875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6819","nonce":181541459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6821","nonce":181541459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6825","nonce":181541459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6823","nonce":181541459}]},"public_addr":"172.21.15.81:6819/181541459","cluster_addr":"172.21.15.81:6821/181541459","heartbeat_back_addr":"172.21.15.81:6825/181541459","heartbeat_front_addr":"172.21.15.81:6823/181541459","state":["exists","up"]},{"osd":3,"uuid":"e5adfd7b-e190-45b4-b8ca-da7c45d21d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6801","nonce":868260375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6803","nonce":868260375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6807","nonce":868260375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6805","nonce":868260375}]},"public_addr":"172.21.15.149:6801/868260375","cluster_addr":"172.21.15.149:6803/868260375","heartbeat_back_addr":"172.21.15.149:6807/868260375","heartbeat_front_addr":"172.21.15.149:6805/868260375","state":["exists","up"]},{"osd":4,"uuid":"a85bb681-b621-4fbf-b77a-cbd090a41a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6809","nonce":1702132597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6811","nonce":1702132597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6815","nonce":1702132597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6813","nonce":1702132597}]},"public_addr":"172.21.15.149:6809/1702132597","cluster_addr":"172.21.15.149:6811/1702132597","heartbeat_back_addr":"172.21.15.149:6815/1702132597","heartbeat_front_addr":"172.21.15.149:6813/1702132597","state":["exists","up"]},{"osd":5,"uuid":"0e83552d-4e6c-4426-9b7c-99d9ad5c45ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6817","nonce":3036205309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6819","nonce":3036205309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6823","nonce":3036205309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6821","nonce":3036205309}]},"public_addr":"172.21.15.149:6817/3036205309","cluster_addr":"172.21.15.149:6819/3036205309","heartbeat_back_addr":"172.21.15.149:6823/3036205309","heartbeat_front_addr":"172.21.15.149:6821/3036205309","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:02:52.797148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:18.980557+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:46.975511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:10.893766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:36.048181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:05:01.848799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:0/988274144":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/1939667879":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/770908409":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/3756300008":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6801/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6800/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/473242716":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6800/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6800/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2438362157":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2249539268":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6801/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6801/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2472307158":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/1030883466":"2024-03-23T07:00:35.069960+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:05:23.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe00c0692a0 msgr2=0x7fe00c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe00c0692a0 0x7fe00c06b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe020066a80 tx=0x7fe008009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe020101d20 msgr2=0x7fe02010eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe020101d20 0x7fe02010eb60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c8c0 tx=0x7fe0100044c0 comp rx=0 tx=0).stop 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 shutdown_connections 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe00c0692a0 0x7fe00c06b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe020101d20 0x7fe02010eb60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 --2- 172.21.15.81:0/3962689916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0201026c0 0x7fe02010f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 >> 172.21.15.81:0/3962689916 conn(0x7fe0200f7870 msgr2=0x7fe0200f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 shutdown_connections 2024-03-22T07:05:23.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:23.134+0000 7fe026000700 1 -- 172.21.15.81:0/3962689916 wait complete. 2024-03-22T07:05:23.803 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:23 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3962689916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:05:23.827 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-22T07:05:23.827 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T07:05:23.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:23 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3962689916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:05:24.082 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:24.506 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:24 smithi081 ceph-mon[29245]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:24.506 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:05:24.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:24 smithi149 ceph-mon[36068]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:24.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:05:25.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.693+0000 7f8258d24700 1 -- 172.21.15.81:0/276162251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 msgr2=0x7f82541020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:25.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.694+0000 7f8258d24700 1 --2- 172.21.15.81:0/276162251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 0x7f82541020f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8244004940 tx=0x7f8244009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:25.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.694+0000 7f8258d24700 1 -- 172.21.15.81:0/276162251 shutdown_connections 2024-03-22T07:05:25.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.694+0000 7f8258d24700 1 --2- 172.21.15.81:0/276162251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82541026c0 0x7f82540fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:25.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.694+0000 7f8258d24700 1 --2- 172.21.15.81:0/276162251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 0x7f82541020f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:25.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.694+0000 7f8258d24700 1 -- 172.21.15.81:0/276162251 >> 172.21.15.81:0/276162251 conn(0x7f82540f7870 msgr2=0x7f82540f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:25.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.694+0000 7f8258d24700 1 -- 172.21.15.81:0/276162251 shutdown_connections 2024-03-22T07:05:25.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.694+0000 7f8258d24700 1 -- 172.21.15.81:0/276162251 wait complete. 2024-03-22T07:05:25.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.694+0000 7f8258d24700 1 Processor -- start 2024-03-22T07:05:25.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.694+0000 7f8258d24700 1 -- start start 2024-03-22T07:05:25.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f8258d24700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 0x7f825410eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f8258d24700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82541026c0 0x7f825410f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f8258d24700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825410f600 con 0x7f8254101d20 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f8258d24700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825410f740 con 0x7f82541026c0 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f825259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 0x7f825410eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f825259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 0x7f825410eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33002/0 (socket says 172.21.15.81:33002) 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f825259c700 1 -- 172.21.15.81:0/1324615130 learned_addr learned my addr 172.21.15.81:0/1324615130 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f8249d9b700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82541026c0 0x7f825410f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f825259c700 1 -- 172.21.15.81:0/1324615130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82541026c0 msgr2=0x7f825410f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f825259c700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82541026c0 0x7f825410f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f825259c700 1 -- 172.21.15.81:0/1324615130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8244005040 con 0x7f8254101d20 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f8249d9b700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82541026c0 0x7f825410f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f825259c700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 0x7f825410eb80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8244009150 tx=0x7f8244009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f824b7fe700 1 -- 172.21.15.81:0/1324615130 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f824400e040 con 0x7f8254101d20 2024-03-22T07:05:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825410f9c0 con 0x7f8254101d20 2024-03-22T07:05:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f824b7fe700 1 -- 172.21.15.81:0/1324615130 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8244013420 con 0x7f8254101d20 2024-03-22T07:05:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f824b7fe700 1 -- 172.21.15.81:0/1324615130 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f824401c630 con 0x7f8254101d20 2024-03-22T07:05:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.695+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8254117ef0 con 0x7f8254101d20 2024-03-22T07:05:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.696+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82540fd470 con 0x7f8254101d20 2024-03-22T07:05:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.697+0000 7f824b7fe700 1 -- 172.21.15.81:0/1324615130 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8244013590 con 0x7f8254101d20 2024-03-22T07:05:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.697+0000 7f824b7fe700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f82400692a0 0x7f824006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.697+0000 7f824b7fe700 1 -- 172.21.15.81:0/1324615130 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8244088110 con 0x7f8254101d20 2024-03-22T07:05:25.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.697+0000 7f8249d9b700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f82400692a0 0x7f824006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:25.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.698+0000 7f8249d9b700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f82400692a0 0x7f824006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f823c007fd0 tx=0x7f823c009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:25.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.702+0000 7f824b7fe700 1 -- 172.21.15.81:0/1324615130 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8244054780 con 0x7f8254101d20 2024-03-22T07:05:25.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.885+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8254066830 con 0x7f8254101d20 2024-03-22T07:05:25.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.886+0000 7f824b7fe700 1 -- 172.21.15.81:0/1324615130 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11070 (secure 0 0 0) 0x7f8244057da0 con 0x7f8254101d20 2024-03-22T07:05:25.886 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:05:25.887 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":40,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","created":"2024-03-22T06:59:35.415160+0000","modified":"2024-03-22T07:05:04.161841+0000","last_up_change":"2024-03-22T07:05:03.156764+0000","last_in_change":"2024-03-22T07:04:44.637120+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:03:49.371549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"69addaec-0295-4219-9e22-5ebf831a65d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3909132761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3909132761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6809","nonce":3909132761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3909132761}]},"public_addr":"172.21.15.81:6803/3909132761","cluster_addr":"172.21.15.81:6805/3909132761","heartbeat_back_addr":"172.21.15.81:6809/3909132761","heartbeat_front_addr":"172.21.15.81:6807/3909132761","state":["exists","up"]},{"osd":1,"uuid":"cd572358-1fd5-4e96-8caf-3d00d6e0608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6811","nonce":3867177766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6813","nonce":3867177766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6817","nonce":3867177766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6815","nonce":3867177766}]},"public_addr":"172.21.15.81:6811/3867177766","cluster_addr":"172.21.15.81:6813/3867177766","heartbeat_back_addr":"172.21.15.81:6817/3867177766","heartbeat_front_addr":"172.21.15.81:6815/3867177766","state":["exists","up"]},{"osd":2,"uuid":"ba6f9b7e-712d-4f4b-8b33-6e051daf2875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6819","nonce":181541459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6821","nonce":181541459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6825","nonce":181541459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6823","nonce":181541459}]},"public_addr":"172.21.15.81:6819/181541459","cluster_addr":"172.21.15.81:6821/181541459","heartbeat_back_addr":"172.21.15.81:6825/181541459","heartbeat_front_addr":"172.21.15.81:6823/181541459","state":["exists","up"]},{"osd":3,"uuid":"e5adfd7b-e190-45b4-b8ca-da7c45d21d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6801","nonce":868260375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6803","nonce":868260375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6807","nonce":868260375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6805","nonce":868260375}]},"public_addr":"172.21.15.149:6801/868260375","cluster_addr":"172.21.15.149:6803/868260375","heartbeat_back_addr":"172.21.15.149:6807/868260375","heartbeat_front_addr":"172.21.15.149:6805/868260375","state":["exists","up"]},{"osd":4,"uuid":"a85bb681-b621-4fbf-b77a-cbd090a41a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6809","nonce":1702132597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6811","nonce":1702132597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6815","nonce":1702132597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6813","nonce":1702132597}]},"public_addr":"172.21.15.149:6809/1702132597","cluster_addr":"172.21.15.149:6811/1702132597","heartbeat_back_addr":"172.21.15.149:6815/1702132597","heartbeat_front_addr":"172.21.15.149:6813/1702132597","state":["exists","up"]},{"osd":5,"uuid":"0e83552d-4e6c-4426-9b7c-99d9ad5c45ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6817","nonce":3036205309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6819","nonce":3036205309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6823","nonce":3036205309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6821","nonce":3036205309}]},"public_addr":"172.21.15.149:6817/3036205309","cluster_addr":"172.21.15.149:6819/3036205309","heartbeat_back_addr":"172.21.15.149:6823/3036205309","heartbeat_front_addr":"172.21.15.149:6821/3036205309","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:02:52.797148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:18.980557+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:46.975511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:10.893766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:36.048181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:05:01.848799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:0/988274144":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/1939667879":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/770908409":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/3756300008":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6801/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6800/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/473242716":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6800/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6800/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2438362157":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2249539268":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6801/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6801/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2472307158":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/1030883466":"2024-03-23T07:00:35.069960+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f82400692a0 msgr2=0x7f824006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f82400692a0 0x7f824006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f823c007fd0 tx=0x7f823c009040 comp rx=0 tx=0).stop 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 msgr2=0x7f825410eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 0x7f825410eb80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8244009150 tx=0x7f8244009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 shutdown_connections 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f82400692a0 0x7f824006b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8254101d20 0x7f825410eb80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 --2- 172.21.15.81:0/1324615130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82541026c0 0x7f825410f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 >> 172.21.15.81:0/1324615130 conn(0x7f82540f7870 msgr2=0x7f82540f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 shutdown_connections 2024-03-22T07:05:25.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:25.888+0000 7f8258d24700 1 -- 172.21.15.81:0/1324615130 wait complete. 2024-03-22T07:05:26.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:26 smithi081 ceph-mon[29245]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:26 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1324615130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:05:26.618 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-22T07:05:26.618 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-22T07:05:26.618 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-22T07:05:26.619 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-22T07:05:26.619 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-22T07:05:26.620 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-22T07:05:26.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:26 smithi149 ceph-mon[36068]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:26.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:26 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1324615130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:05:26.919 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:26.934 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:28.537 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:28 smithi081 ceph-mon[29245]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:28.782 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:28 smithi149 ceph-mon[36068]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:29.310 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:29.312 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.952+0000 7efe0a80e700 1 -- 172.21.15.81:0/2972773888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04102710 msgr2=0x7efe040fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.952+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2972773888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04102710 0x7efe040fbd80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efdf4004940 tx=0x7efdf4009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.952+0000 7efe0a80e700 1 -- 172.21.15.81:0/2972773888 shutdown_connections 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.952+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2972773888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04102710 0x7efe040fbd80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.952+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2972773888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe04101d70 0x7efe04102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.952+0000 7efe0a80e700 1 -- 172.21.15.81:0/2972773888 >> 172.21.15.81:0/2972773888 conn(0x7efe04072bc0 msgr2=0x7efe04072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.952+0000 7efe0a80e700 1 -- 172.21.15.81:0/2972773888 shutdown_connections 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.952+0000 7efe0a80e700 1 -- 172.21.15.81:0/2972773888 wait complete. 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.952+0000 7efe0a80e700 1 Processor -- start 2024-03-22T07:05:29.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe0a80e700 1 -- start start 2024-03-22T07:05:29.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe0a80e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04101d70 0x7efe0410ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:29.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe0a80e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe04102710 0x7efe0410f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:29.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe0a80e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0410f750 con 0x7efe04101d70 2024-03-22T07:05:29.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe0a80e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0410f890 con 0x7efe04102710 2024-03-22T07:05:29.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe03fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04101d70 0x7efe0410ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:29.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe03fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04101d70 0x7efe0410ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33026/0 (socket says 172.21.15.81:33026) 2024-03-22T07:05:29.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe03fff700 1 -- 172.21.15.81:0/2455783010 learned_addr learned my addr 172.21.15.81:0/2455783010 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:29.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe037fe700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe04102710 0x7efe0410f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe03fff700 1 -- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe04102710 msgr2=0x7efe0410f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe03fff700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe04102710 0x7efe0410f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe03fff700 1 -- 172.21.15.81:0/2455783010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4005040 con 0x7efe04101d70 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.953+0000 7efe03fff700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04101d70 0x7efe0410ecd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdf800b790 tx=0x7efdf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.954+0000 7efe00ff9700 1 -- 172.21.15.81:0/2455783010 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf800f770 con 0x7efe04101d70 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.954+0000 7efe00ff9700 1 -- 172.21.15.81:0/2455783010 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdf800fda0 con 0x7efe04101d70 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.954+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe04117be0 con 0x7efe04101d70 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.954+0000 7efe00ff9700 1 -- 172.21.15.81:0/2455783010 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf800d310 con 0x7efe04101d70 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.954+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe041181a0 con 0x7efe04101d70 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.954+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe040fd4c0 con 0x7efe04101d70 2024-03-22T07:05:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.955+0000 7efe00ff9700 1 -- 172.21.15.81:0/2455783010 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7efdf8010420 con 0x7efe04101d70 2024-03-22T07:05:29.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.955+0000 7efe00ff9700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efdec0692a0 0x7efdec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:29.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.955+0000 7efe00ff9700 1 -- 172.21.15.81:0/2455783010 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efdf8086ee0 con 0x7efe04101d70 2024-03-22T07:05:29.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.955+0000 7efe037fe700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efdec0692a0 0x7efdec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:29.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.956+0000 7efe00ff9700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] conn(0x7efdec06ef70 0x7efdec071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:29.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.956+0000 7efe00ff9700 1 -- 172.21.15.81:0/2455783010 --> [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efdec071a30 con 0x7efdec06ef70 2024-03-22T07:05:29.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.956+0000 7efe00ff9700 1 -- 172.21.15.81:0/2455783010 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf808c4e0 con 0x7efe04101d70 2024-03-22T07:05:29.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.956+0000 7efe037fe700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efdec0692a0 0x7efdec06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b3b0 tx=0x7efdf4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:29.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.956+0000 7efe08dab700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] conn(0x7efdec06ef70 0x7efdec071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:29.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.956+0000 7efe08dab700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] conn(0x7efdec06ef70 0x7efdec071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:29.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.957+0000 7efe00ff9700 1 -- 172.21.15.81:0/2455783010 <== osd.3 v2:172.21.15.149:6800/868260375 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7efdec071a30 con 0x7efdec06ef70 2024-03-22T07:05:29.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 --> [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe04117da0 con 0x7efdec06ef70 2024-03-22T07:05:29.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe00ff9700 1 -- 172.21.15.81:0/2455783010 <== osd.3 v2:172.21.15.149:6800/868260375 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efe04117da0 con 0x7efdec06ef70 2024-03-22T07:05:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] conn(0x7efdec06ef70 msgr2=0x7efdec071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] conn(0x7efdec06ef70 0x7efdec071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efdec0692a0 msgr2=0x7efdec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efdec0692a0 0x7efdec06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b3b0 tx=0x7efdf4009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04101d70 msgr2=0x7efe0410ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04101d70 0x7efe0410ecd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efdf800b790 tx=0x7efdf8009250 comp rx=0 tx=0).stop 2024-03-22T07:05:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 shutdown_connections 2024-03-22T07:05:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:6800/868260375,v1:172.21.15.149:6801/868260375] conn(0x7efdec06ef70 0x7efdec071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7efdec0692a0 0x7efdec06b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:29.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe04101d70 0x7efe0410ecd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:29.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 --2- 172.21.15.81:0/2455783010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe04102710 0x7efe0410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:29.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 >> 172.21.15.81:0/2455783010 conn(0x7efe04072bc0 msgr2=0x7efe040faf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:29.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 shutdown_connections 2024-03-22T07:05:29.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:29.972+0000 7efe0a80e700 1 -- 172.21.15.81:0/2455783010 wait complete. 2024-03-22T07:05:30.670 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:30 smithi081 ceph-mon[29245]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:30.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:30 smithi149 ceph-mon[36068]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:31.252 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:31.252 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:32.722 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:32 smithi081 ceph-mon[29245]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:32.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:32 smithi149 ceph-mon[36068]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:33.545 INFO:teuthology.orchestra.run.smithi081.stdout:111669149713 2024-03-22T07:05:33.545 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-22T07:05:33.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.972+0000 7fa8407cd700 1 -- 172.21.15.81:0/1034874190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 msgr2=0x7fa8380fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.972+0000 7fa8407cd700 1 --2- 172.21.15.81:0/1034874190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 0x7fa8380fb940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa828004940 tx=0x7fa828009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.972+0000 7fa8407cd700 1 -- 172.21.15.81:0/1034874190 shutdown_connections 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.972+0000 7fa8407cd700 1 --2- 172.21.15.81:0/1034874190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 0x7fa8380fb940 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.972+0000 7fa8407cd700 1 --2- 172.21.15.81:0/1034874190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa838102f50 0x7fa838103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.972+0000 7fa8407cd700 1 -- 172.21.15.81:0/1034874190 >> 172.21.15.81:0/1034874190 conn(0x7fa8380714c0 msgr2=0x7fa8380718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.972+0000 7fa8407cd700 1 -- 172.21.15.81:0/1034874190 shutdown_connections 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.972+0000 7fa8407cd700 1 -- 172.21.15.81:0/1034874190 wait complete. 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa8407cd700 1 Processor -- start 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa8407cd700 1 -- start start 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa8407cd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 0x7fa8380fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa8407cd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa838102f50 0x7fa8380fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:33.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa8407cd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380fe690 con 0x7fa8380fb4d0 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa8407cd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380fd130 con 0x7fa838102f50 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa83e569700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 0x7fa8380fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa83e569700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 0x7fa8380fc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33042/0 (socket says 172.21.15.81:33042) 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa83e569700 1 -- 172.21.15.81:0/2122245193 learned_addr learned my addr 172.21.15.81:0/2122245193 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa83e569700 1 -- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa838102f50 msgr2=0x7fa8380fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa83dd68700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa838102f50 0x7fa8380fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa83e569700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa838102f50 0x7fa8380fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa83e569700 1 -- 172.21.15.81:0/2122245193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa828005040 con 0x7fa8380fb4d0 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.973+0000 7fa83dd68700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa838102f50 0x7fa8380fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.974+0000 7fa83e569700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 0x7fa8380fc6b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa83400b790 tx=0x7fa834009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.974+0000 7fa82effd700 1 -- 172.21.15.81:0/2122245193 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83400f6f0 con 0x7fa8380fb4d0 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.974+0000 7fa82effd700 1 -- 172.21.15.81:0/2122245193 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa83400fd20 con 0x7fa8380fb4d0 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.974+0000 7fa82effd700 1 -- 172.21.15.81:0/2122245193 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8340103a0 con 0x7fa8380fb4d0 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.974+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8380fd410 con 0x7fa8380fb4d0 2024-03-22T07:05:33.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.974+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83811a1e0 con 0x7fa8380fb4d0 2024-03-22T07:05:33.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.975+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8380fe7f0 con 0x7fa8380fb4d0 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.975+0000 7fa82effd700 1 -- 172.21.15.81:0/2122245193 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa834011400 con 0x7fa8380fb4d0 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.976+0000 7fa82effd700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fa8240692f0 0x7fa82406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.976+0000 7fa82effd700 1 -- 172.21.15.81:0/2122245193 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa834086d90 con 0x7fa8380fb4d0 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.976+0000 7fa83dd68700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fa8240692f0 0x7fa82406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.976+0000 7fa82effd700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] conn(0x7fa82406efc0 0x7fa8240713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.976+0000 7fa82effd700 1 -- 172.21.15.81:0/2122245193 --> [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa824071a80 con 0x7fa82406efc0 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.976+0000 7fa82effd700 1 -- 172.21.15.81:0/2122245193 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa83408c2a0 con 0x7fa8380fb4d0 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.976+0000 7fa83dd68700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fa8240692f0 0x7fa82406b7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa828000c00 tx=0x7fa828009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.976+0000 7fa83ed6a700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] conn(0x7fa82406efc0 0x7fa8240713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.977+0000 7fa83ed6a700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] conn(0x7fa82406efc0 0x7fa8240713d0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:33.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.977+0000 7fa82effd700 1 -- 172.21.15.81:0/2122245193 <== osd.4 v2:172.21.15.149:6808/1702132597 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa824071a80 con 0x7fa82406efc0 2024-03-22T07:05:33.992 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.992+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 --> [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa83805e6c0 con 0x7fa82406efc0 2024-03-22T07:05:33.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.993+0000 7fa82effd700 1 -- 172.21.15.81:0/2122245193 <== osd.4 v2:172.21.15.149:6808/1702132597 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa83805e6c0 con 0x7fa82406efc0 2024-03-22T07:05:33.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.993+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] conn(0x7fa82406efc0 msgr2=0x7fa8240713d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:33.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.993+0000 7fa8407cd700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] conn(0x7fa82406efc0 0x7fa8240713d0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:33.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.993+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fa8240692f0 msgr2=0x7fa82406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:33.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.993+0000 7fa8407cd700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fa8240692f0 0x7fa82406b7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa828000c00 tx=0x7fa828009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:33.994 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 msgr2=0x7fa8380fc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:33.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 0x7fa8380fc6b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa83400b790 tx=0x7fa834009220 comp rx=0 tx=0).stop 2024-03-22T07:05:33.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 shutdown_connections 2024-03-22T07:05:33.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:6808/1702132597,v1:172.21.15.149:6809/1702132597] conn(0x7fa82406efc0 0x7fa8240713d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:33.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fa8240692f0 0x7fa82406b7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:33.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8380fb4d0 0x7fa8380fc6b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:33.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 --2- 172.21.15.81:0/2122245193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa838102f50 0x7fa8380fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:33.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 >> 172.21.15.81:0/2122245193 conn(0x7fa8380714c0 msgr2=0x7fa8380fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:33.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 shutdown_connections 2024-03-22T07:05:33.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:33.994+0000 7fa8407cd700 1 -- 172.21.15.81:0/2122245193 wait complete. 2024-03-22T07:05:34.454 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:34 smithi081 ceph-mon[29245]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:34.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:34 smithi149 ceph-mon[36068]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:34.995 INFO:teuthology.orchestra.run.smithi081.stdout:146028888078 2024-03-22T07:05:34.995 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-22T07:05:36.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 -- 172.21.15.81:0/3129748086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab701037b0 msgr2=0x7fab700fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:36.323 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 --2- 172.21.15.81:0/3129748086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab701037b0 0x7fab700fce20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fab60004940 tx=0x7fab60009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 -- 172.21.15.81:0/3129748086 shutdown_connections 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 --2- 172.21.15.81:0/3129748086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab701037b0 0x7fab700fce20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 --2- 172.21.15.81:0/3129748086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab70102e10 0x7fab701031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 -- 172.21.15.81:0/3129748086 >> 172.21.15.81:0/3129748086 conn(0x7fab700f89a0 msgr2=0x7fab700fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 -- 172.21.15.81:0/3129748086 shutdown_connections 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 -- 172.21.15.81:0/3129748086 wait complete. 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 Processor -- start 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.198+0000 7fab776c1700 1 -- start start 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab776c1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70102e10 0x7fab70113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab776c1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab701037b0 0x7fab701140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab776c1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab701147b0 con 0x7fab70102e10 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab776c1700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7010db10 con 0x7fab701037b0 2024-03-22T07:05:36.324 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab7545d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70102e10 0x7fab70113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab7545d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70102e10 0x7fab70113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33052/0 (socket says 172.21.15.81:33052) 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab7545d700 1 -- 172.21.15.81:0/388774789 learned_addr learned my addr 172.21.15.81:0/388774789 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab74c5c700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab701037b0 0x7fab701140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab7545d700 1 -- 172.21.15.81:0/388774789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab701037b0 msgr2=0x7fab701140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab7545d700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab701037b0 0x7fab701140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab7545d700 1 -- 172.21.15.81:0/388774789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab60005040 con 0x7fab70102e10 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab7545d700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70102e10 0x7fab70113b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b790 tx=0x7fab6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab65ffb700 1 -- 172.21.15.81:0/388774789 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6c00f770 con 0x7fab70102e10 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab65ffb700 1 -- 172.21.15.81:0/388774789 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab6c00fda0 con 0x7fab70102e10 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7010ddf0 con 0x7fab70102e10 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.199+0000 7fab65ffb700 1 -- 172.21.15.81:0/388774789 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6c00d310 con 0x7fab70102e10 2024-03-22T07:05:36.325 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.200+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7010e3b0 con 0x7fab70102e10 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.200+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab7010e980 con 0x7fab70102e10 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.201+0000 7fab65ffb700 1 -- 172.21.15.81:0/388774789 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fab6c00f8d0 con 0x7fab70102e10 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.201+0000 7fab65ffb700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fab5c0692a0 0x7fab5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.201+0000 7fab65ffb700 1 -- 172.21.15.81:0/388774789 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fab6c086dc0 con 0x7fab70102e10 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.201+0000 7fab74c5c700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fab5c0692a0 0x7fab5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.201+0000 7fab65ffb700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] conn(0x7fab5c06ef70 0x7fab5c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.201+0000 7fab65ffb700 1 -- 172.21.15.81:0/388774789 --> [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab5c071a30 con 0x7fab5c06ef70 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.201+0000 7fab65ffb700 1 -- 172.21.15.81:0/388774789 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab6c089430 con 0x7fab70102e10 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.201+0000 7fab75c5e700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] conn(0x7fab5c06ef70 0x7fab5c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.202+0000 7fab74c5c700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fab5c0692a0 0x7fab5c06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fab7010ecd0 tx=0x7fab60009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.202+0000 7fab75c5e700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] conn(0x7fab5c06ef70 0x7fab5c071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.202+0000 7fab65ffb700 1 -- 172.21.15.81:0/388774789 <== osd.1 v2:172.21.15.81:6810/3867177766 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fab5c071a30 con 0x7fab5c06ef70 2024-03-22T07:05:36.326 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 --> [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab700fa360 con 0x7fab5c06ef70 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab65ffb700 1 -- 172.21.15.81:0/388774789 <== osd.1 v2:172.21.15.81:6810/3867177766 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab700fa360 con 0x7fab5c06ef70 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] conn(0x7fab5c06ef70 msgr2=0x7fab5c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] conn(0x7fab5c06ef70 0x7fab5c071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fab5c0692a0 msgr2=0x7fab5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fab5c0692a0 0x7fab5c06b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fab7010ecd0 tx=0x7fab60009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70102e10 msgr2=0x7fab70113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70102e10 0x7fab70113b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b790 tx=0x7fab6c009250 comp rx=0 tx=0).stop 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 shutdown_connections 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6810/3867177766,v1:172.21.15.81:6811/3867177766] conn(0x7fab5c06ef70 0x7fab5c071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:36.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fab5c0692a0 0x7fab5c06b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:36.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab70102e10 0x7fab70113b90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:36.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 --2- 172.21.15.81:0/388774789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab701037b0 0x7fab701140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:36.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.217+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 >> 172.21.15.81:0/388774789 conn(0x7fab700f89a0 msgr2=0x7fab700f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:36.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.218+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 shutdown_connections 2024-03-22T07:05:36.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:34.218+0000 7fab776c1700 1 -- 172.21.15.81:0/388774789 wait complete. 2024-03-22T07:05:36.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:36 smithi081 ceph-mon[29245]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:36.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:36 smithi149 ceph-mon[36068]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:36.951 INFO:teuthology.orchestra.run.smithi081.stdout:60129542174 2024-03-22T07:05:36.951 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-22T07:05:37.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.174+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1377183041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f8101d20 msgr2=0x7f02f81020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.174+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1377183041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f8101d20 0x7f02f81020f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f02e8004940 tx=0x7f02e8009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.174+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1377183041 shutdown_connections 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.174+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1377183041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f80fc4c0 0x7f02f80fc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.174+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1377183041 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f8101d20 0x7f02f81020f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.174+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1377183041 >> 172.21.15.81:0/1377183041 conn(0x7f02f8072310 msgr2=0x7f02f8072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.174+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1377183041 shutdown_connections 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.174+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1377183041 wait complete. 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02fe1ef700 1 Processor -- start 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02fe1ef700 1 -- start start 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02fe1ef700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f80fc4c0 0x7f02f810c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02fe1ef700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f8101d20 0x7f02f810cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02fe1ef700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80fc2d0 con 0x7f02f80fc4c0 2024-03-22T07:05:37.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02fe1ef700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f810d320 con 0x7f02f8101d20 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02f77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f80fc4c0 0x7f02f810c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02f77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f80fc4c0 0x7f02f810c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56348/0 (socket says 172.21.15.81:56348) 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02f77fe700 1 -- 172.21.15.81:0/1895602552 learned_addr learned my addr 172.21.15.81:0/1895602552 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02f6ffd700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f8101d20 0x7f02f810cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02f77fe700 1 -- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f8101d20 msgr2=0x7f02f810cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02f77fe700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f8101d20 0x7f02f810cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.175+0000 7f02f77fe700 1 -- 172.21.15.81:0/1895602552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e8005040 con 0x7f02f80fc4c0 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.176+0000 7f02f77fe700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f80fc4c0 0x7f02f810c8a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02e800b3b0 tx=0x7f02e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.176+0000 7f02dffff700 1 -- 172.21.15.81:0/1895602552 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02e800e040 con 0x7f02f80fc4c0 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.176+0000 7f02dffff700 1 -- 172.21.15.81:0/1895602552 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02e800bbc0 con 0x7f02f80fc4c0 2024-03-22T07:05:37.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.176+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f810d5a0 con 0x7f02f80fc4c0 2024-03-22T07:05:37.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.176+0000 7f02dffff700 1 -- 172.21.15.81:0/1895602552 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02e801c3f0 con 0x7f02f80fc4c0 2024-03-22T07:05:37.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.176+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f811a150 con 0x7f02f80fc4c0 2024-03-22T07:05:37.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.177+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02f80fd4d0 con 0x7f02f80fc4c0 2024-03-22T07:05:37.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.177+0000 7f02dffff700 1 -- 172.21.15.81:0/1895602552 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f02e800bd30 con 0x7f02f80fc4c0 2024-03-22T07:05:37.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.178+0000 7f02dffff700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f02e00692a0 0x7f02e006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:37.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.178+0000 7f02dffff700 1 -- 172.21.15.81:0/1895602552 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f02e8088020 con 0x7f02f80fc4c0 2024-03-22T07:05:37.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.178+0000 7f02f6ffd700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f02e00692a0 0x7f02e006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:37.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.178+0000 7f02dffff700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] conn(0x7f02e006ef70 0x7f02e0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:37.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.178+0000 7f02dffff700 1 -- 172.21.15.81:0/1895602552 --> [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02e0071a30 con 0x7f02e006ef70 2024-03-22T07:05:37.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.178+0000 7f02dffff700 1 -- 172.21.15.81:0/1895602552 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02e808a610 con 0x7f02f80fc4c0 2024-03-22T07:05:37.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.178+0000 7f02f6ffd700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f02e00692a0 0x7f02e006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02ec006fd0 tx=0x7f02ec008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:37.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.179+0000 7f02f7fff700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] conn(0x7f02e006ef70 0x7f02e0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:37.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.179+0000 7f02f7fff700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] conn(0x7f02e006ef70 0x7f02e0071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:37.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.180+0000 7f02dffff700 1 -- 172.21.15.81:0/1895602552 <== osd.5 v2:172.21.15.149:6816/3036205309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f02e0071a30 con 0x7f02e006ef70 2024-03-22T07:05:37.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.194+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 --> [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f02f805e640 con 0x7f02e006ef70 2024-03-22T07:05:37.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02dffff700 1 -- 172.21.15.81:0/1895602552 <== osd.5 v2:172.21.15.149:6816/3036205309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f02f805e640 con 0x7f02e006ef70 2024-03-22T07:05:37.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] conn(0x7f02e006ef70 msgr2=0x7f02e0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] conn(0x7f02e006ef70 0x7f02e0071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f02e00692a0 msgr2=0x7f02e006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f02e00692a0 0x7f02e006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02ec006fd0 tx=0x7f02ec008040 comp rx=0 tx=0).stop 2024-03-22T07:05:37.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f80fc4c0 msgr2=0x7f02f810c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f80fc4c0 0x7f02f810c8a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02e800b3b0 tx=0x7f02e8009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:37.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 shutdown_connections 2024-03-22T07:05:37.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:6816/3036205309,v1:172.21.15.149:6817/3036205309] conn(0x7f02e006ef70 0x7f02e0071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f02e00692a0 0x7f02e006b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02f80fc4c0 0x7f02f810c8a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 --2- 172.21.15.81:0/1895602552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02f8101d20 0x7f02f810cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.195+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 >> 172.21.15.81:0/1895602552 conn(0x7f02f8072310 msgr2=0x7f02f80fa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:37.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.196+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 shutdown_connections 2024-03-22T07:05:37.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.196+0000 7f02fe1ef700 1 -- 172.21.15.81:0/1895602552 wait complete. 2024-03-22T07:05:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 -- 172.21.15.81:0/1379229635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fe7c0 msgr2=0x7f80a4106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 --2- 172.21.15.81:0/1379229635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fe7c0 0x7f80a4106e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f808c004940 tx=0x7f808c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 -- 172.21.15.81:0/1379229635 shutdown_connections 2024-03-22T07:05:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 --2- 172.21.15.81:0/1379229635 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fe7c0 0x7f80a4106e00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 --2- 172.21.15.81:0/1379229635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a40fde20 0x7f80a40fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 -- 172.21.15.81:0/1379229635 >> 172.21.15.81:0/1379229635 conn(0x7f80a40f78d0 msgr2=0x7f80a40f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 -- 172.21.15.81:0/1379229635 shutdown_connections 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 -- 172.21.15.81:0/1379229635 wait complete. 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 Processor -- start 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.393+0000 7f80a928b700 1 -- start start 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a928b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fde20 0x7f80a4113c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a928b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a40fe7c0 0x7f80a4114180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a928b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a4114860 con 0x7f80a40fde20 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a928b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a410dba0 con 0x7f80a40fe7c0 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fde20 0x7f80a4113c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a2ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fde20 0x7f80a4113c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56368/0 (socket says 172.21.15.81:56368) 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a2ffd700 1 -- 172.21.15.81:0/480826821 learned_addr learned my addr 172.21.15.81:0/480826821 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f809a7fc700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a40fe7c0 0x7f80a4114180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a2ffd700 1 -- 172.21.15.81:0/480826821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a40fe7c0 msgr2=0x7f80a4114180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a2ffd700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a40fe7c0 0x7f80a4114180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a2ffd700 1 -- 172.21.15.81:0/480826821 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c005040 con 0x7f80a40fde20 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a2ffd700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fde20 0x7f80a4113c40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f809400b790 tx=0x7f8094009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f809bfff700 1 -- 172.21.15.81:0/480826821 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f809400f6f0 con 0x7f80a40fde20 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a410de20 con 0x7f80a40fde20 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f809bfff700 1 -- 172.21.15.81:0/480826821 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f809400fd20 con 0x7f80a40fde20 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f809bfff700 1 -- 172.21.15.81:0/480826821 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80940103a0 con 0x7f80a40fde20 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.394+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a410e360 con 0x7f80a40fde20 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.395+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80a410e820 con 0x7f80a40fde20 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.396+0000 7f809bfff700 1 -- 172.21.15.81:0/480826821 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8094011400 con 0x7f80a40fde20 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.396+0000 7f809bfff700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f80900692a0 0x7f809006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.396+0000 7f809bfff700 1 -- 172.21.15.81:0/480826821 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8094086c30 con 0x7f80a40fde20 2024-03-22T07:05:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.396+0000 7f809a7fc700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f80900692a0 0x7f809006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:37.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.396+0000 7f809bfff700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] conn(0x7f809006ef70 0x7f8090071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:37.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.396+0000 7f809bfff700 1 -- 172.21.15.81:0/480826821 --> [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8090071a30 con 0x7f809006ef70 2024-03-22T07:05:37.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.396+0000 7f809bfff700 1 -- 172.21.15.81:0/480826821 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f809408c290 con 0x7f80a40fde20 2024-03-22T07:05:37.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.396+0000 7f80a37fe700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] conn(0x7f809006ef70 0x7f8090071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:37.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.397+0000 7f809a7fc700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f80900692a0 0x7f809006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f80a410f4e0 tx=0x7f808c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:37.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.397+0000 7f80a37fe700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] conn(0x7f809006ef70 0x7f8090071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:37.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.397+0000 7f809bfff700 1 -- 172.21.15.81:0/480826821 <== osd.2 v2:172.21.15.81:6818/181541459 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8090071a30 con 0x7f809006ef70 2024-03-22T07:05:37.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.415+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 --> [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f80a4066770 con 0x7f809006ef70 2024-03-22T07:05:37.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.415+0000 7f809bfff700 1 -- 172.21.15.81:0/480826821 <== osd.2 v2:172.21.15.81:6818/181541459 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f80a4066770 con 0x7f809006ef70 2024-03-22T07:05:37.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] conn(0x7f809006ef70 msgr2=0x7f8090071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] conn(0x7f809006ef70 0x7f8090071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f80900692a0 msgr2=0x7f809006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f80900692a0 0x7f809006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f80a410f4e0 tx=0x7f808c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fde20 msgr2=0x7f80a4113c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fde20 0x7f80a4113c40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f809400b790 tx=0x7f8094009250 comp rx=0 tx=0).stop 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 shutdown_connections 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6818/181541459,v1:172.21.15.81:6819/181541459] conn(0x7f809006ef70 0x7f8090071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f80900692a0 0x7f809006b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80a40fde20 0x7f80a4113c40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 --2- 172.21.15.81:0/480826821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80a40fe7c0 0x7f80a4114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 >> 172.21.15.81:0/480826821 conn(0x7f80a40f78d0 msgr2=0x7f80a40ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 shutdown_connections 2024-03-22T07:05:37.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:37.416+0000 7f80a928b700 1 -- 172.21.15.81:0/480826821 wait complete. 2024-03-22T07:05:38.191 INFO:teuthology.orchestra.run.smithi081.stdout:81604378649 2024-03-22T07:05:38.192 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-22T07:05:38.636 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:38.756 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:38 smithi081 ceph-mon[29245]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:38.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:38 smithi149 ceph-mon[36068]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:39.124 INFO:teuthology.orchestra.run.smithi081.stdout:167503724553 2024-03-22T07:05:39.125 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-22T07:05:39.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.351+0000 7f36baeeb700 1 -- 172.21.15.81:0/4084364374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b40708a0 msgr2=0x7f36b4070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:39.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.351+0000 7f36baeeb700 1 --2- 172.21.15.81:0/4084364374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b40708a0 0x7f36b4070d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f36a0004970 tx=0x7f36a0009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:39.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.352+0000 7f36baeeb700 1 -- 172.21.15.81:0/4084364374 shutdown_connections 2024-03-22T07:05:39.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.352+0000 7f36baeeb700 1 --2- 172.21.15.81:0/4084364374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b40708a0 0x7f36b4070d10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:39.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.352+0000 7f36baeeb700 1 --2- 172.21.15.81:0/4084364374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b4071f90 0x7f36b4070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:39.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.352+0000 7f36baeeb700 1 -- 172.21.15.81:0/4084364374 >> 172.21.15.81:0/4084364374 conn(0x7f36b40f89b0 msgr2=0x7f36b40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:39.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.352+0000 7f36baeeb700 1 -- 172.21.15.81:0/4084364374 shutdown_connections 2024-03-22T07:05:39.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.352+0000 7f36baeeb700 1 -- 172.21.15.81:0/4084364374 wait complete. 2024-03-22T07:05:39.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.353+0000 7f36baeeb700 1 Processor -- start 2024-03-22T07:05:39.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.353+0000 7f36baeeb700 1 -- start start 2024-03-22T07:05:39.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.354+0000 7f36baeeb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b40708a0 0x7f36b411a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:39.354 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.354+0000 7f36baeeb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b4071f90 0x7f36b411a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:39.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.354+0000 7f36baeeb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b411ad00 con 0x7f36b40708a0 2024-03-22T07:05:39.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.354+0000 7f36baeeb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b41140d0 con 0x7f36b4071f90 2024-03-22T07:05:39.355 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:39.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.354+0000 7f36b8c87700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b40708a0 0x7f36b411a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:39.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.354+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b4071f90 0x7f36b411a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:39.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.354+0000 7f36b3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b4071f90 0x7f36b411a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.81:42224/0 (socket says 172.21.15.81:42224) 2024-03-22T07:05:39.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.354+0000 7f36b3fff700 1 -- 172.21.15.81:0/3655524015 learned_addr learned my addr 172.21.15.81:0/3655524015 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:39.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.355+0000 7f36b3fff700 1 -- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b40708a0 msgr2=0x7f36b411a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:39.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.355+0000 7f36b3fff700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b40708a0 0x7f36b411a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:39.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.355+0000 7f36b3fff700 1 -- 172.21.15.81:0/3655524015 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a0005040 con 0x7f36b4071f90 2024-03-22T07:05:39.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.355+0000 7f36b8c87700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b40708a0 0x7f36b411a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:05:39.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.355+0000 7f36b3fff700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b4071f90 0x7f36b411a6b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:39.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.355+0000 7f36b17fa700 1 -- 172.21.15.81:0/3655524015 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a000e040 con 0x7f36b4071f90 2024-03-22T07:05:39.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.356+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b4114350 con 0x7f36b4071f90 2024-03-22T07:05:39.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.356+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b4114890 con 0x7f36b4071f90 2024-03-22T07:05:39.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.356+0000 7f36b17fa700 1 -- 172.21.15.81:0/3655524015 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36a000bc10 con 0x7f36b4071f90 2024-03-22T07:05:39.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.356+0000 7f36b17fa700 1 -- 172.21.15.81:0/3655524015 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a001c3f0 con 0x7f36b4071f90 2024-03-22T07:05:39.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.357+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3698000fd0 con 0x7f36b4071f90 2024-03-22T07:05:39.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.358+0000 7f36b17fa700 1 -- 172.21.15.81:0/3655524015 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f36a0013420 con 0x7f36b4071f90 2024-03-22T07:05:39.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.359+0000 7f36b17fa700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f36a40692a0 0x7f36a406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:39.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.359+0000 7f36b17fa700 1 -- 172.21.15.81:0/3655524015 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f36a0088c60 con 0x7f36b4071f90 2024-03-22T07:05:39.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.359+0000 7f36b8c87700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f36a40692a0 0x7f36a406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:39.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.359+0000 7f36b17fa700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] conn(0x7f36a406ef70 0x7f36a4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:39.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.359+0000 7f36b17fa700 1 -- 172.21.15.81:0/3655524015 --> [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36a4071a30 con 0x7f36a406ef70 2024-03-22T07:05:39.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.359+0000 7f36b17fa700 1 -- 172.21.15.81:0/3655524015 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36a0088fa0 con 0x7f36b4071f90 2024-03-22T07:05:39.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.359+0000 7f36b9488700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] conn(0x7f36a406ef70 0x7f36a4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:39.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.360+0000 7f36b8c87700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f36a40692a0 0x7f36a406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f36a8010520 tx=0x7f36a800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:39.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.360+0000 7f36b9488700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] conn(0x7f36a406ef70 0x7f36a4071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:39.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.361+0000 7f36b17fa700 1 -- 172.21.15.81:0/3655524015 <== osd.0 v2:172.21.15.81:6802/3909132761 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f36a4071a30 con 0x7f36a406ef70 2024-03-22T07:05:39.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.379+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 --> [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3698002c90 con 0x7f36a406ef70 2024-03-22T07:05:39.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.379+0000 7f36b17fa700 1 -- 172.21.15.81:0/3655524015 <== osd.0 v2:172.21.15.81:6802/3909132761 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3698002c90 con 0x7f36a406ef70 2024-03-22T07:05:39.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.380+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] conn(0x7f36a406ef70 msgr2=0x7f36a4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:39.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.380+0000 7f36baeeb700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] conn(0x7f36a406ef70 0x7f36a4071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:39.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.380+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f36a40692a0 msgr2=0x7f36a406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:39.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.380+0000 7f36baeeb700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f36a40692a0 0x7f36a406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f36a8010520 tx=0x7f36a800b040 comp rx=0 tx=0).stop 2024-03-22T07:05:39.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.380+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b4071f90 msgr2=0x7f36b411a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:39.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.380+0000 7f36baeeb700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b4071f90 0x7f36b411a6b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a0009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:39.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.381+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 shutdown_connections 2024-03-22T07:05:39.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.381+0000 7f36baeeb700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f36a40692a0 0x7f36a406b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:39.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.381+0000 7f36baeeb700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b40708a0 0x7f36b411a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:39.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.381+0000 7f36baeeb700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.81:6802/3909132761,v1:172.21.15.81:6803/3909132761] conn(0x7f36a406ef70 0x7f36a4071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:39.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.381+0000 7f36baeeb700 1 --2- 172.21.15.81:0/3655524015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b4071f90 0x7f36b411a6b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:39.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.381+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 >> 172.21.15.81:0/3655524015 conn(0x7f36b40f89b0 msgr2=0x7f36b40f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:39.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.381+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 shutdown_connections 2024-03-22T07:05:39.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:39.381+0000 7f36baeeb700 1 -- 172.21.15.81:0/3655524015 wait complete. 2024-03-22T07:05:39.580 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:39.667 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:05:39.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:05:40.688 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:40 smithi081 ceph-mon[29245]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:40.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:40 smithi149 ceph-mon[36068]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.000+0000 7fc12e9e1700 1 -- 172.21.15.81:0/3378189709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 msgr2=0x7fc1280fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.000+0000 7fc12e9e1700 1 --2- 172.21.15.81:0/3378189709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 0x7fc1280fb990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc11c004940 tx=0x7fc11c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.000+0000 7fc12e9e1700 1 -- 172.21.15.81:0/3378189709 shutdown_connections 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.000+0000 7fc12e9e1700 1 --2- 172.21.15.81:0/3378189709 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 0x7fc1280fb990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.000+0000 7fc12e9e1700 1 --2- 172.21.15.81:0/3378189709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc128102fd0 0x7fc1281033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.000+0000 7fc12e9e1700 1 -- 172.21.15.81:0/3378189709 >> 172.21.15.81:0/3378189709 conn(0x7fc128071530 msgr2=0x7fc128071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.000+0000 7fc12e9e1700 1 -- 172.21.15.81:0/3378189709 shutdown_connections 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.000+0000 7fc12e9e1700 1 -- 172.21.15.81:0/3378189709 wait complete. 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.000+0000 7fc12e9e1700 1 Processor -- start 2024-03-22T07:05:42.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc12e9e1700 1 -- start start 2024-03-22T07:05:42.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc12e9e1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 0x7fc1281138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:42.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc12e9e1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc128102fd0 0x7fc128113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:42.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc12e9e1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc128114480 con 0x7fc1280fb540 2024-03-22T07:05:42.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc12e9e1700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12810dac0 con 0x7fc128102fd0 2024-03-22T07:05:42.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc127fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 0x7fc1281138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:42.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc127fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 0x7fc1281138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56396/0 (socket says 172.21.15.81:56396) 2024-03-22T07:05:42.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc127fff700 1 -- 172.21.15.81:0/1989463278 learned_addr learned my addr 172.21.15.81:0/1989463278 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:42.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc1277fe700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc128102fd0 0x7fc128113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc127fff700 1 -- 172.21.15.81:0/1989463278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc128102fd0 msgr2=0x7fc128113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc127fff700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc128102fd0 0x7fc128113e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc127fff700 1 -- 172.21.15.81:0/1989463278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c005040 con 0x7fc1280fb540 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.001+0000 7fc127fff700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 0x7fc1281138f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c8c0 tx=0x7fc1180044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.002+0000 7fc124ff9700 1 -- 172.21.15.81:0/1989463278 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11800a980 con 0x7fc1280fb540 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.002+0000 7fc124ff9700 1 -- 172.21.15.81:0/1989463278 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc11800ce60 con 0x7fc1280fb540 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.002+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12810dda0 con 0x7fc1280fb540 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.002+0000 7fc124ff9700 1 -- 172.21.15.81:0/1989463278 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1180115e0 con 0x7fc1280fb540 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.002+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12810e360 con 0x7fc1280fb540 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.003+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1281053b0 con 0x7fc1280fb540 2024-03-22T07:05:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.003+0000 7fc124ff9700 1 -- 172.21.15.81:0/1989463278 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc11800aae0 con 0x7fc1280fb540 2024-03-22T07:05:42.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.003+0000 7fc124ff9700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc1100692a0 0x7fc11006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:42.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.003+0000 7fc124ff9700 1 -- 172.21.15.81:0/1989463278 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc118086ea0 con 0x7fc1280fb540 2024-03-22T07:05:42.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.003+0000 7fc1277fe700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc1100692a0 0x7fc11006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:42.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.004+0000 7fc1277fe700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc1100692a0 0x7fc11006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00b3b0 tx=0x7fc11c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:42.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.008+0000 7fc124ff9700 1 -- 172.21.15.81:0/1989463278 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc118057040 con 0x7fc1280fb540 2024-03-22T07:05:42.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.199+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc12810a010 con 0x7fc1280fb540 2024-03-22T07:05:42.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.199+0000 7fc124ff9700 1 -- 172.21.15.81:0/1989463278 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc118056bd0 con 0x7fc1280fb540 2024-03-22T07:05:42.199 INFO:teuthology.orchestra.run.smithi081.stdout:146028888079 2024-03-22T07:05:42.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc1100692a0 msgr2=0x7fc11006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:42.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc1100692a0 0x7fc11006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00b3b0 tx=0x7fc11c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:42.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 msgr2=0x7fc1281138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 0x7fc1281138f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c8c0 tx=0x7fc1180044c0 comp rx=0 tx=0).stop 2024-03-22T07:05:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 shutdown_connections 2024-03-22T07:05:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc1100692a0 0x7fc11006b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1280fb540 0x7fc1281138f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 --2- 172.21.15.81:0/1989463278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc128102fd0 0x7fc128113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 >> 172.21.15.81:0/1989463278 conn(0x7fc128071530 msgr2=0x7fc1280fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 shutdown_connections 2024-03-22T07:05:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:42.201+0000 7fc12e9e1700 1 -- 172.21.15.81:0/1989463278 wait complete. 2024-03-22T07:05:42.451 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:42 smithi081 ceph-mon[29245]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:42.451 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:42 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1989463278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T07:05:42.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:42 smithi149 ceph-mon[36068]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:42.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:42 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1989463278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T07:05:43.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888079 for osd.4 2024-03-22T07:05:43.380 DEBUG:teuthology.parallel:result is None 2024-03-22T07:05:43.749 INFO:teuthology.orchestra.run.smithi081.stdout:38654705699 2024-03-22T07:05:43.750 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-22T07:05:44.150 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:44.152 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:44.673 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:44 smithi081 ceph-mon[29245]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:44.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:44 smithi149 ceph-mon[36068]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:45.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 -- 172.21.15.81:0/1239907658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc101d10 msgr2=0x7f39fc1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 --2- 172.21.15.81:0/1239907658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc101d10 0x7f39fc1020e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f39f8004940 tx=0x7f39f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 -- 172.21.15.81:0/1239907658 shutdown_connections 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 --2- 172.21.15.81:0/1239907658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc1026b0 0x7f39fc0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 --2- 172.21.15.81:0/1239907658 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc101d10 0x7f39fc1020e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 -- 172.21.15.81:0/1239907658 >> 172.21.15.81:0/1239907658 conn(0x7f39fc0f7880 msgr2=0x7f39fc0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 -- 172.21.15.81:0/1239907658 shutdown_connections 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 -- 172.21.15.81:0/1239907658 wait complete. 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 Processor -- start 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.997+0000 7f3a0364b700 1 -- start start 2024-03-22T07:05:45.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a0364b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc101d10 0x7f39fc113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a0364b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc1026b0 0x7f39fc1143a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a0364b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc114a80 con 0x7f39fc1026b0 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a0364b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc10ddc0 con 0x7f39fc101d10 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a00be6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc1026b0 0x7f39fc1143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a00be6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc1026b0 0x7f39fc1143a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53674/0 (socket says 172.21.15.81:53674) 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a00be6700 1 -- 172.21.15.81:0/222308637 learned_addr learned my addr 172.21.15.81:0/222308637 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a013e7700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc101d10 0x7f39fc113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a00be6700 1 -- 172.21.15.81:0/222308637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc101d10 msgr2=0x7f39fc113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a00be6700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc101d10 0x7f39fc113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a00be6700 1 -- 172.21.15.81:0/222308637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f8005040 con 0x7f39fc1026b0 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a013e7700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc101d10 0x7f39fc113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a00be6700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc1026b0 0x7f39fc1143a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00b790 tx=0x7f39ec009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f39f1ffb700 1 -- 172.21.15.81:0/222308637 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39ec00f6f0 con 0x7f39fc1026b0 2024-03-22T07:05:46.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f39f1ffb700 1 -- 172.21.15.81:0/222308637 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39ec00fd20 con 0x7f39fc1026b0 2024-03-22T07:05:46.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.998+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc10e040 con 0x7f39fc1026b0 2024-03-22T07:05:46.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.999+0000 7f39f1ffb700 1 -- 172.21.15.81:0/222308637 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39ec0103a0 con 0x7f39fc1026b0 2024-03-22T07:05:46.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:45.999+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc10e600 con 0x7f39fc1026b0 2024-03-22T07:05:46.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.000+0000 7f39f1ffb700 1 -- 172.21.15.81:0/222308637 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f39ec011400 con 0x7f39fc1026b0 2024-03-22T07:05:46.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.000+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc0fd460 con 0x7f39fc1026b0 2024-03-22T07:05:46.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.000+0000 7f39f1ffb700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f39e80690a0 0x7f39e806b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:46.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.000+0000 7f3a013e7700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f39e80690a0 0x7f39e806b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:46.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.000+0000 7f39f1ffb700 1 -- 172.21.15.81:0/222308637 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f39ec086cd0 con 0x7f39fc1026b0 2024-03-22T07:05:46.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.001+0000 7f3a013e7700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f39e80690a0 0x7f39e806b550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39f8004560 tx=0x7f39f8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:46.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.005+0000 7f39f1ffb700 1 -- 172.21.15.81:0/222308637 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f39ec056d50 con 0x7f39fc1026b0 2024-03-22T07:05:46.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.195+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f39fc108e80 con 0x7f39fc1026b0 2024-03-22T07:05:46.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.195+0000 7f39f1ffb700 1 -- 172.21.15.81:0/222308637 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39ec0116b0 con 0x7f39fc1026b0 2024-03-22T07:05:46.196 INFO:teuthology.orchestra.run.smithi081.stdout:111669149716 2024-03-22T07:05:46.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f39e80690a0 msgr2=0x7f39e806b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f39e80690a0 0x7f39e806b550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39f8004560 tx=0x7f39f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc1026b0 msgr2=0x7f39fc1143a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc1026b0 0x7f39fc1143a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00b790 tx=0x7f39ec009220 comp rx=0 tx=0).stop 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 shutdown_connections 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f39e80690a0 0x7f39e806b550 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc101d10 0x7f39fc113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 --2- 172.21.15.81:0/222308637 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc1026b0 0x7f39fc1143a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 >> 172.21.15.81:0/222308637 conn(0x7f39fc0f7880 msgr2=0x7f39fc0f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 shutdown_connections 2024-03-22T07:05:46.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.197+0000 7f3a0364b700 1 -- 172.21.15.81:0/222308637 wait complete. 2024-03-22T07:05:46.567 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:46 smithi081 ceph-mon[29245]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:46.567 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:46 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/222308637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T07:05:46.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.567+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4096191150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 msgr2=0x7fdcec0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:46.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.567+0000 7fdcf18ad700 1 --2- 172.21.15.81:0/4096191150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 0x7fdcec0fe260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009ce0 tx=0x7fdcdc004d10 comp rx=0 tx=0).stop 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.567+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4096191150 shutdown_connections 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.567+0000 7fdcf18ad700 1 --2- 172.21.15.81:0/4096191150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec0fe830 0x7fdcec106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.567+0000 7fdcf18ad700 1 --2- 172.21.15.81:0/4096191150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 0x7fdcec0fe260 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.567+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4096191150 >> 172.21.15.81:0/4096191150 conn(0x7fdcec072bc0 msgr2=0x7fdcec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.567+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4096191150 shutdown_connections 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.567+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4096191150 wait complete. 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.567+0000 7fdcf18ad700 1 Processor -- start 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdcf18ad700 1 -- start start 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdcf18ad700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 0x7fdcec0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdcf18ad700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec0fe830 0x7fdcec0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdcf18ad700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0fc820 con 0x7fdcec0fde90 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdcf18ad700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0fc960 con 0x7fdcec0fe830 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdceaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 0x7fdcec0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdceaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 0x7fdcec0fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53688/0 (socket says 172.21.15.81:53688) 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdceaffd700 1 -- 172.21.15.81:0/4048868185 learned_addr learned my addr 172.21.15.81:0/4048868185 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdce27fc700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec0fe830 0x7fdcec0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdceaffd700 1 -- 172.21.15.81:0/4048868185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec0fe830 msgr2=0x7fdcec0fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdceaffd700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec0fe830 0x7fdcec0fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdceaffd700 1 -- 172.21.15.81:0/4048868185 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc009940 con 0x7fdcec0fde90 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdce27fc700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec0fe830 0x7fdcec0fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdceaffd700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 0x7fdcec0fbd50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00b740 tx=0x7fdcdc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdce3fff700 1 -- 172.21.15.81:0/4048868185 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcdc00e040 con 0x7fdcec0fde90 2024-03-22T07:05:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdce3fff700 1 -- 172.21.15.81:0/4048868185 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcdc0128e0 con 0x7fdcec0fde90 2024-03-22T07:05:46.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.568+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec0fcbb0 con 0x7fdcec0fde90 2024-03-22T07:05:46.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.569+0000 7fdce3fff700 1 -- 172.21.15.81:0/4048868185 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcdc01c7c0 con 0x7fdcec0fde90 2024-03-22T07:05:46.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.569+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec11a280 con 0x7fdcec0fde90 2024-03-22T07:05:46.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.569+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec065e30 con 0x7fdcec0fde90 2024-03-22T07:05:46.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.570+0000 7fdce3fff700 1 -- 172.21.15.81:0/4048868185 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdcdc012410 con 0x7fdcec0fde90 2024-03-22T07:05:46.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.570+0000 7fdce3fff700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdcd80692a0 0x7fdcd806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:46.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.570+0000 7fdce3fff700 1 -- 172.21.15.81:0/4048868185 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdcdc088c40 con 0x7fdcec0fde90 2024-03-22T07:05:46.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.570+0000 7fdce27fc700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdcd80692a0 0x7fdcd806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:46.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.571+0000 7fdce27fc700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdcd80692a0 0x7fdcd806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0fd690 tx=0x7fdcd4009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:46.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.575+0000 7fdce3fff700 1 -- 172.21.15.81:0/4048868185 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdcdc055230 con 0x7fdcec0fde90 2024-03-22T07:05:46.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.776+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdcec109f00 con 0x7fdcec0fde90 2024-03-22T07:05:46.777 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.776+0000 7fdce3fff700 1 -- 172.21.15.81:0/4048868185 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdcdc058850 con 0x7fdcec0fde90 2024-03-22T07:05:46.777 INFO:teuthology.orchestra.run.smithi081.stdout:81604378651 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.778+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdcd80692a0 msgr2=0x7fdcd806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.778+0000 7fdcf18ad700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdcd80692a0 0x7fdcd806b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdcec0fd690 tx=0x7fdcd4009040 comp rx=0 tx=0).stop 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.778+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 msgr2=0x7fdcec0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.778+0000 7fdcf18ad700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 0x7fdcec0fbd50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00b740 tx=0x7fdcdc004d10 comp rx=0 tx=0).stop 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.778+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 shutdown_connections 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.778+0000 7fdcf18ad700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdcd80692a0 0x7fdcd806b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.778+0000 7fdcf18ad700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcec0fde90 0x7fdcec0fbd50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.778+0000 7fdcf18ad700 1 --2- 172.21.15.81:0/4048868185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec0fe830 0x7fdcec0fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.778+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 >> 172.21.15.81:0/4048868185 conn(0x7fdcec072bc0 msgr2=0x7fdcec0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:46.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.779+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 shutdown_connections 2024-03-22T07:05:46.780 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:46.779+0000 7fdcf18ad700 1 -- 172.21.15.81:0/4048868185 wait complete. 2024-03-22T07:05:46.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:46 smithi149 ceph-mon[36068]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:46.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:46 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/222308637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T07:05:47.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.404+0000 7f5d487f3700 1 -- 172.21.15.81:0/3212163985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d40071f90 msgr2=0x7f5d40070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.404+0000 7f5d487f3700 1 --2- 172.21.15.81:0/3212163985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d40071f90 0x7f5d40070360 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c004970 tx=0x7f5d3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 -- 172.21.15.81:0/3212163985 shutdown_connections 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 --2- 172.21.15.81:0/3212163985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d400708a0 0x7f5d40070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 --2- 172.21.15.81:0/3212163985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d40071f90 0x7f5d40070360 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 -- 172.21.15.81:0/3212163985 >> 172.21.15.81:0/3212163985 conn(0x7f5d400f8910 msgr2=0x7f5d400fad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 -- 172.21.15.81:0/3212163985 shutdown_connections 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 -- 172.21.15.81:0/3212163985 wait complete. 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 Processor -- start 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 -- start start 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d400708a0 0x7f5d4011a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d40071f90 0x7f5d4011a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:47.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4011aca0 con 0x7f5d400708a0 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d487f3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40114070 con 0x7f5d40071f90 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.405+0000 7f5d4658f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d400708a0 0x7f5d4011a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d4658f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d400708a0 0x7f5d4011a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53712/0 (socket says 172.21.15.81:53712) 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d4658f700 1 -- 172.21.15.81:0/3258910239 learned_addr learned my addr 172.21.15.81:0/3258910239 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d45d8e700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d40071f90 0x7f5d4011a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d4658f700 1 -- 172.21.15.81:0/3258910239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d40071f90 msgr2=0x7f5d4011a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d4658f700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d40071f90 0x7f5d4011a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d4658f700 1 -- 172.21.15.81:0/3258910239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c005040 con 0x7f5d400708a0 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d4658f700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d400708a0 0x7f5d4011a110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c004f50 tx=0x7f5d3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d36ffd700 1 -- 172.21.15.81:0/3258910239 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d3c00e040 con 0x7f5d400708a0 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d36ffd700 1 -- 172.21.15.81:0/3258910239 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d3c009400 con 0x7f5d400708a0 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d401142f0 con 0x7f5d400708a0 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d36ffd700 1 -- 172.21.15.81:0/3258910239 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d3c01c3f0 con 0x7f5d400708a0 2024-03-22T07:05:47.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.406+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d40114830 con 0x7f5d400708a0 2024-03-22T07:05:47.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.407+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d40071aa0 con 0x7f5d400708a0 2024-03-22T07:05:47.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.408+0000 7f5d36ffd700 1 -- 172.21.15.81:0/3258910239 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5d3c00bc10 con 0x7f5d400708a0 2024-03-22T07:05:47.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.408+0000 7f5d36ffd700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5d2c0692a0 0x7f5d2c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:47.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.408+0000 7f5d45d8e700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5d2c0692a0 0x7f5d2c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:47.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.408+0000 7f5d36ffd700 1 -- 172.21.15.81:0/3258910239 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5d3c088020 con 0x7f5d400708a0 2024-03-22T07:05:47.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.409+0000 7f5d45d8e700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5d2c0692a0 0x7f5d2c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d401159b0 tx=0x7f5d3000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:47.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.412+0000 7f5d36ffd700 1 -- 172.21.15.81:0/3258910239 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5d3c058270 con 0x7f5d400708a0 2024-03-22T07:05:47.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:47 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/4048868185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T07:05:47.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.603+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5d4004bed0 con 0x7f5d400708a0 2024-03-22T07:05:47.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.603+0000 7f5d36ffd700 1 -- 172.21.15.81:0/3258910239 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d3c057e00 con 0x7f5d400708a0 2024-03-22T07:05:47.605 INFO:teuthology.orchestra.run.smithi081.stdout:60129542177 2024-03-22T07:05:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.605+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5d2c0692a0 msgr2=0x7f5d2c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.605+0000 7f5d487f3700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5d2c0692a0 0x7f5d2c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d401159b0 tx=0x7f5d3000b040 comp rx=0 tx=0).stop 2024-03-22T07:05:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.605+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d400708a0 msgr2=0x7f5d4011a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.605+0000 7f5d487f3700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d400708a0 0x7f5d4011a110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c004f50 tx=0x7f5d3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.605+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 shutdown_connections 2024-03-22T07:05:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.605+0000 7f5d487f3700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5d2c0692a0 0x7f5d2c06b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:47.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.605+0000 7f5d487f3700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d400708a0 0x7f5d4011a110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:47.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.605+0000 7f5d487f3700 1 --2- 172.21.15.81:0/3258910239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d40071f90 0x7f5d4011a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:47.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.605+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 >> 172.21.15.81:0/3258910239 conn(0x7f5d400f8910 msgr2=0x7f5d400f9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:47.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.606+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 shutdown_connections 2024-03-22T07:05:47.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:47.606+0000 7f5d487f3700 1 -- 172.21.15.81:0/3258910239 wait complete. 2024-03-22T07:05:47.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:47 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/4048868185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T07:05:48.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378651 for osd.2 2024-03-22T07:05:48.379 DEBUG:teuthology.parallel:result is None 2024-03-22T07:05:48.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:48 smithi081 ceph-mon[29245]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:48.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:48 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3258910239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T07:05:48.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:48 smithi149 ceph-mon[36068]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:48.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:48 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3258910239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T07:05:48.789 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149713 got 111669149716 for osd.3 2024-03-22T07:05:48.789 DEBUG:teuthology.parallel:result is None 2024-03-22T07:05:49.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.233+0000 7fdef2c13700 1 -- 172.21.15.81:0/3347161777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fe830 msgr2=0x7fdeec106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.233+0000 7fdef2c13700 1 --2- 172.21.15.81:0/3347161777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fe830 0x7fdeec106f00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fded8004940 tx=0x7fded8009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.233+0000 7fdef2c13700 1 -- 172.21.15.81:0/3347161777 shutdown_connections 2024-03-22T07:05:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.233+0000 7fdef2c13700 1 --2- 172.21.15.81:0/3347161777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fe830 0x7fdeec106f00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.233+0000 7fdef2c13700 1 --2- 172.21.15.81:0/3347161777 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeec0fde90 0x7fdeec0fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.233+0000 7fdef2c13700 1 -- 172.21.15.81:0/3347161777 >> 172.21.15.81:0/3347161777 conn(0x7fdeec072bc0 msgr2=0x7fdeec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.233+0000 7fdef2c13700 1 -- 172.21.15.81:0/3347161777 shutdown_connections 2024-03-22T07:05:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.233+0000 7fdef2c13700 1 -- 172.21.15.81:0/3347161777 wait complete. 2024-03-22T07:05:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.234+0000 7fdef2c13700 1 Processor -- start 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.234+0000 7fdef2c13700 1 -- start start 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.234+0000 7fdef2c13700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fde90 0x7fdeec0fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.234+0000 7fdef2c13700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeec0fe830 0x7fdeec0fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.234+0000 7fdef2c13700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec0fc8b0 con 0x7fdeec0fde90 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.234+0000 7fdef2c13700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec0fc9f0 con 0x7fdeec0fe830 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.234+0000 7fdef09af700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fde90 0x7fdeec0fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.234+0000 7fdef09af700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fde90 0x7fdeec0fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53736/0 (socket says 172.21.15.81:53736) 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.234+0000 7fdef09af700 1 -- 172.21.15.81:0/1989780340 learned_addr learned my addr 172.21.15.81:0/1989780340 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdeebfff700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeec0fe830 0x7fdeec0fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdef09af700 1 -- 172.21.15.81:0/1989780340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeec0fe830 msgr2=0x7fdeec0fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdef09af700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeec0fe830 0x7fdeec0fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdef09af700 1 -- 172.21.15.81:0/1989780340 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded8005040 con 0x7fdeec0fde90 2024-03-22T07:05:49.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdef09af700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fde90 0x7fdeec0fbd50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdee000b790 tx=0x7fdee0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdee97fa700 1 -- 172.21.15.81:0/1989780340 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee000f6f0 con 0x7fdeec0fde90 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdee97fa700 1 -- 172.21.15.81:0/1989780340 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdee000fd20 con 0x7fdeec0fde90 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeec119e90 con 0x7fdeec0fde90 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdee97fa700 1 -- 172.21.15.81:0/1989780340 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee00103a0 con 0x7fdeec0fde90 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.235+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeec11a2f0 con 0x7fdeec0fde90 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.236+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeec065e30 con 0x7fdeec0fde90 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.237+0000 7fdee97fa700 1 -- 172.21.15.81:0/1989780340 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdee0011400 con 0x7fdeec0fde90 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.237+0000 7fdee97fa700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdedc0692a0 0x7fdedc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.237+0000 7fdeebfff700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdedc0692a0 0x7fdedc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.237+0000 7fdee97fa700 1 -- 172.21.15.81:0/1989780340 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdee0086c10 con 0x7fdeec0fde90 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.237+0000 7fdeebfff700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdedc0692a0 0x7fdedc06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0fd690 tx=0x7fded8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.241+0000 7fdee97fa700 1 -- 172.21.15.81:0/1989780340 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdee0053280 con 0x7fdeec0fde90 2024-03-22T07:05:49.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.429+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdeec05f5f0 con 0x7fdeec0fde90 2024-03-22T07:05:49.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.429+0000 7fdee97fa700 1 -- 172.21.15.81:0/1989780340 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdee00568a0 con 0x7fdeec0fde90 2024-03-22T07:05:49.624 INFO:teuthology.orchestra.run.smithi081.stdout:167503724555 2024-03-22T07:05:49.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdedc0692a0 msgr2=0x7fdedc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:49.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdedc0692a0 0x7fdedc06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0fd690 tx=0x7fded8009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fde90 msgr2=0x7fdeec0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fde90 0x7fdeec0fbd50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdee000b790 tx=0x7fdee0009250 comp rx=0 tx=0).stop 2024-03-22T07:05:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 shutdown_connections 2024-03-22T07:05:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdedc0692a0 0x7fdedc06b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdeec0fde90 0x7fdeec0fbd50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 --2- 172.21.15.81:0/1989780340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdeec0fe830 0x7fdeec0fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 >> 172.21.15.81:0/1989780340 conn(0x7fdeec072bc0 msgr2=0x7fdeec103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 shutdown_connections 2024-03-22T07:05:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:49.431+0000 7fdef2c13700 1 -- 172.21.15.81:0/1989780340 wait complete. 2024-03-22T07:05:49.733 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:49 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1989780340' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T07:05:49.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:49 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1989780340' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T07:05:50.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542177 for osd.1 2024-03-22T07:05:50.155 DEBUG:teuthology.parallel:result is None 2024-03-22T07:05:50.715 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:50 smithi081 ceph-mon[29245]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:50.716 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724555 for osd.5 2024-03-22T07:05:50.716 DEBUG:teuthology.parallel:result is None 2024-03-22T07:05:50.747 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:50.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:50 smithi149 ceph-mon[36068]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:52.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.371+0000 7f669e422700 1 -- 172.21.15.81:0/400719484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698101d70 msgr2=0x7f6698102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:52.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.371+0000 7f669e422700 1 --2- 172.21.15.81:0/400719484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698101d70 0x7f6698102140 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6688004940 tx=0x7f6688009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:52.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.371+0000 7f669e422700 1 -- 172.21.15.81:0/400719484 shutdown_connections 2024-03-22T07:05:52.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.371+0000 7f669e422700 1 --2- 172.21.15.81:0/400719484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698102710 0x7f66980fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.371+0000 7f669e422700 1 --2- 172.21.15.81:0/400719484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698101d70 0x7f6698102140 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.371+0000 7f669e422700 1 -- 172.21.15.81:0/400719484 >> 172.21.15.81:0/400719484 conn(0x7f6698072bc0 msgr2=0x7f6698072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.371+0000 7f669e422700 1 -- 172.21.15.81:0/400719484 shutdown_connections 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.371+0000 7f669e422700 1 -- 172.21.15.81:0/400719484 wait complete. 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.371+0000 7f669e422700 1 Processor -- start 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f669e422700 1 -- start start 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f669e422700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101d70 0x7f669810ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f669e422700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698102710 0x7f669810f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f669e422700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669810f700 con 0x7f6698102710 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f669e422700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669810f840 con 0x7f6698101d70 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f66977fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698102710 0x7f669810f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f66977fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698102710 0x7f669810f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53748/0 (socket says 172.21.15.81:53748) 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f66977fe700 1 -- 172.21.15.81:0/561769367 learned_addr learned my addr 172.21.15.81:0/561769367 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f6697fff700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101d70 0x7f669810ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:52.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f66977fe700 1 -- 172.21.15.81:0/561769367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101d70 msgr2=0x7f669810ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:52.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f66977fe700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101d70 0x7f669810ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:52.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f66977fe700 1 -- 172.21.15.81:0/561769367 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688005040 con 0x7f6698102710 2024-03-22T07:05:52.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f6697fff700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101d70 0x7f669810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:05:52.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.372+0000 7f66977fe700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698102710 0x7f669810f1c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f668c002a50 tx=0x7f668c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:52.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.373+0000 7f6694ff9700 1 -- 172.21.15.81:0/561769367 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f668c007cc0 con 0x7f6698102710 2024-03-22T07:05:52.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.373+0000 7f6694ff9700 1 -- 172.21.15.81:0/561769367 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f668c00f040 con 0x7f6698102710 2024-03-22T07:05:52.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.373+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6698117b30 con 0x7f6698102710 2024-03-22T07:05:52.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.373+0000 7f6694ff9700 1 -- 172.21.15.81:0/561769367 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f668c00daf0 con 0x7f6698102710 2024-03-22T07:05:52.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.373+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66981180c0 con 0x7f6698102710 2024-03-22T07:05:52.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.374+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66980ff080 con 0x7f6698102710 2024-03-22T07:05:52.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.374+0000 7f6694ff9700 1 -- 172.21.15.81:0/561769367 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f668c009810 con 0x7f6698102710 2024-03-22T07:05:52.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.375+0000 7f6694ff9700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f66800692a0 0x7f668006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:52.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.375+0000 7f6694ff9700 1 -- 172.21.15.81:0/561769367 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f668c086b00 con 0x7f6698102710 2024-03-22T07:05:52.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.375+0000 7f6697fff700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f66800692a0 0x7f668006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:52.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.375+0000 7f6697fff700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f66800692a0 0x7f668006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6688004560 tx=0x7f6688009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:52.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.379+0000 7f6694ff9700 1 -- 172.21.15.81:0/561769367 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f668c053170 con 0x7f6698102710 2024-03-22T07:05:52.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.566+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6698108e70 con 0x7f6698102710 2024-03-22T07:05:52.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.567+0000 7f6694ff9700 1 -- 172.21.15.81:0/561769367 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f668c056790 con 0x7f6698102710 2024-03-22T07:05:52.567 INFO:teuthology.orchestra.run.smithi081.stdout:38654705702 2024-03-22T07:05:52.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f66800692a0 msgr2=0x7f668006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:52.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f66800692a0 0x7f668006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6688004560 tx=0x7f6688009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:52.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698102710 msgr2=0x7f669810f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:52.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698102710 0x7f669810f1c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f668c002a50 tx=0x7f668c004600 comp rx=0 tx=0).stop 2024-03-22T07:05:52.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 shutdown_connections 2024-03-22T07:05:52.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f66800692a0 0x7f668006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:52.569 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101d70 0x7f669810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:52.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 --2- 172.21.15.81:0/561769367 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6698102710 0x7f669810f1c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:52.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 >> 172.21.15.81:0/561769367 conn(0x7f6698072bc0 msgr2=0x7f66980fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:52.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 shutdown_connections 2024-03-22T07:05:52.570 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:52.569+0000 7f669e422700 1 -- 172.21.15.81:0/561769367 wait complete. 2024-03-22T07:05:52.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:52 smithi081 ceph-mon[29245]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:52.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:52 smithi149 ceph-mon[36068]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:53.217 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705702 for osd.0 2024-03-22T07:05:53.217 DEBUG:teuthology.parallel:result is None 2024-03-22T07:05:53.217 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-22T07:05:53.218 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T07:05:53.516 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:53.733 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:53 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/561769367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T07:05:53.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:53 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/561769367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T07:05:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:54 smithi081 ceph-mon[29245]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:05:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:54 smithi149 ceph-mon[36068]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:05:55.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 -- 172.21.15.81:0/2958314296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fe830 msgr2=0x7f1ea8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 --2- 172.21.15.81:0/2958314296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fe830 0x7f1ea8106f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1e98004940 tx=0x7f1e98009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 -- 172.21.15.81:0/2958314296 shutdown_connections 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 --2- 172.21.15.81:0/2958314296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fe830 0x7f1ea8106f00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 --2- 172.21.15.81:0/2958314296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea80fde90 0x7f1ea80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 -- 172.21.15.81:0/2958314296 >> 172.21.15.81:0/2958314296 conn(0x7f1ea8072bc0 msgr2=0x7f1ea8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 -- 172.21.15.81:0/2958314296 shutdown_connections 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 -- 172.21.15.81:0/2958314296 wait complete. 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 Processor -- start 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.174+0000 7f1eaf768700 1 -- start start 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1eaf768700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fde90 0x7f1ea80fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:55.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1eaf768700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea80fe830 0x7f1ea80fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1eaf768700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea80fc8b0 con 0x7f1ea80fde90 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1eaf768700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea80fc9f0 con 0x7f1ea80fe830 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1ead504700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fde90 0x7f1ea80fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1ead504700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fde90 0x7f1ea80fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51884/0 (socket says 172.21.15.81:51884) 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1ead504700 1 -- 172.21.15.81:0/3256523523 learned_addr learned my addr 172.21.15.81:0/3256523523 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1eacd03700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea80fe830 0x7f1ea80fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1ead504700 1 -- 172.21.15.81:0/3256523523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea80fe830 msgr2=0x7f1ea80fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1ead504700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea80fe830 0x7f1ea80fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1ead504700 1 -- 172.21.15.81:0/3256523523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e98005040 con 0x7f1ea80fde90 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1ead504700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fde90 0x7f1ea80fbd50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400b790 tx=0x7f1ea4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1e9dffb700 1 -- 172.21.15.81:0/3256523523 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ea400f770 con 0x7f1ea80fde90 2024-03-22T07:05:55.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1e9dffb700 1 -- 172.21.15.81:0/3256523523 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ea400fda0 con 0x7f1ea80fde90 2024-03-22T07:05:55.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea8119e90 con 0x7f1ea80fde90 2024-03-22T07:05:55.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.175+0000 7f1e9dffb700 1 -- 172.21.15.81:0/3256523523 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ea400d310 con 0x7f1ea80fde90 2024-03-22T07:05:55.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.176+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea811a2f0 con 0x7f1ea80fde90 2024-03-22T07:05:55.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.176+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea8065e30 con 0x7f1ea80fde90 2024-03-22T07:05:55.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.177+0000 7f1e9dffb700 1 -- 172.21.15.81:0/3256523523 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1ea4010420 con 0x7f1ea80fde90 2024-03-22T07:05:55.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.177+0000 7f1e9dffb700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e940692a0 0x7f1e9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:55.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.177+0000 7f1e9dffb700 1 -- 172.21.15.81:0/3256523523 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1ea4086e90 con 0x7f1ea80fde90 2024-03-22T07:05:55.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.177+0000 7f1eacd03700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e940692a0 0x7f1e9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:55.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.178+0000 7f1eacd03700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e940692a0 0x7f1e9406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80fd690 tx=0x7f1e98009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:55.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.183+0000 7f1e9dffb700 1 -- 172.21.15.81:0/3256523523 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1ea4053500 con 0x7f1ea80fde90 2024-03-22T07:05:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.364+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ea804bed0 con 0x7f1e940692a0 2024-03-22T07:05:55.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.365+0000 7f1e9dffb700 1 -- 172.21.15.81:0/3256523523 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27970 (secure 0 0 0) 0x7f1ea804bed0 con 0x7f1e940692a0 2024-03-22T07:05:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e940692a0 msgr2=0x7f1e9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e940692a0 0x7f1e9406b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80fd690 tx=0x7f1e98009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fde90 msgr2=0x7f1ea80fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fde90 0x7f1ea80fbd50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400b790 tx=0x7f1ea4009250 comp rx=0 tx=0).stop 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 shutdown_connections 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e940692a0 0x7f1e9406b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea80fde90 0x7f1ea80fbd50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 --2- 172.21.15.81:0/3256523523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ea80fe830 0x7f1ea80fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 >> 172.21.15.81:0/3256523523 conn(0x7f1ea8072bc0 msgr2=0x7f1ea8103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 shutdown_connections 2024-03-22T07:05:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:55.367+0000 7f1eaf768700 1 -- 172.21.15.81:0/3256523523 wait complete. 2024-03-22T07:05:55.369 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-03-22T07:05:55.947 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-03-22T07:05:55.099973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787604,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683308,"statfs":{"total":575970213888,"available":574139707392,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002039"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-22T07:05:03.160253+0000","last_change":"2024-03-22T07:04:37.644983+0000","last_active":"2024-03-22T07:05:03.160253+0000","last_peered":"2024-03-22T07:05:03.160253+0000","last_clean":"2024-03-22T07:05:03.160253+0000","last_became_active":"2024-03-22T07:04:37.644705+0000","last_became_peered":"2024-03-22T07:04:37.644705+0000","last_unstale":"2024-03-22T07:05:03.160253+0000","last_undegraded":"2024-03-22T07:05:03.160253+0000","last_fullsized":"2024-03-22T07:05:03.160253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T07:03:49.813738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T07:03:49.813738+0000","last_clean_scrub_stamp":"2024-03-22T07:03:49.813738+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:24:55.252540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.437},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.34499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:05:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.5},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.48199999999999998}]},{"osd":1,"last update":"Fri Mar 22 07:05:12 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.374}]},{"osd":2,"last update":"Fri Mar 22 07:05:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.35099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:05:52 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34999999999999998,"15min":0.35499999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.316}]},{"osd":1,"last update":"Fri Mar 22 07:05:52 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.34100000000000003,"15min":0.34499999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.25900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.28000000000000003}]},{"osd":3,"last update":"Fri Mar 22 07:05:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Mar 22 07:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 07:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.50600000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.224,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.46400000000000002,"5min":0.495,"15min":0.495},"last":0.25}]},{"osd":2,"last update":"Fri Mar 22 07:04:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.504,"5min":0.504,"15min":0.504},"last":0.25900000000000001}]},{"osd":3,"last update":"Fri Mar 22 07:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39100000000000001}]},{"osd":4,"last update":"Fri Mar 22 07:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32400000000000001},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.318,"15min":0.312},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.376}]},{"osd":2,"last update":"Fri Mar 22 07:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.439},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Mar 22 07:05:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.629}]},{"osd":4,"last update":"Fri Mar 22 07:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T07:05:55.949 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T07:05:56.203 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:56.617 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:56 smithi081 ceph-mon[29245]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:56.618 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:56 smithi081 ceph-mon[29245]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:05:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:56 smithi149 ceph-mon[36068]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:56 smithi149 ceph-mon[36068]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:05:57.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.649+0000 7f8ca043a700 1 -- 172.21.15.81:0/2962685362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98101d70 msgr2=0x7f8c98102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.649+0000 7f8ca043a700 1 --2- 172.21.15.81:0/2962685362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98101d70 0x7f8c98102140 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c94004940 tx=0x7f8c94009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.649+0000 7f8ca043a700 1 -- 172.21.15.81:0/2962685362 shutdown_connections 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.649+0000 7f8ca043a700 1 --2- 172.21.15.81:0/2962685362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c98102710 0x7f8c980fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.649+0000 7f8ca043a700 1 --2- 172.21.15.81:0/2962685362 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98101d70 0x7f8c98102140 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.649+0000 7f8ca043a700 1 -- 172.21.15.81:0/2962685362 >> 172.21.15.81:0/2962685362 conn(0x7f8c98072bc0 msgr2=0x7f8c98072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.649+0000 7f8ca043a700 1 -- 172.21.15.81:0/2962685362 shutdown_connections 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.649+0000 7f8ca043a700 1 -- 172.21.15.81:0/2962685362 wait complete. 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8ca043a700 1 Processor -- start 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8ca043a700 1 -- start start 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8ca043a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c98101d70 0x7f8c98115c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8ca043a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98102710 0x7f8c981161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8ca043a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98116800 con 0x7f8c98102710 2024-03-22T07:05:57.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8ca043a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9810fc80 con 0x7f8c98101d70 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8c9d9d5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98102710 0x7f8c981161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8c9d9d5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98102710 0x7f8c981161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51906/0 (socket says 172.21.15.81:51906) 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8c9d9d5700 1 -- 172.21.15.81:0/916617963 learned_addr learned my addr 172.21.15.81:0/916617963 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.650+0000 7f8c9e1d6700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c98101d70 0x7f8c98115c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8c9d9d5700 1 -- 172.21.15.81:0/916617963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c98101d70 msgr2=0x7f8c98115c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8c9d9d5700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c98101d70 0x7f8c98115c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8c9d9d5700 1 -- 172.21.15.81:0/916617963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94005040 con 0x7f8c98102710 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8c9e1d6700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c98101d70 0x7f8c98115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8c9d9d5700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98102710 0x7f8c981161b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800c800 tx=0x7f8c88004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8c8effd700 1 -- 172.21.15.81:0/916617963 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c8800a760 con 0x7f8c98102710 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8c8effd700 1 -- 172.21.15.81:0/916617963 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c8800ad90 con 0x7f8c98102710 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9810ff60 con 0x7f8c98102710 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8c8effd700 1 -- 172.21.15.81:0/916617963 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c880113a0 con 0x7f8c98102710 2024-03-22T07:05:57.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.651+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c98110520 con 0x7f8c98102710 2024-03-22T07:05:57.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.652+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c980fd4c0 con 0x7f8c98102710 2024-03-22T07:05:57.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.653+0000 7f8c8effd700 1 -- 172.21.15.81:0/916617963 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8c88011610 con 0x7f8c98102710 2024-03-22T07:05:57.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.653+0000 7f8c8effd700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8c840692b0 0x7f8c8406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:05:57.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.653+0000 7f8c9e1d6700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8c840692b0 0x7f8c8406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:05:57.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.653+0000 7f8c8effd700 1 -- 172.21.15.81:0/916617963 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8c88086e80 con 0x7f8c98102710 2024-03-22T07:05:57.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.654+0000 7f8c9e1d6700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8c840692b0 0x7f8c8406b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400b3b0 tx=0x7f8c94009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:05:57.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.657+0000 7f8c8effd700 1 -- 172.21.15.81:0/916617963 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8c88053560 con 0x7f8c98102710 2024-03-22T07:05:57.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.847+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c98108e70 con 0x7f8c840692b0 2024-03-22T07:05:57.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.848+0000 7f8c8effd700 1 -- 172.21.15.81:0/916617963 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28081 (secure 0 0 0) 0x7f8c98108e70 con 0x7f8c840692b0 2024-03-22T07:05:57.849 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.850+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8c840692b0 msgr2=0x7f8c8406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.850+0000 7f8ca043a700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8c840692b0 0x7f8c8406b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400b3b0 tx=0x7f8c94009e30 comp rx=0 tx=0).stop 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.850+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98102710 msgr2=0x7f8c981161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.850+0000 7f8ca043a700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98102710 0x7f8c981161b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800c800 tx=0x7f8c88004470 comp rx=0 tx=0).stop 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.850+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 shutdown_connections 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.850+0000 7f8ca043a700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8c840692b0 0x7f8c8406b760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.850+0000 7f8ca043a700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c98101d70 0x7f8c98115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.850+0000 7f8ca043a700 1 --2- 172.21.15.81:0/916617963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c98102710 0x7f8c981161b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.850+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 >> 172.21.15.81:0/916617963 conn(0x7f8c98072bc0 msgr2=0x7f8c98065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.851+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 shutdown_connections 2024-03-22T07:05:57.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:05:57.851+0000 7f8ca043a700 1 -- 172.21.15.81:0/916617963 wait complete. 2024-03-22T07:05:57.852 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-03-22T07:05:58.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:58 smithi149 ceph-mon[36068]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:58.701 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-03-22T07:05:57.100291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787604,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683308,"statfs":{"total":575970213888,"available":574139707392,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002018"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-22T07:05:03.160253+0000","last_change":"2024-03-22T07:04:37.644983+0000","last_active":"2024-03-22T07:05:03.160253+0000","last_peered":"2024-03-22T07:05:03.160253+0000","last_clean":"2024-03-22T07:05:03.160253+0000","last_became_active":"2024-03-22T07:04:37.644705+0000","last_became_peered":"2024-03-22T07:04:37.644705+0000","last_unstale":"2024-03-22T07:05:03.160253+0000","last_undegraded":"2024-03-22T07:05:03.160253+0000","last_fullsized":"2024-03-22T07:05:03.160253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T07:03:49.813738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T07:03:49.813738+0000","last_clean_scrub_stamp":"2024-03-22T07:03:49.813738+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T07:24:55.252540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.437},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.34499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:05:12 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.5},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.48199999999999998}]},{"osd":1,"last update":"Fri Mar 22 07:05:12 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.374}]},{"osd":2,"last update":"Fri Mar 22 07:05:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Mar 22 07:05:38 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.35099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447508,"statfs":{"total":95995035648,"available":95690248192,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:05:52 2024","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34999999999999998,"15min":0.35499999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.316}]},{"osd":1,"last update":"Fri Mar 22 07:05:52 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.34100000000000003,"15min":0.34499999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.25900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.28000000000000003}]},{"osd":3,"last update":"Fri Mar 22 07:05:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Mar 22 07:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 07:05:26 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.50600000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.192},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.34999999999999998,"15min":0.35199999999999998},"min":{"1min":0.224,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.46400000000000002,"5min":0.495,"15min":0.495},"last":0.23999999999999999}]},{"osd":2,"last update":"Fri Mar 22 07:05:54 2024","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.372,"15min":0.38100000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.48199999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.38900000000000001,"15min":0.40000000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.22700000000000001}]},{"osd":3,"last update":"Fri Mar 22 07:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.443},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Mar 22 07:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.35999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446928,"statfs":{"total":95995035648,"available":95689654272,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32700000000000001,"15min":0.32400000000000001},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.318,"15min":0.312},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.5}]},{"osd":2,"last update":"Fri Mar 22 07:05:54 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.36199999999999999,"15min":0.36399999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.50700000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Mar 22 07:05:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Mar 22 07:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T07:05:58.702 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-22T07:05:58.702 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-22T07:05:58.703 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-22T07:05:58.703 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-22T07:05:58.730 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:58 smithi081 ceph-mon[29245]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:05:58.979 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:05:59.777 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:05:59 smithi081 ceph-mon[29245]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:05:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:05:59 smithi149 ceph-mon[36068]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:06:00.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.509+0000 7f1e7374b700 1 -- 172.21.15.81:0/3971798615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 msgr2=0x7f1e6c106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:00.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.509+0000 7f1e7374b700 1 --2- 172.21.15.81:0/3971798615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 0x7f1e6c106df0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c004940 tx=0x7f1e5c009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:00.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.509+0000 7f1e7374b700 1 -- 172.21.15.81:0/3971798615 shutdown_connections 2024-03-22T07:06:00.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.509+0000 7f1e7374b700 1 --2- 172.21.15.81:0/3971798615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 0x7f1e6c106df0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:00.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.509+0000 7f1e7374b700 1 --2- 172.21.15.81:0/3971798615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e6c0fde10 0x7f1e6c0fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:00.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.509+0000 7f1e7374b700 1 -- 172.21.15.81:0/3971798615 >> 172.21.15.81:0/3971798615 conn(0x7f1e6c0f7860 msgr2=0x7f1e6c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.509+0000 7f1e7374b700 1 -- 172.21.15.81:0/3971798615 shutdown_connections 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.509+0000 7f1e7374b700 1 -- 172.21.15.81:0/3971798615 wait complete. 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e7374b700 1 Processor -- start 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e7374b700 1 -- start start 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e7374b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e6c0fde10 0x7f1e6c113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e7374b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 0x7f1e6c1140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e7374b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c114740 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e7374b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c10db50 con 0x7f1e6c0fde10 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e70ce6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 0x7f1e6c1140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e70ce6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 0x7f1e6c1140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51932/0 (socket says 172.21.15.81:51932) 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e70ce6700 1 -- 172.21.15.81:0/2391433220 learned_addr learned my addr 172.21.15.81:0/2391433220 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e714e7700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e6c0fde10 0x7f1e6c113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:00.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e70ce6700 1 -- 172.21.15.81:0/2391433220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e6c0fde10 msgr2=0x7f1e6c113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:00.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e70ce6700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e6c0fde10 0x7f1e6c113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:00.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e70ce6700 1 -- 172.21.15.81:0/2391433220 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c005040 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e70ce6700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 0x7f1e6c1140f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c004890 tx=0x7f1e5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:00.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e61ffb700 1 -- 172.21.15.81:0/2391433220 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e5c00e040 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e61ffb700 1 -- 172.21.15.81:0/2391433220 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e5c009af0 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6c10ddd0 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.510+0000 7f1e61ffb700 1 -- 172.21.15.81:0/2391433220 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e5c01c630 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.511+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6c10e310 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.512+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6c066770 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.512+0000 7f1e61ffb700 1 -- 172.21.15.81:0/2391433220 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1e5c01c8f0 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.512+0000 7f1e61ffb700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e580692a0 0x7f1e5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:00.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.512+0000 7f1e61ffb700 1 -- 172.21.15.81:0/2391433220 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1e5c088130 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.512+0000 7f1e714e7700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e580692a0 0x7f1e5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:00.514 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.513+0000 7f1e714e7700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e580692a0 0x7f1e5806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1e64004ec0 tx=0x7f1e64009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:00.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.517+0000 7f1e61ffb700 1 -- 172.21.15.81:0/2391433220 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1e5c0547a0 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.743 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.743+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1e6c10f230 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.743 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:00 smithi081 ceph-mon[29245]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:00.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.743+0000 7f1e61ffb700 1 -- 172.21.15.81:0/2391433220 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1e5c057dc0 con 0x7f1e6c0fe7b0 2024-03-22T07:06:00.744 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:00.744 INFO:teuthology.orchestra.run.smithi081.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-22T07:06:00.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e580692a0 msgr2=0x7f1e5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:00.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e580692a0 0x7f1e5806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1e64004ec0 tx=0x7f1e64009040 comp rx=0 tx=0).stop 2024-03-22T07:06:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 msgr2=0x7f1e6c1140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 0x7f1e6c1140f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c004890 tx=0x7f1e5c009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 shutdown_connections 2024-03-22T07:06:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1e580692a0 0x7f1e5806b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e6c0fde10 0x7f1e6c113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 --2- 172.21.15.81:0/2391433220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e6c0fe7b0 0x7f1e6c1140f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 >> 172.21.15.81:0/2391433220 conn(0x7f1e6c0f7860 msgr2=0x7f1e6c0f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 shutdown_connections 2024-03-22T07:06:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:00.745+0000 7f1e7374b700 1 -- 172.21.15.81:0/2391433220 wait complete. 2024-03-22T07:06:00.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:00 smithi149 ceph-mon[36068]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:01.428 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-22T07:06:01.428 INFO:tasks.cephadm:Setup complete, yielding 2024-03-22T07:06:01.428 INFO:teuthology.run_tasks:Running task print... 2024-03-22T07:06:01.437 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-03-22T07:06:01.438 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:06:01.475 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:06:01.476 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-22T07:06:01.735 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:01.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:01 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2391433220' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T07:06:01.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:01 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2391433220' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T07:06:02.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:02 smithi081 ceph-mon[29245]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:02.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:02 smithi149 ceph-mon[36068]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:03.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.396+0000 7f1614774700 1 -- 172.21.15.81:0/1014966450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 msgr2=0x7f160c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.396+0000 7f1614774700 1 --2- 172.21.15.81:0/1014966450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 0x7f160c0fd280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1608004940 tx=0x7f1608009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.396+0000 7f1614774700 1 -- 172.21.15.81:0/1014966450 shutdown_connections 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.396+0000 7f1614774700 1 --2- 172.21.15.81:0/1014966450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f160c0fdfe0 0x7f160c0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.396+0000 7f1614774700 1 --2- 172.21.15.81:0/1014966450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 0x7f160c0fd280 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.396+0000 7f1614774700 1 -- 172.21.15.81:0/1014966450 >> 172.21.15.81:0/1014966450 conn(0x7f160c0f8550 msgr2=0x7f160c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.396+0000 7f1614774700 1 -- 172.21.15.81:0/1014966450 shutdown_connections 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.396+0000 7f1614774700 1 -- 172.21.15.81:0/1014966450 wait complete. 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1614774700 1 Processor -- start 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1614774700 1 -- start start 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1614774700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 0x7f160c071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1614774700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f160c0fdfe0 0x7f160c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1614774700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c070850 con 0x7f160c0fce70 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1614774700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c0709c0 con 0x7f160c0fdfe0 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1612510700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 0x7f160c071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:03.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1612510700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 0x7f160c071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51954/0 (socket says 172.21.15.81:51954) 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1612510700 1 -- 172.21.15.81:0/3535896024 learned_addr learned my addr 172.21.15.81:0/3535896024 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1611d0f700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f160c0fdfe0 0x7f160c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1612510700 1 -- 172.21.15.81:0/3535896024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f160c0fdfe0 msgr2=0x7f160c070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1612510700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f160c0fdfe0 0x7f160c070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1612510700 1 -- 172.21.15.81:0/3535896024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1608005040 con 0x7f160c0fce70 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.397+0000 7f1611d0f700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f160c0fdfe0 0x7f160c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.398+0000 7f1612510700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 0x7f160c071c60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f160800b970 tx=0x7f1608009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.398+0000 7f1602ffd700 1 -- 172.21.15.81:0/3535896024 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f160800e040 con 0x7f160c0fce70 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.398+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160c070c40 con 0x7f160c0fce70 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.398+0000 7f1602ffd700 1 -- 172.21.15.81:0/3535896024 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1608004290 con 0x7f160c0fce70 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.398+0000 7f1602ffd700 1 -- 172.21.15.81:0/3535896024 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f160801c630 con 0x7f160c0fce70 2024-03-22T07:06:03.400 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.398+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c115a50 con 0x7f160c0fce70 2024-03-22T07:06:03.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.399+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160c04c710 con 0x7f160c0fce70 2024-03-22T07:06:03.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.399+0000 7f1602ffd700 1 -- 172.21.15.81:0/3535896024 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f16080139f0 con 0x7f160c0fce70 2024-03-22T07:06:03.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.399+0000 7f1602ffd700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f15f80692a0 0x7f15f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:03.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.400+0000 7f1602ffd700 1 -- 172.21.15.81:0/3535896024 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f16080881f0 con 0x7f160c0fce70 2024-03-22T07:06:03.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.400+0000 7f1611d0f700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f15f80692a0 0x7f15f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:03.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.400+0000 7f1611d0f700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f15f80692a0 0x7f15f806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009b10 tx=0x7f15fc008040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:03.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.404+0000 7f1602ffd700 1 -- 172.21.15.81:0/3535896024 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f16080582f0 con 0x7f160c0fce70 2024-03-22T07:06:03.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.588+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f160c105950 con 0x7f160c0fce70 2024-03-22T07:06:03.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.593+0000 7f1602ffd700 1 -- 172.21.15.81:0/3535896024 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f1608057e80 con 0x7f160c0fce70 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f15f80692a0 msgr2=0x7f15f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f15f80692a0 0x7f15f806b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009b10 tx=0x7f15fc008040 comp rx=0 tx=0).stop 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 msgr2=0x7f160c071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 0x7f160c071c60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f160800b970 tx=0x7f1608009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 shutdown_connections 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f15f80692a0 0x7f15f806b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160c0fce70 0x7f160c071c60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 --2- 172.21.15.81:0/3535896024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f160c0fdfe0 0x7f160c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 >> 172.21.15.81:0/3535896024 conn(0x7f160c0f8550 msgr2=0x7f160c0f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.595+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 shutdown_connections 2024-03-22T07:06:03.596 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:03.596+0000 7f1614774700 1 -- 172.21.15.81:0/3535896024 wait complete. 2024-03-22T07:06:04.467 INFO:teuthology.run_tasks:Running task print... 2024-03-22T07:06:04.476 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-22T07:06:04.477 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:06:04.485 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:06:04.485 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-22T07:06:04.744 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:04.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:04 smithi081 ceph-mon[29245]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:04.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:04 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3535896024' entity='client.admin' 2024-03-22T07:06:04.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:06:04.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:04.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:06:04.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:04 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:04 smithi149 ceph-mon[36068]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:04 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3535896024' entity='client.admin' 2024-03-22T07:06:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:06:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:06:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:04 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.440+0000 7f565e84a700 1 -- 172.21.15.81:0/516617645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580fe450 msgr2=0x7f56580fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.440+0000 7f565e84a700 1 --2- 172.21.15.81:0/516617645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580fe450 0x7f56580fe860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f564c004940 tx=0x7f564c009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.440+0000 7f565e84a700 1 -- 172.21.15.81:0/516617645 shutdown_connections 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.440+0000 7f565e84a700 1 --2- 172.21.15.81:0/516617645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56580feda0 0x7f56580faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.440+0000 7f565e84a700 1 --2- 172.21.15.81:0/516617645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580fe450 0x7f56580fe860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.441+0000 7f565e84a700 1 -- 172.21.15.81:0/516617645 >> 172.21.15.81:0/516617645 conn(0x7f56580f6c10 msgr2=0x7f56580f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.441+0000 7f565e84a700 1 -- 172.21.15.81:0/516617645 shutdown_connections 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.441+0000 7f565e84a700 1 -- 172.21.15.81:0/516617645 wait complete. 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.441+0000 7f565e84a700 1 Processor -- start 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.441+0000 7f565e84a700 1 -- start start 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.441+0000 7f565e84a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56580fe450 0x7f565810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:06.443 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.441+0000 7f565e84a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580feda0 0x7f565810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.441+0000 7f565e84a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565810e4e0 con 0x7f56580feda0 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.441+0000 7f565e84a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5658112e80 con 0x7f56580fe450 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f56577fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580feda0 0x7f565810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f56577fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580feda0 0x7f565810de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46644/0 (socket says 172.21.15.81:46644) 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f56577fe700 1 -- 172.21.15.81:0/695436144 learned_addr learned my addr 172.21.15.81:0/695436144 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f5657fff700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56580fe450 0x7f565810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f56577fe700 1 -- 172.21.15.81:0/695436144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56580fe450 msgr2=0x7f565810d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f56577fe700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56580fe450 0x7f565810d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f56577fe700 1 -- 172.21.15.81:0/695436144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c005040 con 0x7f56580feda0 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f56577fe700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580feda0 0x7f565810de50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f564800b790 tx=0x7f5648009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f5654ff9700 1 -- 172.21.15.81:0/695436144 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f564800f770 con 0x7f56580feda0 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56581130e0 con 0x7f56580feda0 2024-03-22T07:06:06.444 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f5654ff9700 1 -- 172.21.15.81:0/695436144 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f564800fda0 con 0x7f56580feda0 2024-03-22T07:06:06.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f5654ff9700 1 -- 172.21.15.81:0/695436144 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f564800d310 con 0x7f56580feda0 2024-03-22T07:06:06.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.442+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5658113650 con 0x7f56580feda0 2024-03-22T07:06:06.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.443+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565804c710 con 0x7f56580feda0 2024-03-22T07:06:06.445 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.444+0000 7f5654ff9700 1 -- 172.21.15.81:0/695436144 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f564800f8d0 con 0x7f56580feda0 2024-03-22T07:06:06.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.444+0000 7f5654ff9700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f56400692a0 0x7f564006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:06.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.444+0000 7f5654ff9700 1 -- 172.21.15.81:0/695436144 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5648086ed0 con 0x7f56580feda0 2024-03-22T07:06:06.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.444+0000 7f5657fff700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f56400692a0 0x7f564006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:06.446 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.444+0000 7f5657fff700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f56400692a0 0x7f564006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f564c00b3b0 tx=0x7f564c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:06.448 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.448+0000 7f5654ff9700 1 -- 172.21.15.81:0/695436144 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5648056fd0 con 0x7f56580feda0 2024-03-22T07:06:06.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.651+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56581024d0 con 0x7f56400692a0 2024-03-22T07:06:06.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.651+0000 7f5654ff9700 1 -- 172.21.15.81:0/695436144 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f56581024d0 con 0x7f56400692a0 2024-03-22T07:06:06.652 INFO:teuthology.orchestra.run.smithi081.stdout:Backend: cephadm 2024-03-22T07:06:06.652 INFO:teuthology.orchestra.run.smithi081.stdout:Available: Yes 2024-03-22T07:06:06.652 INFO:teuthology.orchestra.run.smithi081.stdout:Paused: No 2024-03-22T07:06:06.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.653+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f56400692a0 msgr2=0x7f564006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:06.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.653+0000 7f565e84a700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f56400692a0 0x7f564006b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f564c00b3b0 tx=0x7f564c009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:06.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.653+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580feda0 msgr2=0x7f565810de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:06.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.653+0000 7f565e84a700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580feda0 0x7f565810de50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f564800b790 tx=0x7f5648009250 comp rx=0 tx=0).stop 2024-03-22T07:06:06.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.654+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 shutdown_connections 2024-03-22T07:06:06.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.654+0000 7f565e84a700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f56400692a0 0x7f564006b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:06.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.654+0000 7f565e84a700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56580fe450 0x7f565810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:06.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.654+0000 7f565e84a700 1 --2- 172.21.15.81:0/695436144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56580feda0 0x7f565810de50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:06.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.654+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 >> 172.21.15.81:0/695436144 conn(0x7f56580f6c10 msgr2=0x7f56580f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:06.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.654+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 shutdown_connections 2024-03-22T07:06:06.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:06.654+0000 7f565e84a700 1 -- 172.21.15.81:0/695436144 wait complete. 2024-03-22T07:06:06.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:06 smithi081 ceph-mon[29245]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:06.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:06 smithi149 ceph-mon[36068]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:07.289 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-22T07:06:07.546 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:07.891 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:07 smithi081 ceph-mon[29245]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:08.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:07 smithi149 ceph-mon[36068]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:08.802 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:08 smithi081 ceph-mon[29245]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 -- 172.21.15.81:0/196340497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040fe450 msgr2=0x7f02040fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 --2- 172.21.15.81:0/196340497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040fe450 0x7f02040fe860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0200004940 tx=0x7f0200009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 -- 172.21.15.81:0/196340497 shutdown_connections 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 --2- 172.21.15.81:0/196340497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02040feda0 0x7f02040faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 --2- 172.21.15.81:0/196340497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040fe450 0x7f02040fe860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 -- 172.21.15.81:0/196340497 >> 172.21.15.81:0/196340497 conn(0x7f02040f6c10 msgr2=0x7f02040f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 -- 172.21.15.81:0/196340497 shutdown_connections 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 -- 172.21.15.81:0/196340497 wait complete. 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 Processor -- start 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 -- start start 2024-03-22T07:06:09.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02040fe450 0x7f02040fda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040feda0 0x7f02040fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02040fc650 con 0x7f02040feda0 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f020ba86700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02040fc7c0 con 0x7f02040fe450 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.004+0000 7f0209021700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040feda0 0x7f02040fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f0209021700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040feda0 0x7f02040fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46658/0 (socket says 172.21.15.81:46658) 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f0209021700 1 -- 172.21.15.81:0/2017699914 learned_addr learned my addr 172.21.15.81:0/2017699914 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f0209822700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02040fe450 0x7f02040fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f0209021700 1 -- 172.21.15.81:0/2017699914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02040fe450 msgr2=0x7f02040fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f0209021700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02040fe450 0x7f02040fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f0209021700 1 -- 172.21.15.81:0/2017699914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0200005040 con 0x7f02040feda0 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f0209021700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040feda0 0x7f02040fc110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01f400b790 tx=0x7f01f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f01fa7fc700 1 -- 172.21.15.81:0/2017699914 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01f400f770 con 0x7f02040feda0 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f01fa7fc700 1 -- 172.21.15.81:0/2017699914 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01f400fda0 con 0x7f02040feda0 2024-03-22T07:06:09.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02040fcaa0 con 0x7f02040feda0 2024-03-22T07:06:09.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f01fa7fc700 1 -- 172.21.15.81:0/2017699914 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01f400d310 con 0x7f02040feda0 2024-03-22T07:06:09.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.005+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0204115a20 con 0x7f02040feda0 2024-03-22T07:06:09.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.006+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020404c710 con 0x7f02040feda0 2024-03-22T07:06:09.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.006+0000 7f01fa7fc700 1 -- 172.21.15.81:0/2017699914 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f01f4010420 con 0x7f02040feda0 2024-03-22T07:06:09.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.007+0000 7f01fa7fc700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f01f00692a0 0x7f01f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:09.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.007+0000 7f01fa7fc700 1 -- 172.21.15.81:0/2017699914 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f01f4086e70 con 0x7f02040feda0 2024-03-22T07:06:09.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.007+0000 7f0209822700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f01f00692a0 0x7f01f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:09.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.007+0000 7f0209822700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f01f00692a0 0x7f01f006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f020000b3b0 tx=0x7f0200009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:09.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.011+0000 7f01fa7fc700 1 -- 172.21.15.81:0/2017699914 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f01f4056f70 con 0x7f02040feda0 2024-03-22T07:06:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:08 smithi149 ceph-mon[36068]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:09.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.208+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f02041024d0 con 0x7f01f00692a0 2024-03-22T07:06:09.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.216+0000 7f01fa7fc700 1 -- 172.21.15.81:0/2017699914 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f02041024d0 con 0x7f01f00692a0 2024-03-22T07:06:09.216 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (3m) 2m ago 5m 26.8M - 0.25.0 c8568f914cd2 f83c4e446eae 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (5m) 2m ago 5m 6660k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (4m) 67s ago 4m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (3m) 2m ago 4m 78.1M - 9.4.7 954c08fa6188 5b1c9b6c3c44 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:9283 running (6m) 2m ago 6m 472M - 17.2.7-726-g9c4e0c34 837124da804f 3ef26f13517a 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283 running (4m) 67s ago 4m 434M - 17.2.7-726-g9c4e0c34 837124da804f eefa675aaa8e 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (6m) 2m ago 6m 41.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (4m) 67s ago 4m 40.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (5m) 2m ago 5m 13.4M - 1.5.0 0da6a335fe13 c04b2c4a7e70 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (4m) 67s ago 4m 16.0M - 1.5.0 0da6a335fe13 fbdea05540dc 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (3m) 2m ago 3m 38.4M 4288M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (2m) 2m ago 2m 39.6M 4288M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (2m) 2m ago 2m 37.0M 4288M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:06:09.217 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (2m) 67s ago 2m 41.4M 1554M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:06:09.218 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (98s) 67s ago 98s 35.6M 1554M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:06:09.218 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (72s) 67s ago 72s 12.1M 1554M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:06:09.218 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (3m) 2m ago 4m 36.1M - 2.43.0 a07b618ecd1d 92a93d9b8284 2024-03-22T07:06:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.218+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f01f00692a0 msgr2=0x7f01f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.218+0000 7f020ba86700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f01f00692a0 0x7f01f006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f020000b3b0 tx=0x7f0200009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.218+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040feda0 msgr2=0x7f02040fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.218+0000 7f020ba86700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040feda0 0x7f02040fc110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01f400b790 tx=0x7f01f4009250 comp rx=0 tx=0).stop 2024-03-22T07:06:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.218+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 shutdown_connections 2024-03-22T07:06:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.218+0000 7f020ba86700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f01f00692a0 0x7f01f006b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.218+0000 7f020ba86700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02040fe450 0x7f02040fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.218+0000 7f020ba86700 1 --2- 172.21.15.81:0/2017699914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f02040feda0 0x7f02040fc110 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:09.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.219+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 >> 172.21.15.81:0/2017699914 conn(0x7f02040f6c10 msgr2=0x7f02040f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:09.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.219+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 shutdown_connections 2024-03-22T07:06:09.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:09.219+0000 7f020ba86700 1 -- 172.21.15.81:0/2017699914 wait complete. 2024-03-22T07:06:09.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:06:09.851 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-22T07:06:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:06:10.104 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:10.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:10 smithi081 ceph-mon[29245]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:10.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:10 smithi081 ceph-mon[29245]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:11.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:10 smithi149 ceph-mon[36068]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:11.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:10 smithi149 ceph-mon[36068]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.660+0000 7ffb73933700 1 -- 172.21.15.81:0/4197221073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fe070 msgr2=0x7ffb6c0fe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.660+0000 7ffb73933700 1 --2- 172.21.15.81:0/4197221073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fe070 0x7ffb6c0fe4e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffb68004940 tx=0x7ffb68009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.660+0000 7ffb73933700 1 -- 172.21.15.81:0/4197221073 shutdown_connections 2024-03-22T07:06:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.660+0000 7ffb73933700 1 --2- 172.21.15.81:0/4197221073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fe070 0x7ffb6c0fe4e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.660+0000 7ffb73933700 1 --2- 172.21.15.81:0/4197221073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb6c0fcf00 0x7ffb6c0fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.660+0000 7ffb73933700 1 -- 172.21.15.81:0/4197221073 >> 172.21.15.81:0/4197221073 conn(0x7ffb6c0f85a0 msgr2=0x7ffb6c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.660+0000 7ffb73933700 1 -- 172.21.15.81:0/4197221073 shutdown_connections 2024-03-22T07:06:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.660+0000 7ffb73933700 1 -- 172.21.15.81:0/4197221073 wait complete. 2024-03-22T07:06:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.660+0000 7ffb73933700 1 Processor -- start 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb73933700 1 -- start start 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb73933700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fcf00 0x7ffb6c10da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb73933700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb6c0fe070 0x7ffb6c10df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb73933700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c10e590 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb73933700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c10ede0 con 0x7ffb6c0fe070 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb716cf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fcf00 0x7ffb6c10da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb716cf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fcf00 0x7ffb6c10da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46670/0 (socket says 172.21.15.81:46670) 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb716cf700 1 -- 172.21.15.81:0/1929966800 learned_addr learned my addr 172.21.15.81:0/1929966800 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb70ece700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb6c0fe070 0x7ffb6c10df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb716cf700 1 -- 172.21.15.81:0/1929966800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb6c0fe070 msgr2=0x7ffb6c10df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb716cf700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb6c0fe070 0x7ffb6c10df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb716cf700 1 -- 172.21.15.81:0/1929966800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb68005040 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb716cf700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fcf00 0x7ffb6c10da50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00c830 tx=0x7ffb5c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb61ffb700 1 -- 172.21.15.81:0/1929966800 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb5c00a760 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb61ffb700 1 -- 172.21.15.81:0/1929966800 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb5c00ad90 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c10f0c0 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.661+0000 7ffb61ffb700 1 -- 172.21.15.81:0/1929966800 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb5c0113a0 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.662+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c10f600 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.663+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c05f5f0 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.663+0000 7ffb61ffb700 1 -- 172.21.15.81:0/1929966800 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ffb5c012400 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.663+0000 7ffb61ffb700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ffb580692a0 0x7ffb5806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.663+0000 7ffb61ffb700 1 -- 172.21.15.81:0/1929966800 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffb5c086e00 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.663+0000 7ffb70ece700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ffb580692a0 0x7ffb5806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.664+0000 7ffb70ece700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ffb580692a0 0x7ffb5806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800b3b0 tx=0x7ffb68009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:11.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.668+0000 7ffb61ffb700 1 -- 172.21.15.81:0/1929966800 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ffb5c056fa0 con 0x7ffb6c0fcf00 2024-03-22T07:06:11.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.860+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb6c10fa30 con 0x7ffb580692a0 2024-03-22T07:06:11.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.864+0000 7ffb61ffb700 1 -- 172.21.15.81:0/1929966800 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7ffb6c10fa30 con 0x7ffb580692a0 2024-03-22T07:06:11.865 INFO:teuthology.orchestra.run.smithi081.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-22T07:06:11.865 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-03-22T07:06:11.865 INFO:teuthology.orchestra.run.smithi081.stdout:crash 2/2 2m ago 5m * 2024-03-22T07:06:11.865 INFO:teuthology.orchestra.run.smithi081.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-03-22T07:06:11.865 INFO:teuthology.orchestra.run.smithi081.stdout:mgr 2/2 2m ago 5m count:2 2024-03-22T07:06:11.866 INFO:teuthology.orchestra.run.smithi081.stdout:mon 2/2 2m ago 5m smithi081:172.21.15.81=smithi081;smithi149:172.21.15.149=smithi149;count:2 2024-03-22T07:06:11.866 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-03-22T07:06:11.866 INFO:teuthology.orchestra.run.smithi081.stdout:osd 6 2m ago - 2024-03-22T07:06:11.866 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-03-22T07:06:11.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ffb580692a0 msgr2=0x7ffb5806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ffb580692a0 0x7ffb5806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800b3b0 tx=0x7ffb68009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fcf00 msgr2=0x7ffb6c10da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fcf00 0x7ffb6c10da50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00c830 tx=0x7ffb5c004470 comp rx=0 tx=0).stop 2024-03-22T07:06:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 shutdown_connections 2024-03-22T07:06:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7ffb580692a0 0x7ffb5806b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:11.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb6c0fcf00 0x7ffb6c10da50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:11.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 --2- 172.21.15.81:0/1929966800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb6c0fe070 0x7ffb6c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:11.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 >> 172.21.15.81:0/1929966800 conn(0x7ffb6c0f85a0 msgr2=0x7ffb6c0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:11.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 shutdown_connections 2024-03-22T07:06:11.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:11.866+0000 7ffb73933700 1 -- 172.21.15.81:0/1929966800 wait complete. 2024-03-22T07:06:12.473 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-22T07:06:12.731 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:12.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:12 smithi081 ceph-mon[29245]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:13.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:12 smithi149 ceph-mon[36068]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:13.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:13 smithi081 ceph-mon[29245]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:14.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:13 smithi149 ceph-mon[36068]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:14.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.356+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/3654031580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c94066220 msgr2=0x7f1c940ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:14.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.356+0000 7f1c9ba1d700 1 --2- 172.21.15.81:0/3654031580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c94066220 0x7f1c940ffa80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c90004940 tx=0x7f1c90009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:14.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.356+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/3654031580 shutdown_connections 2024-03-22T07:06:14.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.356+0000 7f1c9ba1d700 1 --2- 172.21.15.81:0/3654031580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c940fffc0 0x7f1c941023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:14.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.356+0000 7f1c9ba1d700 1 --2- 172.21.15.81:0/3654031580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c94066220 0x7f1c940ffa80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:14.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.356+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/3654031580 >> 172.21.15.81:0/3654031580 conn(0x7f1c940f7430 msgr2=0x7f1c940f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.356+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/3654031580 shutdown_connections 2024-03-22T07:06:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.356+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/3654031580 wait complete. 2024-03-22T07:06:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c9ba1d700 1 Processor -- start 2024-03-22T07:06:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c9ba1d700 1 -- start start 2024-03-22T07:06:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c9ba1d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c94066220 0x7f1c9410dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c9ba1d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c940fffc0 0x7f1c9410e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c9ba1d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9410e690 con 0x7f1c940fffc0 2024-03-22T07:06:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c9ba1d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9410ee30 con 0x7f1c94066220 2024-03-22T07:06:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c98fb8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c940fffc0 0x7f1c9410e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c98fb8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c940fffc0 0x7f1c9410e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46690/0 (socket says 172.21.15.81:46690) 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c98fb8700 1 -- 172.21.15.81:0/348654406 learned_addr learned my addr 172.21.15.81:0/348654406 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c997b9700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c94066220 0x7f1c9410dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c98fb8700 1 -- 172.21.15.81:0/348654406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c94066220 msgr2=0x7f1c9410dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c98fb8700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c94066220 0x7f1c9410dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c98fb8700 1 -- 172.21.15.81:0/348654406 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c90005040 con 0x7f1c940fffc0 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c98fb8700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c940fffc0 0x7f1c9410e000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400b790 tx=0x7f1c84009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c89ffb700 1 -- 172.21.15.81:0/348654406 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c8400f6f0 con 0x7f1c940fffc0 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9410f110 con 0x7f1c940fffc0 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c89ffb700 1 -- 172.21.15.81:0/348654406 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c8400fd20 con 0x7f1c940fffc0 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.357+0000 7f1c89ffb700 1 -- 172.21.15.81:0/348654406 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c840103a0 con 0x7f1c940fffc0 2024-03-22T07:06:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.358+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9410f6d0 con 0x7f1c940fffc0 2024-03-22T07:06:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.359+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9405f5f0 con 0x7f1c940fffc0 2024-03-22T07:06:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.359+0000 7f1c89ffb700 1 -- 172.21.15.81:0/348654406 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1c84011400 con 0x7f1c940fffc0 2024-03-22T07:06:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.359+0000 7f1c89ffb700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1c800692a0 0x7f1c8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.359+0000 7f1c89ffb700 1 -- 172.21.15.81:0/348654406 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1c84086d10 con 0x7f1c940fffc0 2024-03-22T07:06:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.360+0000 7f1c997b9700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1c800692a0 0x7f1c8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.360+0000 7f1c997b9700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1c800692a0 0x7f1c8006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000b970 tx=0x7f1c90009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:14.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.363+0000 7f1c89ffb700 1 -- 172.21.15.81:0/348654406 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1c84056d90 con 0x7f1c940fffc0 2024-03-22T07:06:14.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.554+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c9410f9c0 con 0x7f1c800692a0 2024-03-22T07:06:14.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.556+0000 7f1c89ffb700 1 -- 172.21.15.81:0/348654406 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f1c9410f9c0 con 0x7f1c800692a0 2024-03-22T07:06:14.556 INFO:teuthology.orchestra.run.smithi081.stdout:HOST ADDR LABELS STATUS 2024-03-22T07:06:14.556 INFO:teuthology.orchestra.run.smithi081.stdout:smithi081 172.21.15.81 2024-03-22T07:06:14.556 INFO:teuthology.orchestra.run.smithi081.stdout:smithi149 172.21.15.149 2024-03-22T07:06:14.556 INFO:teuthology.orchestra.run.smithi081.stdout:2 hosts in cluster 2024-03-22T07:06:14.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1c800692a0 msgr2=0x7f1c8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:14.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1c800692a0 0x7f1c8006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000b970 tx=0x7f1c90009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:14.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c940fffc0 msgr2=0x7f1c9410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:14.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c940fffc0 0x7f1c9410e000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400b790 tx=0x7f1c84009220 comp rx=0 tx=0).stop 2024-03-22T07:06:14.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 shutdown_connections 2024-03-22T07:06:14.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f1c800692a0 0x7f1c8006b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:14.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c94066220 0x7f1c9410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:14.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 --2- 172.21.15.81:0/348654406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c940fffc0 0x7f1c9410e000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:14.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 >> 172.21.15.81:0/348654406 conn(0x7f1c940f7430 msgr2=0x7f1c940f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:14.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 shutdown_connections 2024-03-22T07:06:14.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:14.558+0000 7f1c9ba1d700 1 -- 172.21.15.81:0/348654406 wait complete. 2024-03-22T07:06:14.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:14 smithi081 ceph-mon[29245]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:14 smithi149 ceph-mon[36068]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:15.204 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-22T07:06:15.459 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:15.898 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:15 smithi081 ceph-mon[29245]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:16.003 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:15 smithi149 ceph-mon[36068]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:16.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:16 smithi081 ceph-mon[29245]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:17.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.011+0000 7fe7822e9700 1 -- 172.21.15.81:0/1468503401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 msgr2=0x7fe77c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:17.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.011+0000 7fe7822e9700 1 --2- 172.21.15.81:0/1468503401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 0x7fe77c0fc0e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe764004940 tx=0x7fe764009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:17.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.011+0000 7fe7822e9700 1 -- 172.21.15.81:0/1468503401 shutdown_connections 2024-03-22T07:06:17.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.011+0000 7fe7822e9700 1 --2- 172.21.15.81:0/1468503401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe77c0fcf80 0x7fe77c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:17.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.011+0000 7fe7822e9700 1 --2- 172.21.15.81:0/1468503401 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 0x7fe77c0fc0e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:17.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.011+0000 7fe7822e9700 1 -- 172.21.15.81:0/1468503401 >> 172.21.15.81:0/1468503401 conn(0x7fe77c0f7430 msgr2=0x7fe77c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:17.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.011+0000 7fe7822e9700 1 -- 172.21.15.81:0/1468503401 shutdown_connections 2024-03-22T07:06:17.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.011+0000 7fe7822e9700 1 -- 172.21.15.81:0/1468503401 wait complete. 2024-03-22T07:06:17.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.011+0000 7fe7822e9700 1 Processor -- start 2024-03-22T07:06:17.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe7822e9700 1 -- start start 2024-03-22T07:06:17.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe7822e9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 0x7fe77c0ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:17.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe7822e9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe77c0fcf80 0x7fe77c0fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe7822e9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c0fe850 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe7822e9700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c0fe990 con 0x7fe77c0fcf80 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe77bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 0x7fe77c0ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe77bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 0x7fe77c0ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49332/0 (socket says 172.21.15.81:49332) 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe77bfff700 1 -- 172.21.15.81:0/4034332727 learned_addr learned my addr 172.21.15.81:0/4034332727 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe77b7fe700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe77c0fcf80 0x7fe77c0fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe77bfff700 1 -- 172.21.15.81:0/4034332727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe77c0fcf80 msgr2=0x7fe77c0fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe77bfff700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe77c0fcf80 0x7fe77c0fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe77bfff700 1 -- 172.21.15.81:0/4034332727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe764005040 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe77bfff700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 0x7fe77c0ffda0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe76400b3b0 tx=0x7fe764009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.012+0000 7fe778ff9700 1 -- 172.21.15.81:0/4034332727 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76400e040 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.013+0000 7fe778ff9700 1 -- 172.21.15.81:0/4034332727 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76400bbf0 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.013+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c0fec10 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.013+0000 7fe778ff9700 1 -- 172.21.15.81:0/4034332727 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76401c630 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.013+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77c1159f0 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.013+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77c101b10 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.014+0000 7fe778ff9700 1 -- 172.21.15.81:0/4034332727 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe76400bd60 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.014+0000 7fe778ff9700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe7680692a0 0x7fe76806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:17.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.014+0000 7fe778ff9700 1 -- 172.21.15.81:0/4034332727 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe764087b70 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.014+0000 7fe77b7fe700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe7680692a0 0x7fe76806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:17.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.015+0000 7fe77b7fe700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe7680692a0 0x7fe76806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe77c100380 tx=0x7fe76c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:17.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.019+0000 7fe778ff9700 1 -- 172.21.15.81:0/4034332727 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe7640541e0 con 0x7fe77c0fbcd0 2024-03-22T07:06:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:16 smithi149 ceph-mon[36068]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:17.210 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.210+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe77c104930 con 0x7fe7680692a0 2024-03-22T07:06:17.212 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.212+0000 7fe778ff9700 1 -- 172.21.15.81:0/4034332727 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fe77c104930 con 0x7fe7680692a0 2024-03-22T07:06:17.212 INFO:teuthology.orchestra.run.smithi081.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-22T07:06:17.212 INFO:teuthology.orchestra.run.smithi081.stdout:smithi081 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400N7400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-22T07:06:17.212 INFO:teuthology.orchestra.run.smithi081.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 71s ago Has a FileSystem, LVM detected 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe7680692a0 msgr2=0x7fe76806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe7680692a0 0x7fe76806b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe77c100380 tx=0x7fe76c00b040 comp rx=0 tx=0).stop 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 msgr2=0x7fe77c0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 0x7fe77c0ffda0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe76400b3b0 tx=0x7fe764009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 shutdown_connections 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe7680692a0 0x7fe76806b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe77c0fbcd0 0x7fe77c0ffda0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 --2- 172.21.15.81:0/4034332727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe77c0fcf80 0x7fe77c0fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 >> 172.21.15.81:0/4034332727 conn(0x7fe77c0f7430 msgr2=0x7fe77c065b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 shutdown_connections 2024-03-22T07:06:17.215 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:17.214+0000 7fe7822e9700 1 -- 172.21.15.81:0/4034332727 wait complete. 2024-03-22T07:06:17.803 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:06:17.813 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:06:17.813 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-22T07:06:18.074 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:18.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:18 smithi081 ceph-mon[29245]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:18.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:18 smithi081 ceph-mon[29245]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:19.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:18 smithi149 ceph-mon[36068]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:19.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:18 smithi149 ceph-mon[36068]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:19.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.586+0000 7f17cdd84700 1 -- 172.21.15.81:0/3602242963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 msgr2=0x7f17c80fb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:19.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.586+0000 7f17cdd84700 1 --2- 172.21.15.81:0/3602242963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 0x7f17c80fb880 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f17b8004940 tx=0x7f17b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.586+0000 7f17cdd84700 1 -- 172.21.15.81:0/3602242963 shutdown_connections 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.586+0000 7f17cdd84700 1 --2- 172.21.15.81:0/3602242963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17c80fc720 0x7f17c80fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.586+0000 7f17cdd84700 1 --2- 172.21.15.81:0/3602242963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 0x7f17c80fb880 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.586+0000 7f17cdd84700 1 -- 172.21.15.81:0/3602242963 >> 172.21.15.81:0/3602242963 conn(0x7f17c80f6bd0 msgr2=0x7f17c80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17cdd84700 1 -- 172.21.15.81:0/3602242963 shutdown_connections 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17cdd84700 1 -- 172.21.15.81:0/3602242963 wait complete. 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17cdd84700 1 Processor -- start 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17cdd84700 1 -- start start 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17cdd84700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 0x7f17c810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17cdd84700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17c80fc720 0x7f17c810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17cdd84700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c810e4e0 con 0x7f17c80fb470 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17cdd84700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8112e80 con 0x7f17c80fc720 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17c77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 0x7f17c810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17c77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 0x7f17c810d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49348/0 (socket says 172.21.15.81:49348) 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.587+0000 7f17c77fe700 1 -- 172.21.15.81:0/367639564 learned_addr learned my addr 172.21.15.81:0/367639564 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.588+0000 7f17c77fe700 1 -- 172.21.15.81:0/367639564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17c80fc720 msgr2=0x7f17c810de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.588+0000 7f17c77fe700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17c80fc720 0x7f17c810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.588+0000 7f17c77fe700 1 -- 172.21.15.81:0/367639564 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b8005040 con 0x7f17c80fb470 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.588+0000 7f17c77fe700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 0x7f17c810d910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b970 tx=0x7f17b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.588+0000 7f17affff700 1 -- 172.21.15.81:0/367639564 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b800e040 con 0x7f17c80fb470 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.588+0000 7f17affff700 1 -- 172.21.15.81:0/367639564 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17b800bcc0 con 0x7f17c80fb470 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.588+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c8113080 con 0x7f17c80fb470 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.588+0000 7f17affff700 1 -- 172.21.15.81:0/367639564 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b8013660 con 0x7f17c80fb470 2024-03-22T07:06:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.588+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c81135f0 con 0x7f17c80fb470 2024-03-22T07:06:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.589+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c804c710 con 0x7f17c80fb470 2024-03-22T07:06:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.590+0000 7f17affff700 1 -- 172.21.15.81:0/367639564 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f17b801ca00 con 0x7f17c80fb470 2024-03-22T07:06:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.590+0000 7f17affff700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f17b00692a0 0x7f17b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.590+0000 7f17c6ffd700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f17b00692a0 0x7f17b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.590+0000 7f17affff700 1 -- 172.21.15.81:0/367639564 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f17b8087b70 con 0x7f17c80fb470 2024-03-22T07:06:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.590+0000 7f17c6ffd700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f17b00692a0 0x7f17b006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f17bc006fd0 tx=0x7f17bc006db0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.594+0000 7f17affff700 1 -- 172.21.15.81:0/367639564 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f17b8057d20 con 0x7f17c80fb470 2024-03-22T07:06:19.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:19.808+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f17c80f8260 con 0x7f17b00692a0 2024-03-22T07:06:21.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:20 smithi081 ceph-mon[29245]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:21.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:20 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-22T07:06:21.101 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:20 smithi149 ceph-mon[36068]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:20 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-22T07:06:21.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.679+0000 7f17affff700 1 -- 172.21.15.81:0/367639564 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f17c80f8260 con 0x7f17b00692a0 2024-03-22T07:06:21.683 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.683+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f17b00692a0 msgr2=0x7f17b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:21.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.683+0000 7f17cdd84700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f17b00692a0 0x7f17b006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f17bc006fd0 tx=0x7f17bc006db0 comp rx=0 tx=0).stop 2024-03-22T07:06:21.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.683+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 msgr2=0x7f17c810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:21.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.683+0000 7f17cdd84700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 0x7f17c810d910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b970 tx=0x7f17b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:21.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.683+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 shutdown_connections 2024-03-22T07:06:21.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.683+0000 7f17cdd84700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f17b00692a0 0x7f17b006b750 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f17bc006fd0 tx=0x7f17bc006db0 comp rx=0 tx=0).stop 2024-03-22T07:06:21.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.683+0000 7f17cdd84700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17c80fb470 0x7f17c810d910 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:21.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.683+0000 7f17cdd84700 1 --2- 172.21.15.81:0/367639564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f17c80fc720 0x7f17c810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:21.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.683+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 >> 172.21.15.81:0/367639564 conn(0x7f17c80f6bd0 msgr2=0x7f17c80f7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:21.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.684+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 shutdown_connections 2024-03-22T07:06:21.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:21.685+0000 7f17cdd84700 1 -- 172.21.15.81:0/367639564 wait complete. 2024-03-22T07:06:22.031 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:21 smithi081 ceph-mon[29245]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:22.031 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:21 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-22T07:06:22.031 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:21 smithi081 ceph-mon[29245]: osdmap e41: 6 total, 6 up, 6 in 2024-03-22T07:06:22.031 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:21 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-22T07:06:22.031 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:21 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081[29240]: 2024-03-22T07:06:21.657+0000 7f9a7c98d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:06:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:21 smithi149 ceph-mon[36068]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:06:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:21 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-22T07:06:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:21 smithi149 ceph-mon[36068]: osdmap e41: 6 total, 6 up, 6 in 2024-03-22T07:06:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:21 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-22T07:06:22.383 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T07:06:22.894 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: pgmap v170: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:22.894 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: osdmap e42: 6 total, 6 up, 6 in 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: osdmap e43: 6 total, 6 up, 6 in 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: fsmap cephfs:0 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: Saving service mds.cephfs spec with placement count:4 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:22.895 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:06:22.896 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:22.896 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.bnzfrv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:06:22.896 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.bnzfrv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:06:22.896 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:22.896 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:22 smithi081 ceph-mon[29245]: Deploying daemon mds.cephfs.smithi081.bnzfrv on smithi081 2024-03-22T07:06:23.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: pgmap v170: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:23.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-22T07:06:23.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-22T07:06:23.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: osdmap e42: 6 total, 6 up, 6 in 2024-03-22T07:06:23.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-22T07:06:23.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: osdmap e43: 6 total, 6 up, 6 in 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: fsmap cephfs:0 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: Saving service mds.cephfs spec with placement count:4 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.bnzfrv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.bnzfrv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:23.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:22 smithi149 ceph-mon[36068]: Deploying daemon mds.cephfs.smithi081.bnzfrv on smithi081 2024-03-22T07:06:23.523 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:23.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:23 smithi081 ceph-mon[29245]: osdmap e44: 6 total, 6 up, 6 in 2024-03-22T07:06:24.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:23 smithi149 ceph-mon[36068]: osdmap e44: 6 total, 6 up, 6 in 2024-03-22T07:06:24.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:24 smithi081 ceph-mon[29245]: pgmap v174: 65 pgs: 7 creating+peering, 22 active+clean, 36 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:24.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:24 smithi081 ceph-mon[29245]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-22T07:06:24.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:24.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:06:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:24 smithi149 ceph-mon[36068]: pgmap v174: 65 pgs: 7 creating+peering, 22 active+clean, 36 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:24 smithi149 ceph-mon[36068]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-22T07:06:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:06:25.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 -- 172.21.15.81:0/600429384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fcf80 msgr2=0x7fee3c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:25.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 --2- 172.21.15.81:0/600429384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fcf80 0x7fee3c0fd3f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fee2c004940 tx=0x7fee2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:25.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 -- 172.21.15.81:0/600429384 shutdown_connections 2024-03-22T07:06:25.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 --2- 172.21.15.81:0/600429384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fcf80 0x7fee3c0fd3f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:25.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 --2- 172.21.15.81:0/600429384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee3c0fbcd0 0x7fee3c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:25.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 -- 172.21.15.81:0/600429384 >> 172.21.15.81:0/600429384 conn(0x7fee3c0f7450 msgr2=0x7fee3c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:25.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 -- 172.21.15.81:0/600429384 shutdown_connections 2024-03-22T07:06:25.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 -- 172.21.15.81:0/600429384 wait complete. 2024-03-22T07:06:25.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 Processor -- start 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 -- start start 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fbcd0 0x7fee3c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee3c0fcf80 0x7fee3c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c10e440 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.225+0000 7fee43ca2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c10e5b0 con 0x7fee3c0fcf80 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee41a3e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fbcd0 0x7fee3c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee41a3e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fbcd0 0x7fee3c10d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58468/0 (socket says 172.21.15.81:58468) 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee41a3e700 1 -- 172.21.15.81:0/2903907775 learned_addr learned my addr 172.21.15.81:0/2903907775 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee4123d700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee3c0fcf80 0x7fee3c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee41a3e700 1 -- 172.21.15.81:0/2903907775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee3c0fcf80 msgr2=0x7fee3c10de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee41a3e700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee3c0fcf80 0x7fee3c10de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee41a3e700 1 -- 172.21.15.81:0/2903907775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c005040 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee41a3e700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fbcd0 0x7fee3c10d900 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fee3800c8c0 tx=0x7fee380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee327fc700 1 -- 172.21.15.81:0/2903907775 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee3800a760 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.236 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c112fb0 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee327fc700 1 -- 172.21.15.81:0/2903907775 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee3800ad90 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee327fc700 1 -- 172.21.15.81:0/2903907775 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee380113a0 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.226+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c113540 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.227+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c101b10 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.228+0000 7fee327fc700 1 -- 172.21.15.81:0/2903907775 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fee38012400 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.228+0000 7fee327fc700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fee280692a0 0x7fee2806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.228+0000 7fee327fc700 1 -- 172.21.15.81:0/2903907775 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fee380871e0 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.228+0000 7fee4123d700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fee280692a0 0x7fee2806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.228+0000 7fee4123d700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fee280692a0 0x7fee2806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00b970 tx=0x7fee2c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:25.237 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.232+0000 7fee327fc700 1 -- 172.21.15.81:0/2903907775 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fee38056fa0 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.468+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fee3c04c710 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.468+0000 7fee327fc700 1 -- 172.21.15.81:0/2903907775 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fee38056b30 con 0x7fee3c0fbcd0 2024-03-22T07:06:25.469 INFO:teuthology.orchestra.run.smithi081.stdout:e2 2024-03-22T07:06:25.469 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:06:25.469 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:06:25.469 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:06:25.469 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:25.469 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 2 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:06:21.658439+0000 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:06:25.470 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 0 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 1 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:in 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:up {} 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 0 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:25.471 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:25.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fee280692a0 msgr2=0x7fee2806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:25.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fee280692a0 0x7fee2806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00b970 tx=0x7fee2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:25.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fbcd0 msgr2=0x7fee3c10d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:25.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fbcd0 0x7fee3c10d900 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fee3800c8c0 tx=0x7fee380044c0 comp rx=0 tx=0).stop 2024-03-22T07:06:25.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 shutdown_connections 2024-03-22T07:06:25.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fee280692a0 0x7fee2806b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:25.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3c0fbcd0 0x7fee3c10d900 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:25.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 --2- 172.21.15.81:0/2903907775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee3c0fcf80 0x7fee3c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:25.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 >> 172.21.15.81:0/2903907775 conn(0x7fee3c0f7450 msgr2=0x7fee3c0f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:25.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 shutdown_connections 2024-03-22T07:06:25.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:25.470+0000 7fee43ca2700 1 -- 172.21.15.81:0/2903907775 wait complete. 2024-03-22T07:06:25.473 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 2 2024-03-22T07:06:26.181 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:06:26.190 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:06:26.191 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-03-22T07:06:26.447 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: pgmap v175: 65 pgs: 7 creating+peering, 44 active+clean, 14 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2903907775' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.kdtxfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.kdtxfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: Deploying daemon mds.cephfs.smithi149.kdtxfw on smithi149 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6826/2831536232,v1:172.21.15.81:6827/2831536232] up:boot 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: daemon mds.cephfs.smithi081.bnzfrv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: Cluster is now healthy 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: fsmap cephfs:0 1 up:standby 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: fsmap cephfs:1 {0=cephfs.smithi081.bnzfrv=up:creating} 2024-03-22T07:06:26.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:26 smithi149 ceph-mon[36068]: daemon mds.cephfs.smithi081.bnzfrv is now active in filesystem cephfs as rank 0 2024-03-22T07:06:26.760 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: pgmap v175: 65 pgs: 7 creating+peering, 44 active+clean, 14 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:06:26.760 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:26.760 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:26.760 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2903907775' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:06:26.760 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:26.760 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.kdtxfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:06:26.760 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.kdtxfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:06:26.760 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: Deploying daemon mds.cephfs.smithi149.kdtxfw on smithi149 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6826/2831536232,v1:172.21.15.81:6827/2831536232] up:boot 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: daemon mds.cephfs.smithi081.bnzfrv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: Cluster is now healthy 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: fsmap cephfs:0 1 up:standby 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: fsmap cephfs:1 {0=cephfs.smithi081.bnzfrv=up:creating} 2024-03-22T07:06:26.761 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:26 smithi081 ceph-mon[29245]: daemon mds.cephfs.smithi081.bnzfrv is now active in filesystem cephfs as rank 0 2024-03-22T07:06:27.788 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:27 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6826/2831536232,v1:172.21.15.81:6827/2831536232] up:active 2024-03-22T07:06:27.789 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:27 smithi081 ceph-mon[29245]: fsmap cephfs:1 {0=cephfs.smithi081.bnzfrv=up:active} 2024-03-22T07:06:27.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.793+0000 7fd6e0247700 1 -- 172.21.15.81:0/4037967507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fdfe0 msgr2=0x7fd6d80fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:27.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.793+0000 7fd6e0247700 1 --2- 172.21.15.81:0/4037967507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fdfe0 0x7fd6d80fe450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8004940 tx=0x7fd6c8009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:27.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.793+0000 7fd6e0247700 1 -- 172.21.15.81:0/4037967507 shutdown_connections 2024-03-22T07:06:27.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.793+0000 7fd6e0247700 1 --2- 172.21.15.81:0/4037967507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fdfe0 0x7fd6d80fe450 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:27.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.793+0000 7fd6e0247700 1 --2- 172.21.15.81:0/4037967507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6d80fce70 0x7fd6d80fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:27.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.793+0000 7fd6e0247700 1 -- 172.21.15.81:0/4037967507 >> 172.21.15.81:0/4037967507 conn(0x7fd6d80f8550 msgr2=0x7fd6d80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:27.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.793+0000 7fd6e0247700 1 -- 172.21.15.81:0/4037967507 shutdown_connections 2024-03-22T07:06:27.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.793+0000 7fd6e0247700 1 -- 172.21.15.81:0/4037967507 wait complete. 2024-03-22T07:06:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.794+0000 7fd6e0247700 1 Processor -- start 2024-03-22T07:06:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.794+0000 7fd6e0247700 1 -- start start 2024-03-22T07:06:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.794+0000 7fd6e0247700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fce70 0x7fd6d810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.794+0000 7fd6e0247700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6d80fdfe0 0x7fd6d810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.794+0000 7fd6e0247700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d810e460 con 0x7fd6d80fce70 2024-03-22T07:06:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.794+0000 7fd6e0247700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d810e5d0 con 0x7fd6d80fdfe0 2024-03-22T07:06:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.794+0000 7fd6ddfe3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fce70 0x7fd6d810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.794+0000 7fd6ddfe3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fce70 0x7fd6d810d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58506/0 (socket says 172.21.15.81:58506) 2024-03-22T07:06:27.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.794+0000 7fd6ddfe3700 1 -- 172.21.15.81:0/3163212319 learned_addr learned my addr 172.21.15.81:0/3163212319 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6dd7e2700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6d80fdfe0 0x7fd6d810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6ddfe3700 1 -- 172.21.15.81:0/3163212319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6d80fdfe0 msgr2=0x7fd6d810de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6ddfe3700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6d80fdfe0 0x7fd6d810de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6ddfe3700 1 -- 172.21.15.81:0/3163212319 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8005040 con 0x7fd6d80fce70 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6ddfe3700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fce70 0x7fd6d810d920 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4002a50 tx=0x7fd6d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6ce7fc700 1 -- 172.21.15.81:0/3163212319 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d4007cc0 con 0x7fd6d80fce70 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d8112fd0 con 0x7fd6d80fce70 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6ce7fc700 1 -- 172.21.15.81:0/3163212319 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6d400f040 con 0x7fd6d80fce70 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6ce7fc700 1 -- 172.21.15.81:0/3163212319 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d400da30 con 0x7fd6d80fce70 2024-03-22T07:06:27.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.795+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d8113560 con 0x7fd6d80fce70 2024-03-22T07:06:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.796+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d804c710 con 0x7fd6d80fce70 2024-03-22T07:06:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.797+0000 7fd6ce7fc700 1 -- 172.21.15.81:0/3163212319 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd6d400db90 con 0x7fd6d80fce70 2024-03-22T07:06:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.797+0000 7fd6ce7fc700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd6c40692a0 0x7fd6c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.797+0000 7fd6ce7fc700 1 -- 172.21.15.81:0/3163212319 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fd6d4086f40 con 0x7fd6d80fce70 2024-03-22T07:06:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.797+0000 7fd6dd7e2700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd6c40692a0 0x7fd6c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:27.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.798+0000 7fd6dd7e2700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd6c40692a0 0x7fd6c406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800b970 tx=0x7fd6c8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:27.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:27.802+0000 7fd6ce7fc700 1 -- 172.21.15.81:0/3163212319 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd6d4056c80 con 0x7fd6d80fce70 2024-03-22T07:06:28.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.043+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fd6d81058d0 con 0x7fd6d80fce70 2024-03-22T07:06:28.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:27 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6826/2831536232,v1:172.21.15.81:6827/2831536232] up:active 2024-03-22T07:06:28.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:27 smithi149 ceph-mon[36068]: fsmap cephfs:1 {0=cephfs.smithi081.bnzfrv=up:active} 2024-03-22T07:06:28.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.687+0000 7fd6ce7fc700 1 -- 172.21.15.81:0/3163212319 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7fd6d81058d0 con 0x7fd6d80fce70 2024-03-22T07:06:28.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.690+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd6c40692a0 msgr2=0x7fd6c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:28.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.690+0000 7fd6e0247700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd6c40692a0 0x7fd6c406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800b970 tx=0x7fd6c8009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:28.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.690+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fce70 msgr2=0x7fd6d810d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:28.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.690+0000 7fd6e0247700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fce70 0x7fd6d810d920 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4002a50 tx=0x7fd6d40046b0 comp rx=0 tx=0).stop 2024-03-22T07:06:28.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.691+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 shutdown_connections 2024-03-22T07:06:28.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.691+0000 7fd6e0247700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fd6c40692a0 0x7fd6c406b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:28.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.691+0000 7fd6e0247700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80fce70 0x7fd6d810d920 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:28.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.691+0000 7fd6e0247700 1 --2- 172.21.15.81:0/3163212319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6d80fdfe0 0x7fd6d810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:28.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.691+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 >> 172.21.15.81:0/3163212319 conn(0x7fd6d80f8550 msgr2=0x7fd6d80f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:28.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.691+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 shutdown_connections 2024-03-22T07:06:28.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:28.691+0000 7fd6e0247700 1 -- 172.21.15.81:0/3163212319 wait complete. 2024-03-22T07:06:28.709 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:28 smithi081 ceph-mon[29245]: pgmap v176: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T07:06:28.709 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:28 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:28.710 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:28 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:28.710 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:28 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:28.710 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:28 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.zfgxzy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:06:28.710 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:28 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.zfgxzy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:06:28.710 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:28 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:28.710 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:28 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3163212319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-22T07:06:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:28 smithi149 ceph-mon[36068]: pgmap v176: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T07:06:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:28 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:28 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:28 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:29.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:28 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.zfgxzy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:06:29.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:28 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.zfgxzy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:06:29.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:28 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:29.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:28 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3163212319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-22T07:06:29.716 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:06:29.726 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:06:29.726 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: Deploying daemon mds.cephfs.smithi081.zfgxzy on smithi081 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:boot 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3163212319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: daemon mds.cephfs.smithi149.kdtxfw assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: fsmap cephfs:1 {0=cephfs.smithi081.bnzfrv=up:active} 1 up:standby 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:06:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:creating} 2024-03-22T07:06:30.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:29 smithi081 ceph-mon[29245]: daemon mds.cephfs.smithi149.kdtxfw is now active in filesystem cephfs as rank 1 2024-03-22T07:06:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: Deploying daemon mds.cephfs.smithi081.zfgxzy on smithi081 2024-03-22T07:06:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T07:06:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:boot 2024-03-22T07:06:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3163212319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-22T07:06:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: daemon mds.cephfs.smithi149.kdtxfw assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T07:06:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:06:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T07:06:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: fsmap cephfs:1 {0=cephfs.smithi081.bnzfrv=up:active} 1 up:standby 2024-03-22T07:06:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:06:30.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:creating} 2024-03-22T07:06:30.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:29 smithi149 ceph-mon[36068]: daemon mds.cephfs.smithi149.kdtxfw is now active in filesystem cephfs as rank 1 2024-03-22T07:06:30.532 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:30.924 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:30 smithi081 ceph-mon[29245]: pgmap v177: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 823 B/s wr, 2 op/s 2024-03-22T07:06:30.924 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:30 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:active 2024-03-22T07:06:30.924 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:30 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2024-03-22T07:06:31.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:30 smithi149 ceph-mon[36068]: pgmap v177: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 823 B/s wr, 2 op/s 2024-03-22T07:06:31.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:30 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:active 2024-03-22T07:06:31.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:30 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.368+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2341162282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fbcd0 msgr2=0x7fdc740fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.368+0000 7fdc7c54f700 1 --2- 172.21.15.81:0/2341162282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fbcd0 0x7fdc740fc0e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdc70004940 tx=0x7fdc70009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.368+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2341162282 shutdown_connections 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.368+0000 7fdc7c54f700 1 --2- 172.21.15.81:0/2341162282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc740fcf80 0x7fdc740fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.368+0000 7fdc7c54f700 1 --2- 172.21.15.81:0/2341162282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fbcd0 0x7fdc740fc0e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.368+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2341162282 >> 172.21.15.81:0/2341162282 conn(0x7fdc740f7450 msgr2=0x7fdc740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2341162282 shutdown_connections 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2341162282 wait complete. 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc7c54f700 1 Processor -- start 2024-03-22T07:06:32.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc7c54f700 1 -- start start 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc7c54f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc740fbcd0 0x7fdc74066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc7c54f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fcf80 0x7fdc741126e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc7c54f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc74112cb0 con 0x7fdc740fcf80 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc7c54f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc74112df0 con 0x7fdc740fbcd0 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc79aea700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fcf80 0x7fdc741126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc79aea700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fcf80 0x7fdc741126e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58528/0 (socket says 172.21.15.81:58528) 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.369+0000 7fdc79aea700 1 -- 172.21.15.81:0/2388500008 learned_addr learned my addr 172.21.15.81:0/2388500008 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc7a2eb700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc740fbcd0 0x7fdc74066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc79aea700 1 -- 172.21.15.81:0/2388500008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc740fbcd0 msgr2=0x7fdc74066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:32.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc79aea700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc740fbcd0 0x7fdc74066d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:32.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc79aea700 1 -- 172.21.15.81:0/2388500008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc70005040 con 0x7fdc740fcf80 2024-03-22T07:06:32.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc79aea700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fcf80 0x7fdc741126e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400b790 tx=0x7fdc64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:32.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc6affd700 1 -- 172.21.15.81:0/2388500008 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc6400f770 con 0x7fdc740fcf80 2024-03-22T07:06:32.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc6affd700 1 -- 172.21.15.81:0/2388500008 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc6400fda0 con 0x7fdc740fcf80 2024-03-22T07:06:32.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc741130d0 con 0x7fdc740fcf80 2024-03-22T07:06:32.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc6affd700 1 -- 172.21.15.81:0/2388500008 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc6400d310 con 0x7fdc740fcf80 2024-03-22T07:06:32.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.370+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc74113690 con 0x7fdc740fcf80 2024-03-22T07:06:32.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.371+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7404c710 con 0x7fdc740fcf80 2024-03-22T07:06:32.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.371+0000 7fdc6affd700 1 -- 172.21.15.81:0/2388500008 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdc64010420 con 0x7fdc740fcf80 2024-03-22T07:06:32.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.372+0000 7fdc6affd700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdc600692a0 0x7fdc6006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:32.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.372+0000 7fdc7a2eb700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdc600692a0 0x7fdc6006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:32.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.372+0000 7fdc6affd700 1 -- 172.21.15.81:0/2388500008 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fdc640871b0 con 0x7fdc740fcf80 2024-03-22T07:06:32.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.372+0000 7fdc7a2eb700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdc600692a0 0x7fdc6006b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000b3b0 tx=0x7fdc70009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:32.376 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.376+0000 7fdc6affd700 1 -- 172.21.15.81:0/2388500008 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdc64056f70 con 0x7fdc740fcf80 2024-03-22T07:06:32.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.614+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdc74104840 con 0x7fdc740fcf80 2024-03-22T07:06:32.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.699+0000 7fdc6affd700 1 -- 172.21.15.81:0/2388500008 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdc64056b00 con 0x7fdc740fcf80 2024-03-22T07:06:32.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdc600692a0 msgr2=0x7fdc6006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdc600692a0 0x7fdc6006b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000b3b0 tx=0x7fdc70009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fcf80 msgr2=0x7fdc741126e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fcf80 0x7fdc741126e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400b790 tx=0x7fdc64009250 comp rx=0 tx=0).stop 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 shutdown_connections 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fdc600692a0 0x7fdc6006b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc740fbcd0 0x7fdc74066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 --2- 172.21.15.81:0/2388500008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc740fcf80 0x7fdc741126e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 >> 172.21.15.81:0/2388500008 conn(0x7fdc740f7450 msgr2=0x7fdc740f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 shutdown_connections 2024-03-22T07:06:32.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:32.702+0000 7fdc7c54f700 1 -- 172.21.15.81:0/2388500008 wait complete. 2024-03-22T07:06:32.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:32 smithi081 ceph-mon[29245]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 6 op/s 2024-03-22T07:06:32.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:32 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2388500008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-22T07:06:32.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:32 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:32.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:32 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:32.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:32 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:32.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:32 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.ffewvr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:06:32.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:32 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.ffewvr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:06:32.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:32 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:33.083 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:32 smithi149 ceph-mon[36068]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 6 op/s 2024-03-22T07:06:33.083 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:32 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2388500008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-03-22T07:06:33.083 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:32 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:33.083 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:32 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:33.083 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:32 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:33.083 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:32 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.ffewvr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:06:33.083 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:32 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.ffewvr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:06:33.083 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:32 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:33.315 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:06:33.325 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:06:33.325 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-22T07:06:33.586 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:33.967 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:33 smithi149 ceph-mon[36068]: Deploying daemon mds.cephfs.smithi149.ffewvr on smithi149 2024-03-22T07:06:33.967 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:33 smithi149 ceph-mon[36068]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-22T07:06:33.967 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:33 smithi149 ceph-mon[36068]: Cluster is now healthy 2024-03-22T07:06:33.967 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:33 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:boot 2024-03-22T07:06:33.967 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:33 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2388500008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-22T07:06:33.968 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:33 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:active} 1 up:standby 2024-03-22T07:06:33.968 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:33 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:06:34.018 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:33 smithi081 ceph-mon[29245]: Deploying daemon mds.cephfs.smithi149.ffewvr on smithi149 2024-03-22T07:06:34.018 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:33 smithi081 ceph-mon[29245]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-22T07:06:34.018 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:33 smithi081 ceph-mon[29245]: Cluster is now healthy 2024-03-22T07:06:34.018 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:33 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:boot 2024-03-22T07:06:34.018 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:33 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2388500008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-03-22T07:06:34.018 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:33 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:active} 1 up:standby 2024-03-22T07:06:34.019 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:33 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:06:34.941 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:34 smithi081 ceph-mon[29245]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s wr, 8 op/s 2024-03-22T07:06:34.941 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:34 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:35.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:34 smithi149 ceph-mon[36068]: pgmap v179: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s wr, 8 op/s 2024-03-22T07:06:35.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:34 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:35.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.179+0000 7f5dce332700 1 -- 172.21.15.81:0/4008879775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fbcd0 msgr2=0x7f5dc80fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.179+0000 7f5dce332700 1 --2- 172.21.15.81:0/4008879775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fbcd0 0x7f5dc80fc0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5db8004940 tx=0x7f5db8009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 -- 172.21.15.81:0/4008879775 shutdown_connections 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 --2- 172.21.15.81:0/4008879775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dc80fcf80 0x7f5dc80fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 --2- 172.21.15.81:0/4008879775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fbcd0 0x7f5dc80fc0e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 -- 172.21.15.81:0/4008879775 >> 172.21.15.81:0/4008879775 conn(0x7f5dc80f7430 msgr2=0x7f5dc80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 -- 172.21.15.81:0/4008879775 shutdown_connections 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 -- 172.21.15.81:0/4008879775 wait complete. 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 Processor -- start 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 -- start start 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dc80fbcd0 0x7f5dc810fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fcf80 0x7f5dc8110130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8110730 con 0x7f5dc80fcf80 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.180+0000 7f5dce332700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8110f30 con 0x7f5dc80fbcd0 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fcf80 0x7f5dc8110130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dc80fbcd0 0x7f5dc810fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dc80fbcd0 0x7f5dc810fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.81:55484/0 (socket says 172.21.15.81:55484) 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fcf80 0x7f5dc8110130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44204/0 (socket says 172.21.15.81:44204) 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc7fff700 1 -- 172.21.15.81:0/1542685184 learned_addr learned my addr 172.21.15.81:0/1542685184 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc77fe700 1 -- 172.21.15.81:0/1542685184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dc80fbcd0 msgr2=0x7f5dc810fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc77fe700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dc80fbcd0 0x7f5dc810fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc77fe700 1 -- 172.21.15.81:0/1542685184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8005040 con 0x7f5dc80fcf80 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc77fe700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fcf80 0x7f5dc8110130 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc002a50 tx=0x7f5dbc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc4ff9700 1 -- 172.21.15.81:0/1542685184 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dbc007cc0 con 0x7f5dc80fcf80 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc8111210 con 0x7f5dc80fcf80 2024-03-22T07:06:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc4ff9700 1 -- 172.21.15.81:0/1542685184 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5dbc00f040 con 0x7f5dc80fcf80 2024-03-22T07:06:35.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dc4ff9700 1 -- 172.21.15.81:0/1542685184 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dbc009530 con 0x7f5dc80fcf80 2024-03-22T07:06:35.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.181+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc8111750 con 0x7f5dc80fcf80 2024-03-22T07:06:35.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.182+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc8101910 con 0x7f5dc80fcf80 2024-03-22T07:06:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.183+0000 7f5dc4ff9700 1 -- 172.21.15.81:0/1542685184 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5dbc0097f0 con 0x7f5dc80fcf80 2024-03-22T07:06:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.183+0000 7f5dc4ff9700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5db00692a0 0x7f5db006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.184+0000 7f5dc7fff700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5db00692a0 0x7f5db006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.184+0000 7f5dc4ff9700 1 -- 172.21.15.81:0/1542685184 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5dbc086ed0 con 0x7f5dc80fcf80 2024-03-22T07:06:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.184+0000 7f5dc7fff700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5db00692a0 0x7f5db006b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5db8004f90 tx=0x7f5db8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:35.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.188+0000 7f5dc4ff9700 1 -- 172.21.15.81:0/1542685184 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5dbc056c10 con 0x7f5dc80fcf80 2024-03-22T07:06:35.441 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.440+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f5dc805f5f0 con 0x7f5dc80fcf80 2024-03-22T07:06:35.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.710+0000 7f5dc4ff9700 1 -- 172.21.15.81:0/1542685184 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v10) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5dbc0567a0 con 0x7f5dc80fcf80 2024-03-22T07:06:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5db00692a0 msgr2=0x7f5db006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5db00692a0 0x7f5db006b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5db8004f90 tx=0x7f5db8009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fcf80 msgr2=0x7f5dc8110130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fcf80 0x7f5dc8110130 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc002a50 tx=0x7f5dbc004600 comp rx=0 tx=0).stop 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 shutdown_connections 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5db00692a0 0x7f5db006b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5dc80fbcd0 0x7f5dc810fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 --2- 172.21.15.81:0/1542685184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5dc80fcf80 0x7f5dc8110130 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 >> 172.21.15.81:0/1542685184 conn(0x7f5dc80f7430 msgr2=0x7f5dc8065af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 shutdown_connections 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:35.712+0000 7f5dce332700 1 -- 172.21.15.81:0/1542685184 wait complete. 2024-03-22T07:06:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:inline data disabled 2024-03-22T07:06:36.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 6 op/s 2024-03-22T07:06:36.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:06:36.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1542685184' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-22T07:06:36.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] up:boot 2024-03-22T07:06:36.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1542685184' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-22T07:06:36.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:36.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:35 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:06:36.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:36.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 6 op/s 2024-03-22T07:06:36.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:06:36.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1542685184' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-22T07:06:36.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] up:boot 2024-03-22T07:06:36.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1542685184' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-22T07:06:36.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:36.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:35 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:06:36.400 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:06:36.421 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:06:36.421 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T07:06:36.740 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:38.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:standby 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:active 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: Dropping low affinity active daemon mds.cephfs.smithi081.bnzfrv in favor of higher affinity standby. 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: Replacing daemon mds.cephfs.smithi081.bnzfrv as rank 0 with standby daemon mds.cephfs.smithi081.zfgxzy 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: osdmap e45: 6 total, 6 up, 6 in 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: fsmap cephfs:2/2 {0=cephfs.smithi081.zfgxzy=up:replay,1=cephfs.smithi149.kdtxfw=up:active} 1 up:standby 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:38 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:standby 2024-03-22T07:06:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:active 2024-03-22T07:06:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: Dropping low affinity active daemon mds.cephfs.smithi081.bnzfrv in favor of higher affinity standby. 2024-03-22T07:06:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: Replacing daemon mds.cephfs.smithi081.bnzfrv as rank 0 with standby daemon mds.cephfs.smithi081.zfgxzy 2024-03-22T07:06:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T07:06:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: osdmap e45: 6 total, 6 up, 6 in 2024-03-22T07:06:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: fsmap cephfs:2/2 {0=cephfs.smithi081.zfgxzy=up:replay,1=cephfs.smithi149.kdtxfw=up:active} 1 up:standby 2024-03-22T07:06:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-22T07:06:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:38 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:38.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.886+0000 7fc1a31b2700 1 -- 172.21.15.81:0/457791004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fdfe0 msgr2=0x7fc19c0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:38.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.886+0000 7fc1a31b2700 1 --2- 172.21.15.81:0/457791004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fdfe0 0x7fc19c0fe450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc188004940 tx=0x7fc188009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:38.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.887+0000 7fc1a31b2700 1 -- 172.21.15.81:0/457791004 shutdown_connections 2024-03-22T07:06:38.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.887+0000 7fc1a31b2700 1 --2- 172.21.15.81:0/457791004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fdfe0 0x7fc19c0fe450 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:38.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.887+0000 7fc1a31b2700 1 --2- 172.21.15.81:0/457791004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19c0fce70 0x7fc19c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:38.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.887+0000 7fc1a31b2700 1 -- 172.21.15.81:0/457791004 >> 172.21.15.81:0/457791004 conn(0x7fc19c0f8550 msgr2=0x7fc19c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:38.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.887+0000 7fc1a31b2700 1 -- 172.21.15.81:0/457791004 shutdown_connections 2024-03-22T07:06:38.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.887+0000 7fc1a31b2700 1 -- 172.21.15.81:0/457791004 wait complete. 2024-03-22T07:06:38.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.887+0000 7fc1a31b2700 1 Processor -- start 2024-03-22T07:06:38.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.887+0000 7fc1a31b2700 1 -- start start 2024-03-22T07:06:38.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a31b2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fce70 0x7fc19c071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:38.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a31b2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19c0fdfe0 0x7fc19c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:38.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a31b2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c070850 con 0x7fc19c0fce70 2024-03-22T07:06:38.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a31b2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0709c0 con 0x7fc19c0fdfe0 2024-03-22T07:06:38.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a0f4e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fce70 0x7fc19c071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a0f4e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fce70 0x7fc19c071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44234/0 (socket says 172.21.15.81:44234) 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a0f4e700 1 -- 172.21.15.81:0/1163454710 learned_addr learned my addr 172.21.15.81:0/1163454710 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc19bfff700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19c0fdfe0 0x7fc19c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a0f4e700 1 -- 172.21.15.81:0/1163454710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19c0fdfe0 msgr2=0x7fc19c070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a0f4e700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19c0fdfe0 0x7fc19c070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a0f4e700 1 -- 172.21.15.81:0/1163454710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc188005040 con 0x7fc19c0fce70 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc19bfff700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19c0fdfe0 0x7fc19c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a0f4e700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fce70 0x7fc19c071c60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc19000b790 tx=0x7fc190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1997fa700 1 -- 172.21.15.81:0/1163454710 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc19000f770 con 0x7fc19c0fce70 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c070ca0 con 0x7fc19c0fce70 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1997fa700 1 -- 172.21.15.81:0/1163454710 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc19000fda0 con 0x7fc19c0fce70 2024-03-22T07:06:38.891 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1997fa700 1 -- 172.21.15.81:0/1163454710 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc19000d310 con 0x7fc19c0fce70 2024-03-22T07:06:38.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.888+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c115a40 con 0x7fc19c0fce70 2024-03-22T07:06:38.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.889+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c04c710 con 0x7fc19c0fce70 2024-03-22T07:06:38.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.890+0000 7fc1997fa700 1 -- 172.21.15.81:0/1163454710 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc190010420 con 0x7fc19c0fce70 2024-03-22T07:06:38.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.890+0000 7fc1997fa700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc18c0692a0 0x7fc18c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:38.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.890+0000 7fc1997fa700 1 -- 172.21.15.81:0/1163454710 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc190087200 con 0x7fc19c0fce70 2024-03-22T07:06:38.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.890+0000 7fc19bfff700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc18c0692a0 0x7fc18c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:38.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.891+0000 7fc19bfff700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc18c0692a0 0x7fc18c06b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b3b0 tx=0x7fc188009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:38.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:38.894+0000 7fc1997fa700 1 -- 172.21.15.81:0/1163454710 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc190056f70 con 0x7fc19c0fce70 2024-03-22T07:06:39.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.142+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc19c115bd0 con 0x7fc19c0fce70 2024-03-22T07:06:39.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.143+0000 7fc1997fa700 1 -- 172.21.15.81:0/1163454710 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1821 (secure 0 0 0) 0x7fc190056b00 con 0x7fc19c0fce70 2024-03-22T07:06:39.143 INFO:teuthology.orchestra.run.smithi081.stdout:e14 2024-03-22T07:06:39.143 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:06:39.143 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:06:39.143 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 14 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:06:39.052285+0000 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:06:39.144 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:06:39.145 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:reconnect seq 4 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:06:39.146 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:06:39.146 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:39.146 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:39.146 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:06:39.146 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:39.146 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:06:39.146 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc18c0692a0 msgr2=0x7fc18c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc18c0692a0 0x7fc18c06b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc18800b3b0 tx=0x7fc188009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fce70 msgr2=0x7fc19c071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fce70 0x7fc19c071c60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc19000b790 tx=0x7fc190009250 comp rx=0 tx=0).stop 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 shutdown_connections 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fc18c0692a0 0x7fc18c06b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc19c0fce70 0x7fc19c071c60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 --2- 172.21.15.81:0/1163454710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19c0fdfe0 0x7fc19c070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 >> 172.21.15.81:0/1163454710 conn(0x7fc19c0f8550 msgr2=0x7fc19c0f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 shutdown_connections 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:39.145+0000 7fc1a31b2700 1 -- 172.21.15.81:0/1163454710 wait complete. 2024-03-22T07:06:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 14 2024-03-22T07:06:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:39 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:resolve 2024-03-22T07:06:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:39 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] up:boot 2024-03-22T07:06:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:39 smithi081 ceph-mon[29245]: fsmap cephfs:2/2 {0=cephfs.smithi081.zfgxzy=up:resolve,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:06:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:39 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:resolve 2024-03-22T07:06:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:39 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] up:boot 2024-03-22T07:06:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:39 smithi149 ceph-mon[36068]: fsmap cephfs:2/2 {0=cephfs.smithi081.zfgxzy=up:resolve,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:06:39.786 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-22T07:06:40.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:40 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:reconnect 2024-03-22T07:06:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:40 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] up:standby 2024-03-22T07:06:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:40 smithi081 ceph-mon[29245]: fsmap cephfs:2/2 {0=cephfs.smithi081.zfgxzy=up:reconnect,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:40 smithi081 ceph-mon[29245]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-22T07:06:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:06:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:40 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1163454710' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:06:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:40 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:reconnect 2024-03-22T07:06:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:40 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] up:standby 2024-03-22T07:06:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:40 smithi149 ceph-mon[36068]: fsmap cephfs:2/2 {0=cephfs.smithi081.zfgxzy=up:reconnect,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:40 smithi149 ceph-mon[36068]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 614 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-22T07:06:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:40 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:40 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:06:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:40 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:40 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1163454710' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:06:40.107 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:41.144 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:41 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:rejoin 2024-03-22T07:06:41.144 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:41 smithi081 ceph-mon[29245]: fsmap cephfs:2/2 {0=cephfs.smithi081.zfgxzy=up:rejoin,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:41.144 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:41 smithi081 ceph-mon[29245]: daemon mds.cephfs.smithi081.zfgxzy is now active in filesystem cephfs as rank 0 2024-03-22T07:06:41.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:41 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:rejoin 2024-03-22T07:06:41.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:41 smithi149 ceph-mon[36068]: fsmap cephfs:2/2 {0=cephfs.smithi081.zfgxzy=up:rejoin,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:41.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:41 smithi149 ceph-mon[36068]: daemon mds.cephfs.smithi081.zfgxzy is now active in filesystem cephfs as rank 0 2024-03-22T07:06:42.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:42 smithi081 ceph-mon[29245]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T07:06:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:42 smithi081 ceph-mon[29245]: Cluster is now healthy 2024-03-22T07:06:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:42 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:active 2024-03-22T07:06:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:42 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:42 smithi081 ceph-mon[29245]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 819 B/s rd, 511 B/s wr, 3 op/s 2024-03-22T07:06:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:42 smithi149 ceph-mon[36068]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T07:06:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:42 smithi149 ceph-mon[36068]: Cluster is now healthy 2024-03-22T07:06:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:42 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:active 2024-03-22T07:06:42.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:42 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:06:42.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:42 smithi149 ceph-mon[36068]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 819 B/s rd, 511 B/s wr, 3 op/s 2024-03-22T07:06:42.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.901+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/2992500622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 msgr2=0x7f9b740fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:42.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.901+0000 7f9b7a0bb700 1 --2- 172.21.15.81:0/2992500622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 0x7f9b740fc050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9b68004940 tx=0x7f9b68009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:42.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.901+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/2992500622 shutdown_connections 2024-03-22T07:06:42.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.901+0000 7f9b7a0bb700 1 --2- 172.21.15.81:0/2992500622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b740fcef0 0x7f9b740fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:42.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.901+0000 7f9b7a0bb700 1 --2- 172.21.15.81:0/2992500622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 0x7f9b740fc050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:42.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.901+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/2992500622 >> 172.21.15.81:0/2992500622 conn(0x7f9b740f73c0 msgr2=0x7f9b740f97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:42.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.901+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/2992500622 shutdown_connections 2024-03-22T07:06:42.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.901+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/2992500622 wait complete. 2024-03-22T07:06:42.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.901+0000 7f9b7a0bb700 1 Processor -- start 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b7a0bb700 1 -- start start 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b7a0bb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 0x7f9b7410d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b7a0bb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b740fcef0 0x7f9b7410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b7a0bb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7410e520 con 0x7f9b740fbc40 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b7a0bb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7410ed20 con 0x7f9b740fcef0 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b737fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 0x7f9b7410d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b737fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 0x7f9b7410d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44258/0 (socket says 172.21.15.81:44258) 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b737fe700 1 -- 172.21.15.81:0/501118582 learned_addr learned my addr 172.21.15.81:0/501118582 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b72ffd700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b740fcef0 0x7f9b7410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b737fe700 1 -- 172.21.15.81:0/501118582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b740fcef0 msgr2=0x7f9b7410df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b737fe700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b740fcef0 0x7f9b7410df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b737fe700 1 -- 172.21.15.81:0/501118582 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b68005040 con 0x7f9b740fbc40 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.902+0000 7f9b737fe700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 0x7f9b7410d9e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800b3b0 tx=0x7f9b68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:42.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.903+0000 7f9b5bfff700 1 -- 172.21.15.81:0/501118582 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b6800e040 con 0x7f9b740fbc40 2024-03-22T07:06:42.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.903+0000 7f9b5bfff700 1 -- 172.21.15.81:0/501118582 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b6800bbf0 con 0x7f9b740fbc40 2024-03-22T07:06:42.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.903+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7410efa0 con 0x7f9b740fbc40 2024-03-22T07:06:42.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.903+0000 7f9b5bfff700 1 -- 172.21.15.81:0/501118582 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b6801c630 con 0x7f9b740fbc40 2024-03-22T07:06:42.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.903+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7410f4e0 con 0x7f9b740fbc40 2024-03-22T07:06:42.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.904+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b74101a80 con 0x7f9b740fbc40 2024-03-22T07:06:42.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.904+0000 7f9b5bfff700 1 -- 172.21.15.81:0/501118582 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9b6800bd60 con 0x7f9b740fbc40 2024-03-22T07:06:42.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.904+0000 7f9b5bfff700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9b5c0692a0 0x7f9b5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:42.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.905+0000 7f9b5bfff700 1 -- 172.21.15.81:0/501118582 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9b68087f00 con 0x7f9b740fbc40 2024-03-22T07:06:42.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.905+0000 7f9b72ffd700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9b5c0692a0 0x7f9b5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:42.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.905+0000 7f9b72ffd700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9b5c0692a0 0x7f9b5c06b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9b64004f40 tx=0x7f9b6400b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:42.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:42.909+0000 7f9b5bfff700 1 -- 172.21.15.81:0/501118582 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9b68057c70 con 0x7f9b740fbc40 2024-03-22T07:06:43.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.153+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b7405f5f0 con 0x7f9b740fbc40 2024-03-22T07:06:43.154 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.154+0000 7f9b5bfff700 1 -- 172.21.15.81:0/501118582 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4706 (secure 0 0 0) 0x7f9b68057800 con 0x7f9b740fbc40 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9b5c0692a0 msgr2=0x7f9b5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9b5c0692a0 0x7f9b5c06b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9b64004f40 tx=0x7f9b6400b040 comp rx=0 tx=0).stop 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 msgr2=0x7f9b7410d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 0x7f9b7410d9e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800b3b0 tx=0x7f9b68009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 shutdown_connections 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9b5c0692a0 0x7f9b5c06b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b740fbc40 0x7f9b7410d9e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 --2- 172.21.15.81:0/501118582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b740fcef0 0x7f9b7410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 >> 172.21.15.81:0/501118582 conn(0x7f9b740f73c0 msgr2=0x7f9b74065c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:43.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 shutdown_connections 2024-03-22T07:06:43.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:43.156+0000 7f9b7a0bb700 1 -- 172.21.15.81:0/501118582 wait complete. 2024-03-22T07:06:43.158 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 16 2024-03-22T07:06:43.167 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:06:43.801 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-22T07:06:44.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:44.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:44.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:44.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:06:44.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:44.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:44 smithi149 ceph-mon[36068]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-22T07:06:44.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:44 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/501118582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:06:44.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:44 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:06:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:06:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:44 smithi081 ceph-mon[29245]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-22T07:06:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:44 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/501118582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:06:44.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:44 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:06:44.632 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:46.308 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:46 smithi081 ceph-mon[29245]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-22T07:06:46.308 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:46 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:46.308 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:46 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:46.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:46 smithi149 ceph-mon[36068]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-22T07:06:46.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:46 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:46.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:46 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:46.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.748+0000 7f8ae2080700 1 -- 172.21.15.81:0/3656438773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0662e0 msgr2=0x7f8adc0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:46.750 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.748+0000 7f8ae2080700 1 --2- 172.21.15.81:0/3656438773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0662e0 0x7f8adc0ff9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8acc004940 tx=0x7f8acc009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.748+0000 7f8ae2080700 1 -- 172.21.15.81:0/3656438773 shutdown_connections 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.748+0000 7f8ae2080700 1 --2- 172.21.15.81:0/3656438773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc0fff00 0x7f8adc102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.748+0000 7f8ae2080700 1 --2- 172.21.15.81:0/3656438773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0662e0 0x7f8adc0ff9c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.748+0000 7f8ae2080700 1 -- 172.21.15.81:0/3656438773 >> 172.21.15.81:0/3656438773 conn(0x7f8adc0f7430 msgr2=0x7f8adc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.748+0000 7f8ae2080700 1 -- 172.21.15.81:0/3656438773 shutdown_connections 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.748+0000 7f8ae2080700 1 -- 172.21.15.81:0/3656438773 wait complete. 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8ae2080700 1 Processor -- start 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8ae2080700 1 -- start start 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8ae2080700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc0662e0 0x7f8adc10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8ae2080700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0fff00 0x7f8adc10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8ae2080700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc10e540 con 0x7f8adc0fff00 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8ae2080700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc10ed40 con 0x7f8adc0662e0 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8adaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0fff00 0x7f8adc10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:46.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8adaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0fff00 0x7f8adc10df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33168/0 (socket says 172.21.15.81:33168) 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8adaffd700 1 -- 172.21.15.81:0/3967300843 learned_addr learned my addr 172.21.15.81:0/3967300843 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.749+0000 7f8adb7fe700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc0662e0 0x7f8adc10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8adaffd700 1 -- 172.21.15.81:0/3967300843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc0662e0 msgr2=0x7f8adc10da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8adaffd700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc0662e0 0x7f8adc10da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8adaffd700 1 -- 172.21.15.81:0/3967300843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc005040 con 0x7f8adc0fff00 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8adb7fe700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc0662e0 0x7f8adc10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8adaffd700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0fff00 0x7f8adc10df40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000eb40 tx=0x7f8ad000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8ac3fff700 1 -- 172.21.15.81:0/3967300843 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad0007cc0 con 0x7f8adc0fff00 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8ac3fff700 1 -- 172.21.15.81:0/3967300843 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ad000f040 con 0x7f8adc0fff00 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc10efc0 con 0x7f8adc0fff00 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8ac3fff700 1 -- 172.21.15.81:0/3967300843 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad000ca10 con 0x7f8adc0fff00 2024-03-22T07:06:46.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.750+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc10f500 con 0x7f8adc0fff00 2024-03-22T07:06:46.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.751+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc05f5f0 con 0x7f8adc0fff00 2024-03-22T07:06:46.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.751+0000 7f8ac3fff700 1 -- 172.21.15.81:0/3967300843 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8ad000cb70 con 0x7f8adc0fff00 2024-03-22T07:06:46.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.752+0000 7f8ac3fff700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8ac40692a0 0x7f8ac406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:46.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.752+0000 7f8adb7fe700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8ac40692a0 0x7f8ac406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:46.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.752+0000 7f8ac3fff700 1 -- 172.21.15.81:0/3967300843 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8ad0086d20 con 0x7f8adc0fff00 2024-03-22T07:06:46.753 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.752+0000 7f8adb7fe700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8ac40692a0 0x7f8ac406b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8acc004890 tx=0x7f8acc009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:46.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:46.756+0000 7f8ac3fff700 1 -- 172.21.15.81:0/3967300843 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8ad004e190 con 0x7f8adc0fff00 2024-03-22T07:06:47.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.040+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f8adc04bed0 con 0x7f8adc0fff00 2024-03-22T07:06:47.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.041+0000 7f8ac3fff700 1 -- 172.21.15.81:0/3967300843 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v16) v1 ==== 78+0+99 (secure 0 0 0) 0x7f8ad0056b40 con 0x7f8adc0fff00 2024-03-22T07:06:47.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8ac40692a0 msgr2=0x7f8ac406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:47.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8ac40692a0 0x7f8ac406b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8acc004890 tx=0x7f8acc009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:47.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0fff00 msgr2=0x7f8adc10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:47.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0fff00 0x7f8adc10df40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000eb40 tx=0x7f8ad000b040 comp rx=0 tx=0).stop 2024-03-22T07:06:47.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 shutdown_connections 2024-03-22T07:06:47.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f8ac40692a0 0x7f8ac406b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:47.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8adc0662e0 0x7f8adc10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:47.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 --2- 172.21.15.81:0/3967300843 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8adc0fff00 0x7f8adc10df40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:47.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 >> 172.21.15.81:0/3967300843 conn(0x7f8adc0f7430 msgr2=0x7f8adc0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:47.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 shutdown_connections 2024-03-22T07:06:47.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.043+0000 7f8ae2080700 1 -- 172.21.15.81:0/3967300843 wait complete. 2024-03-22T07:06:47.053 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:06:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:47 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3967300843' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-22T07:06:47.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:47 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3967300843' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-22T07:06:47.704 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-22T07:06:47.717 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T07:06:47.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.861+0000 7fe34ec65640 1 -- 172.21.15.81:0/2780710080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 msgr2=0x7fe3480fb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:47.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.861+0000 7fe34ec65640 1 --2- 172.21.15.81:0/2780710080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 0x7fe3480fb4a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe340009c50 tx=0x7fe340004d10 comp rx=0 tx=0).stop 2024-03-22T07:06:47.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.861+0000 7fe34ec65640 1 -- 172.21.15.81:0/2780710080 shutdown_connections 2024-03-22T07:06:47.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.861+0000 7fe34ec65640 1 --2- 172.21.15.81:0/2780710080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3480fc360 0x7fe3480fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:47.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.861+0000 7fe34ec65640 1 --2- 172.21.15.81:0/2780710080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 0x7fe3480fb4a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:47.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.861+0000 7fe34ec65640 1 -- 172.21.15.81:0/2780710080 >> 172.21.15.81:0/2780710080 conn(0x7fe3480f6850 msgr2=0x7fe3480f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:47.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.861+0000 7fe34ec65640 1 -- 172.21.15.81:0/2780710080 shutdown_connections 2024-03-22T07:06:47.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.861+0000 7fe34ec65640 1 -- 172.21.15.81:0/2780710080 wait complete. 2024-03-22T07:06:47.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe34ec65640 1 Processor -- start 2024-03-22T07:06:47.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe34ec65640 1 -- start start 2024-03-22T07:06:47.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe34ec65640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 0x7fe348065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:47.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe34ec65640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3480fc360 0x7fe348111ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:47.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe34ec65640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe348112000 con 0x7fe3480fb090 2024-03-22T07:06:47.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe34ec65640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe348112140 con 0x7fe3480fc360 2024-03-22T07:06:47.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe34ca01640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 0x7fe348065fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:47.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe34ca01640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 0x7fe348065fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33184/0 (socket says 172.21.15.81:33184) 2024-03-22T07:06:47.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe34ca01640 1 -- 172.21.15.81:0/2951208419 learned_addr learned my addr 172.21.15.81:0/2951208419 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:47.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.862+0000 7fe347fff640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3480fc360 0x7fe348111ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:47.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.863+0000 7fe34ca01640 1 -- 172.21.15.81:0/2951208419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3480fc360 msgr2=0x7fe348111ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.863+0000 7fe34ca01640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3480fc360 0x7fe348111ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.863+0000 7fe34ca01640 1 -- 172.21.15.81:0/2951208419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400098b0 con 0x7fe3480fb090 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.863+0000 7fe34ca01640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 0x7fe348065fd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe340009c20 tx=0x7fe340004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.863+0000 7fe3457fa640 1 -- 172.21.15.81:0/2951208419 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe34003a070 con 0x7fe3480fb090 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.863+0000 7fe3457fa640 1 -- 172.21.15.81:0/2951208419 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe340035b20 con 0x7fe3480fb090 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.863+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe348112340 con 0x7fe3480fb090 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.863+0000 7fe3457fa640 1 -- 172.21.15.81:0/2951208419 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe34003ea60 con 0x7fe3480fb090 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.863+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3481128e0 con 0x7fe3480fb090 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.864+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe348105590 con 0x7fe3480fb090 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.864+0000 7fe3457fa640 1 -- 172.21.15.81:0/2951208419 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe340032070 con 0x7fe3480fb090 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.865+0000 7fe3457fa640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe334069210 0x7fe33406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.865+0000 7fe347fff640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe334069210 0x7fe33406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:47.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.865+0000 7fe3457fa640 1 -- 172.21.15.81:0/2951208419 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe3400a9a30 con 0x7fe3480fb090 2024-03-22T07:06:47.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.865+0000 7fe347fff640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe334069210 0x7fe33406b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe348065d80 tx=0x7fe33800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:47.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:47.869+0000 7fe3457fa640 1 -- 172.21.15.81:0/2951208419 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe340075d50 con 0x7fe3480fb090 2024-03-22T07:06:48.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.052+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe348103c00 con 0x7fe3480fb090 2024-03-22T07:06:48.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.053+0000 7fe3457fa640 1 -- 172.21.15.81:0/2951208419 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14385 (secure 0 0 0) 0x7fe340079370 con 0x7fe3480fb090 2024-03-22T07:06:48.054 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:48.054 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":45,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","created":"2024-03-22T06:59:35.415160+0000","modified":"2024-03-22T07:06:37.046429+0000","last_up_change":"2024-03-22T07:05:03.156764+0000","last_in_change":"2024-03-22T07:04:44.637120+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:03:49.371549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T07:06:20.045033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T07:06:20.879130+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"69addaec-0295-4219-9e22-5ebf831a65d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6803","nonce":3909132761}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6805","nonce":3909132761}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6809","nonce":3909132761}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3909132761},{"type":"v1","addr":"172.21.15.81:6807","nonce":3909132761}]},"public_addr":"172.21.15.81:6803/3909132761","cluster_addr":"172.21.15.81:6805/3909132761","heartbeat_back_addr":"172.21.15.81:6809/3909132761","heartbeat_front_addr":"172.21.15.81:6807/3909132761","state":["exists","up"]},{"osd":1,"uuid":"cd572358-1fd5-4e96-8caf-3d00d6e0608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6811","nonce":3867177766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6813","nonce":3867177766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6817","nonce":3867177766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":3867177766},{"type":"v1","addr":"172.21.15.81:6815","nonce":3867177766}]},"public_addr":"172.21.15.81:6811/3867177766","cluster_addr":"172.21.15.81:6813/3867177766","heartbeat_back_addr":"172.21.15.81:6817/3867177766","heartbeat_front_addr":"172.21.15.81:6815/3867177766","state":["exists","up"]},{"osd":2,"uuid":"ba6f9b7e-712d-4f4b-8b33-6e051daf2875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6819","nonce":181541459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6821","nonce":181541459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6825","nonce":181541459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":181541459},{"type":"v1","addr":"172.21.15.81:6823","nonce":181541459}]},"public_addr":"172.21.15.81:6819/181541459","cluster_addr":"172.21.15.81:6821/181541459","heartbeat_back_addr":"172.21.15.81:6825/181541459","heartbeat_front_addr":"172.21.15.81:6823/181541459","state":["exists","up"]},{"osd":3,"uuid":"e5adfd7b-e190-45b4-b8ca-da7c45d21d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6801","nonce":868260375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6803","nonce":868260375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6807","nonce":868260375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":868260375},{"type":"v1","addr":"172.21.15.149:6805","nonce":868260375}]},"public_addr":"172.21.15.149:6801/868260375","cluster_addr":"172.21.15.149:6803/868260375","heartbeat_back_addr":"172.21.15.149:6807/868260375","heartbeat_front_addr":"172.21.15.149:6805/868260375","state":["exists","up"]},{"osd":4,"uuid":"a85bb681-b621-4fbf-b77a-cbd090a41a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6809","nonce":1702132597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6811","nonce":1702132597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6815","nonce":1702132597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1702132597},{"type":"v1","addr":"172.21.15.149:6813","nonce":1702132597}]},"public_addr":"172.21.15.149:6809/1702132597","cluster_addr":"172.21.15.149:6811/1702132597","heartbeat_back_addr":"172.21.15.149:6815/1702132597","heartbeat_front_addr":"172.21.15.149:6813/1702132597","state":["exists","up"]},{"osd":5,"uuid":"0e83552d-4e6c-4426-9b7c-99d9ad5c45ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6817","nonce":3036205309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6819","nonce":3036205309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6823","nonce":3036205309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3036205309},{"type":"v1","addr":"172.21.15.149:6821","nonce":3036205309}]},"public_addr":"172.21.15.149:6817/3036205309","cluster_addr":"172.21.15.149:6819/3036205309","heartbeat_back_addr":"172.21.15.149:6823/3036205309","heartbeat_front_addr":"172.21.15.149:6821/3036205309","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:02:52.797148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:18.980557+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:46.975511+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:10.893766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:36.048181+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:05:01.848799+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:0/1030883466":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/2472307158":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6801/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6801/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/2249539268":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/2438362157":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6800/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6800/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/473242716":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6827/2831536232":"2024-03-23T07:06:37.046400+0000","172.21.15.81:6800/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6801/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/3756300008":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/770908409":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/1939667879":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/988274144":"2024-03-23T07:01:39.049837+0000","172.21.15.81:6826/2831536232":"2024-03-23T07:06:37.046400+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:06:48.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.055+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe334069210 msgr2=0x7fe33406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.055+0000 7fe34ec65640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe334069210 0x7fe33406b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe348065d80 tx=0x7fe33800b040 comp rx=0 tx=0).stop 2024-03-22T07:06:48.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.055+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 msgr2=0x7fe348065fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.055+0000 7fe34ec65640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 0x7fe348065fd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe340009c20 tx=0x7fe340004d10 comp rx=0 tx=0).stop 2024-03-22T07:06:48.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.056+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 shutdown_connections 2024-03-22T07:06:48.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.056+0000 7fe34ec65640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe334069210 0x7fe33406b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.056+0000 7fe34ec65640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3480fc360 0x7fe348111ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.056+0000 7fe34ec65640 1 --2- 172.21.15.81:0/2951208419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3480fb090 0x7fe348065fd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.056+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 >> 172.21.15.81:0/2951208419 conn(0x7fe3480f6850 msgr2=0x7fe3480ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:48.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.056+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 shutdown_connections 2024-03-22T07:06:48.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.056+0000 7fe34ec65640 1 -- 172.21.15.81:0/2951208419 wait complete. 2024-03-22T07:06:48.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:03:49.371549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T07:06:20.045033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T07:06:20.879130+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T07:06:48.071 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T07:06:48.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 -- 172.21.15.81:0/1131595099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fd800 msgr2=0x7f11400fdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 --2- 172.21.15.81:0/1131595099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fd800 0x7f11400fdc10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f113c0048d0 tx=0x7f113c009da0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 -- 172.21.15.81:0/1131595099 shutdown_connections 2024-03-22T07:06:48.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 --2- 172.21.15.81:0/1131595099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11400fe150 0x7f11400fa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 --2- 172.21.15.81:0/1131595099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fd800 0x7f11400fdc10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 -- 172.21.15.81:0/1131595099 >> 172.21.15.81:0/1131595099 conn(0x7f11400f6000 msgr2=0x7f11400f8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:48.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 -- 172.21.15.81:0/1131595099 shutdown_connections 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 -- 172.21.15.81:0/1131595099 wait complete. 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 Processor -- start 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.171+0000 7f114879d640 1 -- start start 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f114879d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11400fd800 0x7f114010cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f114879d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fe150 0x7f114010d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f114879d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114010d8a0 con 0x7f11400fe150 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f114879d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140112260 con 0x7f11400fd800 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1145d38640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fe150 0x7f114010d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1145d38640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fe150 0x7f114010d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33198/0 (socket says 172.21.15.81:33198) 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1145d38640 1 -- 172.21.15.81:0/3995137214 learned_addr learned my addr 172.21.15.81:0/3995137214 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1146539640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11400fd800 0x7f114010cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.174 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1145d38640 1 -- 172.21.15.81:0/3995137214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11400fd800 msgr2=0x7f114010cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1145d38640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11400fd800 0x7f114010cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1145d38640 1 -- 172.21.15.81:0/3995137214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c005040 con 0x7f11400fe150 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1146539640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11400fd800 0x7f114010cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1145d38640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fe150 0x7f114010d250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f113000c800 tx=0x7f1130004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1136ffd640 1 -- 172.21.15.81:0/3995137214 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f113000a820 con 0x7f11400fe150 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1136ffd640 1 -- 172.21.15.81:0/3995137214 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f113000ae50 con 0x7f11400fe150 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f1136ffd640 1 -- 172.21.15.81:0/3995137214 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11300115e0 con 0x7f11400fe150 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1140112400 con 0x7f11400fe150 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.172+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11401128d0 con 0x7f11400fe150 2024-03-22T07:06:48.175 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.173+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114004bcc0 con 0x7f11400fe150 2024-03-22T07:06:48.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.174+0000 7f1136ffd640 1 -- 172.21.15.81:0/3995137214 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1130012400 con 0x7f11400fe150 2024-03-22T07:06:48.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.174+0000 7f1136ffd640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f112c069210 0x7f112c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.174+0000 7f1136ffd640 1 -- 172.21.15.81:0/3995137214 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1130087460 con 0x7f11400fe150 2024-03-22T07:06:48.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.174+0000 7f1146539640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f112c069210 0x7f112c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.174+0000 7f1146539640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f112c069210 0x7f112c06b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f113c004540 tx=0x7f113c009da0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:48.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.178+0000 7f1136ffd640 1 -- 172.21.15.81:0/3995137214 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1130053740 con 0x7f11400fe150 2024-03-22T07:06:48.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.375+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1140112b50 con 0x7f11400fe150 2024-03-22T07:06:48.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.375+0000 7f1136ffd640 1 -- 172.21.15.81:0/3995137214 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1140112b50 con 0x7f11400fe150 2024-03-22T07:06:48.376 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f112c069210 msgr2=0x7f112c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f112c069210 0x7f112c06b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f113c004540 tx=0x7f113c009da0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fe150 msgr2=0x7f114010d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fe150 0x7f114010d250 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f113000c800 tx=0x7f1130004430 comp rx=0 tx=0).stop 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 shutdown_connections 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f112c069210 0x7f112c06b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11400fe150 0x7f114010d250 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 --2- 172.21.15.81:0/3995137214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11400fd800 0x7f114010cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 >> 172.21.15.81:0/3995137214 conn(0x7f11400f6000 msgr2=0x7f11400f75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 shutdown_connections 2024-03-22T07:06:48.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.377+0000 7f114879d640 1 -- 172.21.15.81:0/3995137214 wait complete. 2024-03-22T07:06:48.387 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:48 smithi081 ceph-mon[29245]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-22T07:06:48.388 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:48 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2951208419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:06:48.389 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T07:06:48.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.497+0000 7fac2af9d640 1 -- 172.21.15.81:0/2311815960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fc690 msgr2=0x7fac240fcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.497+0000 7fac2af9d640 1 --2- 172.21.15.81:0/2311815960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fc690 0x7fac240fcaa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fac180048d0 tx=0x7fac18009da0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.497+0000 7fac2af9d640 1 -- 172.21.15.81:0/2311815960 shutdown_connections 2024-03-22T07:06:48.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.497+0000 7fac2af9d640 1 --2- 172.21.15.81:0/2311815960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac240fcfe0 0x7fac240fb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.497+0000 7fac2af9d640 1 --2- 172.21.15.81:0/2311815960 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fc690 0x7fac240fcaa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.497+0000 7fac2af9d640 1 -- 172.21.15.81:0/2311815960 >> 172.21.15.81:0/2311815960 conn(0x7fac240f5ff0 msgr2=0x7fac240f8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:48.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.497+0000 7fac2af9d640 1 -- 172.21.15.81:0/2311815960 shutdown_connections 2024-03-22T07:06:48.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.497+0000 7fac2af9d640 1 -- 172.21.15.81:0/2311815960 wait complete. 2024-03-22T07:06:48.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac2af9d640 1 Processor -- start 2024-03-22T07:06:48.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac2af9d640 1 -- start start 2024-03-22T07:06:48.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac2af9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac240fc690 0x7fac2410bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac2af9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fcfe0 0x7fac2410c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac2af9d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2410c740 con 0x7fac240fcfe0 2024-03-22T07:06:48.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac2af9d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac24111140 con 0x7fac240fc690 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac23fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fcfe0 0x7fac2410c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac23fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fcfe0 0x7fac2410c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33212/0 (socket says 172.21.15.81:33212) 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac23fff640 1 -- 172.21.15.81:0/95786539 learned_addr learned my addr 172.21.15.81:0/95786539 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.498+0000 7fac28d39640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac240fc690 0x7fac2410bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac23fff640 1 -- 172.21.15.81:0/95786539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac240fc690 msgr2=0x7fac2410bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac23fff640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac240fc690 0x7fac2410bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac23fff640 1 -- 172.21.15.81:0/95786539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18005040 con 0x7fac240fcfe0 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac28d39640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac240fc690 0x7fac2410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac23fff640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fcfe0 0x7fac2410c0f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fac10002a50 tx=0x7fac10004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac217fa640 1 -- 172.21.15.81:0/95786539 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac10007c30 con 0x7fac240fcfe0 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac217fa640 1 -- 172.21.15.81:0/95786539 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac1000f040 con 0x7fac240fcfe0 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac241112e0 con 0x7fac240fcfe0 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac217fa640 1 -- 172.21.15.81:0/95786539 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac1000db50 con 0x7fac240fcfe0 2024-03-22T07:06:48.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.499+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac241117d0 con 0x7fac240fcfe0 2024-03-22T07:06:48.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.500+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac24105590 con 0x7fac240fcfe0 2024-03-22T07:06:48.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.501+0000 7fac217fa640 1 -- 172.21.15.81:0/95786539 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fac1001c400 con 0x7fac240fcfe0 2024-03-22T07:06:48.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.501+0000 7fac217fa640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fac14069210 0x7fac1406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.502+0000 7fac217fa640 1 -- 172.21.15.81:0/95786539 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fac10086a20 con 0x7fac240fcfe0 2024-03-22T07:06:48.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.502+0000 7fac28d39640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fac14069210 0x7fac1406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.502+0000 7fac28d39640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fac14069210 0x7fac1406b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac180317e0 tx=0x7fac18009da0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:48.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.506+0000 7fac217fa640 1 -- 172.21.15.81:0/95786539 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fac1004d190 con 0x7fac240fcfe0 2024-03-22T07:06:48.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:48 smithi149 ceph-mon[36068]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-22T07:06:48.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:48 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2951208419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:06:48.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.675+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fac24103c00 con 0x7fac240fcfe0 2024-03-22T07:06:48.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.676+0000 7fac217fa640 1 -- 172.21.15.81:0/95786539 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7fac10052d30 con 0x7fac240fcfe0 2024-03-22T07:06:48.676 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-03-22T07:06:48.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fac14069210 msgr2=0x7fac1406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fac14069210 0x7fac1406b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fac180317e0 tx=0x7fac18009da0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fcfe0 msgr2=0x7fac2410c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fcfe0 0x7fac2410c0f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fac10002a50 tx=0x7fac10004600 comp rx=0 tx=0).stop 2024-03-22T07:06:48.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 shutdown_connections 2024-03-22T07:06:48.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fac14069210 0x7fac1406b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac240fcfe0 0x7fac2410c0f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 --2- 172.21.15.81:0/95786539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac240fc690 0x7fac2410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 >> 172.21.15.81:0/95786539 conn(0x7fac240f5ff0 msgr2=0x7fac240ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:48.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 shutdown_connections 2024-03-22T07:06:48.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.678+0000 7fac2af9d640 1 -- 172.21.15.81:0/95786539 wait complete. 2024-03-22T07:06:48.689 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.790+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/3902091895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fc1c0 msgr2=0x7fe6e40fc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.790+0000 7fe6ea0ae640 1 --2- 172.21.15.81:0/3902091895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fc1c0 0x7fe6e40fc5d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40048d0 tx=0x7fe6d4009da0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.790+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/3902091895 shutdown_connections 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.790+0000 7fe6ea0ae640 1 --2- 172.21.15.81:0/3902091895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e40fd490 0x7fe6e40fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.790+0000 7fe6ea0ae640 1 --2- 172.21.15.81:0/3902091895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fc1c0 0x7fe6e40fc5d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.790+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/3902091895 >> 172.21.15.81:0/3902091895 conn(0x7fe6e40f7980 msgr2=0x7fe6e40f9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.790+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/3902091895 shutdown_connections 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.791+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/3902091895 wait complete. 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.791+0000 7fe6ea0ae640 1 Processor -- start 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.791+0000 7fe6ea0ae640 1 -- start start 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.791+0000 7fe6ea0ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e40fc1c0 0x7fe6e410ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.791+0000 7fe6ea0ae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fd490 0x7fe6e410d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.793 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.791+0000 7fe6ea0ae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e410d870 con 0x7fe6e40fd490 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.791+0000 7fe6ea0ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e4112230 con 0x7fe6e40fc1c0 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.791+0000 7fe6e2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fd490 0x7fe6e410d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6e2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fd490 0x7fe6e410d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33236/0 (socket says 172.21.15.81:33236) 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6e2ffd640 1 -- 172.21.15.81:0/1097922903 learned_addr learned my addr 172.21.15.81:0/1097922903 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6e37fe640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e40fc1c0 0x7fe6e410ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6e2ffd640 1 -- 172.21.15.81:0/1097922903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e40fc1c0 msgr2=0x7fe6e410ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6e2ffd640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e40fc1c0 0x7fe6e410ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6e2ffd640 1 -- 172.21.15.81:0/1097922903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d4005040 con 0x7fe6e40fd490 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6e37fe640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e40fc1c0 0x7fe6e410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6e2ffd640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fd490 0x7fe6e410d1f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800c860 tx=0x7fe6d8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6cbfff640 1 -- 172.21.15.81:0/1097922903 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6d800aa30 con 0x7fe6e40fd490 2024-03-22T07:06:48.794 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e41123d0 con 0x7fe6e40fd490 2024-03-22T07:06:48.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6cbfff640 1 -- 172.21.15.81:0/1097922903 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6d800ce50 con 0x7fe6e40fd490 2024-03-22T07:06:48.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6cbfff640 1 -- 172.21.15.81:0/1097922903 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6d80115e0 con 0x7fe6e40fd490 2024-03-22T07:06:48.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.792+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e41128c0 con 0x7fe6e40fd490 2024-03-22T07:06:48.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.793+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e404bcc0 con 0x7fe6e40fd490 2024-03-22T07:06:48.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.794+0000 7fe6cbfff640 1 -- 172.21.15.81:0/1097922903 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe6d800ab90 con 0x7fe6e40fd490 2024-03-22T07:06:48.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.794+0000 7fe6cbfff640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6cc069210 0x7fe6cc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.794+0000 7fe6cbfff640 1 -- 172.21.15.81:0/1097922903 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe6d8087440 con 0x7fe6e40fd490 2024-03-22T07:06:48.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.794+0000 7fe6e37fe640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6cc069210 0x7fe6cc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.796 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.795+0000 7fe6e37fe640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6cc069210 0x7fe6cc06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4000c00 tx=0x7fe6d4009da0 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.798+0000 7fe6cbfff640 1 -- 172.21.15.81:0/1097922903 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe6d8053720 con 0x7fe6e40fd490 2024-03-22T07:06:48.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.975+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe6e4104d10 con 0x7fe6e40fd490 2024-03-22T07:06:48.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.976+0000 7fe6cbfff640 1 -- 172.21.15.81:0/1097922903 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe6d800ae40 con 0x7fe6e40fd490 2024-03-22T07:06:48.976 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-03-22T07:06:48.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6cc069210 msgr2=0x7fe6cc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6cc069210 0x7fe6cc06b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4000c00 tx=0x7fe6d4009da0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fd490 msgr2=0x7fe6e410d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fd490 0x7fe6e410d1f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800c860 tx=0x7fe6d8004430 comp rx=0 tx=0).stop 2024-03-22T07:06:48.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 shutdown_connections 2024-03-22T07:06:48.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe6cc069210 0x7fe6cc06b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6e40fd490 0x7fe6e410d1f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 --2- 172.21.15.81:0/1097922903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6e40fc1c0 0x7fe6e410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 >> 172.21.15.81:0/1097922903 conn(0x7fe6e40f7980 msgr2=0x7fe6e40f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:48.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 shutdown_connections 2024-03-22T07:06:48.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:48.978+0000 7fe6ea0ae640 1 -- 172.21.15.81:0/1097922903 wait complete. 2024-03-22T07:06:48.989 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T07:06:49.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.087+0000 7f4cc106e640 1 -- 172.21.15.81:0/1531279557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 msgr2=0x7f4cbc0fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:49.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.087+0000 7f4cc106e640 1 --2- 172.21.15.81:0/1531279557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 0x7f4cbc0fc7e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00048d0 tx=0x7f4cb0009da0 comp rx=0 tx=0).stop 2024-03-22T07:06:49.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.087+0000 7f4cc106e640 1 -- 172.21.15.81:0/1531279557 shutdown_connections 2024-03-22T07:06:49.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.087+0000 7f4cc106e640 1 --2- 172.21.15.81:0/1531279557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 0x7f4cbc0fc7e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:49.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.087+0000 7f4cc106e640 1 --2- 172.21.15.81:0/1531279557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cbc0fb0a0 0x7f4cbc0fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:49.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.087+0000 7f4cc106e640 1 -- 172.21.15.81:0/1531279557 >> 172.21.15.81:0/1531279557 conn(0x7f4cbc0f6840 msgr2=0x7f4cbc0f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:49.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.087+0000 7f4cc106e640 1 -- 172.21.15.81:0/1531279557 shutdown_connections 2024-03-22T07:06:49.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.087+0000 7f4cc106e640 1 -- 172.21.15.81:0/1531279557 wait complete. 2024-03-22T07:06:49.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cc106e640 1 Processor -- start 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cc106e640 1 -- start start 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cc106e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cbc0fb0a0 0x7f4cbc064e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cc106e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 0x7f4cbc111a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cc106e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc111fb0 con 0x7f4cbc0fc370 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cc106e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc1120f0 con 0x7f4cbc0fb0a0 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cba59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 0x7f4cbc111a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cba59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 0x7f4cbc111a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33260/0 (socket says 172.21.15.81:33260) 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cba59c640 1 -- 172.21.15.81:0/3229860119 learned_addr learned my addr 172.21.15.81:0/3229860119 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cba59c640 1 -- 172.21.15.81:0/3229860119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cbc0fb0a0 msgr2=0x7f4cbc064e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cba59c640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cbc0fb0a0 0x7f4cbc064e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cba59c640 1 -- 172.21.15.81:0/3229860119 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb0005040 con 0x7f4cbc0fc370 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.088+0000 7f4cba59c640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 0x7f4cbc111a70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0007dd0 tx=0x7f4cb0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.089+0000 7f4ca37fe640 1 -- 172.21.15.81:0/3229860119 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb00094c0 con 0x7f4cbc0fc370 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.089+0000 7f4ca37fe640 1 -- 172.21.15.81:0/3229860119 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cb0009620 con 0x7f4cbc0fc370 2024-03-22T07:06:49.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.089+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cbc112340 con 0x7f4cbc0fc370 2024-03-22T07:06:49.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.089+0000 7f4ca37fe640 1 -- 172.21.15.81:0/3229860119 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb003b640 con 0x7f4cbc0fc370 2024-03-22T07:06:49.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.089+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc112860 con 0x7f4cbc0fc370 2024-03-22T07:06:49.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.090+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cbc1055a0 con 0x7f4cbc0fc370 2024-03-22T07:06:49.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.090+0000 7f4ca37fe640 1 -- 172.21.15.81:0/3229860119 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f4cb002f070 con 0x7f4cbc0fc370 2024-03-22T07:06:49.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.090+0000 7f4ca37fe640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f4ca4069210 0x7f4ca406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:49.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.090+0000 7f4cbad9d640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f4ca4069210 0x7f4ca406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:49.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.090+0000 7f4ca37fe640 1 -- 172.21.15.81:0/3229860119 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4cb00a97b0 con 0x7f4cbc0fc370 2024-03-22T07:06:49.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.091+0000 7f4cbad9d640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f4ca4069210 0x7f4ca406b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4004c80 tx=0x7f4cb4009c50 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:49.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.095+0000 7f4ca37fe640 1 -- 172.21.15.81:0/3229860119 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4cb0075a10 con 0x7f4cbc0fc370 2024-03-22T07:06:49.257 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:49 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3995137214' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:06:49.257 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:49 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/95786539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:06:49.257 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:49 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1097922903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:06:49.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.309+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cbc103c10 con 0x7f4cbc0fc370 2024-03-22T07:06:49.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.310+0000 7f4ca37fe640 1 -- 172.21.15.81:0/3229860119 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4706 (secure 0 0 0) 0x7f4cb0079030 con 0x7f4cbc0fc370 2024-03-22T07:06:49.310 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:06:49.310 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14496,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/4012792796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":4012792796},{"type":"v1","addr":"172.21.15.81:6827","nonce":4012792796}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:06:41.064175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi081.zfgxzy","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.81:6829/268578316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":268578316},{"type":"v1","addr":"172.21.15.81:6829","nonce":268578316}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24251":{"gid":24251,"name":"cephfs.smithi149.kdtxfw","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6825/3521694486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3521694486},{"type":"v1","addr":"172.21.15.149:6825","nonce":3521694486}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-03-22T07:06:49.310 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 16 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f4ca4069210 msgr2=0x7f4ca406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f4ca4069210 0x7f4ca406b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4004c80 tx=0x7f4cb4009c50 comp rx=0 tx=0).stop 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 msgr2=0x7f4cbc111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 0x7f4cbc111a70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0007dd0 tx=0x7f4cb0009da0 comp rx=0 tx=0).stop 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 shutdown_connections 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f4ca4069210 0x7f4ca406b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4cbc0fc370 0x7f4cbc111a70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 --2- 172.21.15.81:0/3229860119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4cbc0fb0a0 0x7f4cbc064e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 >> 172.21.15.81:0/3229860119 conn(0x7f4cbc0f6840 msgr2=0x7f4cbc0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 shutdown_connections 2024-03-22T07:06:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:49.312+0000 7f4cc106e640 1 -- 172.21.15.81:0/3229860119 wait complete. 2024-03-22T07:06:49.323 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 18} 2024-03-22T07:06:49.324 INFO:teuthology.run_tasks:Running task kclient... 2024-03-22T07:06:49.346 INFO:tasks.kclient:Mounting kernel clients... 2024-03-22T07:06:49.346 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-22T07:06:49.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:49.346 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-22T07:06:49.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:49.362 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2024-03-22T07:06:49.430 INFO:teuthology.orchestra.run.smithi081.stderr:Cannot find device "ceph-brx" 2024-03-22T07:06:49.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:06:49.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:49.432 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-03-22T07:06:49.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:49.453 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link delete ceph-brx 2024-03-22T07:06:49.522 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:49 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3995137214' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:06:49.523 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:49 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/95786539' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:06:49.523 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:49 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1097922903' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:06:49.529 INFO:teuthology.orchestra.run.smithi149.stderr:Cannot find device "ceph-brx" 2024-03-22T07:06:49.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:06:49.531 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-22T07:06:49.531 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T07:06:49.532 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T07:06:49.532 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-22T07:06:49.823 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:50.538 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:50 smithi081 ceph-mon[29245]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-22T07:06:50.538 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:50 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3229860119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:06:50.538 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:50.539 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:50.539 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:50.539 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:06:50.539 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:50 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:50 smithi149 ceph-mon[36068]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-22T07:06:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:50 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3229860119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:06:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:50 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:50 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:50 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:06:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:50 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:06:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:50 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.305+0000 7f369855e700 1 -- 172.21.15.81:0/1051760507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36901015c0 msgr2=0x7f3690101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.305+0000 7f369855e700 1 --2- 172.21.15.81:0/1051760507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36901015c0 0x7f3690101990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3680004940 tx=0x7f3680009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.305+0000 7f369855e700 1 -- 172.21.15.81:0/1051760507 shutdown_connections 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.305+0000 7f369855e700 1 --2- 172.21.15.81:0/1051760507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36900fb650 0x7f36900fbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.305+0000 7f369855e700 1 --2- 172.21.15.81:0/1051760507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36901015c0 0x7f3690101990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.305+0000 7f369855e700 1 -- 172.21.15.81:0/1051760507 >> 172.21.15.81:0/1051760507 conn(0x7f3690071530 msgr2=0x7f3690071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.305+0000 7f369855e700 1 -- 172.21.15.81:0/1051760507 shutdown_connections 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.305+0000 7f369855e700 1 -- 172.21.15.81:0/1051760507 wait complete. 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f369855e700 1 Processor -- start 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f369855e700 1 -- start start 2024-03-22T07:06:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f369855e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36900fb650 0x7f3690113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f369855e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36901015c0 0x7f3690113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f369855e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36901144a0 con 0x7f36900fb650 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f369855e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369010dae0 con 0x7f36901015c0 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f36962fa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36900fb650 0x7f3690113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f36962fa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36900fb650 0x7f3690113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33280/0 (socket says 172.21.15.81:33280) 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f36962fa700 1 -- 172.21.15.81:0/2302222814 learned_addr learned my addr 172.21.15.81:0/2302222814 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f36962fa700 1 -- 172.21.15.81:0/2302222814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36901015c0 msgr2=0x7f3690113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f3695af9700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36901015c0 0x7f3690113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f36962fa700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36901015c0 0x7f3690113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f36962fa700 1 -- 172.21.15.81:0/2302222814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3680005040 con 0x7f36900fb650 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f36962fa700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36900fb650 0x7f3690113910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f368000b8b0 tx=0x7f3680009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f3686ffd700 1 -- 172.21.15.81:0/2302222814 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f368000e040 con 0x7f36900fb650 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f3686ffd700 1 -- 172.21.15.81:0/2302222814 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3680004290 con 0x7f36900fb650 2024-03-22T07:06:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.306+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369010dd60 con 0x7f36900fb650 2024-03-22T07:06:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.307+0000 7f3686ffd700 1 -- 172.21.15.81:0/2302222814 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f368001c3f0 con 0x7f36900fb650 2024-03-22T07:06:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.307+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369010e320 con 0x7f36900fb650 2024-03-22T07:06:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.307+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36900fcd20 con 0x7f36900fb650 2024-03-22T07:06:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.308+0000 7f3686ffd700 1 -- 172.21.15.81:0/2302222814 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f36800137b0 con 0x7f36900fb650 2024-03-22T07:06:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.308+0000 7f3686ffd700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f367c0692a0 0x7f367c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.308+0000 7f3686ffd700 1 -- 172.21.15.81:0/2302222814 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3680088540 con 0x7f36900fb650 2024-03-22T07:06:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.309+0000 7f3695af9700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f367c0692a0 0x7f367c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.309+0000 7f3695af9700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f367c0692a0 0x7f367c06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36901007b0 tx=0x7f368c009040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.313+0000 7f3686ffd700 1 -- 172.21.15.81:0/2302222814 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f36800582b0 con 0x7f36900fb650 2024-03-22T07:06:51.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.541+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f369010efc0 con 0x7f36900fb650 2024-03-22T07:06:51.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.542+0000 7f3686ffd700 1 -- 172.21.15.81:0/2302222814 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7f3680057e40 con 0x7f36900fb650 2024-03-22T07:06:51.542 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f367c0692a0 msgr2=0x7f367c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f367c0692a0 0x7f367c06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36901007b0 tx=0x7f368c009040 comp rx=0 tx=0).stop 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36900fb650 msgr2=0x7f3690113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36900fb650 0x7f3690113910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f368000b8b0 tx=0x7f3680009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 shutdown_connections 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f367c0692a0 0x7f367c06b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36900fb650 0x7f3690113910 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 --2- 172.21.15.81:0/2302222814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36901015c0 0x7f3690113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 >> 172.21.15.81:0/2302222814 conn(0x7f3690071530 msgr2=0x7f36900fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 shutdown_connections 2024-03-22T07:06:51.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:51.544+0000 7f369855e700 1 -- 172.21.15.81:0/2302222814 wait complete. 2024-03-22T07:06:52.196 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T07:06:52.196 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T07:06:52.196 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2024-03-22T07:06:52.196 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T07:06:52.196 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T07:06:52.196 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T07:06:52.196 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T07:06:52.196 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T07:06:52.196 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T07:06:52.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:52.197 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2024-03-22T07:06:52.213 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42222sec preferred_lft 42222sec 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2024-03-22T07:06:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:06:52.214 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T07:06:52.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:52.215 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-22T07:06:52.215 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-22T07:06:52.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add name ceph-brx type bridge 2024-03-22T07:06:52.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr flush dev ceph-brx 2024-03-22T07:06:52.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set ceph-brx up 2024-03-22T07:06:52.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T07:06:52.215 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-22T07:06:52.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:52.385 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T07:06:52.411 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-03-22T07:06:52.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:52.413 DEBUG:teuthology.orchestra.run.smithi081:> route 2024-03-22T07:06:52.506 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2024-03-22T07:06:52.506 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T07:06:52.506 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T07:06:52.506 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T07:06:52.506 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T07:06:52.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:52.507 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-22T07:06:52.507 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-22T07:06:52.507 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T07:06:52.507 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T07:06:52.507 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T07:06:52.507 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-22T07:06:52.532 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:52 smithi081 ceph-mon[29245]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-22T07:06:52.532 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:52 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2302222814' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T07:06:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:52 smithi149 ceph-mon[36068]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-22T07:06:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:52 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2302222814' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T07:06:52.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:52.757 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-22T07:06:52.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:52.812 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list-id 2024-03-22T07:06:52.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:52.867 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-22T07:06:52.867 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-22T07:06:52.867 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T07:06:52.867 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-22T07:06:52.867 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-22T07:06:52.949 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-22T07:06:52.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:52.950 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-22T07:06:52.950 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-22T07:06:52.950 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-22T07:06:52.950 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T07:06:52.950 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-22T07:06:52.950 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-22T07:06:52.950 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-22T07:06:52.950 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-22T07:06:53.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:53.084 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2024-03-22T07:06:53.084 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2024-03-22T07:06:53.084 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2024-03-22T07:06:53.084 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T07:06:53.084 DEBUG:teuthology.orchestra.run.smithi081:> ') 2024-03-22T07:06:53.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T07:06:53.125 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T07:06:53.179 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T07:06:53.180 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T07:06:53.180 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T07:06:53.234 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T07:06:53.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:53.234 DEBUG:teuthology.orchestra.run.smithi081:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T07:06:53.324 INFO:teuthology.orchestra.run.smithi081.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T07:06:53.325 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T07:06:53.325 INFO:teuthology.orchestra.run.smithi081.stdout:invalid new device string format 2024-03-22T07:06:53.444 INFO:teuthology.orchestra.run.smithi081.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T07:06:53.444 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T07:06:53.444 INFO:teuthology.orchestra.run.smithi081.stdout:invalid new device string format 2024-03-22T07:06:53.445 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: resolved to: "172.21.15.81:3300,172.21.15.149:3300" 2024-03-22T07:06:53.445 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T07:06:53.452 INFO:teuthology.orchestra.run.smithi081.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T07:06:53.453 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T07:06:53.453 INFO:teuthology.orchestra.run.smithi081.stdout:invalid new device string format 2024-03-22T07:06:53.453 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: resolved to: "172.21.15.81:3300,172.21.15.149:3300" 2024-03-22T07:06:53.453 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T07:06:53.453 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: resolved to: "172.21.15.81:6789,172.21.15.149:6789" 2024-03-22T07:06:53.453 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.81:6789,172.21.15.149:6789:/ 2024-03-22T07:06:53.453 INFO:teuthology.orchestra.run.smithi081.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=a42e8632-e819-11ee-95cd-87774f69a715" will pass to kernel 2024-03-22T07:06:53.454 INFO:teuthology.orchestra.run.smithi081.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-22T07:06:53.454 INFO:teuthology.orchestra.run.smithi081.stdout: You just mounted a file system that supports labels which does not 2024-03-22T07:06:53.454 INFO:teuthology.orchestra.run.smithi081.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-22T07:06:53.454 INFO:teuthology.orchestra.run.smithi081.stdout: applications will generate AVC messages and not be allowed access to 2024-03-22T07:06:53.454 INFO:teuthology.orchestra.run.smithi081.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-22T07:06:53.455 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T07:06:53.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:53.456 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T07:06:53.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:53.483 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:06:53.545 INFO:teuthology.orchestra.run.smithi081.stdout:client14534 2024-03-22T07:06:53.545 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:06:53.545 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:06:53.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:53.601 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:06:53.663 INFO:teuthology.orchestra.run.smithi081.stdout:client14534 2024-03-22T07:06:53.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:53.664 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:06:53.723 INFO:teuthology.orchestra.run.smithi081.stdout:client14534 2024-03-22T07:06:53.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:53.724 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/a42e8632-e819-11ee-95cd-87774f69a715.client14534/status') 2024-03-22T07:06:53.793 INFO:teuthology.orchestra.run.smithi081.stdout:instance: client.14534 (3)192.168.0.1:0/323690204 2024-03-22T07:06:53.793 INFO:teuthology.orchestra.run.smithi081.stdout:blocklisted: false 2024-03-22T07:06:53.793 INFO:teuthology.orchestra.run.smithi081.stderr:0+1 records in 2024-03-22T07:06:53.793 INFO:teuthology.orchestra.run.smithi081.stderr:0+1 records out 2024-03-22T07:06:53.793 INFO:teuthology.orchestra.run.smithi081.stderr:69 bytes copied, 2.0956e-05 s, 3.3 MB/s 2024-03-22T07:06:53.794 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-22T07:06:53.794 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T07:06:53.795 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T07:06:53.795 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-22T07:06:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:54.516 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:54 smithi081 ceph-mon[29245]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-22T07:06:54.517 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:54.517 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:06:54.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:54 smithi149 ceph-mon[36068]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-22T07:06:54.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:06:54.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:06:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.662+0000 7f5f0d02f700 1 -- 172.21.15.81:0/2249192734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f08102e00 msgr2=0x7f5f081031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.662+0000 7f5f0d02f700 1 --2- 172.21.15.81:0/2249192734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f08102e00 0x7f5f081031d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009ce0 tx=0x7f5f00004d10 comp rx=0 tx=0).stop 2024-03-22T07:06:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.662+0000 7f5f0d02f700 1 -- 172.21.15.81:0/2249192734 shutdown_connections 2024-03-22T07:06:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.662+0000 7f5f0d02f700 1 --2- 172.21.15.81:0/2249192734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f081037a0 0x7f5f080fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.662+0000 7f5f0d02f700 1 --2- 172.21.15.81:0/2249192734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f08102e00 0x7f5f081031d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.662+0000 7f5f0d02f700 1 -- 172.21.15.81:0/2249192734 >> 172.21.15.81:0/2249192734 conn(0x7f5f080f89b0 msgr2=0x7f5f080fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.662+0000 7f5f0d02f700 1 -- 172.21.15.81:0/2249192734 shutdown_connections 2024-03-22T07:06:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.662+0000 7f5f0d02f700 1 -- 172.21.15.81:0/2249192734 wait complete. 2024-03-22T07:06:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.663+0000 7f5f0d02f700 1 Processor -- start 2024-03-22T07:06:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.663+0000 7f5f0d02f700 1 -- start start 2024-03-22T07:06:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.663+0000 7f5f0d02f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f08102e00 0x7f5f08113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.663+0000 7f5f0d02f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f081037a0 0x7f5f08114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.663+0000 7f5f0d02f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f08114680 con 0x7f5f081037a0 2024-03-22T07:06:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.663+0000 7f5f0d02f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0810dcb0 con 0x7f5f08102e00 2024-03-22T07:06:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.663+0000 7f5f0659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f081037a0 0x7f5f08114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.663+0000 7f5f0659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f081037a0 0x7f5f08114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51202/0 (socket says 172.21.15.81:51202) 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.663+0000 7f5f0659c700 1 -- 172.21.15.81:0/254944606 learned_addr learned my addr 172.21.15.81:0/254944606 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5f06d9d700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f08102e00 0x7f5f08113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5f0659c700 1 -- 172.21.15.81:0/254944606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f08102e00 msgr2=0x7f5f08113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5f0659c700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f08102e00 0x7f5f08113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5f0659c700 1 -- 172.21.15.81:0/254944606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f00009940 con 0x7f5f081037a0 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5f06d9d700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f08102e00 0x7f5f08113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5f0659c700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f081037a0 0x7f5f08114030 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5efc002a50 tx=0x7f5efc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5eef7fe700 1 -- 172.21.15.81:0/254944606 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5efc007cc0 con 0x7f5f081037a0 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5eef7fe700 1 -- 172.21.15.81:0/254944606 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5efc00f040 con 0x7f5f081037a0 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0810df90 con 0x7f5f081037a0 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5eef7fe700 1 -- 172.21.15.81:0/254944606 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5efc00daf0 con 0x7f5f081037a0 2024-03-22T07:06:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.664+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0810e550 con 0x7f5f081037a0 2024-03-22T07:06:55.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.665+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f08100110 con 0x7f5f081037a0 2024-03-22T07:06:55.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.665+0000 7f5eef7fe700 1 -- 172.21.15.81:0/254944606 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5efc009810 con 0x7f5f081037a0 2024-03-22T07:06:55.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.666+0000 7f5eef7fe700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5ef00692a0 0x7f5ef006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:55.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.666+0000 7f5eef7fe700 1 -- 172.21.15.81:0/254944606 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5efc086eb0 con 0x7f5f081037a0 2024-03-22T07:06:55.667 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.666+0000 7f5f06d9d700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5ef00692a0 0x7f5ef006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:55.668 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.666+0000 7f5f06d9d700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5ef00692a0 0x7f5ef006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f000048c0 tx=0x7f5f00004d10 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:55.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.670+0000 7f5eef7fe700 1 -- 172.21.15.81:0/254944606 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5efc056c20 con 0x7f5f081037a0 2024-03-22T07:06:55.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.900+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5f0810f140 con 0x7f5f081037a0 2024-03-22T07:06:55.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.900+0000 7f5eef7fe700 1 -- 172.21.15.81:0/254944606 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+83 (secure 0 0 0) 0x7f5efc0567b0 con 0x7f5f081037a0 2024-03-22T07:06:55.901 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-22T07:06:55.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5ef00692a0 msgr2=0x7f5ef006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:55.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5ef00692a0 0x7f5ef006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f000048c0 tx=0x7f5f00004d10 comp rx=0 tx=0).stop 2024-03-22T07:06:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f081037a0 msgr2=0x7f5f08114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f081037a0 0x7f5f08114030 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5efc002a50 tx=0x7f5efc004600 comp rx=0 tx=0).stop 2024-03-22T07:06:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 shutdown_connections 2024-03-22T07:06:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f5ef00692a0 0x7f5ef006b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f08102e00 0x7f5f08113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 --2- 172.21.15.81:0/254944606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f081037a0 0x7f5f08114030 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 >> 172.21.15.81:0/254944606 conn(0x7f5f080f89b0 msgr2=0x7f5f081073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 shutdown_connections 2024-03-22T07:06:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:55.902+0000 7f5f0d02f700 1 -- 172.21.15.81:0/254944606 wait complete. 2024-03-22T07:06:56.444 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T07:06:56.445 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T07:06:56.445 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi149.front.sepia.ceph.com 2024-03-22T07:06:56.445 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-22T07:06:56.445 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T07:06:56.445 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T07:06:56.445 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T07:06:56.445 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T07:06:56.445 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-22T07:06:56.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:56.445 DEBUG:teuthology.orchestra.run.smithi149:> ip addr 2024-03-22T07:06:56.464 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T07:06:56.464 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T07:06:56.464 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T07:06:56.464 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:06:56.464 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-03-22T07:06:56.464 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:06:56.464 INFO:teuthology.orchestra.run.smithi149.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:06:56.464 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a4 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:06:56.464 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f0 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a5 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f1 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:56 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:57 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout: inet 172.21.15.149/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft 42264sec preferred_lft 42264sec 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::ec4:7aff:fe88:7d57/64 scope link 2024-03-22T07:06:56.465 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:06:56.465 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T07:06:56.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:56.466 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec bash -c ' 2024-03-22T07:06:56.466 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-03-22T07:06:56.466 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add name ceph-brx type bridge 2024-03-22T07:06:56.466 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip addr flush dev ceph-brx 2024-03-22T07:06:56.466 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set ceph-brx up 2024-03-22T07:06:56.466 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T07:06:56.466 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-03-22T07:06:56.533 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:56 smithi149 ceph-mon[36068]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-22T07:06:56.533 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:56 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/254944606' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T07:06:56.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:56 smithi081 ceph-mon[29245]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-22T07:06:56.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:56 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/254944606' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T07:06:56.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:56.628 DEBUG:teuthology.orchestra.run.smithi149:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T07:06:56.695 INFO:teuthology.orchestra.run.smithi149.stdout:1 2024-03-22T07:06:56.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:56.697 DEBUG:teuthology.orchestra.run.smithi149:> route 2024-03-22T07:06:56.774 INFO:teuthology.orchestra.run.smithi149.stdout:Kernel IP routing table 2024-03-22T07:06:56.774 INFO:teuthology.orchestra.run.smithi149.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T07:06:56.774 INFO:teuthology.orchestra.run.smithi149.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T07:06:56.774 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T07:06:56.774 INFO:teuthology.orchestra.run.smithi149.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T07:06:56.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:56.775 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec bash -c ' 2024-03-22T07:06:56.775 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-03-22T07:06:56.775 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T07:06:56.775 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T07:06:56.775 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T07:06:56.775 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-03-22T07:06:57.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.034 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-03-22T07:06:57.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.049 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list-id 2024-03-22T07:06:57.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.104 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec bash -c ' 2024-03-22T07:06:57.104 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-03-22T07:06:57.104 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T07:06:57.104 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-22T07:06:57.104 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-03-22T07:06:57.186 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-22T07:06:57.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.187 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec bash -c ' 2024-03-22T07:06:57.187 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-03-22T07:06:57.187 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-22T07:06:57.187 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T07:06:57.187 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-22T07:06:57.187 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-22T07:06:57.187 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-22T07:06:57.187 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-03-22T07:06:57.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.328 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec bash -c ' 2024-03-22T07:06:57.328 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-03-22T07:06:57.328 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set brx.0 up 2024-03-22T07:06:57.328 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T07:06:57.328 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-03-22T07:06:57.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T07:06:57.370 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-22T07:06:57.424 INFO:teuthology.orchestra.run.smithi149.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-22T07:06:57.425 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T07:06:57.425 DEBUG:teuthology.orchestra.run.smithi149:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-22T07:06:57.479 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T07:06:57.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.479 DEBUG:teuthology.orchestra.run.smithi149:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T07:06:57.571 INFO:teuthology.orchestra.run.smithi149.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T07:06:57.571 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T07:06:57.571 INFO:teuthology.orchestra.run.smithi149.stdout:invalid new device string format 2024-03-22T07:06:57.688 INFO:teuthology.orchestra.run.smithi149.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T07:06:57.688 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T07:06:57.688 INFO:teuthology.orchestra.run.smithi149.stdout:invalid new device string format 2024-03-22T07:06:57.688 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: resolved to: "172.21.15.81:3300,172.21.15.149:3300" 2024-03-22T07:06:57.688 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T07:06:57.692 INFO:teuthology.orchestra.run.smithi149.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T07:06:57.692 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T07:06:57.692 INFO:teuthology.orchestra.run.smithi149.stdout:invalid new device string format 2024-03-22T07:06:57.692 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: resolved to: "172.21.15.81:3300,172.21.15.149:3300" 2024-03-22T07:06:57.693 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T07:06:57.693 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: resolved to: "172.21.15.81:6789,172.21.15.149:6789" 2024-03-22T07:06:57.693 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.81:6789,172.21.15.149:6789:/ 2024-03-22T07:06:57.693 INFO:teuthology.orchestra.run.smithi149.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=a42e8632-e819-11ee-95cd-87774f69a715" will pass to kernel 2024-03-22T07:06:57.693 INFO:teuthology.orchestra.run.smithi149.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-22T07:06:57.693 INFO:teuthology.orchestra.run.smithi149.stdout: You just mounted a file system that supports labels which does not 2024-03-22T07:06:57.694 INFO:teuthology.orchestra.run.smithi149.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-22T07:06:57.694 INFO:teuthology.orchestra.run.smithi149.stdout: applications will generate AVC messages and not be allowed access to 2024-03-22T07:06:57.694 INFO:teuthology.orchestra.run.smithi149.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-22T07:06:57.695 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T07:06:57.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.695 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-22T07:06:57.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.722 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:06:57.778 INFO:teuthology.orchestra.run.smithi149.stdout:client14540 2024-03-22T07:06:57.778 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:06:57.778 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:06:57.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.835 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:06:57.891 INFO:teuthology.orchestra.run.smithi149.stdout:client14540 2024-03-22T07:06:57.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.891 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:06:57.948 INFO:teuthology.orchestra.run.smithi149.stdout:client14540 2024-03-22T07:06:57.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:06:57.948 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/a42e8632-e819-11ee-95cd-87774f69a715.client14540/status') 2024-03-22T07:06:58.015 INFO:teuthology.orchestra.run.smithi149.stdout:instance: client.14540 (3)172.21.15.149:0/2343999153 2024-03-22T07:06:58.016 INFO:teuthology.orchestra.run.smithi149.stdout:blocklisted: false 2024-03-22T07:06:58.016 INFO:teuthology.orchestra.run.smithi149.stderr:0+1 records in 2024-03-22T07:06:58.016 INFO:teuthology.orchestra.run.smithi149.stderr:0+1 records out 2024-03-22T07:06:58.016 INFO:teuthology.orchestra.run.smithi149.stderr:72 bytes copied, 2.1828e-05 s, 3.3 MB/s 2024-03-22T07:06:58.017 INFO:teuthology.run_tasks:Running task print... 2024-03-22T07:06:58.027 INFO:teuthology.task.print:**** done client 2024-03-22T07:06:58.027 INFO:teuthology.run_tasks:Running task parallel... 2024-03-22T07:06:58.054 INFO:teuthology.task.parallel:starting parallel... 2024-03-22T07:06:58.054 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-22T07:06:58.055 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T07:06:58.055 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:06:58.055 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-22T07:06:58.055 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-22T07:06:58.056 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-22T07:06:58.059 INFO:tasks.workunit:Pulling workunits from ref 768f6cd6e95d0c7a626042719c230df59bdba9d3 2024-03-22T07:06:58.059 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-22T07:06:58.060 INFO:tasks.workunit:timeout=3h 2024-03-22T07:06:58.060 INFO:tasks.workunit:cleanup=True 2024-03-22T07:06:58.060 DEBUG:teuthology.orchestra.run.smithi081:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-22T07:06:58.077 INFO:teuthology.orchestra.run.smithi081.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-22T07:06:58.077 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-22T07:06:58.077 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 51h/81d Inode: 1 Links: 2 2024-03-22T07:06:58.077 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:06:58.077 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-22T07:06:58.077 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-22T07:06:58.077 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-03-22 07:06:25.680261270 +0000 2024-03-22T07:06:58.077 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-03-22 07:06:53.481287475 +0000 2024-03-22T07:06:58.077 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-03-22 07:06:25.680261270 +0000 2024-03-22T07:06:58.077 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-22T07:06:58.078 DEBUG:teuthology.orchestra.run.smithi081:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-22T07:06:58.149 DEBUG:teuthology.orchestra.run.smithi149:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-22T07:06:58.166 INFO:teuthology.orchestra.run.smithi149.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-22T07:06:58.166 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-03-22T07:06:58.166 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-03-22T07:06:58.166 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:06:58.166 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-22T07:06:58.166 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-22T07:06:58.166 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-22 07:06:58.142199680 +0000 2024-03-22T07:06:58.166 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-22 07:06:58.142199680 +0000 2024-03-22T07:06:58.166 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-22 07:06:25.680261270 +0000 2024-03-22T07:06:58.167 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-22T07:06:58.167 DEBUG:teuthology.orchestra.run.smithi149:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-22T07:06:58.240 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 768f6cd6e95d0c7a626042719c230df59bdba9d3 2024-03-22T07:06:58.240 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 768f6cd6e95d0c7a626042719c230df59bdba9d3 2024-03-22T07:06:58.256 INFO:tasks.workunit.client.0.smithi081.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-22T07:06:58.295 INFO:tasks.workunit.client.1.smithi149.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-22T07:06:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:06:58.460 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:06:58 smithi081 ceph-mon[29245]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 0 B/s wr, 0 op/s 2024-03-22T07:06:58.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:06:58 smithi149 ceph-mon[36068]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 0 B/s wr, 0 op/s 2024-03-22T07:06:59.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.958+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1797344122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b80fbce0 msgr2=0x7fe1b80fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:59.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.958+0000 7fe1bdd2d700 1 --2- 172.21.15.81:0/1797344122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b80fbce0 0x7fe1b80fc0f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8004940 tx=0x7fe1a8009e30 comp rx=0 tx=0).stop 2024-03-22T07:06:59.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1797344122 shutdown_connections 2024-03-22T07:06:59.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 --2- 172.21.15.81:0/1797344122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b80fcf90 0x7fe1b80fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:59.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 --2- 172.21.15.81:0/1797344122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b80fbce0 0x7fe1b80fc0f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:59.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1797344122 >> 172.21.15.81:0/1797344122 conn(0x7fe1b80f7440 msgr2=0x7fe1b80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1797344122 shutdown_connections 2024-03-22T07:06:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1797344122 wait complete. 2024-03-22T07:06:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 Processor -- start 2024-03-22T07:06:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 -- start start 2024-03-22T07:06:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b80fbce0 0x7fe1b810c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b80fcf90 0x7fe1b810cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b810d300 con 0x7fe1b80fcf90 2024-03-22T07:06:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.959+0000 7fe1bdd2d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b810d470 con 0x7fe1b80fbce0 2024-03-22T07:06:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.960+0000 7fe1b77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b80fbce0 0x7fe1b810c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.960+0000 7fe1b77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b80fbce0 0x7fe1b810c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.81:34526/0 (socket says 172.21.15.81:34526) 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.960+0000 7fe1b77fe700 1 -- 172.21.15.81:0/1810556114 learned_addr learned my addr 172.21.15.81:0/1810556114 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.960+0000 7fe1b6ffd700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b80fcf90 0x7fe1b810cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.960+0000 7fe1b77fe700 1 -- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b80fcf90 msgr2=0x7fe1b810cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.960+0000 7fe1b77fe700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b80fcf90 0x7fe1b810cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.960+0000 7fe1b77fe700 1 -- 172.21.15.81:0/1810556114 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a8005040 con 0x7fe1b80fbce0 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.960+0000 7fe1b6ffd700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b80fcf90 0x7fe1b810cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.960+0000 7fe1b77fe700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b80fbce0 0x7fe1b810c7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8013870 tx=0x7fe1a8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.961+0000 7fe19ffff700 1 -- 172.21.15.81:0/1810556114 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1a8021070 con 0x7fe1b80fbce0 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.961+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b8111e70 con 0x7fe1b80fbce0 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.961+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b8112320 con 0x7fe1b80fbce0 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.961+0000 7fe19ffff700 1 -- 172.21.15.81:0/1810556114 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1a8009430 con 0x7fe1b80fbce0 2024-03-22T07:06:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.961+0000 7fe19ffff700 1 -- 172.21.15.81:0/1810556114 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1a800e070 con 0x7fe1b80fbce0 2024-03-22T07:06:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.961+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b804c710 con 0x7fe1b80fbce0 2024-03-22T07:06:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.962+0000 7fe19ffff700 1 -- 172.21.15.81:0/1810556114 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe1a8026780 con 0x7fe1b80fbce0 2024-03-22T07:06:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.962+0000 7fe19ffff700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe1a00692a0 0x7fe1a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.963+0000 7fe19ffff700 1 -- 172.21.15.81:0/1810556114 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe1a8088440 con 0x7fe1b80fbce0 2024-03-22T07:06:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.963+0000 7fe1b6ffd700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe1a00692a0 0x7fe1a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:59.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.963+0000 7fe1b6ffd700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe1a00692a0 0x7fe1a006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009b80 tx=0x7fe1ac00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:59.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:06:59.967+0000 7fe19ffff700 1 -- 172.21.15.81:0/1810556114 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe1a80581b0 con 0x7fe1b80fbce0 2024-03-22T07:07:00.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.162+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fe1b8112880 con 0x7fe1b80fbce0 2024-03-22T07:07:00.163 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.163+0000 7fe19ffff700 1 -- 172.21.15.81:0/1810556114 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fe1a8004490 con 0x7fe1b80fbce0 2024-03-22T07:07:00.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe1a00692a0 msgr2=0x7fe1a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:00.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe1a00692a0 0x7fe1a006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009b80 tx=0x7fe1ac00b040 comp rx=0 tx=0).stop 2024-03-22T07:07:00.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b80fbce0 msgr2=0x7fe1b810c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:00.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b80fbce0 0x7fe1b810c7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8013870 tx=0x7fe1a8009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:00.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 shutdown_connections 2024-03-22T07:07:00.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe1a00692a0 0x7fe1a006b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:00.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1b80fbce0 0x7fe1b810c7c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:00.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 --2- 172.21.15.81:0/1810556114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1b80fcf90 0x7fe1b810cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:00.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 >> 172.21.15.81:0/1810556114 conn(0x7fe1b80f7440 msgr2=0x7fe1b80f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:00.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 shutdown_connections 2024-03-22T07:07:00.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:00.165+0000 7fe1bdd2d700 1 -- 172.21.15.81:0/1810556114 wait complete. 2024-03-22T07:07:00.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:00 smithi081 ceph-mon[29245]: pgmap v193: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T07:07:00.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:00 smithi149 ceph-mon[36068]: pgmap v193: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T07:07:00.719 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-22T07:07:01.017 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:07:02.540 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:02 smithi081 ceph-mon[29245]: pgmap v194: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T07:07:02.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:02 smithi149 ceph-mon[36068]: pgmap v194: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-03-22T07:07:02.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 -- 172.21.15.81:0/384517372 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c1078e0 msgr2=0x7f327c109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:02.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 --2- 172.21.15.81:0/384517372 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c1078e0 0x7f327c109cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f326c004940 tx=0x7f326c009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:02.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 -- 172.21.15.81:0/384517372 shutdown_connections 2024-03-22T07:07:02.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 --2- 172.21.15.81:0/384517372 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c1078e0 0x7f327c109cc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:02.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 --2- 172.21.15.81:0/384517372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f327c104fc0 0x7f327c1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:02.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 -- 172.21.15.81:0/384517372 >> 172.21.15.81:0/384517372 conn(0x7f327c069de0 msgr2=0x7f327c06c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:02.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 -- 172.21.15.81:0/384517372 shutdown_connections 2024-03-22T07:07:02.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 -- 172.21.15.81:0/384517372 wait complete. 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 Processor -- start 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 -- start start 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c104fc0 0x7f327c116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.797+0000 7f328191e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f327c1078e0 0x7f327c1166c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f328191e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c116cc0 con 0x7f327c104fc0 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f328191e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c116e30 con 0x7f327c1078e0 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f327affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c104fc0 0x7f327c116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f327affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c104fc0 0x7f327c116180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51232/0 (socket says 172.21.15.81:51232) 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f327affd700 1 -- 172.21.15.81:0/196394487 learned_addr learned my addr 172.21.15.81:0/196394487 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f327a7fc700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f327c1078e0 0x7f327c1166c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f327affd700 1 -- 172.21.15.81:0/196394487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f327c1078e0 msgr2=0x7f327c1166c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f327affd700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f327c1078e0 0x7f327c1166c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f327affd700 1 -- 172.21.15.81:0/196394487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c005040 con 0x7f327c104fc0 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f327affd700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c104fc0 0x7f327c116180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3274001d80 tx=0x7f3274009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f32637fe700 1 -- 172.21.15.81:0/196394487 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3274007d10 con 0x7f327c104fc0 2024-03-22T07:07:02.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f32637fe700 1 -- 172.21.15.81:0/196394487 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3274007e70 con 0x7f327c104fc0 2024-03-22T07:07:02.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c11b890 con 0x7f327c104fc0 2024-03-22T07:07:02.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f32637fe700 1 -- 172.21.15.81:0/196394487 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3274014640 con 0x7f327c104fc0 2024-03-22T07:07:02.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.798+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c11be50 con 0x7f327c104fc0 2024-03-22T07:07:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.799+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c10fb40 con 0x7f327c104fc0 2024-03-22T07:07:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.800+0000 7f32637fe700 1 -- 172.21.15.81:0/196394487 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f327400e960 con 0x7f327c104fc0 2024-03-22T07:07:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.800+0000 7f32637fe700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f32640692a0 0x7f326406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.800+0000 7f327a7fc700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f32640692a0 0x7f326406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.800+0000 7f32637fe700 1 -- 172.21.15.81:0/196394487 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3274086fc0 con 0x7f327c104fc0 2024-03-22T07:07:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.801+0000 7f327a7fc700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f32640692a0 0x7f326406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f326c00b970 tx=0x7f326c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:02.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.805+0000 7f32637fe700 1 -- 172.21.15.81:0/196394487 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3274056d60 con 0x7f327c104fc0 2024-03-22T07:07:02.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.992+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f327c04c710 con 0x7f327c104fc0 2024-03-22T07:07:02.993 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.993+0000 7f32637fe700 1 -- 172.21.15.81:0/196394487 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f32740568f0 con 0x7f327c104fc0 2024-03-22T07:07:02.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f32640692a0 msgr2=0x7f326406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:02.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f32640692a0 0x7f326406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f326c00b970 tx=0x7f326c009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:02.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c104fc0 msgr2=0x7f327c116180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:02.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c104fc0 0x7f327c116180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3274001d80 tx=0x7f3274009ec0 comp rx=0 tx=0).stop 2024-03-22T07:07:02.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 shutdown_connections 2024-03-22T07:07:02.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f32640692a0 0x7f326406b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:02.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f327c104fc0 0x7f327c116180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:02.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 --2- 172.21.15.81:0/196394487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f327c1078e0 0x7f327c1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:02.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 >> 172.21.15.81:0/196394487 conn(0x7f327c069de0 msgr2=0x7f327c06c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:02.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 shutdown_connections 2024-03-22T07:07:02.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:02.995+0000 7f328191e700 1 -- 172.21.15.81:0/196394487 wait complete. 2024-03-22T07:07:03.664 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-22T07:07:03.924 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:07:04.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:04 smithi081 ceph-mon[29245]: pgmap v195: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-03-22T07:07:04.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:04 smithi149 ceph-mon[36068]: pgmap v195: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-03-22T07:07:05.553 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.553+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/1850467035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fcf80 msgr2=0x7f9c080fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:05.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.553+0000 7f9c0f3f6700 1 --2- 172.21.15.81:0/1850467035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fcf80 0x7f9c080fd3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8004940 tx=0x7f9bf8009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:05.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.553+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/1850467035 shutdown_connections 2024-03-22T07:07:05.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.553+0000 7f9c0f3f6700 1 --2- 172.21.15.81:0/1850467035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fcf80 0x7f9c080fd3f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.553+0000 7f9c0f3f6700 1 --2- 172.21.15.81:0/1850467035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c080fbcd0 0x7f9c080fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.553+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/1850467035 >> 172.21.15.81:0/1850467035 conn(0x7f9c080f7430 msgr2=0x7f9c080f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:05.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.553+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/1850467035 shutdown_connections 2024-03-22T07:07:05.554 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.553+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/1850467035 wait complete. 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0f3f6700 1 Processor -- start 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0f3f6700 1 -- start start 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0f3f6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fbcd0 0x7f9c08110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0f3f6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c080fcf80 0x7f9c08112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0f3f6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08066400 con 0x7f9c080fbcd0 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0f3f6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c08066540 con 0x7f9c080fcf80 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0d192700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fbcd0 0x7f9c08110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0d192700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fbcd0 0x7f9c08110710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34866/0 (socket says 172.21.15.81:34866) 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0d192700 1 -- 172.21.15.81:0/3562312222 learned_addr learned my addr 172.21.15.81:0/3562312222 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0c991700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c080fcf80 0x7f9c08112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0d192700 1 -- 172.21.15.81:0/3562312222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c080fcf80 msgr2=0x7f9c08112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:05.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0d192700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c080fcf80 0x7f9c08112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.554+0000 7f9c0d192700 1 -- 172.21.15.81:0/3562312222 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf8005040 con 0x7f9c080fbcd0 2024-03-22T07:07:05.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.555+0000 7f9c0d192700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fbcd0 0x7f9c08110710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400b7c0 tx=0x7f9c04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:05.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.555+0000 7f9bfdffb700 1 -- 172.21.15.81:0/3562312222 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c0400f6f0 con 0x7f9c080fbcd0 2024-03-22T07:07:05.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.555+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c081132c0 con 0x7f9c080fbcd0 2024-03-22T07:07:05.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.555+0000 7f9bfdffb700 1 -- 172.21.15.81:0/3562312222 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c0400fd20 con 0x7f9c080fbcd0 2024-03-22T07:07:05.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.555+0000 7f9bfdffb700 1 -- 172.21.15.81:0/3562312222 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c040103a0 con 0x7f9c080fbcd0 2024-03-22T07:07:05.557 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.555+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08113880 con 0x7f9c080fbcd0 2024-03-22T07:07:05.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.556+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0805f5f0 con 0x7f9c080fbcd0 2024-03-22T07:07:05.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.556+0000 7f9bfdffb700 1 -- 172.21.15.81:0/3562312222 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9c04011400 con 0x7f9c080fbcd0 2024-03-22T07:07:05.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.557+0000 7f9bfdffb700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9bf40692a0 0x7f9bf406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:05.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.557+0000 7f9bfdffb700 1 -- 172.21.15.81:0/3562312222 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9c04087040 con 0x7f9c080fbcd0 2024-03-22T07:07:05.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.557+0000 7f9c0c991700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9bf40692a0 0x7f9bf406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:05.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.557+0000 7f9c0c991700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9bf40692a0 0x7f9bf406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8000c00 tx=0x7f9bf8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:05.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.561+0000 7f9bfdffb700 1 -- 172.21.15.81:0/3562312222 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9c04053320 con 0x7f9c080fbcd0 2024-03-22T07:07:05.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.748+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f9c0804bed0 con 0x7f9c080fbcd0 2024-03-22T07:07:05.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.749+0000 7f9bfdffb700 1 -- 172.21.15.81:0/3562312222 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f9c04056940 con 0x7f9c080fbcd0 2024-03-22T07:07:05.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9bf40692a0 msgr2=0x7f9bf406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:05.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9bf40692a0 0x7f9bf406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8000c00 tx=0x7f9bf8009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:05.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fbcd0 msgr2=0x7f9c08110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:05.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fbcd0 0x7f9c08110710 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400b7c0 tx=0x7f9c04009250 comp rx=0 tx=0).stop 2024-03-22T07:07:05.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 shutdown_connections 2024-03-22T07:07:05.751 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f9bf40692a0 0x7f9bf406b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c080fbcd0 0x7f9c08110710 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 --2- 172.21.15.81:0/3562312222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c080fcf80 0x7f9c08112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 >> 172.21.15.81:0/3562312222 conn(0x7f9c080f7430 msgr2=0x7f9c080f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:05.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 shutdown_connections 2024-03-22T07:07:05.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:05.751+0000 7f9c0f3f6700 1 -- 172.21.15.81:0/3562312222 wait complete. 2024-03-22T07:07:06.368 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:06 smithi081 ceph-mon[29245]: pgmap v196: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-03-22T07:07:06.369 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-22T07:07:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:06 smithi149 ceph-mon[36068]: pgmap v196: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-03-22T07:07:06.627 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:07:08.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.296+0000 7fcdfda0f700 1 -- 172.21.15.81:0/4144088366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 msgr2=0x7fcdf80fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:08.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.296+0000 7fcdfda0f700 1 --2- 172.21.15.81:0/4144088366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 0x7fcdf80fe450 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcdec004940 tx=0x7fcdec009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:08.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.296+0000 7fcdfda0f700 1 -- 172.21.15.81:0/4144088366 shutdown_connections 2024-03-22T07:07:08.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.296+0000 7fcdfda0f700 1 --2- 172.21.15.81:0/4144088366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 0x7fcdf80fe450 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:08.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.296+0000 7fcdfda0f700 1 --2- 172.21.15.81:0/4144088366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdf80fce70 0x7fcdf80fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:08.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.296+0000 7fcdfda0f700 1 -- 172.21.15.81:0/4144088366 >> 172.21.15.81:0/4144088366 conn(0x7fcdf80f8550 msgr2=0x7fcdf80fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.297+0000 7fcdfda0f700 1 -- 172.21.15.81:0/4144088366 shutdown_connections 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.297+0000 7fcdfda0f700 1 -- 172.21.15.81:0/4144088366 wait complete. 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.297+0000 7fcdfda0f700 1 Processor -- start 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.297+0000 7fcdfda0f700 1 -- start start 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.297+0000 7fcdfda0f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdf80fce70 0x7fcdf810d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.297+0000 7fcdfda0f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 0x7fcdf810de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.297+0000 7fcdfda0f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf810e400 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.297+0000 7fcdfda0f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf810e570 con 0x7fcdf80fce70 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdf67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 0x7fcdf810de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdf67fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 0x7fcdf810de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34872/0 (socket says 172.21.15.81:34872) 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdf67fc700 1 -- 172.21.15.81:0/460981993 learned_addr learned my addr 172.21.15.81:0/460981993 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdf6ffd700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdf80fce70 0x7fcdf810d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:08.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdf67fc700 1 -- 172.21.15.81:0/460981993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdf80fce70 msgr2=0x7fcdf810d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:08.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdf67fc700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdf80fce70 0x7fcdf810d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:08.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdf67fc700 1 -- 172.21.15.81:0/460981993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec005040 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdf67fc700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 0x7fcdf810de00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009150 tx=0x7fcdec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:08.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcddf7fe700 1 -- 172.21.15.81:0/460981993 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdec00e040 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcddf7fe700 1 -- 172.21.15.81:0/460981993 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdec004290 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf8112f70 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcddf7fe700 1 -- 172.21.15.81:0/460981993 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdec01c630 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.298+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf8113500 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.299+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf804c710 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.299+0000 7fcddf7fe700 1 -- 172.21.15.81:0/460981993 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcdec00bbf0 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.300+0000 7fcddf7fe700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcde00692a0 0x7fcde006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:08.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.300+0000 7fcddf7fe700 1 -- 172.21.15.81:0/460981993 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcdec087d90 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.300+0000 7fcdf6ffd700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcde00692a0 0x7fcde006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:08.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.300+0000 7fcdf6ffd700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcde00692a0 0x7fcde006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009fd0 tx=0x7fcde800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:08.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.304+0000 7fcddf7fe700 1 -- 172.21.15.81:0/460981993 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcdec057ba0 con 0x7fcdf80fdfe0 2024-03-22T07:07:08.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:08.503+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdf8102280 con 0x7fcde00692a0 2024-03-22T07:07:08.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:08 smithi081 ceph-mon[29245]: pgmap v197: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-22T07:07:08.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:08 smithi149 ceph-mon[36068]: pgmap v197: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-22T07:07:09.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:09 smithi081 ceph-mon[29245]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:09.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:07:09.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:09 smithi149 ceph-mon[36068]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:09.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:07:10.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:10 smithi081 ceph-mon[29245]: pgmap v198: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-22T07:07:10.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:10 smithi149 ceph-mon[36068]: pgmap v198: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-03-22T07:07:12.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:12 smithi081 ceph-mon[29245]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T07:07:12.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:12 smithi149 ceph-mon[36068]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-03-22T07:07:14.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:14 smithi081 ceph-mon[29245]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-22T07:07:14.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:14 smithi149 ceph-mon[36068]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-22T07:07:16.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:16 smithi081 ceph-mon[29245]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T07:07:16.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:16 smithi149 ceph-mon[36068]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T07:07:18.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:18 smithi081 ceph-mon[29245]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T07:07:18.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:18 smithi149 ceph-mon[36068]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T07:07:20.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:20 smithi081 ceph-mon[29245]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:20.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:20 smithi149 ceph-mon[36068]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:22.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:22 smithi081 ceph-mon[29245]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:22.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:22 smithi149 ceph-mon[36068]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:24.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:24 smithi081 ceph-mon[29245]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:24.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:07:24.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:24 smithi149 ceph-mon[36068]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:24.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:07:26.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:26 smithi081 ceph-mon[29245]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:26.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:26 smithi149 ceph-mon[36068]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:28 smithi081 ceph-mon[29245]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:28.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:28 smithi149 ceph-mon[36068]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:30.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:30 smithi081 ceph-mon[29245]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:30.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:30 smithi149 ceph-mon[36068]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:32.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:32 smithi081 ceph-mon[29245]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:32.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:32 smithi149 ceph-mon[36068]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:34 smithi081 ceph-mon[29245]: pgmap v210: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-22T07:07:34.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:34 smithi149 ceph-mon[36068]: pgmap v210: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-22T07:07:36.565 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:36 smithi081 ceph-mon[29245]: pgmap v211: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-22T07:07:36.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:36 smithi149 ceph-mon[36068]: pgmap v211: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-03-22T07:07:37.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.060+0000 7fcddf7fe700 1 -- 172.21.15.81:0/460981993 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fcdf8102280 con 0x7fcde00692a0 2024-03-22T07:07:37.060 INFO:teuthology.orchestra.run.smithi081.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:07:37.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcde00692a0 msgr2=0x7fcde006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:37.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcde00692a0 0x7fcde006b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009fd0 tx=0x7fcde800b040 comp rx=0 tx=0).stop 2024-03-22T07:07:37.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 msgr2=0x7fcdf810de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:37.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 0x7fcdf810de00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009150 tx=0x7fcdec009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:37.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 shutdown_connections 2024-03-22T07:07:37.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcde00692a0 0x7fcde006b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:37.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdf80fce70 0x7fcdf810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:37.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 --2- 172.21.15.81:0/460981993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcdf80fdfe0 0x7fcdf810de00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:37.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 >> 172.21.15.81:0/460981993 conn(0x7fcdf80f8550 msgr2=0x7fcdf80f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:37.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 shutdown_connections 2024-03-22T07:07:37.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:37.062+0000 7fcdfda0f700 1 -- 172.21.15.81:0/460981993 wait complete. 2024-03-22T07:07:37.785 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-22T07:07:38.063 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:38 smithi081 ceph-mon[29245]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:07:38.063 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:38 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:07:38.064 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:38 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:38.064 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:38 smithi081 ceph-mon[29245]: pgmap v212: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:07:38.138 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:07:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:38 smithi149 ceph-mon[36068]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:07:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:38 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:07:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:38 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:38 smithi149 ceph-mon[36068]: pgmap v212: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:07:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:07:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:07:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:07:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:39 smithi081 ceph-mon[29245]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:07:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:07:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:07:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:07:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:39 smithi149 ceph-mon[36068]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:07:40.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.054+0000 7fec0db46700 1 -- 172.21.15.81:0/2257932175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec08071e60 msgr2=0x7fec08070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.054+0000 7fec0db46700 1 --2- 172.21.15.81:0/2257932175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec08071e60 0x7fec08070370 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fec00009ce0 tx=0x7fec00004d10 comp rx=0 tx=0).stop 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.054+0000 7fec0db46700 1 -- 172.21.15.81:0/2257932175 shutdown_connections 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.054+0000 7fec0db46700 1 --2- 172.21.15.81:0/2257932175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec080708b0 0x7fec08070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.054+0000 7fec0db46700 1 --2- 172.21.15.81:0/2257932175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec08071e60 0x7fec08070370 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.054+0000 7fec0db46700 1 -- 172.21.15.81:0/2257932175 >> 172.21.15.81:0/2257932175 conn(0x7fec080f8550 msgr2=0x7fec080fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.054+0000 7fec0db46700 1 -- 172.21.15.81:0/2257932175 shutdown_connections 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.054+0000 7fec0db46700 1 -- 172.21.15.81:0/2257932175 wait complete. 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.055+0000 7fec0db46700 1 Processor -- start 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.055+0000 7fec0db46700 1 -- start start 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.055+0000 7fec0db46700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec080708b0 0x7fec0810fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.055+0000 7fec0db46700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec08071e60 0x7fec08110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.055+0000 7fec0db46700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec08110790 con 0x7fec080708b0 2024-03-22T07:07:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.055+0000 7fec0db46700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec08110fe0 con 0x7fec08071e60 2024-03-22T07:07:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.056+0000 7fec077fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec080708b0 0x7fec0810fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.056+0000 7fec077fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec080708b0 0x7fec0810fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38412/0 (socket says 172.21.15.81:38412) 2024-03-22T07:07:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.056+0000 7fec077fe700 1 -- 172.21.15.81:0/63037091 learned_addr learned my addr 172.21.15.81:0/63037091 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.056+0000 7fec06ffd700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec08071e60 0x7fec08110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.056+0000 7fec077fe700 1 -- 172.21.15.81:0/63037091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec08071e60 msgr2=0x7fec08110190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.056+0000 7fec077fe700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec08071e60 0x7fec08110190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.056+0000 7fec077fe700 1 -- 172.21.15.81:0/63037091 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec00009940 con 0x7fec080708b0 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.056+0000 7fec077fe700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec080708b0 0x7fec0810fc50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fec000098e0 tx=0x7fec00004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.056+0000 7febeffff700 1 -- 172.21.15.81:0/63037091 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0000e040 con 0x7fec080708b0 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.057+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec08111260 con 0x7fec080708b0 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.057+0000 7febeffff700 1 -- 172.21.15.81:0/63037091 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec000049f0 con 0x7fec080708b0 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.057+0000 7febeffff700 1 -- 172.21.15.81:0/63037091 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec00012c00 con 0x7fec080708b0 2024-03-22T07:07:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.057+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec081116e0 con 0x7fec080708b0 2024-03-22T07:07:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.058+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0805f5f0 con 0x7fec080708b0 2024-03-22T07:07:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.058+0000 7febeffff700 1 -- 172.21.15.81:0/63037091 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fec00025400 con 0x7fec080708b0 2024-03-22T07:07:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.059+0000 7febeffff700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7febf00692a0 0x7febf006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.059+0000 7febeffff700 1 -- 172.21.15.81:0/63037091 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fec000893b0 con 0x7fec080708b0 2024-03-22T07:07:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.059+0000 7fec06ffd700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7febf00692a0 0x7febf006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:40.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.059+0000 7fec06ffd700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7febf00692a0 0x7febf006b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7febfc009b80 tx=0x7febfc00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:40.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.067+0000 7febeffff700 1 -- 172.21.15.81:0/63037091 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fec00059190 con 0x7fec080708b0 2024-03-22T07:07:40.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:40 smithi081 ceph-mon[29245]: pgmap v213: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:07:40.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:40 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:07:40.351 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:40 smithi149 ceph-mon[36068]: pgmap v213: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:07:40.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:40 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:07:40.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.355+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec08111ab0 con 0x7febf00692a0 2024-03-22T07:07:40.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.357+0000 7febeffff700 1 -- 172.21.15.81:0/63037091 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fec08111ab0 con 0x7febf00692a0 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7febf00692a0 msgr2=0x7febf006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7febf00692a0 0x7febf006b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7febfc009b80 tx=0x7febfc00b040 comp rx=0 tx=0).stop 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec080708b0 msgr2=0x7fec0810fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec080708b0 0x7fec0810fc50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fec000098e0 tx=0x7fec00004d10 comp rx=0 tx=0).stop 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 shutdown_connections 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7febf00692a0 0x7febf006b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec080708b0 0x7fec0810fc50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 --2- 172.21.15.81:0/63037091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fec08071e60 0x7fec08110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 >> 172.21.15.81:0/63037091 conn(0x7fec080f8550 msgr2=0x7fec080f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.359+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 shutdown_connections 2024-03-22T07:07:40.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.360+0000 7fec0db46700 1 -- 172.21.15.81:0/63037091 wait complete. 2024-03-22T07:07:40.370 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:07:40.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.482+0000 7f2c68025700 1 -- 172.21.15.81:0/2258722645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c60100010 msgr2=0x7f2c601023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.482+0000 7f2c68025700 1 --2- 172.21.15.81:0/2258722645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c60100010 0x7f2c601023f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2c50004940 tx=0x7f2c50009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:40.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.482+0000 7f2c68025700 1 -- 172.21.15.81:0/2258722645 shutdown_connections 2024-03-22T07:07:40.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.482+0000 7f2c68025700 1 --2- 172.21.15.81:0/2258722645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c60100010 0x7f2c601023f0 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2c50004940 tx=0x7f2c50009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:40.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.482+0000 7f2c68025700 1 --2- 172.21.15.81:0/2258722645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c60066af0 0x7f2c600ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.482+0000 7f2c68025700 1 -- 172.21.15.81:0/2258722645 >> 172.21.15.81:0/2258722645 conn(0x7f2c600f74e0 msgr2=0x7f2c600f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:40.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.482+0000 7f2c68025700 1 -- 172.21.15.81:0/2258722645 shutdown_connections 2024-03-22T07:07:40.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.482+0000 7f2c68025700 1 -- 172.21.15.81:0/2258722645 wait complete. 2024-03-22T07:07:40.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.483+0000 7f2c68025700 1 Processor -- start 2024-03-22T07:07:40.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.483+0000 7f2c68025700 1 -- start start 2024-03-22T07:07:40.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.483+0000 7f2c68025700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c60066af0 0x7f2c6006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:40.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.483+0000 7f2c68025700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c6006fb30 0x7f2c6006b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:40.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.483+0000 7f2c68025700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6006fff0 con 0x7f2c6006fb30 2024-03-22T07:07:40.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.483+0000 7f2c68025700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6006b550 con 0x7f2c60066af0 2024-03-22T07:07:40.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.483+0000 7f2c65dc1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c60066af0 0x7f2c6006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:40.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.483+0000 7f2c65dc1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c60066af0 0x7f2c6006f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.81:34386/0 (socket says 172.21.15.81:34386) 2024-03-22T07:07:40.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.484+0000 7f2c65dc1700 1 -- 172.21.15.81:0/887409002 learned_addr learned my addr 172.21.15.81:0/887409002 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:07:40.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.484+0000 7f2c655c0700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c6006fb30 0x7f2c6006b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.484+0000 7f2c65dc1700 1 -- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c6006fb30 msgr2=0x7f2c6006b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.484+0000 7f2c65dc1700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c6006fb30 0x7f2c6006b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.484+0000 7f2c65dc1700 1 -- 172.21.15.81:0/887409002 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50005040 con 0x7f2c60066af0 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.484+0000 7f2c655c0700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c6006fb30 0x7f2c6006b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.484+0000 7f2c65dc1700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c60066af0 0x7f2c6006f5f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00eb40 tx=0x7f2c5c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.485+0000 7f2c567fc700 1 -- 172.21.15.81:0/887409002 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c5c007cc0 con 0x7f2c60066af0 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.485+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6006b7d0 con 0x7f2c60066af0 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.485+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6006c3a0 con 0x7f2c60066af0 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.485+0000 7f2c567fc700 1 -- 172.21.15.81:0/887409002 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c5c00f040 con 0x7f2c60066af0 2024-03-22T07:07:40.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.485+0000 7f2c567fc700 1 -- 172.21.15.81:0/887409002 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c5c00cb30 con 0x7f2c60066af0 2024-03-22T07:07:40.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.486+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c440052e0 con 0x7f2c60066af0 2024-03-22T07:07:40.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.487+0000 7f2c567fc700 1 -- 172.21.15.81:0/887409002 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2c5c00cc90 con 0x7f2c60066af0 2024-03-22T07:07:40.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.488+0000 7f2c567fc700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2c4c0692a0 0x7f2c4c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:40.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.488+0000 7f2c567fc700 1 -- 172.21.15.81:0/887409002 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2c5c086d60 con 0x7f2c60066af0 2024-03-22T07:07:40.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.488+0000 7f2c655c0700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2c4c0692a0 0x7f2c4c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:40.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.489+0000 7f2c655c0700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2c4c0692a0 0x7f2c4c06b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c50000c00 tx=0x7f2c50009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:40.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.496+0000 7f2c567fc700 1 -- 172.21.15.81:0/887409002 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2c5c08b070 con 0x7f2c60066af0 2024-03-22T07:07:40.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.766+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c44000bf0 con 0x7f2c4c0692a0 2024-03-22T07:07:40.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.768+0000 7f2c567fc700 1 -- 172.21.15.81:0/887409002 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f2c44000bf0 con 0x7f2c4c0692a0 2024-03-22T07:07:40.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2c4c0692a0 msgr2=0x7f2c4c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2c4c0692a0 0x7f2c4c06b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c50000c00 tx=0x7f2c50009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:40.771 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c60066af0 msgr2=0x7f2c6006f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c60066af0 0x7f2c6006f5f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00eb40 tx=0x7f2c5c00b040 comp rx=0 tx=0).stop 2024-03-22T07:07:40.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 shutdown_connections 2024-03-22T07:07:40.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f2c4c0692a0 0x7f2c4c06b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2c60066af0 0x7f2c6006f5f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 --2- 172.21.15.81:0/887409002 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c6006fb30 0x7f2c6006b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 >> 172.21.15.81:0/887409002 conn(0x7f2c600f74e0 msgr2=0x7f2c600f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:40.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 shutdown_connections 2024-03-22T07:07:40.772 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.771+0000 7f2c68025700 1 -- 172.21.15.81:0/887409002 wait complete. 2024-03-22T07:07:40.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.902+0000 7f7f1a16e700 1 -- 172.21.15.81:0/1474031263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 msgr2=0x7f7f14101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.902+0000 7f7f1a16e700 1 --2- 172.21.15.81:0/1474031263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 0x7f7f14101df0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7f04004940 tx=0x7f7f04009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:40.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.902+0000 7f7f1a16e700 1 -- 172.21.15.81:0/1474031263 shutdown_connections 2024-03-22T07:07:40.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.902+0000 7f7f1a16e700 1 --2- 172.21.15.81:0/1474031263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f14102330 0x7f7f14104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.902+0000 7f7f1a16e700 1 --2- 172.21.15.81:0/1474031263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 0x7f7f14101df0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.902+0000 7f7f1a16e700 1 -- 172.21.15.81:0/1474031263 >> 172.21.15.81:0/1474031263 conn(0x7f7f140f74a0 msgr2=0x7f7f140f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:40.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.902+0000 7f7f1a16e700 1 -- 172.21.15.81:0/1474031263 shutdown_connections 2024-03-22T07:07:40.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.902+0000 7f7f1a16e700 1 -- 172.21.15.81:0/1474031263 wait complete. 2024-03-22T07:07:40.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.903+0000 7f7f1a16e700 1 Processor -- start 2024-03-22T07:07:40.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.903+0000 7f7f1a16e700 1 -- start start 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.903+0000 7f7f1a16e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 0x7f7f1410fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.903+0000 7f7f1a16e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f14102330 0x7f7f141100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.903+0000 7f7f1a16e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141106a0 con 0x7f7f14066f50 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.903+0000 7f7f1a16e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14110810 con 0x7f7f14102330 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.903+0000 7f7f137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 0x7f7f1410fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.904+0000 7f7f137fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 0x7f7f1410fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38440/0 (socket says 172.21.15.81:38440) 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.904+0000 7f7f137fe700 1 -- 172.21.15.81:0/3545661351 learned_addr learned my addr 172.21.15.81:0/3545661351 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.904+0000 7f7f137fe700 1 -- 172.21.15.81:0/3545661351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f14102330 msgr2=0x7f7f141100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.904+0000 7f7f137fe700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f14102330 0x7f7f141100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:40.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.904+0000 7f7f137fe700 1 -- 172.21.15.81:0/3545661351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04005040 con 0x7f7f14066f50 2024-03-22T07:07:40.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.904+0000 7f7f137fe700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 0x7f7f1410fb60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7f04000c00 tx=0x7f7f04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:40.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.904+0000 7f7efbfff700 1 -- 172.21.15.81:0/3545661351 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0400e070 con 0x7f7f14066f50 2024-03-22T07:07:40.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.904+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f141151c0 con 0x7f7f14066f50 2024-03-22T07:07:40.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.905+0000 7f7efbfff700 1 -- 172.21.15.81:0/3545661351 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f0401c400 con 0x7f7f14066f50 2024-03-22T07:07:40.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.905+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f14115720 con 0x7f7f14066f50 2024-03-22T07:07:40.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.905+0000 7f7efbfff700 1 -- 172.21.15.81:0/3545661351 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0400b650 con 0x7f7f14066f50 2024-03-22T07:07:40.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.906+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f000052e0 con 0x7f7f14066f50 2024-03-22T07:07:40.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.906+0000 7f7efbfff700 1 -- 172.21.15.81:0/3545661351 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7f0401c570 con 0x7f7f14066f50 2024-03-22T07:07:40.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.907+0000 7f7efbfff700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f7efc0692f0 0x7f7efc06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:40.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.907+0000 7f7efbfff700 1 -- 172.21.15.81:0/3545661351 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f7f040888d0 con 0x7f7f14066f50 2024-03-22T07:07:40.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.907+0000 7f7f12ffd700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f7efc0692f0 0x7f7efc06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:40.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.908+0000 7f7f12ffd700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f7efc0692f0 0x7f7efc06b7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009fd0 tx=0x7f7f0800b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:40.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:40.916+0000 7f7efbfff700 1 -- 172.21.15.81:0/3545661351 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7f040585a0 con 0x7f7f14066f50 2024-03-22T07:07:41.211 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.210+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f00000bf0 con 0x7f7efc0692f0 2024-03-22T07:07:41.228 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.228+0000 7f7efbfff700 1 -- 172.21.15.81:0/3545661351 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f7f00000bf0 con 0x7f7efc0692f0 2024-03-22T07:07:41.228 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:07:41.228 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (5m) 51s ago 6m 25.9M - 0.25.0 c8568f914cd2 f83c4e446eae 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (6m) 51s ago 6m 6660k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (5m) 56s ago 5m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (5m) 51s ago 6m 82.4M - 9.4.7 954c08fa6188 5b1c9b6c3c44 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (76s) 51s ago 75s 10.4M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (69s) 51s ago 68s 16.5M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (66s) 56s ago 66s 13.5M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (73s) 56s ago 73s 16.0M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:9283 running (7m) 51s ago 7m 480M - 17.2.7-726-g9c4e0c34 837124da804f 3ef26f13517a 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283 running (5m) 56s ago 5m 435M - 17.2.7-726-g9c4e0c34 837124da804f eefa675aaa8e 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (8m) 51s ago 8m 47.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (5m) 56s ago 5m 38.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (6m) 51s ago 6m 13.7M - 1.5.0 0da6a335fe13 c04b2c4a7e70 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (5m) 56s ago 5m 15.3M - 1.5.0 0da6a335fe13 fbdea05540dc 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (4m) 51s ago 4m 45.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:07:41.229 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (4m) 51s ago 4m 46.5M 4288M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:07:41.230 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (3m) 51s ago 3m 43.0M 4288M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:07:41.230 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (3m) 56s ago 3m 47.0M 1554M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:07:41.230 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (3m) 56s ago 3m 41.4M 1554M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:07:41.230 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (2m) 56s ago 2m 41.5M 1554M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:07:41.230 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (5m) 51s ago 6m 56.0M - 2.43.0 a07b618ecd1d 92a93d9b8284 2024-03-22T07:07:41.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f7efc0692f0 msgr2=0x7f7efc06b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:41.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f7efc0692f0 0x7f7efc06b7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009fd0 tx=0x7f7f0800b040 comp rx=0 tx=0).stop 2024-03-22T07:07:41.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 msgr2=0x7f7f1410fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:41.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 0x7f7f1410fb60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7f04000c00 tx=0x7f7f04009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:41.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 shutdown_connections 2024-03-22T07:07:41.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f7efc0692f0 0x7f7efc06b7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f14066f50 0x7f7f1410fb60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 --2- 172.21.15.81:0/3545661351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7f14102330 0x7f7f141100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 >> 172.21.15.81:0/3545661351 conn(0x7f7f140f74a0 msgr2=0x7f7f140f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:41.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 shutdown_connections 2024-03-22T07:07:41.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.231+0000 7f7f1a16e700 1 -- 172.21.15.81:0/3545661351 wait complete. 2024-03-22T07:07:41.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.382+0000 7fea1c8c9700 1 -- 172.21.15.81:0/487433972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 msgr2=0x7fea180ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:41.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.382+0000 7fea1c8c9700 1 --2- 172.21.15.81:0/487433972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 0x7fea180ffa20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fea08004940 tx=0x7fea08009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:41.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.383+0000 7fea1c8c9700 1 -- 172.21.15.81:0/487433972 shutdown_connections 2024-03-22T07:07:41.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.383+0000 7fea1c8c9700 1 --2- 172.21.15.81:0/487433972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea180fff60 0x7fea18102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.383+0000 7fea1c8c9700 1 --2- 172.21.15.81:0/487433972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 0x7fea180ffa20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.383+0000 7fea1c8c9700 1 -- 172.21.15.81:0/487433972 >> 172.21.15.81:0/487433972 conn(0x7fea180f7430 msgr2=0x7fea180f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:41.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.383+0000 7fea1c8c9700 1 -- 172.21.15.81:0/487433972 shutdown_connections 2024-03-22T07:07:41.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.383+0000 7fea1c8c9700 1 -- 172.21.15.81:0/487433972 wait complete. 2024-03-22T07:07:41.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.383+0000 7fea1c8c9700 1 Processor -- start 2024-03-22T07:07:41.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.383+0000 7fea1c8c9700 1 -- start start 2024-03-22T07:07:41.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1c8c9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 0x7fea180fd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1c8c9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea180fff60 0x7fea180fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 0x7fea180fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 0x7fea180fd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38460/0 (socket says 172.21.15.81:38460) 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1659c700 1 -- 172.21.15.81:0/1993364914 learned_addr learned my addr 172.21.15.81:0/1993364914 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea180fd820 con 0x7fea18066e90 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea18111610 con 0x7fea180fff60 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea15d9b700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea180fff60 0x7fea180fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1659c700 1 -- 172.21.15.81:0/1993364914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea180fff60 msgr2=0x7fea180fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1659c700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea180fff60 0x7fea180fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea1659c700 1 -- 172.21.15.81:0/1993364914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea08005040 con 0x7fea18066e90 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.384+0000 7fea15d9b700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea180fff60 0x7fea180fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.385+0000 7fea1659c700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 0x7fea180fd220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fea08000c00 tx=0x7fea08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:41.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.385+0000 7fea06ffd700 1 -- 172.21.15.81:0/1993364914 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea0800e070 con 0x7fea18066e90 2024-03-22T07:07:41.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.385+0000 7fea06ffd700 1 -- 172.21.15.81:0/1993364914 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea0801c400 con 0x7fea18066e90 2024-03-22T07:07:41.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.385+0000 7fea06ffd700 1 -- 172.21.15.81:0/1993364914 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea0800b650 con 0x7fea18066e90 2024-03-22T07:07:41.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.385+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea180fbe60 con 0x7fea18066e90 2024-03-22T07:07:41.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.385+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea180fc340 con 0x7fea18066e90 2024-03-22T07:07:41.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.386+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1804c710 con 0x7fea18066e90 2024-03-22T07:07:41.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.387+0000 7fea06ffd700 1 -- 172.21.15.81:0/1993364914 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fea0801c570 con 0x7fea18066e90 2024-03-22T07:07:41.389 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.388+0000 7fea06ffd700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fea000692a0 0x7fea0006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:41.389 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.388+0000 7fea06ffd700 1 -- 172.21.15.81:0/1993364914 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fea08088760 con 0x7fea18066e90 2024-03-22T07:07:41.389 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.388+0000 7fea15d9b700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fea000692a0 0x7fea0006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:41.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.388+0000 7fea15d9b700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fea000692a0 0x7fea0006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea0c004600 tx=0x7fea0c00b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:41.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.396+0000 7fea06ffd700 1 -- 172.21.15.81:0/1993364914 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fea08058570 con 0x7fea18066e90 2024-03-22T07:07:41.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:41 smithi081 ceph-mon[29245]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:41.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:41 smithi149 ceph-mon[36068]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:41.846 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.846+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fea1811a210 con 0x7fea18066e90 2024-03-22T07:07:41.847 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.847+0000 7fea06ffd700 1 -- 172.21.15.81:0/1993364914 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fea0801c820 con 0x7fea18066e90 2024-03-22T07:07:41.847 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:07:41.847 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:07:41.847 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:07:41.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:07:41.847 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:07:41.847 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:07:41.848 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:07:41.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fea000692a0 msgr2=0x7fea0006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:41.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fea000692a0 0x7fea0006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea0c004600 tx=0x7fea0c00b040 comp rx=0 tx=0).stop 2024-03-22T07:07:41.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 msgr2=0x7fea180fd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:41.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 0x7fea180fd220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fea08000c00 tx=0x7fea08009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:41.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 shutdown_connections 2024-03-22T07:07:41.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fea000692a0 0x7fea0006b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea18066e90 0x7fea180fd220 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 --2- 172.21.15.81:0/1993364914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea180fff60 0x7fea180fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 >> 172.21.15.81:0/1993364914 conn(0x7fea180f7430 msgr2=0x7fea180f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:41.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 shutdown_connections 2024-03-22T07:07:41.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.849+0000 7fea1c8c9700 1 -- 172.21.15.81:0/1993364914 wait complete. 2024-03-22T07:07:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.972+0000 7f3fce6a4700 1 -- 172.21.15.81:0/4060007017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc8066b80 msgr2=0x7f3fc80ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.972+0000 7f3fce6a4700 1 --2- 172.21.15.81:0/4060007017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc8066b80 0x7f3fc80ffa70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8004940 tx=0x7f3fb8009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.972+0000 7f3fce6a4700 1 -- 172.21.15.81:0/4060007017 shutdown_connections 2024-03-22T07:07:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.972+0000 7f3fce6a4700 1 --2- 172.21.15.81:0/4060007017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fc80fffb0 0x7f3fc81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.972+0000 7f3fce6a4700 1 --2- 172.21.15.81:0/4060007017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc8066b80 0x7f3fc80ffa70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.972+0000 7f3fce6a4700 1 -- 172.21.15.81:0/4060007017 >> 172.21.15.81:0/4060007017 conn(0x7f3fc80f7480 msgr2=0x7f3fc80f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.972+0000 7f3fce6a4700 1 -- 172.21.15.81:0/4060007017 shutdown_connections 2024-03-22T07:07:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.972+0000 7f3fce6a4700 1 -- 172.21.15.81:0/4060007017 wait complete. 2024-03-22T07:07:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fce6a4700 1 Processor -- start 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fce6a4700 1 -- start start 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fce6a4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fc8066b80 0x7f3fc810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fce6a4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc80fffb0 0x7f3fc810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fc77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc80fffb0 0x7f3fc810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fc77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc80fffb0 0x7f3fc810df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38486/0 (socket says 172.21.15.81:38486) 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fce6a4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc810e580 con 0x7f3fc80fffb0 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fc7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fc8066b80 0x7f3fc810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fce6a4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc810ed80 con 0x7f3fc8066b80 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fc7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fc8066b80 0x7f3fc810da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.81:34428/0 (socket says 172.21.15.81:34428) 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.973+0000 7f3fc7fff700 1 -- 172.21.15.81:0/2468614430 learned_addr learned my addr 172.21.15.81:0/2468614430 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.974+0000 7f3fc77fe700 1 -- 172.21.15.81:0/2468614430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fc8066b80 msgr2=0x7f3fc810da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.974+0000 7f3fc77fe700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fc8066b80 0x7f3fc810da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:41.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.974+0000 7f3fc77fe700 1 -- 172.21.15.81:0/2468614430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb8005040 con 0x7f3fc80fffb0 2024-03-22T07:07:41.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.974+0000 7f3fc7fff700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fc8066b80 0x7f3fc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:07:41.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.974+0000 7f3fc77fe700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc80fffb0 0x7f3fc810df80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009fd0 tx=0x7f3fbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:41.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.974+0000 7f3fc4ff9700 1 -- 172.21.15.81:0/2468614430 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fbc007ea0 con 0x7f3fc80fffb0 2024-03-22T07:07:41.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.974+0000 7f3fc4ff9700 1 -- 172.21.15.81:0/2468614430 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fbc00f040 con 0x7f3fc80fffb0 2024-03-22T07:07:41.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.974+0000 7f3fc4ff9700 1 -- 172.21.15.81:0/2468614430 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fbc013b70 con 0x7f3fc80fffb0 2024-03-22T07:07:41.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.974+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc810f000 con 0x7f3fc80fffb0 2024-03-22T07:07:41.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.975+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc810f480 con 0x7f3fc80fffb0 2024-03-22T07:07:41.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.976+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc805f5f0 con 0x7f3fc80fffb0 2024-03-22T07:07:41.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.976+0000 7f3fc4ff9700 1 -- 172.21.15.81:0/2468614430 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3fbc013cd0 con 0x7f3fc80fffb0 2024-03-22T07:07:41.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.977+0000 7f3fc4ff9700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3fb0069370 0x7f3fb006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:41.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.977+0000 7f3fc4ff9700 1 -- 172.21.15.81:0/2468614430 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3fbc08afa0 con 0x7f3fc80fffb0 2024-03-22T07:07:41.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.977+0000 7f3fc7fff700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3fb0069370 0x7f3fb006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:41.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.977+0000 7f3fc7fff700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3fb0069370 0x7f3fb006b820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800b0d0 tx=0x7f3fb8009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:41.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:41.983+0000 7f3fc4ff9700 1 -- 172.21.15.81:0/2468614430 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3fbc05adb0 con 0x7f3fc80fffb0 2024-03-22T07:07:42.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.276+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fc810f7c0 con 0x7f3fb0069370 2024-03-22T07:07:42.278 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.278+0000 7f3fc4ff9700 1 -- 172.21.15.81:0/2468614430 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f3fc810f7c0 con 0x7f3fb0069370 2024-03-22T07:07:42.278 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:07:42.279 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:07:42.279 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:07:42.279 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:07:42.279 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [], 2024-03-22T07:07:42.279 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "0/2 daemons upgraded", 2024-03-22T07:07:42.279 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 image on host smithi149", 2024-03-22T07:07:42.279 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:07:42.279 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:07:42.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3fb0069370 msgr2=0x7f3fb006b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:42.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3fb0069370 0x7f3fb006b820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800b0d0 tx=0x7f3fb8009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:42.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc80fffb0 msgr2=0x7f3fc810df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:42.281 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc80fffb0 0x7f3fc810df80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009fd0 tx=0x7f3fbc00b040 comp rx=0 tx=0).stop 2024-03-22T07:07:42.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 shutdown_connections 2024-03-22T07:07:42.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f3fb0069370 0x7f3fb006b820 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:42.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fc8066b80 0x7f3fc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:42.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 --2- 172.21.15.81:0/2468614430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fc80fffb0 0x7f3fc810df80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:42.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 >> 172.21.15.81:0/2468614430 conn(0x7f3fc80f7480 msgr2=0x7f3fc80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:42.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 shutdown_connections 2024-03-22T07:07:42.282 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:07:42.281+0000 7f3fce6a4700 1 -- 172.21.15.81:0/2468614430 wait complete. 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: pgmap v214: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (unknown) 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: Upgrade: Need to upgrade myself (mgr.smithi081.bcxarw) 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 on smithi149 2024-03-22T07:07:42.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:42 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1993364914' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:07:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: pgmap v214: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:07:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:07:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (unknown) 2024-03-22T07:07:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-22T07:07:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:07:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: Upgrade: Need to upgrade myself (mgr.smithi081.bcxarw) 2024-03-22T07:07:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 on smithi149 2024-03-22T07:07:42.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:42 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1993364914' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:07:43.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:43 smithi081 ceph-mon[29245]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:43.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:43 smithi149 ceph-mon[36068]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:44.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:44 smithi081 ceph-mon[29245]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T07:07:44.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:44 smithi149 ceph-mon[36068]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-03-22T07:07:46.654 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:46 smithi149 ceph-mon[36068]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T07:07:46.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:46 smithi081 ceph-mon[29245]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T07:07:48.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:48 smithi081 ceph-mon[29245]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T07:07:48.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:48 smithi149 ceph-mon[36068]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-03-22T07:07:50.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:50 smithi081 ceph-mon[29245]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T07:07:50.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:50 smithi149 ceph-mon[36068]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T07:07:52.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:52 smithi081 ceph-mon[29245]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T07:07:52.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:52 smithi149 ceph-mon[36068]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T07:07:54.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:54 smithi081 ceph-mon[29245]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T07:07:54.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:54 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:07:54.851 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:54 smithi149 ceph-mon[36068]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-03-22T07:07:54.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:54 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:07:56.620 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:56 smithi081 ceph-mon[29245]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:56.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:56 smithi149 ceph-mon[36068]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:58.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:07:58 smithi081 ceph-mon[29245]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:07:58.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:07:58 smithi149 ceph-mon[36068]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:00.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:00 smithi081 ceph-mon[29245]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:00.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:00 smithi149 ceph-mon[36068]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:02.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:02 smithi081 ceph-mon[29245]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:02.851 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:02 smithi149 ceph-mon[36068]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr:Note: switching to '768f6cd6e95d0c7a626042719c230df59bdba9d3'. 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr: 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr:state without impacting any branches by switching back to a branch. 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr: 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr: 2024-03-22T07:08:03.850 INFO:tasks.workunit.client.0.smithi081.stderr: git switch -c 2024-03-22T07:08:03.851 INFO:tasks.workunit.client.0.smithi081.stderr: 2024-03-22T07:08:03.851 INFO:tasks.workunit.client.0.smithi081.stderr:Or undo this operation with: 2024-03-22T07:08:03.851 INFO:tasks.workunit.client.0.smithi081.stderr: 2024-03-22T07:08:03.851 INFO:tasks.workunit.client.0.smithi081.stderr: git switch - 2024-03-22T07:08:03.851 INFO:tasks.workunit.client.0.smithi081.stderr: 2024-03-22T07:08:03.851 INFO:tasks.workunit.client.0.smithi081.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-22T07:08:03.851 INFO:tasks.workunit.client.0.smithi081.stderr: 2024-03-22T07:08:03.851 INFO:tasks.workunit.client.0.smithi081.stderr:HEAD is now at 768f6cd6e95 Merge pull request #56339 from zdover23/wip-doc-2024-03-20-backport-56333-to-squid 2024-03-22T07:08:03.854 DEBUG:teuthology.orchestra.run.smithi081:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-22T07:08:03.883 INFO:tasks.workunit.client.0.smithi081.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-22T07:08:03.885 INFO:tasks.workunit.client.0.smithi081.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T07:08:03.885 INFO:tasks.workunit.client.0.smithi081.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-22T07:08:04.733 INFO:tasks.workunit.client.0.smithi081.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-22T07:08:04.790 INFO:tasks.workunit.client.0.smithi081.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-22T07:08:04.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:04 smithi081 ceph-mon[29245]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:04.830 INFO:tasks.workunit.client.0.smithi081.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T07:08:04.831 INFO:tasks.workunit.client.0.smithi081.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T07:08:04.831 INFO:tasks.workunit.client.0.smithi081.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-22T07:08:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:04 smithi149 ceph-mon[36068]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:04.870 INFO:tasks.workunit.client.0.smithi081.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T07:08:04.874 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:08:04.874 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-22T07:08:04.929 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-22T07:08:04.932 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-22T07:08:04.932 DEBUG:teuthology.orchestra.run.smithi081:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=768f6cd6e95d0c7a626042719c230df59bdba9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-22T07:08:04.990 INFO:tasks.workunit.client.0.smithi081.stderr:+ mkdir -p fsstress 2024-03-22T07:08:04.991 INFO:tasks.workunit.client.0.smithi081.stderr:+ pushd fsstress 2024-03-22T07:08:04.992 INFO:tasks.workunit.client.0.smithi081.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T07:08:04.992 INFO:tasks.workunit.client.0.smithi081.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-22T07:08:05.686 INFO:tasks.workunit.client.0.smithi081.stderr:+ tar xzf ltp-full.tgz 2024-03-22T07:08:06.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:06 smithi081 ceph-mon[29245]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:06.851 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:06 smithi149 ceph-mon[36068]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:08.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:08 smithi081 ceph-mon[29245]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:08.782 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:08 smithi149 ceph-mon[36068]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:08:09.574 INFO:tasks.workunit.client.1.smithi149.stderr:Updating files: 90% (11223/12393) Updating files: 91% (11278/12393) Updating files: 92% (11402/12393) Updating files: 93% (11526/12393) Updating files: 94% (11650/12393) Updating files: 95% (11774/12393) Updating files: 96% (11898/12393) Updating files: 97% (12022/12393) Updating files: 98% (12146/12393) Updating files: 99% (12270/12393) Updating files: 100% (12393/12393) Updating files: 100% (12393/12393), done. 2024-03-22T07:08:09.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:09 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:08:09.851 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:09 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr:Note: switching to '768f6cd6e95d0c7a626042719c230df59bdba9d3'. 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr:state without impacting any branches by switching back to a branch. 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr: git switch -c 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-03-22T07:08:10.268 INFO:tasks.workunit.client.1.smithi149.stderr:Or undo this operation with: 2024-03-22T07:08:10.269 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-03-22T07:08:10.269 INFO:tasks.workunit.client.1.smithi149.stderr: git switch - 2024-03-22T07:08:10.269 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-03-22T07:08:10.269 INFO:tasks.workunit.client.1.smithi149.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-22T07:08:10.269 INFO:tasks.workunit.client.1.smithi149.stderr: 2024-03-22T07:08:10.269 INFO:tasks.workunit.client.1.smithi149.stderr:HEAD is now at 768f6cd6e95 Merge pull request #56339 from zdover23/wip-doc-2024-03-20-backport-56333-to-squid 2024-03-22T07:08:10.273 DEBUG:teuthology.orchestra.run.smithi149:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-22T07:08:10.438 INFO:tasks.workunit.client.1.smithi149.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-22T07:08:10.445 INFO:tasks.workunit.client.1.smithi149.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-22T07:08:10.445 INFO:tasks.workunit.client.1.smithi149.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-22T07:08:10.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:10 smithi081 ceph-mon[29245]: pgmap v228: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-03-22T07:08:10.851 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:10 smithi149 ceph-mon[36068]: pgmap v228: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.394+0000 7fe171e81700 1 -- 172.21.15.81:0/2333730603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fbce0 msgr2=0x7fe16c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.394+0000 7fe171e81700 1 --2- 172.21.15.81:0/2333730603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fbce0 0x7fe16c0fc0f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe15c004940 tx=0x7fe15c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.394+0000 7fe171e81700 1 -- 172.21.15.81:0/2333730603 shutdown_connections 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.394+0000 7fe171e81700 1 --2- 172.21.15.81:0/2333730603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16c0fcf90 0x7fe16c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.394+0000 7fe171e81700 1 --2- 172.21.15.81:0/2333730603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fbce0 0x7fe16c0fc0f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.394+0000 7fe171e81700 1 -- 172.21.15.81:0/2333730603 >> 172.21.15.81:0/2333730603 conn(0x7fe16c0f7440 msgr2=0x7fe16c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.394+0000 7fe171e81700 1 -- 172.21.15.81:0/2333730603 shutdown_connections 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.394+0000 7fe171e81700 1 -- 172.21.15.81:0/2333730603 wait complete. 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.394+0000 7fe171e81700 1 Processor -- start 2024-03-22T07:08:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe171e81700 1 -- start start 2024-03-22T07:08:12.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe171e81700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16c0fbce0 0x7fe16c112ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:12.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe171e81700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fcf90 0x7fe16c115020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:12.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe171e81700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c066ad0 con 0x7fe16c0fcf90 2024-03-22T07:08:12.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe171e81700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c066c10 con 0x7fe16c0fbce0 2024-03-22T07:08:12.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe16affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fcf90 0x7fe16c115020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:12.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe16affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fcf90 0x7fe16c115020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51438/0 (socket says 172.21.15.81:51438) 2024-03-22T07:08:12.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe16b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16c0fbce0 0x7fe16c112ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:12.396 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe16affd700 1 -- 172.21.15.81:0/1471055361 learned_addr learned my addr 172.21.15.81:0/1471055361 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:12.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe16b7fe700 1 -- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fcf90 msgr2=0x7fe16c115020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:12.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe16b7fe700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fcf90 0x7fe16c115020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:12.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe16b7fe700 1 -- 172.21.15.81:0/1471055361 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c005040 con 0x7fe16c0fbce0 2024-03-22T07:08:12.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe16affd700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fcf90 0x7fe16c115020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:08:12.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.395+0000 7fe16b7fe700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16c0fbce0 0x7fe16c112ad0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000c00 tx=0x7fe15c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:12.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.396+0000 7fe153fff700 1 -- 172.21.15.81:0/1471055361 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe15c00e030 con 0x7fe16c0fbce0 2024-03-22T07:08:12.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.396+0000 7fe153fff700 1 -- 172.21.15.81:0/1471055361 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe15c013de0 con 0x7fe16c0fbce0 2024-03-22T07:08:12.397 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.396+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c115560 con 0x7fe16c0fbce0 2024-03-22T07:08:12.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.396+0000 7fe153fff700 1 -- 172.21.15.81:0/1471055361 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe15c01ce50 con 0x7fe16c0fbce0 2024-03-22T07:08:12.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.396+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c1158b0 con 0x7fe16c0fbce0 2024-03-22T07:08:12.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.397+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c05f5f0 con 0x7fe16c0fbce0 2024-03-22T07:08:12.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.397+0000 7fe153fff700 1 -- 172.21.15.81:0/1471055361 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe15c00b4c0 con 0x7fe16c0fbce0 2024-03-22T07:08:12.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.397+0000 7fe153fff700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe154069230 0x7fe15406b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:12.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.398+0000 7fe16affd700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe154069230 0x7fe15406b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:12.398 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.398+0000 7fe153fff700 1 -- 172.21.15.81:0/1471055361 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe15c088c80 con 0x7fe16c0fbce0 2024-03-22T07:08:12.399 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.398+0000 7fe16affd700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe154069230 0x7fe15406b6e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe1600041d0 tx=0x7fe16000b040 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:12.402 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.402+0000 7fe153fff700 1 -- 172.21.15.81:0/1471055361 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe15c054f90 con 0x7fe16c0fbce0 2024-03-22T07:08:12.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.649+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe16c101260 con 0x7fe154069230 2024-03-22T07:08:12.650 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:12 smithi081 ceph-mon[29245]: pgmap v229: 65 pgs: 65 active+clean; 10 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 850 KiB/s wr, 38 op/s 2024-03-22T07:08:12.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.651+0000 7fe153fff700 1 -- 172.21.15.81:0/1471055361 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fe16c101260 con 0x7fe154069230 2024-03-22T07:08:12.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe154069230 msgr2=0x7fe15406b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:12.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe154069230 0x7fe15406b6e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe1600041d0 tx=0x7fe16000b040 comp rx=0 tx=0).stop 2024-03-22T07:08:12.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16c0fbce0 msgr2=0x7fe16c112ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:12.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16c0fbce0 0x7fe16c112ad0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000c00 tx=0x7fe15c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:12.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 shutdown_connections 2024-03-22T07:08:12.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fe154069230 0x7fe15406b6e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:12.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16c0fbce0 0x7fe16c112ad0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:12.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 --2- 172.21.15.81:0/1471055361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe16c0fcf90 0x7fe16c115020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:12.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 >> 172.21.15.81:0/1471055361 conn(0x7fe16c0f7440 msgr2=0x7fe16c0f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:12.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 shutdown_connections 2024-03-22T07:08:12.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.653+0000 7fe171e81700 1 -- 172.21.15.81:0/1471055361 wait complete. 2024-03-22T07:08:12.667 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:08:12.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.761+0000 7f1205bbb700 1 -- 172.21.15.81:0/471055036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fe0d0 msgr2=0x7f12000fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:12.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.761+0000 7f1205bbb700 1 --2- 172.21.15.81:0/471055036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fe0d0 0x7f12000fe540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f11f4004940 tx=0x7f11f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:12.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.761+0000 7f1205bbb700 1 -- 172.21.15.81:0/471055036 shutdown_connections 2024-03-22T07:08:12.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.761+0000 7f1205bbb700 1 --2- 172.21.15.81:0/471055036 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fe0d0 0x7f12000fe540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:12.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.761+0000 7f1205bbb700 1 --2- 172.21.15.81:0/471055036 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12000fcee0 0x7f12000fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:12.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.761+0000 7f1205bbb700 1 -- 172.21.15.81:0/471055036 >> 172.21.15.81:0/471055036 conn(0x7f12000f8560 msgr2=0x7f12000fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:12.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.761+0000 7f1205bbb700 1 -- 172.21.15.81:0/471055036 shutdown_connections 2024-03-22T07:08:12.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.761+0000 7f1205bbb700 1 -- 172.21.15.81:0/471055036 wait complete. 2024-03-22T07:08:12.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f1205bbb700 1 Processor -- start 2024-03-22T07:08:12.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f1205bbb700 1 -- start start 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f1205bbb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fcee0 0x7f1200071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f1205bbb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12000fe0d0 0x7f1200070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f1205bbb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12000708a0 con 0x7f12000fcee0 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f1205bbb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12000709e0 con 0x7f12000fe0d0 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f11ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fcee0 0x7f1200071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f11ff7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fcee0 0x7f1200071dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51462/0 (socket says 172.21.15.81:51462) 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f11ff7fe700 1 -- 172.21.15.81:0/1438596005 learned_addr learned my addr 172.21.15.81:0/1438596005 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.762+0000 7f11feffd700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12000fe0d0 0x7f1200070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.763+0000 7f11ff7fe700 1 -- 172.21.15.81:0/1438596005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12000fe0d0 msgr2=0x7f1200070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.763+0000 7f11ff7fe700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12000fe0d0 0x7f1200070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.763+0000 7f11ff7fe700 1 -- 172.21.15.81:0/1438596005 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f4005040 con 0x7f12000fcee0 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.763+0000 7f11ff7fe700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fcee0 0x7f1200071dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c8b0 tx=0x7f11f0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.763+0000 7f11e7fff700 1 -- 172.21.15.81:0/1438596005 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f00106f0 con 0x7f12000fcee0 2024-03-22T07:08:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.763+0000 7f11e7fff700 1 -- 172.21.15.81:0/1438596005 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11f0010d20 con 0x7f12000fcee0 2024-03-22T07:08:12.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.763+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1200070c90 con 0x7f12000fcee0 2024-03-22T07:08:12.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.763+0000 7f11e7fff700 1 -- 172.21.15.81:0/1438596005 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f0012e90 con 0x7f12000fcee0 2024-03-22T07:08:12.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.763+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1200115af0 con 0x7f12000fcee0 2024-03-22T07:08:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.764+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120005f5f0 con 0x7f12000fcee0 2024-03-22T07:08:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.765+0000 7f11e7fff700 1 -- 172.21.15.81:0/1438596005 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f11f0010850 con 0x7f12000fcee0 2024-03-22T07:08:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.765+0000 7f11e7fff700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f11e80692a0 0x7f11e806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.765+0000 7f11e7fff700 1 -- 172.21.15.81:0/1438596005 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f11f0087110 con 0x7f12000fcee0 2024-03-22T07:08:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.765+0000 7f11feffd700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f11e80692a0 0x7f11e806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.766+0000 7f11feffd700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f11e80692a0 0x7f11e806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11f400b970 tx=0x7f11f4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:12.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:12.773+0000 7f11e7fff700 1 -- 172.21.15.81:0/1438596005 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f11f00533f0 con 0x7f12000fcee0 2024-03-22T07:08:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:12 smithi149 ceph-mon[36068]: pgmap v229: 65 pgs: 65 active+clean; 10 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 850 KiB/s wr, 38 op/s 2024-03-22T07:08:13.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.086+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12001024e0 con 0x7f11e80692a0 2024-03-22T07:08:13.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.087+0000 7f11e7fff700 1 -- 172.21.15.81:0/1438596005 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f12001024e0 con 0x7f11e80692a0 2024-03-22T07:08:13.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f11e80692a0 msgr2=0x7f11e806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f11e80692a0 0x7f11e806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11f400b970 tx=0x7f11f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:13.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fcee0 msgr2=0x7f1200071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fcee0 0x7f1200071dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f11f000c8b0 tx=0x7f11f0004470 comp rx=0 tx=0).stop 2024-03-22T07:08:13.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 shutdown_connections 2024-03-22T07:08:13.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f11e80692a0 0x7f11e806b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12000fcee0 0x7f1200071dd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 --2- 172.21.15.81:0/1438596005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12000fe0d0 0x7f1200070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 >> 172.21.15.81:0/1438596005 conn(0x7f12000f8560 msgr2=0x7f12000f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:13.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 shutdown_connections 2024-03-22T07:08:13.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.090+0000 7f1205bbb700 1 -- 172.21.15.81:0/1438596005 wait complete. 2024-03-22T07:08:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.193+0000 7f507d544700 1 -- 172.21.15.81:0/2669578494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fdff0 msgr2=0x7f50780fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.193+0000 7f507d544700 1 --2- 172.21.15.81:0/2669578494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fdff0 0x7f50780fe460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f506c004940 tx=0x7f506c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.193+0000 7f507d544700 1 -- 172.21.15.81:0/2669578494 shutdown_connections 2024-03-22T07:08:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.193+0000 7f507d544700 1 --2- 172.21.15.81:0/2669578494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fdff0 0x7f50780fe460 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.193+0000 7f507d544700 1 --2- 172.21.15.81:0/2669578494 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fce80 0x7f50780fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.193+0000 7f507d544700 1 -- 172.21.15.81:0/2669578494 >> 172.21.15.81:0/2669578494 conn(0x7f50780f8540 msgr2=0x7f50780fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.194+0000 7f507d544700 1 -- 172.21.15.81:0/2669578494 shutdown_connections 2024-03-22T07:08:13.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.194+0000 7f507d544700 1 -- 172.21.15.81:0/2669578494 wait complete. 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.194+0000 7f507d544700 1 Processor -- start 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.194+0000 7f507d544700 1 -- start start 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f507d544700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fce80 0x7f507810fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f507d544700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fdff0 0x7f5078110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f507d544700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078110770 con 0x7f50780fce80 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f507d544700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078110fc0 con 0x7f50780fdff0 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f5076ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fce80 0x7f507810fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f5076ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fce80 0x7f507810fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51476/0 (socket says 172.21.15.81:51476) 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f5076ffd700 1 -- 172.21.15.81:0/2279296814 learned_addr learned my addr 172.21.15.81:0/2279296814 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:13.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f50767fc700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fdff0 0x7f5078110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f5076ffd700 1 -- 172.21.15.81:0/2279296814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fdff0 msgr2=0x7f5078110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f5076ffd700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fdff0 0x7f5078110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f5076ffd700 1 -- 172.21.15.81:0/2279296814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c005040 con 0x7f50780fce80 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f50767fc700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fdff0 0x7f5078110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f5076ffd700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fce80 0x7f507810fc30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5068002a50 tx=0x7f50680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.195+0000 7f505f7fe700 1 -- 172.21.15.81:0/2279296814 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5068007cc0 con 0x7f50780fce80 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.196+0000 7f505f7fe700 1 -- 172.21.15.81:0/2279296814 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f506800f040 con 0x7f50780fce80 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.196+0000 7f505f7fe700 1 -- 172.21.15.81:0/2279296814 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5068009640 con 0x7f50780fce80 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.196+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50781112a0 con 0x7f50780fce80 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.196+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50781116d0 con 0x7f50780fce80 2024-03-22T07:08:13.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.197+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5078101980 con 0x7f50780fce80 2024-03-22T07:08:13.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.197+0000 7f505f7fe700 1 -- 172.21.15.81:0/2279296814 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f50680097a0 con 0x7f50780fce80 2024-03-22T07:08:13.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.198+0000 7f505f7fe700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:13.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.198+0000 7f505f7fe700 1 -- 172.21.15.81:0/2279296814 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f5068087010 con 0x7f50780fce80 2024-03-22T07:08:13.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.198+0000 7f50767fc700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:13.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.198+0000 7f50767fc700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f506c00b970 tx=0x7f506c009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:13.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.203+0000 7f505f7fe700 1 -- 172.21.15.81:0/2279296814 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5068056d00 con 0x7f50780fce80 2024-03-22T07:08:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.434+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5078111a00 con 0x7f50600692a0 2024-03-22T07:08:13.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.451+0000 7f505f7fe700 1 -- 172.21.15.81:0/2279296814 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f5078111a00 con 0x7f50600692a0 2024-03-22T07:08:13.451 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:08:13.451 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (5m) 83s ago 7m 25.9M - 0.25.0 c8568f914cd2 f83c4e446eae 2024-03-22T07:08:13.451 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (7m) 83s ago 7m 6660k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:08:13.451 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (6m) 88s ago 6m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (5m) 83s ago 6m 82.4M - 9.4.7 954c08fa6188 5b1c9b6c3c44 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (108s) 83s ago 108s 10.4M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (101s) 83s ago 100s 16.5M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (98s) 88s ago 98s 13.5M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (105s) 88s ago 105s 16.0M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:9283 running (8m) 83s ago 8m 480M - 17.2.7-726-g9c4e0c34 837124da804f 3ef26f13517a 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283 running (6m) 88s ago 6m 435M - 17.2.7-726-g9c4e0c34 837124da804f eefa675aaa8e 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (8m) 83s ago 8m 47.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (6m) 88s ago 6m 38.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (7m) 83s ago 7m 13.7M - 1.5.0 0da6a335fe13 c04b2c4a7e70 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (6m) 88s ago 6m 15.3M - 1.5.0 0da6a335fe13 fbdea05540dc 2024-03-22T07:08:13.452 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (5m) 83s ago 5m 45.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:08:13.453 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (4m) 83s ago 4m 46.5M 4288M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:08:13.453 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (4m) 83s ago 4m 43.0M 4288M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:08:13.453 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (4m) 88s ago 4m 47.0M 1554M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:08:13.453 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (3m) 88s ago 3m 41.4M 1554M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:08:13.453 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (3m) 88s ago 3m 41.5M 1554M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:08:13.453 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (5m) 83s ago 6m 56.0M - 2.43.0 a07b618ecd1d 92a93d9b8284 2024-03-22T07:08:13.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 msgr2=0x7f506006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f506c00b970 tx=0x7f506c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:13.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fce80 msgr2=0x7f507810fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fce80 0x7f507810fc30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5068002a50 tx=0x7f50680046b0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 shutdown_connections 2024-03-22T07:08:13.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f50600692a0 0x7f506006b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50780fce80 0x7f507810fc30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 --2- 172.21.15.81:0/2279296814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50780fdff0 0x7f5078110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 >> 172.21.15.81:0/2279296814 conn(0x7f50780f8540 msgr2=0x7f50780f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:13.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 shutdown_connections 2024-03-22T07:08:13.455 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.454+0000 7f507d544700 1 -- 172.21.15.81:0/2279296814 wait complete. 2024-03-22T07:08:13.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.562+0000 7fcf40801700 1 -- 172.21.15.81:0/3482804001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c0fff80 msgr2=0x7fcf3c102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.562+0000 7fcf40801700 1 --2- 172.21.15.81:0/3482804001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c0fff80 0x7fcf3c102360 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcf30004940 tx=0x7fcf30009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:13.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.562+0000 7fcf40801700 1 -- 172.21.15.81:0/3482804001 shutdown_connections 2024-03-22T07:08:13.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.562+0000 7fcf40801700 1 --2- 172.21.15.81:0/3482804001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c0fff80 0x7fcf3c102360 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.562+0000 7fcf40801700 1 --2- 172.21.15.81:0/3482804001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c066260 0x7fcf3c0ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.562+0000 7fcf40801700 1 -- 172.21.15.81:0/3482804001 >> 172.21.15.81:0/3482804001 conn(0x7fcf3c0f7430 msgr2=0x7fcf3c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:13.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.562+0000 7fcf40801700 1 -- 172.21.15.81:0/3482804001 shutdown_connections 2024-03-22T07:08:13.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.562+0000 7fcf40801700 1 -- 172.21.15.81:0/3482804001 wait complete. 2024-03-22T07:08:13.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.563+0000 7fcf40801700 1 Processor -- start 2024-03-22T07:08:13.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.563+0000 7fcf40801700 1 -- start start 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.563+0000 7fcf40801700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c066260 0x7fcf3c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.563+0000 7fcf40801700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c0fff80 0x7fcf3c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.563+0000 7fcf40801700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c10e570 con 0x7fcf3c066260 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.563+0000 7fcf40801700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c10edc0 con 0x7fcf3c0fff80 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.563+0000 7fcf3a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c066260 0x7fcf3c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf3a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c066260 0x7fcf3c10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51498/0 (socket says 172.21.15.81:51498) 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf3a59c700 1 -- 172.21.15.81:0/1433688008 learned_addr learned my addr 172.21.15.81:0/1433688008 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf39d9b700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c0fff80 0x7fcf3c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf3a59c700 1 -- 172.21.15.81:0/1433688008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c0fff80 msgr2=0x7fcf3c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.565 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf3a59c700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c0fff80 0x7fcf3c10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf3a59c700 1 -- 172.21.15.81:0/1433688008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30005040 con 0x7fcf3c066260 2024-03-22T07:08:13.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf39d9b700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c0fff80 0x7fcf3c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:08:13.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf3a59c700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c066260 0x7fcf3c10da30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c8b0 tx=0x7fcf2c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:13.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf2affd700 1 -- 172.21.15.81:0/1433688008 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf2c00a760 con 0x7fcf3c066260 2024-03-22T07:08:13.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf2affd700 1 -- 172.21.15.81:0/1433688008 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf2c00ad90 con 0x7fcf3c066260 2024-03-22T07:08:13.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c10f0a0 con 0x7fcf3c066260 2024-03-22T07:08:13.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf2affd700 1 -- 172.21.15.81:0/1433688008 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf2c0113a0 con 0x7fcf3c066260 2024-03-22T07:08:13.566 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.564+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c10f5e0 con 0x7fcf3c066260 2024-03-22T07:08:13.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.565+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c05f5f0 con 0x7fcf3c066260 2024-03-22T07:08:13.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.566+0000 7fcf2affd700 1 -- 172.21.15.81:0/1433688008 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcf2c012400 con 0x7fcf3c066260 2024-03-22T07:08:13.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.567+0000 7fcf2affd700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcf240692b0 0x7fcf2406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:13.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.567+0000 7fcf2affd700 1 -- 172.21.15.81:0/1433688008 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcf2c087270 con 0x7fcf3c066260 2024-03-22T07:08:13.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.567+0000 7fcf39d9b700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcf240692b0 0x7fcf2406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:13.567 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.567+0000 7fcf39d9b700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcf240692b0 0x7fcf2406b760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000b3b0 tx=0x7fcf30009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:13.571 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.571+0000 7fcf2affd700 1 -- 172.21.15.81:0/1433688008 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcf2c056fe0 con 0x7fcf3c066260 2024-03-22T07:08:13.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:13 smithi081 ceph-mon[29245]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:13.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:13 smithi149 ceph-mon[36068]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:13.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.896+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcf3c10f9c0 con 0x7fcf3c066260 2024-03-22T07:08:13.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.896+0000 7fcf2affd700 1 -- 172.21.15.81:0/1433688008 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fcf2c056b70 con 0x7fcf3c066260 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-22T07:08:13.897 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:08:13.898 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:08:13.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.899+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcf240692b0 msgr2=0x7fcf2406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.899+0000 7fcf40801700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcf240692b0 0x7fcf2406b760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000b3b0 tx=0x7fcf30009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:13.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.899+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c066260 msgr2=0x7fcf3c10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:13.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.899+0000 7fcf40801700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c066260 0x7fcf3c10da30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c8b0 tx=0x7fcf2c004470 comp rx=0 tx=0).stop 2024-03-22T07:08:13.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.900+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 shutdown_connections 2024-03-22T07:08:13.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.900+0000 7fcf40801700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7fcf240692b0 0x7fcf2406b760 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.900+0000 7fcf40801700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf3c066260 0x7fcf3c10da30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.900+0000 7fcf40801700 1 --2- 172.21.15.81:0/1433688008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c0fff80 0x7fcf3c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:13.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.900+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 >> 172.21.15.81:0/1433688008 conn(0x7fcf3c0f7430 msgr2=0x7fcf3c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:13.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.900+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 shutdown_connections 2024-03-22T07:08:13.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:13.900+0000 7fcf40801700 1 -- 172.21.15.81:0/1433688008 wait complete. 2024-03-22T07:08:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.032+0000 7f47e6412700 1 -- 172.21.15.81:0/3762032743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fcf80 msgr2=0x7f47e00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.032+0000 7f47e6412700 1 --2- 172.21.15.81:0/3762032743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fcf80 0x7f47e00fd3f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47d4004940 tx=0x7f47d4009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.032+0000 7f47e6412700 1 -- 172.21.15.81:0/3762032743 shutdown_connections 2024-03-22T07:08:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.032+0000 7f47e6412700 1 --2- 172.21.15.81:0/3762032743 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fcf80 0x7f47e00fd3f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.032+0000 7f47e6412700 1 --2- 172.21.15.81:0/3762032743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47e00fbcd0 0x7f47e00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.032+0000 7f47e6412700 1 -- 172.21.15.81:0/3762032743 >> 172.21.15.81:0/3762032743 conn(0x7f47e00f7430 msgr2=0x7f47e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.032+0000 7f47e6412700 1 -- 172.21.15.81:0/3762032743 shutdown_connections 2024-03-22T07:08:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.032+0000 7f47e6412700 1 -- 172.21.15.81:0/3762032743 wait complete. 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.033+0000 7f47e6412700 1 Processor -- start 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.033+0000 7f47e6412700 1 -- start start 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.033+0000 7f47e6412700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fbcd0 0x7f47e01106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.033+0000 7f47e6412700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47e00fcf80 0x7f47e0112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.033+0000 7f47e6412700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e00663b0 con 0x7f47e00fbcd0 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.033+0000 7f47e6412700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e00664f0 con 0x7f47e00fcf80 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.033+0000 7f47dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fbcd0 0x7f47e01106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fbcd0 0x7f47e01106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51510/0 (socket says 172.21.15.81:51510) 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47dffff700 1 -- 172.21.15.81:0/3995089129 learned_addr learned my addr 172.21.15.81:0/3995089129 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47df7fe700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47e00fcf80 0x7f47e0112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:14.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47dffff700 1 -- 172.21.15.81:0/3995089129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47e00fcf80 msgr2=0x7f47e0112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:14.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47dffff700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47e00fcf80 0x7f47e0112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47dffff700 1 -- 172.21.15.81:0/3995089129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4005040 con 0x7f47e00fbcd0 2024-03-22T07:08:14.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47dffff700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fbcd0 0x7f47e01106c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f47d0002a50 tx=0x7f47d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:14.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47dcff9700 1 -- 172.21.15.81:0/3995089129 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d0007cc0 con 0x7f47e00fbcd0 2024-03-22T07:08:14.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e0113210 con 0x7f47e00fbcd0 2024-03-22T07:08:14.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47dcff9700 1 -- 172.21.15.81:0/3995089129 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47d000f040 con 0x7f47e00fbcd0 2024-03-22T07:08:14.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47dcff9700 1 -- 172.21.15.81:0/3995089129 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d000da30 con 0x7f47e00fbcd0 2024-03-22T07:08:14.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.034+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e01137d0 con 0x7f47e00fbcd0 2024-03-22T07:08:14.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.036+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e005f5f0 con 0x7f47e00fbcd0 2024-03-22T07:08:14.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.036+0000 7f47dcff9700 1 -- 172.21.15.81:0/3995089129 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f47d000db90 con 0x7f47e00fbcd0 2024-03-22T07:08:14.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.036+0000 7f47dcff9700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f47c80692a0 0x7f47c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:14.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.037+0000 7f47dcff9700 1 -- 172.21.15.81:0/3995089129 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f47d0086eb0 con 0x7f47e00fbcd0 2024-03-22T07:08:14.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.037+0000 7f47df7fe700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f47c80692a0 0x7f47c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:14.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.037+0000 7f47df7fe700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f47c80692a0 0x7f47c806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b970 tx=0x7f47d4009e30 comp rx=0 tx=0).ready entity=mgr.14212 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:14.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.041+0000 7f47dcff9700 1 -- 172.21.15.81:0/3995089129 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f47d0053190 con 0x7f47e00fbcd0 2024-03-22T07:08:14.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.360+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 --> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47e0101360 con 0x7f47c80692a0 2024-03-22T07:08:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.361+0000 7f47dcff9700 1 -- 172.21.15.81:0/3995089129 <== mgr.14212 v2:172.21.15.81:6800/742540357 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f47e0101360 con 0x7f47c80692a0 2024-03-22T07:08:14.362 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:08:14.362 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:08:14.362 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:08:14.362 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:08:14.362 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [], 2024-03-22T07:08:14.362 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "0/2 daemons upgraded", 2024-03-22T07:08:14.362 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 image on host smithi149", 2024-03-22T07:08:14.363 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:08:14.363 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.364+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f47c80692a0 msgr2=0x7f47c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.364+0000 7f47e6412700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f47c80692a0 0x7f47c806b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f47d400b970 tx=0x7f47d4009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.364+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fbcd0 msgr2=0x7f47e01106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.364+0000 7f47e6412700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fbcd0 0x7f47e01106c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f47d0002a50 tx=0x7f47d00046b0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.364+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 shutdown_connections 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.364+0000 7f47e6412700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:6800/742540357,v1:172.21.15.81:6801/742540357] conn(0x7f47c80692a0 0x7f47c806b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.365+0000 7f47e6412700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e00fbcd0 0x7f47e01106c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.365+0000 7f47e6412700 1 --2- 172.21.15.81:0/3995089129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47e00fcf80 0x7f47e0112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.365+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 >> 172.21.15.81:0/3995089129 conn(0x7f47e00f7430 msgr2=0x7f47e00f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:14.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.365+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 shutdown_connections 2024-03-22T07:08:14.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:14.365+0000 7f47e6412700 1 -- 172.21.15.81:0/3995089129 wait complete. 2024-03-22T07:08:14.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:14 smithi081 ceph-mon[29245]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:14 smithi081 ceph-mon[29245]: pgmap v230: 65 pgs: 65 active+clean; 34 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.9 MiB/s wr, 239 op/s 2024-03-22T07:08:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:14 smithi081 ceph-mon[29245]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:14 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1433688008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:08:14.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:14 smithi149 ceph-mon[36068]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:14.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:14 smithi149 ceph-mon[36068]: pgmap v230: 65 pgs: 65 active+clean; 34 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.9 MiB/s wr, 239 op/s 2024-03-22T07:08:14.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:14 smithi149 ceph-mon[36068]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:14.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:14 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1433688008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:08:15.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:15 smithi081 ceph-mon[29245]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:15.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:15 smithi149 ceph-mon[36068]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:16.753 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:16 smithi149 ceph-mon[36068]: pgmap v231: 65 pgs: 65 active+clean; 51 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s wr, 410 op/s 2024-03-22T07:08:16.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:16 smithi081 ceph-mon[29245]: pgmap v231: 65 pgs: 65 active+clean; 51 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s wr, 410 op/s 2024-03-22T07:08:18.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:18 smithi081 ceph-mon[29245]: pgmap v232: 65 pgs: 65 active+clean; 58 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.0 MiB/s wr, 462 op/s 2024-03-22T07:08:18.783 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:18 smithi149 ceph-mon[36068]: pgmap v232: 65 pgs: 65 active+clean; 58 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.0 MiB/s wr, 462 op/s 2024-03-22T07:08:19.450 INFO:tasks.workunit.client.1.smithi149.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-22T07:08:19.595 INFO:tasks.workunit.client.1.smithi149.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-22T07:08:19.637 INFO:tasks.workunit.client.1.smithi149.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-22T07:08:19.638 INFO:tasks.workunit.client.1.smithi149.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-22T07:08:19.638 INFO:tasks.workunit.client.1.smithi149.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-22T07:08:19.680 INFO:tasks.workunit.client.1.smithi149.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-22T07:08:19.684 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:08:19.684 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-22T07:08:19.701 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-22T07:08:19.702 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-22T07:08:19.702 DEBUG:teuthology.orchestra.run.smithi149:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=768f6cd6e95d0c7a626042719c230df59bdba9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-22T07:08:19.724 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:19 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:08:19.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:19 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:08:20.160 INFO:tasks.workunit.client.0.smithi081.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-22T07:08:20.160 INFO:tasks.workunit.client.0.smithi081.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T07:08:20.160 INFO:tasks.workunit.client.0.smithi081.stderr:+ make 2024-03-22T07:08:20.191 INFO:tasks.workunit.client.1.smithi149.stderr:+ mkdir -p fsstress 2024-03-22T07:08:20.192 INFO:tasks.workunit.client.1.smithi149.stderr:+ pushd fsstress 2024-03-22T07:08:20.193 INFO:tasks.workunit.client.1.smithi149.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T07:08:20.193 INFO:tasks.workunit.client.1.smithi149.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-22T07:08:20.251 INFO:tasks.workunit.client.0.smithi081.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-22T07:08:20.668 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:20 smithi149 ceph-mon[36068]: pgmap v233: 65 pgs: 65 active+clean; 66 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.7 MiB/s wr, 470 op/s 2024-03-22T07:08:20.669 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:20 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:20.669 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:20 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:08:20.669 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:20 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:08:20.669 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:20 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:08:20.761 INFO:tasks.workunit.client.1.smithi149.stderr:+ tar xzf ltp-full.tgz 2024-03-22T07:08:20.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:20 smithi081 ceph-mon[29245]: pgmap v233: 65 pgs: 65 active+clean; 66 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 5.7 MiB/s wr, 470 op/s 2024-03-22T07:08:20.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:20 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:20.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:20 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:08:20.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:20 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:08:20.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:20 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:08:21.514 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:21 smithi149 ceph-mon[36068]: Upgrade: Updating mgr.smithi149.sgmtoa 2024-03-22T07:08:21.515 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:21 smithi149 ceph-mon[36068]: Deploying daemon mgr.smithi149.sgmtoa on smithi149 2024-03-22T07:08:21.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:21 smithi081 ceph-mon[29245]: Upgrade: Updating mgr.smithi149.sgmtoa 2024-03-22T07:08:21.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:21 smithi081 ceph-mon[29245]: Deploying daemon mgr.smithi149.sgmtoa on smithi149 2024-03-22T07:08:21.958 INFO:tasks.workunit.client.0.smithi081.stderr:++ readlink -f fsstress 2024-03-22T07:08:21.959 INFO:tasks.workunit.client.0.smithi081.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-22T07:08:21.959 INFO:tasks.workunit.client.0.smithi081.stderr:+ popd 2024-03-22T07:08:21.959 INFO:tasks.workunit.client.0.smithi081.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T07:08:21.960 INFO:tasks.workunit.client.0.smithi081.stderr:+ popd 2024-03-22T07:08:21.960 INFO:tasks.workunit.client.0.smithi081.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-22T07:08:21.960 INFO:tasks.workunit.client.0.smithi081.stderr:++ mktemp -d -p . 2024-03-22T07:08:21.961 INFO:tasks.workunit.client.0.smithi081.stderr:+ T=./tmp.TCwELreJ9l 2024-03-22T07:08:21.961 INFO:tasks.workunit.client.0.smithi081.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.TCwELreJ9l -l 1 -n 1000 -p 10 -v 2024-03-22T07:08:21.963 INFO:tasks.workunit.client.0.smithi081.stdout:seed = 1710463311 2024-03-22T07:08:21.966 INFO:tasks.workunit.client.0.smithi081.stdout:0/0: fdatasync - no filename 2024-03-22T07:08:21.966 INFO:tasks.workunit.client.0.smithi081.stdout:0/1: readlink - no filename 2024-03-22T07:08:21.966 INFO:tasks.workunit.client.0.smithi081.stdout:0/2: dwrite - no filename 2024-03-22T07:08:21.966 INFO:tasks.workunit.client.0.smithi081.stdout:0/3: chown . 47 1 2024-03-22T07:08:21.966 INFO:tasks.workunit.client.0.smithi081.stdout:0/4: unlink - no file 2024-03-22T07:08:21.967 INFO:tasks.workunit.client.0.smithi081.stdout:0/5: dwrite - no filename 2024-03-22T07:08:21.967 INFO:tasks.workunit.client.0.smithi081.stdout:0/6: truncate - no filename 2024-03-22T07:08:21.967 INFO:tasks.workunit.client.0.smithi081.stdout:0/7: truncate - no filename 2024-03-22T07:08:21.967 INFO:tasks.workunit.client.0.smithi081.stdout:0/8: dread - no filename 2024-03-22T07:08:21.967 INFO:tasks.workunit.client.0.smithi081.stdout:0/9: link - no file 2024-03-22T07:08:21.967 INFO:tasks.workunit.client.0.smithi081.stdout:1/0: unlink - no file 2024-03-22T07:08:21.967 INFO:tasks.workunit.client.0.smithi081.stdout:1/1: dwrite - no filename 2024-03-22T07:08:21.968 INFO:tasks.workunit.client.0.smithi081.stdout:0/10: creat f0 x:0 0 0 2024-03-22T07:08:21.969 INFO:tasks.workunit.client.0.smithi081.stdout:2/0: read - no filename 2024-03-22T07:08:21.971 INFO:tasks.workunit.client.0.smithi081.stdout:1/2: creat f0 x:0 0 0 2024-03-22T07:08:21.971 INFO:tasks.workunit.client.0.smithi081.stdout:1/3: dread - f0 zero size 2024-03-22T07:08:21.971 INFO:tasks.workunit.client.0.smithi081.stdout:1/4: creat f1 x:0 0 0 2024-03-22T07:08:21.971 INFO:tasks.workunit.client.0.smithi081.stdout:1/5: creat f2 x:0 0 0 2024-03-22T07:08:21.971 INFO:tasks.workunit.client.0.smithi081.stdout:3/0: truncate - no filename 2024-03-22T07:08:21.972 INFO:tasks.workunit.client.0.smithi081.stdout:3/1: dwrite - no filename 2024-03-22T07:08:21.972 INFO:tasks.workunit.client.0.smithi081.stdout:3/2: read - no filename 2024-03-22T07:08:21.972 INFO:tasks.workunit.client.0.smithi081.stdout:3/3: chown . 1057201899 1 2024-03-22T07:08:21.972 INFO:tasks.workunit.client.0.smithi081.stdout:3/4: dread - no filename 2024-03-22T07:08:21.972 INFO:tasks.workunit.client.0.smithi081.stdout:3/5: link - no file 2024-03-22T07:08:21.972 INFO:tasks.workunit.client.0.smithi081.stdout:3/6: dwrite - no filename 2024-03-22T07:08:21.972 INFO:tasks.workunit.client.0.smithi081.stdout:3/7: unlink - no file 2024-03-22T07:08:21.972 INFO:tasks.workunit.client.0.smithi081.stdout:2/1: creat f0 x:0 0 0 2024-03-22T07:08:21.975 INFO:tasks.workunit.client.0.smithi081.stdout:1/6: link f0 f3 0 2024-03-22T07:08:21.975 INFO:tasks.workunit.client.0.smithi081.stdout:1/7: chown f0 65198949 1 2024-03-22T07:08:21.976 INFO:tasks.workunit.client.0.smithi081.stdout:4/0: rename - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/1: write - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/2: fsync - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/3: write - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/4: rename - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/5: dwrite - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/6: chown . 118 1 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/7: write - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/8: readlink - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/9: dwrite - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/10: read - no filename 2024-03-22T07:08:21.977 INFO:tasks.workunit.client.0.smithi081.stdout:4/11: unlink - no file 2024-03-22T07:08:21.978 INFO:tasks.workunit.client.0.smithi081.stdout:3/8: symlink l0 0 2024-03-22T07:08:21.978 INFO:tasks.workunit.client.0.smithi081.stdout:2/2: symlink l1 0 2024-03-22T07:08:21.979 INFO:tasks.workunit.client.0.smithi081.stdout:1/8: unlink f3 0 2024-03-22T07:08:21.980 INFO:tasks.workunit.client.0.smithi081.stdout:5/0: rename - no filename 2024-03-22T07:08:21.980 INFO:tasks.workunit.client.0.smithi081.stdout:5/1: truncate - no filename 2024-03-22T07:08:21.980 INFO:tasks.workunit.client.0.smithi081.stdout:5/2: stat - no entries 2024-03-22T07:08:21.980 INFO:tasks.workunit.client.0.smithi081.stdout:5/3: dwrite - no filename 2024-03-22T07:08:21.981 INFO:tasks.workunit.client.0.smithi081.stdout:5/4: unlink - no file 2024-03-22T07:08:21.981 INFO:tasks.workunit.client.0.smithi081.stdout:4/12: creat f0 x:0 0 0 2024-03-22T07:08:21.982 INFO:tasks.workunit.client.0.smithi081.stdout:3/9: creat f1 x:0 0 0 2024-03-22T07:08:21.984 INFO:tasks.workunit.client.0.smithi081.stdout:2/3: rename f0 to f2 0 2024-03-22T07:08:21.984 INFO:tasks.workunit.client.0.smithi081.stdout:2/4: creat f3 x:0 0 0 2024-03-22T07:08:21.984 INFO:tasks.workunit.client.0.smithi081.stdout:2/5: stat f2 0 2024-03-22T07:08:21.984 INFO:tasks.workunit.client.0.smithi081.stdout:2/6: readlink l1 0 2024-03-22T07:08:21.985 INFO:tasks.workunit.client.0.smithi081.stdout:1/9: rename f2 to f4 0 2024-03-22T07:08:21.986 INFO:tasks.workunit.client.0.smithi081.stdout:6/0: readlink - no filename 2024-03-22T07:08:21.986 INFO:tasks.workunit.client.0.smithi081.stdout:6/1: dwrite - no filename 2024-03-22T07:08:21.987 INFO:tasks.workunit.client.0.smithi081.stdout:5/5: creat f0 x:0 0 0 2024-03-22T07:08:21.988 INFO:tasks.workunit.client.0.smithi081.stdout:5/6: creat f1 x:0 0 0 2024-03-22T07:08:21.988 INFO:tasks.workunit.client.0.smithi081.stdout:5/7: readlink - no filename 2024-03-22T07:08:21.988 INFO:tasks.workunit.client.0.smithi081.stdout:5/8: dread - f0 zero size 2024-03-22T07:08:21.988 INFO:tasks.workunit.client.0.smithi081.stdout:5/9: dread - f0 zero size 2024-03-22T07:08:21.989 INFO:tasks.workunit.client.0.smithi081.stdout:3/10: mknod c2 0 2024-03-22T07:08:21.993 INFO:tasks.workunit.client.0.smithi081.stdout:1/10: unlink f0 0 2024-03-22T07:08:21.994 INFO:tasks.workunit.client.0.smithi081.stdout:7/0: dwrite - no filename 2024-03-22T07:08:21.994 INFO:tasks.workunit.client.0.smithi081.stdout:7/1: fsync - no filename 2024-03-22T07:08:21.996 INFO:tasks.workunit.client.0.smithi081.stdout:1/11: symlink l5 0 2024-03-22T07:08:21.996 INFO:tasks.workunit.client.0.smithi081.stdout:1/12: read - f4 zero size 2024-03-22T07:08:22.001 INFO:tasks.workunit.client.0.smithi081.stdout:1/13: mkdir d6 0 2024-03-22T07:08:22.001 INFO:tasks.workunit.client.0.smithi081.stdout:7/2: creat f0 x:0 0 0 2024-03-22T07:08:22.002 INFO:tasks.workunit.client.0.smithi081.stdout:7/3: stat f0 0 2024-03-22T07:08:22.002 INFO:tasks.workunit.client.0.smithi081.stdout:7/4: creat f1 x:0 0 0 2024-03-22T07:08:22.002 INFO:tasks.workunit.client.0.smithi081.stdout:7/5: creat f2 x:0 0 0 2024-03-22T07:08:22.007 INFO:tasks.workunit.client.0.smithi081.stdout:0/11: dwrite f0 [0,4194304] 0 2024-03-22T07:08:22.007 INFO:tasks.workunit.client.0.smithi081.stdout:0/12: chown f0 7925 1 2024-03-22T07:08:22.008 INFO:tasks.workunit.client.0.smithi081.stdout:0/13: creat f1 x:0 0 0 2024-03-22T07:08:22.008 INFO:tasks.workunit.client.0.smithi081.stdout:0/14: rmdir - no directory 2024-03-22T07:08:22.008 INFO:tasks.workunit.client.0.smithi081.stdout:0/15: creat f2 x:0 0 0 2024-03-22T07:08:22.009 INFO:tasks.workunit.client.0.smithi081.stdout:1/14: link l5 d6/l7 0 2024-03-22T07:08:22.016 INFO:tasks.workunit.client.0.smithi081.stdout:8/0: creat f0 x:0 0 0 2024-03-22T07:08:22.017 INFO:tasks.workunit.client.0.smithi081.stdout:8/1: truncate f0 91396 0 2024-03-22T07:08:22.017 INFO:tasks.workunit.client.0.smithi081.stdout:8/2: creat f1 x:0 0 0 2024-03-22T07:08:22.017 INFO:tasks.workunit.client.0.smithi081.stdout:8/3: dread - f1 zero size 2024-03-22T07:08:22.018 INFO:tasks.workunit.client.0.smithi081.stdout:9/0: dread - no filename 2024-03-22T07:08:22.018 INFO:tasks.workunit.client.0.smithi081.stdout:9/1: write - no filename 2024-03-22T07:08:22.018 INFO:tasks.workunit.client.0.smithi081.stdout:9/2: fsync - no filename 2024-03-22T07:08:22.018 INFO:tasks.workunit.client.0.smithi081.stdout:9/3: dwrite - no filename 2024-03-22T07:08:22.018 INFO:tasks.workunit.client.0.smithi081.stdout:9/4: fsync - no filename 2024-03-22T07:08:22.028 INFO:tasks.workunit.client.0.smithi081.stdout:1/15: mknod d6/c8 0 2024-03-22T07:08:22.154 INFO:tasks.workunit.client.0.smithi081.stdout:5/10: dwrite f0 [0,4194304] 0 2024-03-22T07:08:22.154 INFO:tasks.workunit.client.0.smithi081.stdout:5/11: creat f2 x:0 0 0 2024-03-22T07:08:22.295 INFO:tasks.workunit.client.0.smithi081.stdout:0/16: dwrite f2 [0,4194304] 0 2024-03-22T07:08:22.326 INFO:tasks.workunit.client.0.smithi081.stdout:4/13: fsync f0 0 2024-03-22T07:08:22.330 INFO:tasks.workunit.client.0.smithi081.stdout:2/7: unlink f2 0 2024-03-22T07:08:22.330 INFO:tasks.workunit.client.0.smithi081.stdout:2/8: truncate f3 647899 0 2024-03-22T07:08:22.330 INFO:tasks.workunit.client.0.smithi081.stdout:2/9: write f3 [993874,58357] 0 2024-03-22T07:08:22.330 INFO:tasks.workunit.client.0.smithi081.stdout:2/10: creat f4 x:0 0 0 2024-03-22T07:08:22.330 INFO:tasks.workunit.client.0.smithi081.stdout:2/11: creat f5 x:0 0 0 2024-03-22T07:08:22.333 INFO:tasks.workunit.client.0.smithi081.stdout:1/16: rename f4 to d6/f9 0 2024-03-22T07:08:22.430 INFO:tasks.workunit.client.0.smithi081.stdout:3/11: dwrite f1 [0,4194304] 0 2024-03-22T07:08:22.495 INFO:tasks.workunit.client.0.smithi081.stdout:5/12: write f0 [4927151,65682] 0 2024-03-22T07:08:22.495 INFO:tasks.workunit.client.0.smithi081.stdout:5/13: fsync f1 0 2024-03-22T07:08:22.558 INFO:tasks.workunit.client.0.smithi081.stdout:0/17: mkdir d3 0 2024-03-22T07:08:22.617 INFO:tasks.workunit.client.0.smithi081.stdout:8/4: dwrite f0 [0,4194304] 0 2024-03-22T07:08:22.659 INFO:tasks.workunit.client.0.smithi081.stdout:7/6: dwrite f2 [0,4194304] 0 2024-03-22T07:08:22.816 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:22 smithi149 ceph-mon[36068]: pgmap v234: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 5.5 MiB/s wr, 470 op/s 2024-03-22T07:08:22.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:22 smithi081 ceph-mon[29245]: pgmap v234: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 5.5 MiB/s wr, 470 op/s 2024-03-22T07:08:22.967 INFO:tasks.workunit.client.0.smithi081.stdout:4/14: dwrite f0 [0,4194304] 0 2024-03-22T07:08:22.975 INFO:tasks.workunit.client.0.smithi081.stdout:4/15: write f0 [2885424,36183] 0 2024-03-22T07:08:22.975 INFO:tasks.workunit.client.0.smithi081.stdout:4/16: write f0 [3440952,104288] 0 2024-03-22T07:08:22.975 INFO:tasks.workunit.client.0.smithi081.stdout:4/17: creat f1 x:0 0 0 2024-03-22T07:08:23.130 INFO:tasks.workunit.client.0.smithi081.stdout:3/12: dwrite f1 [0,4194304] 0 2024-03-22T07:08:23.130 INFO:tasks.workunit.client.0.smithi081.stdout:3/13: readlink l0 0 2024-03-22T07:08:23.538 INFO:tasks.workunit.client.0.smithi081.stdout:5/14: unlink f0 0 2024-03-22T07:08:23.538 INFO:tasks.workunit.client.0.smithi081.stdout:5/15: creat f3 x:0 0 0 2024-03-22T07:08:24.378 INFO:tasks.workunit.client.0.smithi081.stdout:8/5: mkdir d2 0 2024-03-22T07:08:24.600 INFO:tasks.workunit.client.0.smithi081.stdout:7/7: mknod c3 0 2024-03-22T07:08:24.600 INFO:tasks.workunit.client.0.smithi081.stdout:7/8: fsync f0 0 2024-03-22T07:08:24.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:24 smithi081 ceph-mon[29245]: pgmap v235: 65 pgs: 65 active+clean; 89 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.8 MiB/s wr, 516 op/s 2024-03-22T07:08:24.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:24 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:08:24.851 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:24 smithi149 ceph-mon[36068]: pgmap v235: 65 pgs: 65 active+clean; 89 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 6.8 MiB/s wr, 516 op/s 2024-03-22T07:08:24.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:24 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:08:26.724 INFO:tasks.workunit.client.0.smithi081.stdout:4/18: symlink l2 0 2024-03-22T07:08:26.746 INFO:tasks.workunit.client.0.smithi081.stdout:4/19: dwrite f1 [0,4194304] 0 2024-03-22T07:08:26.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:26 smithi081 ceph-mon[29245]: pgmap v236: 65 pgs: 65 active+clean; 114 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 6.9 MiB/s wr, 486 op/s 2024-03-22T07:08:26.828 INFO:tasks.workunit.client.0.smithi081.stdout:4/20: dread f1 [0,4194304] 0 2024-03-22T07:08:26.846 INFO:tasks.workunit.client.0.smithi081.stdout:4/21: dwrite f0 [0,4194304] 0 2024-03-22T07:08:26.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:26 smithi149 ceph-mon[36068]: pgmap v236: 65 pgs: 65 active+clean; 114 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 6.9 MiB/s wr, 486 op/s 2024-03-22T07:08:26.855 INFO:tasks.workunit.client.0.smithi081.stdout:4/22: dread f0 [0,4194304] 0 2024-03-22T07:08:27.640 INFO:tasks.workunit.client.0.smithi081.stdout:3/14: mknod c3 0 2024-03-22T07:08:27.640 INFO:tasks.workunit.client.0.smithi081.stdout:3/15: creat f4 x:0 0 0 2024-03-22T07:08:27.758 INFO:tasks.workunit.client.0.smithi081.stdout:5/16: mkdir d4 0 2024-03-22T07:08:27.793 INFO:tasks.workunit.client.0.smithi081.stdout:0/18: truncate f2 3660466 0 2024-03-22T07:08:27.794 INFO:tasks.workunit.client.0.smithi081.stdout:0/19: fsync f1 0 2024-03-22T07:08:27.794 INFO:tasks.workunit.client.0.smithi081.stdout:0/20: fdatasync f1 0 2024-03-22T07:08:27.795 INFO:tasks.workunit.client.0.smithi081.stdout:8/6: mkdir d2/d3 0 2024-03-22T07:08:27.798 INFO:tasks.workunit.client.0.smithi081.stdout:7/9: rename f0 to f4 0 2024-03-22T07:08:27.798 INFO:tasks.workunit.client.0.smithi081.stdout:7/10: fsync f1 0 2024-03-22T07:08:27.804 INFO:tasks.workunit.client.0.smithi081.stdout:7/11: read f2 [2291649,101030] 0 2024-03-22T07:08:27.821 INFO:tasks.workunit.client.0.smithi081.stdout:0/21: dwrite f1 [0,4194304] 0 2024-03-22T07:08:27.830 INFO:tasks.workunit.client.0.smithi081.stdout:7/12: dwrite f4 [0,4194304] 0 2024-03-22T07:08:27.837 INFO:tasks.workunit.client.0.smithi081.stdout:0/22: write f1 [3549909,124758] 0 2024-03-22T07:08:27.837 INFO:tasks.workunit.client.0.smithi081.stdout:0/23: chown f0 67065 1 2024-03-22T07:08:27.839 INFO:tasks.workunit.client.0.smithi081.stdout:7/13: write f4 [3486947,1758] 0 2024-03-22T07:08:27.839 INFO:tasks.workunit.client.0.smithi081.stdout:7/14: chown c3 23 1 2024-03-22T07:08:27.861 INFO:tasks.workunit.client.0.smithi081.stdout:3/16: symlink l5 0 2024-03-22T07:08:27.863 INFO:tasks.workunit.client.0.smithi081.stdout:5/17: mknod d4/c5 0 2024-03-22T07:08:27.867 INFO:tasks.workunit.client.0.smithi081.stdout:8/7: creat d2/d3/f4 x:0 0 0 2024-03-22T07:08:27.869 INFO:tasks.workunit.client.0.smithi081.stdout:4/23: link f1 f3 0 2024-03-22T07:08:27.869 INFO:tasks.workunit.client.0.smithi081.stdout:4/24: stat f1 0 2024-03-22T07:08:27.869 INFO:tasks.workunit.client.0.smithi081.stdout:4/25: chown f0 3996 1 2024-03-22T07:08:27.878 INFO:tasks.workunit.client.0.smithi081.stdout:0/24: rename f0 to d3/f4 0 2024-03-22T07:08:27.880 INFO:tasks.workunit.client.0.smithi081.stdout:7/15: symlink l5 0 2024-03-22T07:08:27.880 INFO:tasks.workunit.client.0.smithi081.stdout:7/16: chown f1 12086 1 2024-03-22T07:08:27.882 INFO:tasks.workunit.client.0.smithi081.stdout:8/8: mknod d2/c5 0 2024-03-22T07:08:27.887 INFO:tasks.workunit.client.0.smithi081.stdout:0/25: mknod d3/c5 0 2024-03-22T07:08:27.888 INFO:tasks.workunit.client.0.smithi081.stdout:7/17: link f1 f6 0 2024-03-22T07:08:27.888 INFO:tasks.workunit.client.0.smithi081.stdout:7/18: read f2 [427622,25848] 0 2024-03-22T07:08:27.890 INFO:tasks.workunit.client.0.smithi081.stdout:0/26: rmdir d3 39 2024-03-22T07:08:27.891 INFO:tasks.workunit.client.0.smithi081.stdout:7/19: dread f2 [0,4194304] 0 2024-03-22T07:08:27.897 INFO:tasks.workunit.client.0.smithi081.stdout:7/20: link f2 f7 0 2024-03-22T07:08:27.897 INFO:tasks.workunit.client.0.smithi081.stdout:7/21: chown f7 15327 1 2024-03-22T07:08:27.901 INFO:tasks.workunit.client.0.smithi081.stdout:3/17: dwrite f1 [0,4194304] 0 2024-03-22T07:08:27.901 INFO:tasks.workunit.client.0.smithi081.stdout:3/18: stat l5 0 2024-03-22T07:08:27.901 INFO:tasks.workunit.client.0.smithi081.stdout:3/19: creat f6 x:0 0 0 2024-03-22T07:08:27.901 INFO:tasks.workunit.client.0.smithi081.stdout:3/20: dread - f6 zero size 2024-03-22T07:08:27.901 INFO:tasks.workunit.client.0.smithi081.stdout:3/21: creat f7 x:0 0 0 2024-03-22T07:08:27.901 INFO:tasks.workunit.client.0.smithi081.stdout:3/22: dread - f4 zero size 2024-03-22T07:08:27.904 INFO:tasks.workunit.client.0.smithi081.stdout:4/26: write f1 [963003,111412] 0 2024-03-22T07:08:27.904 INFO:tasks.workunit.client.0.smithi081.stdout:4/27: chown f0 0 1 2024-03-22T07:08:27.904 INFO:tasks.workunit.client.0.smithi081.stdout:4/28: readlink l2 0 2024-03-22T07:08:27.906 INFO:tasks.workunit.client.0.smithi081.stdout:7/22: symlink l8 0 2024-03-22T07:08:27.907 INFO:tasks.workunit.client.0.smithi081.stdout:7/23: write f2 [5048027,97864] 0 2024-03-22T07:08:27.907 INFO:tasks.workunit.client.0.smithi081.stdout:7/24: read - f6 zero size 2024-03-22T07:08:27.915 INFO:tasks.workunit.client.0.smithi081.stdout:3/23: symlink l8 0 2024-03-22T07:08:27.924 INFO:tasks.workunit.client.0.smithi081.stdout:7/25: mkdir d9 0 2024-03-22T07:08:27.924 INFO:tasks.workunit.client.0.smithi081.stdout:7/26: getdents d9 0 2024-03-22T07:08:27.937 INFO:tasks.workunit.client.0.smithi081.stdout:7/27: dwrite f6 [0,4194304] 0 2024-03-22T07:08:27.937 INFO:tasks.workunit.client.0.smithi081.stdout:7/28: chown c3 15203175 1 2024-03-22T07:08:27.937 INFO:tasks.workunit.client.0.smithi081.stdout:7/29: stat f4 0 2024-03-22T07:08:27.952 INFO:tasks.workunit.client.0.smithi081.stdout:7/30: dwrite f6 [0,4194304] 0 2024-03-22T07:08:27.954 INFO:tasks.workunit.client.0.smithi081.stdout:7/31: symlink d9/la 0 2024-03-22T07:08:27.956 INFO:tasks.workunit.client.0.smithi081.stdout:7/32: symlink d9/lb 0 2024-03-22T07:08:27.957 INFO:tasks.workunit.client.0.smithi081.stdout:7/33: mkdir d9/dc 0 2024-03-22T07:08:27.959 INFO:tasks.workunit.client.0.smithi081.stdout:7/34: mknod d9/cd 0 2024-03-22T07:08:27.960 INFO:tasks.workunit.client.0.smithi081.stdout:7/35: write f7 [1216555,76344] 0 2024-03-22T07:08:27.962 INFO:tasks.workunit.client.0.smithi081.stdout:7/36: rename l5 to d9/le 0 2024-03-22T07:08:27.965 INFO:tasks.workunit.client.0.smithi081.stdout:7/37: creat d9/dc/ff x:0 0 0 2024-03-22T07:08:28.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:28 smithi081 ceph-mon[29245]: pgmap v237: 65 pgs: 65 active+clean; 149 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 341 KiB/s rd, 8.8 MiB/s wr, 510 op/s 2024-03-22T07:08:28.741 INFO:tasks.workunit.client.0.smithi081.stdout:5/18: fsync f3 0 2024-03-22T07:08:28.742 INFO:tasks.workunit.client.0.smithi081.stdout:5/19: mknod d4/c6 0 2024-03-22T07:08:28.762 INFO:tasks.workunit.client.0.smithi081.stdout:5/20: dwrite f2 [0,4194304] 0 2024-03-22T07:08:28.762 INFO:tasks.workunit.client.0.smithi081.stdout:5/21: stat d4 0 2024-03-22T07:08:28.767 INFO:tasks.workunit.client.0.smithi081.stdout:5/22: mknod d4/c7 0 2024-03-22T07:08:28.767 INFO:tasks.workunit.client.0.smithi081.stdout:5/23: rename d4 to d4/d8 22 2024-03-22T07:08:28.784 INFO:tasks.workunit.client.0.smithi081.stdout:5/24: dwrite f2 [0,4194304] 0 2024-03-22T07:08:28.788 INFO:tasks.workunit.client.0.smithi081.stdout:5/25: mkdir d4/d9 0 2024-03-22T07:08:28.839 INFO:tasks.workunit.client.0.smithi081.stdout:4/29: fsync f1 0 2024-03-22T07:08:28.841 INFO:tasks.workunit.client.0.smithi081.stdout:4/30: link l2 l4 0 2024-03-22T07:08:28.843 INFO:tasks.workunit.client.0.smithi081.stdout:4/31: symlink l5 0 2024-03-22T07:08:28.846 INFO:tasks.workunit.client.0.smithi081.stdout:0/27: fsync d3/f4 0 2024-03-22T07:08:28.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:28 smithi149 ceph-mon[36068]: pgmap v237: 65 pgs: 65 active+clean; 149 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 341 KiB/s rd, 8.8 MiB/s wr, 510 op/s 2024-03-22T07:08:28.853 INFO:tasks.workunit.client.0.smithi081.stdout:4/32: dread f3 [0,4194304] 0 2024-03-22T07:08:28.853 INFO:tasks.workunit.client.0.smithi081.stdout:4/33: fdatasync f1 0 2024-03-22T07:08:28.879 INFO:tasks.workunit.client.0.smithi081.stdout:4/34: dwrite f3 [0,4194304] 0 2024-03-22T07:08:28.879 INFO:tasks.workunit.client.0.smithi081.stdout:4/35: stat f3 0 2024-03-22T07:08:28.907 INFO:tasks.workunit.client.0.smithi081.stdout:4/36: dwrite f3 [0,4194304] 0 2024-03-22T07:08:28.907 INFO:tasks.workunit.client.0.smithi081.stdout:4/37: chown f1 2296 1 2024-03-22T07:08:28.907 INFO:tasks.workunit.client.0.smithi081.stdout:7/38: dwrite f1 [4194304,4194304] 0 2024-03-22T07:08:28.909 INFO:tasks.workunit.client.0.smithi081.stdout:4/38: link l4 l6 0 2024-03-22T07:08:28.912 INFO:tasks.workunit.client.0.smithi081.stdout:4/39: dread f0 [0,4194304] 0 2024-03-22T07:08:28.912 INFO:tasks.workunit.client.0.smithi081.stdout:4/40: creat f7 x:0 0 0 2024-03-22T07:08:28.913 INFO:tasks.workunit.client.0.smithi081.stdout:7/39: mkdir d9/d10 0 2024-03-22T07:08:28.913 INFO:tasks.workunit.client.0.smithi081.stdout:7/40: dread - d9/dc/ff zero size 2024-03-22T07:08:28.918 INFO:tasks.workunit.client.0.smithi081.stdout:7/41: link d9/dc/ff d9/f11 0 2024-03-22T07:08:28.918 INFO:tasks.workunit.client.0.smithi081.stdout:7/42: chown d9/d10 4390 1 2024-03-22T07:08:28.935 INFO:tasks.workunit.client.0.smithi081.stdout:4/41: dwrite f3 [0,4194304] 0 2024-03-22T07:08:28.935 INFO:tasks.workunit.client.0.smithi081.stdout:4/42: rmdir - no directory 2024-03-22T07:08:28.939 INFO:tasks.workunit.client.0.smithi081.stdout:4/43: dread f0 [0,4194304] 0 2024-03-22T07:08:28.939 INFO:tasks.workunit.client.0.smithi081.stdout:4/44: creat f8 x:0 0 0 2024-03-22T07:08:28.939 INFO:tasks.workunit.client.0.smithi081.stdout:4/45: rmdir - no directory 2024-03-22T07:08:28.939 INFO:tasks.workunit.client.0.smithi081.stdout:4/46: dread - f7 zero size 2024-03-22T07:08:28.939 INFO:tasks.workunit.client.0.smithi081.stdout:4/47: chown l5 12557938 1 2024-03-22T07:08:28.946 INFO:tasks.workunit.client.0.smithi081.stdout:7/43: dwrite f4 [0,4194304] 0 2024-03-22T07:08:28.954 INFO:tasks.workunit.client.0.smithi081.stdout:7/44: write f6 [1038940,2339] 0 2024-03-22T07:08:28.957 INFO:tasks.workunit.client.0.smithi081.stdout:4/48: write f3 [1279372,30767] 0 2024-03-22T07:08:28.957 INFO:tasks.workunit.client.0.smithi081.stdout:4/49: chown l4 1961420029 1 2024-03-22T07:08:28.957 INFO:tasks.workunit.client.0.smithi081.stdout:4/50: truncate f7 493127 0 2024-03-22T07:08:28.957 INFO:tasks.workunit.client.0.smithi081.stdout:4/51: write f3 [4246457,87424] 0 2024-03-22T07:08:28.957 INFO:tasks.workunit.client.0.smithi081.stdout:4/52: write f3 [2645475,5973] 0 2024-03-22T07:08:28.969 INFO:tasks.workunit.client.0.smithi081.stdout:7/45: dwrite f7 [0,4194304] 0 2024-03-22T07:08:28.976 INFO:tasks.workunit.client.0.smithi081.stdout:7/46: write f2 [1996906,126265] 0 2024-03-22T07:08:28.983 INFO:tasks.workunit.client.0.smithi081.stdout:7/47: dread f4 [0,4194304] 0 2024-03-22T07:08:29.731 INFO:tasks.workunit.client.0.smithi081.stdout:0/28: dwrite f2 [0,4194304] 0 2024-03-22T07:08:29.732 INFO:tasks.workunit.client.0.smithi081.stdout:0/29: mknod d3/c6 0 2024-03-22T07:08:29.733 INFO:tasks.workunit.client.0.smithi081.stdout:0/30: creat d3/f7 x:0 0 0 2024-03-22T07:08:29.735 INFO:tasks.workunit.client.0.smithi081.stdout:0/31: rename f1 to d3/f8 0 2024-03-22T07:08:29.737 INFO:tasks.workunit.client.0.smithi081.stdout:0/32: symlink d3/l9 0 2024-03-22T07:08:29.757 INFO:tasks.workunit.client.0.smithi081.stdout:4/53: truncate f1 598728 0 2024-03-22T07:08:29.757 INFO:tasks.workunit.client.0.smithi081.stdout:4/54: write f8 [27644,40116] 0 2024-03-22T07:08:29.758 INFO:tasks.workunit.client.0.smithi081.stdout:4/55: rename f8 to f9 0 2024-03-22T07:08:29.758 INFO:tasks.workunit.client.0.smithi081.stdout:4/56: truncate f9 974232 0 2024-03-22T07:08:29.759 INFO:tasks.workunit.client.0.smithi081.stdout:4/57: mkdir da 0 2024-03-22T07:08:29.759 INFO:tasks.workunit.client.0.smithi081.stdout:4/58: chown f0 122 1 2024-03-22T07:08:29.766 INFO:tasks.workunit.client.0.smithi081.stdout:7/48: truncate f4 3085270 0 2024-03-22T07:08:29.767 INFO:tasks.workunit.client.0.smithi081.stdout:7/49: mknod d9/c12 0 2024-03-22T07:08:29.796 INFO:tasks.workunit.client.0.smithi081.stdout:7/50: dwrite f2 [4194304,4194304] 0 2024-03-22T07:08:29.796 INFO:tasks.workunit.client.0.smithi081.stdout:7/51: chown c3 2183457 1 2024-03-22T07:08:29.803 INFO:tasks.workunit.client.0.smithi081.stdout:7/52: dread f1 [0,4194304] 0 2024-03-22T07:08:29.805 INFO:tasks.workunit.client.0.smithi081.stdout:7/53: rmdir d9/dc 39 2024-03-22T07:08:29.808 INFO:tasks.workunit.client.0.smithi081.stdout:7/54: link c3 d9/dc/c13 0 2024-03-22T07:08:29.808 INFO:tasks.workunit.client.0.smithi081.stdout:7/55: readlink d9/la 0 2024-03-22T07:08:29.808 INFO:tasks.workunit.client.0.smithi081.stdout:7/56: readlink d9/le 0 2024-03-22T07:08:29.813 INFO:tasks.workunit.client.0.smithi081.stdout:7/57: write f1 [7071367,79963] 0 2024-03-22T07:08:29.814 INFO:tasks.workunit.client.0.smithi081.stdout:7/58: mkdir d9/d14 0 2024-03-22T07:08:29.815 INFO:tasks.workunit.client.0.smithi081.stdout:7/59: mkdir d9/d15 0 2024-03-22T07:08:29.816 INFO:tasks.workunit.client.0.smithi081.stdout:7/60: mkdir d9/d10/d16 0 2024-03-22T07:08:29.817 INFO:tasks.workunit.client.0.smithi081.stdout:7/61: rename d9/dc/ff to d9/d15/f17 0 2024-03-22T07:08:29.818 INFO:tasks.workunit.client.0.smithi081.stdout:7/62: creat d9/d15/f18 x:0 0 0 2024-03-22T07:08:29.818 INFO:tasks.workunit.client.0.smithi081.stdout:7/63: chown d9/dc/c13 113656 1 2024-03-22T07:08:29.819 INFO:tasks.workunit.client.0.smithi081.stdout:7/64: mknod d9/c19 0 2024-03-22T07:08:29.820 INFO:tasks.workunit.client.0.smithi081.stdout:7/65: creat d9/dc/f1a x:0 0 0 2024-03-22T07:08:29.839 INFO:tasks.workunit.client.0.smithi081.stdout:7/66: dwrite f1 [0,4194304] 0 2024-03-22T07:08:29.840 INFO:tasks.workunit.client.0.smithi081.stdout:7/67: symlink d9/d14/l1b 0 2024-03-22T07:08:29.840 INFO:tasks.workunit.client.0.smithi081.stdout:7/68: write d9/dc/f1a [60045,113220] 0 2024-03-22T07:08:29.840 INFO:tasks.workunit.client.0.smithi081.stdout:7/69: fsync f6 0 2024-03-22T07:08:29.840 INFO:tasks.workunit.client.0.smithi081.stdout:7/70: fdatasync f4 0 2024-03-22T07:08:29.840 INFO:tasks.workunit.client.0.smithi081.stdout:7/71: read f7 [2488586,32418] 0 2024-03-22T07:08:29.841 INFO:tasks.workunit.client.0.smithi081.stdout:7/72: mknod d9/d10/c1c 0 2024-03-22T07:08:29.842 INFO:tasks.workunit.client.0.smithi081.stdout:7/73: mknod d9/d10/c1d 0 2024-03-22T07:08:29.867 INFO:tasks.workunit.client.0.smithi081.stdout:7/74: dwrite f2 [4194304,4194304] 0 2024-03-22T07:08:29.867 INFO:tasks.workunit.client.0.smithi081.stdout:7/75: write f6 [6011404,116894] 0 2024-03-22T07:08:30.030 INFO:tasks.workunit.client.0.smithi081.stdout:0/33: rmdir d3 39 2024-03-22T07:08:30.031 INFO:tasks.workunit.client.0.smithi081.stdout:4/59: link f9 da/fb 0 2024-03-22T07:08:30.033 INFO:tasks.workunit.client.0.smithi081.stdout:4/60: creat da/fc x:0 0 0 2024-03-22T07:08:30.033 INFO:tasks.workunit.client.0.smithi081.stdout:4/61: mkdir da/dd 0 2024-03-22T07:08:30.040 INFO:tasks.workunit.client.0.smithi081.stdout:7/76: rename d9/d15 to d9/d10/d16/d1e 0 2024-03-22T07:08:30.049 INFO:tasks.workunit.client.0.smithi081.stdout:4/62: getdents da 0 2024-03-22T07:08:30.049 INFO:tasks.workunit.client.0.smithi081.stdout:4/63: chown da 69 1 2024-03-22T07:08:30.050 INFO:tasks.workunit.client.0.smithi081.stdout:4/64: rename f7 to da/fe 0 2024-03-22T07:08:30.051 INFO:tasks.workunit.client.0.smithi081.stdout:4/65: truncate f0 3905099 0 2024-03-22T07:08:30.085 INFO:tasks.workunit.client.0.smithi081.stdout:0/34: dwrite d3/f8 [0,4194304] 0 2024-03-22T07:08:30.086 INFO:tasks.workunit.client.0.smithi081.stdout:0/35: symlink d3/la 0 2024-03-22T07:08:30.086 INFO:tasks.workunit.client.0.smithi081.stdout:0/36: chown d3/c5 189 1 2024-03-22T07:08:30.086 INFO:tasks.workunit.client.0.smithi081.stdout:0/37: truncate d3/f7 584550 0 2024-03-22T07:08:30.090 INFO:tasks.workunit.client.0.smithi081.stdout:4/66: dwrite f0 [0,4194304] 0 2024-03-22T07:08:30.091 INFO:tasks.workunit.client.0.smithi081.stdout:7/77: dwrite f7 [4194304,4194304] 0 2024-03-22T07:08:30.092 INFO:tasks.workunit.client.0.smithi081.stdout:4/67: stat da/fc 0 2024-03-22T07:08:30.095 INFO:tasks.workunit.client.0.smithi081.stdout:7/78: fsync f1 0 2024-03-22T07:08:30.095 INFO:tasks.workunit.client.0.smithi081.stdout:7/79: creat d9/dc/f1f x:0 0 0 2024-03-22T07:08:30.101 INFO:tasks.workunit.client.0.smithi081.stdout:7/80: symlink d9/l20 0 2024-03-22T07:08:30.105 INFO:tasks.workunit.client.0.smithi081.stdout:7/81: rename d9/d14/l1b to d9/d10/l21 0 2024-03-22T07:08:30.107 INFO:tasks.workunit.client.0.smithi081.stdout:7/82: mknod d9/d10/c22 0 2024-03-22T07:08:30.107 INFO:tasks.workunit.client.0.smithi081.stdout:7/83: fsync f1 0 2024-03-22T07:08:30.242 INFO:tasks.workunit.client.0.smithi081.stdout:0/38: getdents d3 0 2024-03-22T07:08:30.249 INFO:tasks.workunit.client.0.smithi081.stdout:0/39: dread f2 [0,4194304] 0 2024-03-22T07:08:30.253 INFO:tasks.workunit.client.0.smithi081.stdout:7/84: truncate f1 2210801 0 2024-03-22T07:08:30.256 INFO:tasks.workunit.client.0.smithi081.stdout:7/85: write f4 [2920251,21088] 0 2024-03-22T07:08:30.258 INFO:tasks.workunit.client.0.smithi081.stdout:7/86: symlink d9/d10/d16/d1e/l23 0 2024-03-22T07:08:30.259 INFO:tasks.workunit.client.0.smithi081.stdout:4/68: truncate da/fb 949034 0 2024-03-22T07:08:30.262 INFO:tasks.workunit.client.0.smithi081.stdout:7/87: creat d9/d14/f24 x:0 0 0 2024-03-22T07:08:30.262 INFO:tasks.workunit.client.0.smithi081.stdout:7/88: creat d9/d14/f25 x:0 0 0 2024-03-22T07:08:30.263 INFO:tasks.workunit.client.0.smithi081.stdout:4/69: rmdir da 39 2024-03-22T07:08:30.266 INFO:tasks.workunit.client.0.smithi081.stdout:7/89: symlink d9/d14/l26 0 2024-03-22T07:08:30.270 INFO:tasks.workunit.client.0.smithi081.stdout:7/90: unlink d9/la 0 2024-03-22T07:08:30.270 INFO:tasks.workunit.client.0.smithi081.stdout:7/91: write d9/dc/f1a [524183,107716] 0 2024-03-22T07:08:30.270 INFO:tasks.workunit.client.0.smithi081.stdout:7/92: dread - d9/f11 zero size 2024-03-22T07:08:30.270 INFO:tasks.workunit.client.0.smithi081.stdout:7/93: truncate f7 8472968 0 2024-03-22T07:08:30.271 INFO:tasks.workunit.client.0.smithi081.stdout:4/70: mkdir da/df 0 2024-03-22T07:08:30.276 INFO:tasks.workunit.client.0.smithi081.stdout:4/71: mknod da/c10 0 2024-03-22T07:08:30.282 INFO:tasks.workunit.client.0.smithi081.stdout:4/72: symlink da/dd/l11 0 2024-03-22T07:08:30.285 INFO:tasks.workunit.client.0.smithi081.stdout:4/73: mknod da/df/c12 0 2024-03-22T07:08:30.285 INFO:tasks.workunit.client.0.smithi081.stdout:4/74: truncate f3 1119152 0 2024-03-22T07:08:30.285 INFO:tasks.workunit.client.0.smithi081.stdout:4/75: write f3 [373410,74161] 0 2024-03-22T07:08:30.292 INFO:tasks.workunit.client.0.smithi081.stdout:7/94: dwrite f6 [0,4194304] 0 2024-03-22T07:08:30.295 INFO:tasks.workunit.client.0.smithi081.stdout:4/76: dread f0 [0,4194304] 0 2024-03-22T07:08:30.298 INFO:tasks.workunit.client.0.smithi081.stdout:7/95: mkdir d9/d10/d16/d1e/d27 0 2024-03-22T07:08:30.298 INFO:tasks.workunit.client.0.smithi081.stdout:7/96: write d9/d14/f25 [822742,71132] 0 2024-03-22T07:08:30.298 INFO:tasks.workunit.client.0.smithi081.stdout:7/97: write d9/d10/d16/d1e/f17 [336997,15817] 0 2024-03-22T07:08:30.300 INFO:tasks.workunit.client.0.smithi081.stdout:4/77: symlink da/df/l13 0 2024-03-22T07:08:30.302 INFO:tasks.workunit.client.0.smithi081.stdout:7/98: creat d9/d10/d16/d1e/f28 x:0 0 0 2024-03-22T07:08:30.304 INFO:tasks.workunit.client.0.smithi081.stdout:4/78: rmdir da 39 2024-03-22T07:08:30.307 INFO:tasks.workunit.client.0.smithi081.stdout:4/79: dread da/fb [0,4194304] 0 2024-03-22T07:08:30.334 INFO:tasks.workunit.client.0.smithi081.stdout:4/80: getdents da/df 0 2024-03-22T07:08:30.334 INFO:tasks.workunit.client.0.smithi081.stdout:4/81: fsync da/fe 0 2024-03-22T07:08:30.338 INFO:tasks.workunit.client.0.smithi081.stdout:4/82: symlink da/dd/l14 0 2024-03-22T07:08:30.338 INFO:tasks.workunit.client.0.smithi081.stdout:4/83: readlink l5 0 2024-03-22T07:08:30.342 INFO:tasks.workunit.client.0.smithi081.stdout:4/84: symlink da/l15 0 2024-03-22T07:08:30.344 INFO:tasks.workunit.client.0.smithi081.stdout:4/85: rmdir da/df 39 2024-03-22T07:08:30.344 INFO:tasks.workunit.client.0.smithi081.stdout:4/86: stat da/df/c12 0 2024-03-22T07:08:30.365 INFO:tasks.workunit.client.0.smithi081.stdout:4/87: dwrite da/fb [0,4194304] 0 2024-03-22T07:08:30.365 INFO:tasks.workunit.client.0.smithi081.stdout:4/88: chown da/df 354989525 1 2024-03-22T07:08:30.368 INFO:tasks.workunit.client.0.smithi081.stdout:4/89: link da/dd/l14 da/df/l16 0 2024-03-22T07:08:30.368 INFO:tasks.workunit.client.0.smithi081.stdout:4/90: chown f9 833 1 2024-03-22T07:08:30.396 INFO:tasks.workunit.client.0.smithi081.stdout:4/91: write f9 [4239470,22567] 0 2024-03-22T07:08:30.398 INFO:tasks.workunit.client.0.smithi081.stdout:4/92: symlink da/df/l17 0 2024-03-22T07:08:30.400 INFO:tasks.workunit.client.0.smithi081.stdout:4/93: mkdir da/d18 0 2024-03-22T07:08:30.446 INFO:tasks.workunit.client.0.smithi081.stdout:4/94: truncate da/fb 1167370 0 2024-03-22T07:08:30.446 INFO:tasks.workunit.client.0.smithi081.stdout:4/95: fsync f1 0 2024-03-22T07:08:30.790 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:30 smithi149 ceph-mon[36068]: pgmap v238: 65 pgs: 65 active+clean; 166 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 683 KiB/s rd, 9.6 MiB/s wr, 576 op/s 2024-03-22T07:08:30.790 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:30 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:30.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:30 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:30.791 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:30 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:30.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:30 smithi081 ceph-mon[29245]: pgmap v238: 65 pgs: 65 active+clean; 166 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 683 KiB/s rd, 9.6 MiB/s wr, 576 op/s 2024-03-22T07:08:30.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:30 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:30.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:30 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:30.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:30 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:31.042 INFO:tasks.workunit.client.0.smithi081.stdout:4/96: symlink da/l19 0 2024-03-22T07:08:31.046 INFO:tasks.workunit.client.0.smithi081.stdout:4/97: link da/df/l16 da/l1a 0 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:3/24: sync 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:2/12: sync 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:7/99: sync 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:5/26: sync 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:6/2: sync 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:0/40: sync 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:1/17: sync 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:6/3: fsync - no filename 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:7/100: truncate d9/d14/f24 383466 0 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:6/4: chown . 4235 1 2024-03-22T07:08:31.651 INFO:tasks.workunit.client.0.smithi081.stdout:6/5: chown . 173 1 2024-03-22T07:08:31.652 INFO:tasks.workunit.client.0.smithi081.stdout:9/5: sync 2024-03-22T07:08:31.652 INFO:tasks.workunit.client.0.smithi081.stdout:8/9: sync 2024-03-22T07:08:31.652 INFO:tasks.workunit.client.0.smithi081.stdout:7/101: dread - d9/dc/f1f zero size 2024-03-22T07:08:31.652 INFO:tasks.workunit.client.0.smithi081.stdout:5/27: write f1 [770112,86015] 0 2024-03-22T07:08:31.653 INFO:tasks.workunit.client.0.smithi081.stdout:0/41: mknod d3/cb 0 2024-03-22T07:08:31.655 INFO:tasks.workunit.client.0.smithi081.stdout:1/18: creat d6/fa x:0 0 0 2024-03-22T07:08:31.655 INFO:tasks.workunit.client.0.smithi081.stdout:1/19: creat d6/fb x:0 0 0 2024-03-22T07:08:31.656 INFO:tasks.workunit.client.0.smithi081.stdout:1/20: creat d6/fc x:0 0 0 2024-03-22T07:08:31.656 INFO:tasks.workunit.client.0.smithi081.stdout:1/21: truncate d6/fa 996828 0 2024-03-22T07:08:31.657 INFO:tasks.workunit.client.0.smithi081.stdout:8/10: creat d2/f6 x:0 0 0 2024-03-22T07:08:31.657 INFO:tasks.workunit.client.0.smithi081.stdout:8/11: write d2/d3/f4 [193397,125746] 0 2024-03-22T07:08:31.657 INFO:tasks.workunit.client.0.smithi081.stdout:8/12: chown f1 57377 1 2024-03-22T07:08:31.657 INFO:tasks.workunit.client.0.smithi081.stdout:8/13: truncate d2/f6 982934 0 2024-03-22T07:08:31.659 INFO:tasks.workunit.client.0.smithi081.stdout:5/28: write f2 [4117292,26004] 0 2024-03-22T07:08:31.659 INFO:tasks.workunit.client.0.smithi081.stdout:5/29: stat d4/c6 0 2024-03-22T07:08:31.659 INFO:tasks.workunit.client.0.smithi081.stdout:5/30: dread - f3 zero size 2024-03-22T07:08:31.659 INFO:tasks.workunit.client.0.smithi081.stdout:6/6: symlink l0 0 2024-03-22T07:08:31.659 INFO:tasks.workunit.client.0.smithi081.stdout:6/7: write - no filename 2024-03-22T07:08:31.659 INFO:tasks.workunit.client.0.smithi081.stdout:6/8: dread - no filename 2024-03-22T07:08:31.659 INFO:tasks.workunit.client.0.smithi081.stdout:6/9: dwrite - no filename 2024-03-22T07:08:31.660 INFO:tasks.workunit.client.0.smithi081.stdout:9/6: creat f0 x:0 0 0 2024-03-22T07:08:31.660 INFO:tasks.workunit.client.0.smithi081.stdout:9/7: chown f0 261852492 1 2024-03-22T07:08:31.660 INFO:tasks.workunit.client.0.smithi081.stdout:9/8: creat f1 x:0 0 0 2024-03-22T07:08:31.663 INFO:tasks.workunit.client.0.smithi081.stdout:6/10: link l0 l1 0 2024-03-22T07:08:31.663 INFO:tasks.workunit.client.0.smithi081.stdout:6/11: truncate - no filename 2024-03-22T07:08:31.663 INFO:tasks.workunit.client.0.smithi081.stdout:6/12: truncate - no filename 2024-03-22T07:08:31.664 INFO:tasks.workunit.client.0.smithi081.stdout:6/13: dread - no filename 2024-03-22T07:08:31.666 INFO:tasks.workunit.client.0.smithi081.stdout:6/14: mknod c2 0 2024-03-22T07:08:31.666 INFO:tasks.workunit.client.0.smithi081.stdout:6/15: read - no filename 2024-03-22T07:08:31.666 INFO:tasks.workunit.client.0.smithi081.stdout:6/16: write - no filename 2024-03-22T07:08:31.668 INFO:tasks.workunit.client.0.smithi081.stdout:6/17: unlink l0 0 2024-03-22T07:08:31.668 INFO:tasks.workunit.client.0.smithi081.stdout:6/18: truncate - no filename 2024-03-22T07:08:31.669 INFO:tasks.workunit.client.0.smithi081.stdout:6/19: mkdir d3 0 2024-03-22T07:08:31.695 INFO:tasks.workunit.client.0.smithi081.stdout:7/102: dwrite d9/dc/f1a [0,4194304] 0 2024-03-22T07:08:31.695 INFO:tasks.workunit.client.0.smithi081.stdout:5/31: dread f1 [0,4194304] 0 2024-03-22T07:08:31.698 INFO:tasks.workunit.client.0.smithi081.stdout:7/103: rename d9/c19 to d9/c29 0 2024-03-22T07:08:31.699 INFO:tasks.workunit.client.0.smithi081.stdout:7/104: mkdir d9/d10/d2a 0 2024-03-22T07:08:31.700 INFO:tasks.workunit.client.0.smithi081.stdout:7/105: symlink d9/d10/d16/l2b 0 2024-03-22T07:08:31.700 INFO:tasks.workunit.client.0.smithi081.stdout:7/106: write d9/d10/d16/d1e/f17 [1070070,31566] 0 2024-03-22T07:08:31.701 INFO:tasks.workunit.client.0.smithi081.stdout:7/107: rename d9/d10/d16/d1e/f17 to d9/d10/d16/d1e/f2c 0 2024-03-22T07:08:31.701 INFO:tasks.workunit.client.0.smithi081.stdout:7/108: mknod d9/d10/d2a/c2d 0 2024-03-22T07:08:31.703 INFO:tasks.workunit.client.0.smithi081.stdout:7/109: creat d9/d10/f2e x:0 0 0 2024-03-22T07:08:31.704 INFO:tasks.workunit.client.0.smithi081.stdout:7/110: mknod d9/d10/d16/c2f 0 2024-03-22T07:08:31.704 INFO:tasks.workunit.client.0.smithi081.stdout:7/111: fdatasync d9/d10/d16/d1e/f28 0 2024-03-22T07:08:31.704 INFO:tasks.workunit.client.0.smithi081.stdout:7/112: mknod d9/d14/c30 0 2024-03-22T07:08:31.706 INFO:tasks.workunit.client.0.smithi081.stdout:7/113: mknod d9/d10/d16/d1e/c31 0 2024-03-22T07:08:31.706 INFO:tasks.workunit.client.0.smithi081.stdout:7/114: creat d9/d10/d16/d1e/f32 x:0 0 0 2024-03-22T07:08:31.708 INFO:tasks.workunit.client.0.smithi081.stdout:2/13: dwrite f4 [0,4194304] 0 2024-03-22T07:08:31.709 INFO:tasks.workunit.client.0.smithi081.stdout:7/115: rename d9/d14/f25 to d9/dc/f33 0 2024-03-22T07:08:31.709 INFO:tasks.workunit.client.0.smithi081.stdout:7/116: dread - d9/d10/d16/d1e/f32 zero size 2024-03-22T07:08:31.709 INFO:tasks.workunit.client.0.smithi081.stdout:3/25: dwrite f7 [0,4194304] 0 2024-03-22T07:08:31.709 INFO:tasks.workunit.client.0.smithi081.stdout:7/117: write d9/d10/d16/d1e/f2c [80327,5625] 0 2024-03-22T07:08:31.709 INFO:tasks.workunit.client.0.smithi081.stdout:7/118: chown d9/d10/d2a/c2d 267905998 1 2024-03-22T07:08:31.711 INFO:tasks.workunit.client.0.smithi081.stdout:8/14: dwrite d2/f6 [0,4194304] 0 2024-03-22T07:08:31.711 INFO:tasks.workunit.client.0.smithi081.stdout:8/15: truncate f1 468493 0 2024-03-22T07:08:31.713 INFO:tasks.workunit.client.0.smithi081.stdout:3/26: mknod c9 0 2024-03-22T07:08:31.713 INFO:tasks.workunit.client.0.smithi081.stdout:3/27: write f4 [95274,124778] 0 2024-03-22T07:08:31.720 INFO:tasks.workunit.client.0.smithi081.stdout:3/28: dread f1 [0,4194304] 0 2024-03-22T07:08:31.733 INFO:tasks.workunit.client.0.smithi081.stdout:2/14: dread f4 [0,4194304] 0 2024-03-22T07:08:31.733 INFO:tasks.workunit.client.0.smithi081.stdout:2/15: rmdir - no directory 2024-03-22T07:08:31.739 INFO:tasks.workunit.client.0.smithi081.stdout:0/42: dwrite d3/f8 [0,4194304] 0 2024-03-22T07:08:31.739 INFO:tasks.workunit.client.0.smithi081.stdout:5/32: dwrite f1 [0,4194304] 0 2024-03-22T07:08:31.739 INFO:tasks.workunit.client.0.smithi081.stdout:5/33: rename d4/d9 to d4/d9/da 22 2024-03-22T07:08:31.739 INFO:tasks.workunit.client.0.smithi081.stdout:5/34: write f3 [212448,129417] 0 2024-03-22T07:08:31.744 INFO:tasks.workunit.client.0.smithi081.stdout:7/119: dwrite d9/d10/d16/d1e/f32 [0,4194304] 0 2024-03-22T07:08:31.745 INFO:tasks.workunit.client.0.smithi081.stdout:2/16: read f4 [3333767,82180] 0 2024-03-22T07:08:31.745 INFO:tasks.workunit.client.0.smithi081.stdout:2/17: chown f3 9182 1 2024-03-22T07:08:31.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/18: chown l1 954 1 2024-03-22T07:08:31.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/19: creat f6 x:0 0 0 2024-03-22T07:08:31.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/20: chown f5 64335081 1 2024-03-22T07:08:31.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/21: stat f5 0 2024-03-22T07:08:31.747 INFO:tasks.workunit.client.0.smithi081.stdout:3/29: dwrite f7 [0,4194304] 0 2024-03-22T07:08:31.747 INFO:tasks.workunit.client.0.smithi081.stdout:3/30: write f6 [880318,96754] 0 2024-03-22T07:08:31.750 INFO:tasks.workunit.client.0.smithi081.stdout:2/22: mknod c7 0 2024-03-22T07:08:31.754 INFO:tasks.workunit.client.0.smithi081.stdout:3/31: rename c2 to ca 0 2024-03-22T07:08:31.754 INFO:tasks.workunit.client.0.smithi081.stdout:3/32: chown f4 1273835925 1 2024-03-22T07:08:31.757 INFO:tasks.workunit.client.0.smithi081.stdout:3/33: mkdir db 0 2024-03-22T07:08:31.778 INFO:tasks.workunit.client.0.smithi081.stdout:2/23: dwrite f3 [0,4194304] 0 2024-03-22T07:08:31.780 INFO:tasks.workunit.client.0.smithi081.stdout:5/35: dwrite f3 [0,4194304] 0 2024-03-22T07:08:31.783 INFO:tasks.workunit.client.0.smithi081.stdout:2/24: mknod c8 0 2024-03-22T07:08:31.783 INFO:tasks.workunit.client.0.smithi081.stdout:2/25: dread - f6 zero size 2024-03-22T07:08:31.785 INFO:tasks.workunit.client.0.smithi081.stdout:5/36: getdents d4 0 2024-03-22T07:08:31.790 INFO:tasks.workunit.client.0.smithi081.stdout:3/34: dwrite f6 [0,4194304] 0 2024-03-22T07:08:31.791 INFO:tasks.workunit.client.0.smithi081.stdout:2/26: link c8 c9 0 2024-03-22T07:08:31.791 INFO:tasks.workunit.client.0.smithi081.stdout:2/27: stat f6 0 2024-03-22T07:08:31.793 INFO:tasks.workunit.client.0.smithi081.stdout:3/35: creat db/fc x:0 0 0 2024-03-22T07:08:31.795 INFO:tasks.workunit.client.0.smithi081.stdout:2/28: mkdir da 0 2024-03-22T07:08:31.820 INFO:tasks.workunit.client.0.smithi081.stdout:3/36: dwrite f6 [0,4194304] 0 2024-03-22T07:08:31.820 INFO:tasks.workunit.client.0.smithi081.stdout:3/37: write f4 [621975,56630] 0 2024-03-22T07:08:31.821 INFO:tasks.workunit.client.0.smithi081.stdout:3/38: creat db/fd x:0 0 0 2024-03-22T07:08:31.964 INFO:tasks.workunit.client.0.smithi081.stdout:0/43: getdents d3 0 2024-03-22T07:08:31.966 INFO:tasks.workunit.client.0.smithi081.stdout:1/22: rmdir d6 39 2024-03-22T07:08:31.968 INFO:tasks.workunit.client.0.smithi081.stdout:1/23: creat d6/fd x:0 0 0 2024-03-22T07:08:31.969 INFO:tasks.workunit.client.0.smithi081.stdout:9/9: getdents . 0 2024-03-22T07:08:31.969 INFO:tasks.workunit.client.0.smithi081.stdout:9/10: dread - f0 zero size 2024-03-22T07:08:31.972 INFO:tasks.workunit.client.0.smithi081.stdout:9/11: mkdir d2 0 2024-03-22T07:08:31.983 INFO:tasks.workunit.client.0.smithi081.stdout:9/12: rename f0 to d2/f3 0 2024-03-22T07:08:31.983 INFO:tasks.workunit.client.0.smithi081.stdout:9/13: chown d2 586340873 1 2024-03-22T07:08:31.986 INFO:tasks.workunit.client.0.smithi081.stdout:9/14: creat d2/f4 x:0 0 0 2024-03-22T07:08:31.988 INFO:tasks.workunit.client.0.smithi081.stdout:0/44: dwrite d3/f7 [0,4194304] 0 2024-03-22T07:08:31.990 INFO:tasks.workunit.client.0.smithi081.stdout:9/15: mkdir d2/d5 0 2024-03-22T07:08:31.995 INFO:tasks.workunit.client.0.smithi081.stdout:9/16: write f1 [598764,15456] 0 2024-03-22T07:08:31.995 INFO:tasks.workunit.client.0.smithi081.stdout:9/17: write d2/f3 [485453,108773] 0 2024-03-22T07:08:31.997 INFO:tasks.workunit.client.0.smithi081.stdout:0/45: dread d3/f8 [0,4194304] 0 2024-03-22T07:08:31.997 INFO:tasks.workunit.client.0.smithi081.stdout:7/120: truncate d9/d10/d16/d1e/f2c 1054925 0 2024-03-22T07:08:31.997 INFO:tasks.workunit.client.0.smithi081.stdout:7/121: chown d9/d10/d16/d1e/l23 657 1 2024-03-22T07:08:31.997 INFO:tasks.workunit.client.0.smithi081.stdout:7/122: write d9/dc/f1f [187479,120548] 0 2024-03-22T07:08:31.997 INFO:tasks.workunit.client.0.smithi081.stdout:1/24: dwrite d6/fd [0,4194304] 0 2024-03-22T07:08:31.999 INFO:tasks.workunit.client.0.smithi081.stdout:5/37: truncate f1 1625744 0 2024-03-22T07:08:31.999 INFO:tasks.workunit.client.0.smithi081.stdout:5/38: write f2 [1908245,59661] 0 2024-03-22T07:08:31.999 INFO:tasks.workunit.client.0.smithi081.stdout:9/18: creat d2/d5/f6 x:0 0 0 2024-03-22T07:08:31.999 INFO:tasks.workunit.client.0.smithi081.stdout:9/19: read - d2/d5/f6 zero size 2024-03-22T07:08:32.001 INFO:tasks.workunit.client.0.smithi081.stdout:0/46: mkdir d3/dc 0 2024-03-22T07:08:32.003 INFO:tasks.workunit.client.0.smithi081.stdout:7/123: mknod d9/c34 0 2024-03-22T07:08:32.004 INFO:tasks.workunit.client.0.smithi081.stdout:7/124: readlink d9/l20 0 2024-03-22T07:08:32.016 INFO:tasks.workunit.client.0.smithi081.stdout:1/25: mknod d6/ce 0 2024-03-22T07:08:32.017 INFO:tasks.workunit.client.0.smithi081.stdout:7/125: dwrite f1 [0,4194304] 0 2024-03-22T07:08:32.017 INFO:tasks.workunit.client.0.smithi081.stdout:7/126: creat d9/d10/d16/d1e/f35 x:0 0 0 2024-03-22T07:08:32.018 INFO:tasks.workunit.client.0.smithi081.stdout:7/127: fdatasync d9/f11 0 2024-03-22T07:08:32.018 INFO:tasks.workunit.client.0.smithi081.stdout:9/20: rename f1 to d2/d5/f7 0 2024-03-22T07:08:32.019 INFO:tasks.workunit.client.0.smithi081.stdout:9/21: write d2/d5/f7 [105105,2111] 0 2024-03-22T07:08:32.019 INFO:tasks.workunit.client.0.smithi081.stdout:9/22: creat d2/f8 x:0 0 0 2024-03-22T07:08:32.021 INFO:tasks.workunit.client.0.smithi081.stdout:0/47: creat d3/dc/fd x:0 0 0 2024-03-22T07:08:32.024 INFO:tasks.workunit.client.0.smithi081.stdout:1/26: mknod d6/cf 0 2024-03-22T07:08:32.033 INFO:tasks.workunit.client.0.smithi081.stdout:7/128: link d9/lb d9/d10/d2a/l36 0 2024-03-22T07:08:32.033 INFO:tasks.workunit.client.0.smithi081.stdout:7/129: chown d9/d10/d2a/c2d 15080 1 2024-03-22T07:08:32.033 INFO:tasks.workunit.client.0.smithi081.stdout:7/130: stat d9/d10/f2e 0 2024-03-22T07:08:32.033 INFO:tasks.workunit.client.0.smithi081.stdout:9/23: dwrite d2/f8 [0,4194304] 0 2024-03-22T07:08:32.038 INFO:tasks.workunit.client.0.smithi081.stdout:0/48: link d3/f4 d3/dc/fe 0 2024-03-22T07:08:32.038 INFO:tasks.workunit.client.0.smithi081.stdout:0/49: readlink d3/l9 0 2024-03-22T07:08:32.038 INFO:tasks.workunit.client.0.smithi081.stdout:9/24: dread d2/d5/f7 [0,4194304] 0 2024-03-22T07:08:32.043 INFO:tasks.workunit.client.0.smithi081.stdout:7/131: stat d9/le 0 2024-03-22T07:08:32.043 INFO:tasks.workunit.client.0.smithi081.stdout:7/132: chown l8 305 1 2024-03-22T07:08:32.043 INFO:tasks.workunit.client.0.smithi081.stdout:7/133: write d9/dc/f1f [1319512,6026] 0 2024-03-22T07:08:32.049 INFO:tasks.workunit.client.0.smithi081.stdout:4/98: sync 2024-03-22T07:08:32.049 INFO:tasks.workunit.client.0.smithi081.stdout:4/99: fdatasync f9 0 2024-03-22T07:08:32.053 INFO:tasks.workunit.client.0.smithi081.stdout:0/50: rmdir d3 39 2024-03-22T07:08:32.053 INFO:tasks.workunit.client.0.smithi081.stdout:0/51: creat d3/dc/ff x:0 0 0 2024-03-22T07:08:32.053 INFO:tasks.workunit.client.0.smithi081.stdout:5/39: dread f1 [0,4194304] 0 2024-03-22T07:08:32.055 INFO:tasks.workunit.client.0.smithi081.stdout:4/100: dread f9 [0,4194304] 0 2024-03-22T07:08:32.055 INFO:tasks.workunit.client.0.smithi081.stdout:9/25: symlink d2/l9 0 2024-03-22T07:08:32.056 INFO:tasks.workunit.client.0.smithi081.stdout:9/26: stat d2/f3 0 2024-03-22T07:08:32.057 INFO:tasks.workunit.client.0.smithi081.stdout:6/20: sync 2024-03-22T07:08:32.057 INFO:tasks.workunit.client.0.smithi081.stdout:6/21: fdatasync - no filename 2024-03-22T07:08:32.057 INFO:tasks.workunit.client.0.smithi081.stdout:6/22: write - no filename 2024-03-22T07:08:32.057 INFO:tasks.workunit.client.0.smithi081.stdout:6/23: read - no filename 2024-03-22T07:08:32.061 INFO:tasks.workunit.client.0.smithi081.stdout:5/40: write f3 [2716959,97698] 0 2024-03-22T07:08:32.061 INFO:tasks.workunit.client.0.smithi081.stdout:5/41: stat f2 0 2024-03-22T07:08:32.063 INFO:tasks.workunit.client.0.smithi081.stdout:7/134: symlink d9/d10/l37 0 2024-03-22T07:08:32.070 INFO:tasks.workunit.client.0.smithi081.stdout:4/101: getdents da 0 2024-03-22T07:08:32.072 INFO:tasks.workunit.client.0.smithi081.stdout:1/27: getdents d6 0 2024-03-22T07:08:32.072 INFO:tasks.workunit.client.0.smithi081.stdout:1/28: readlink l5 0 2024-03-22T07:08:32.072 INFO:tasks.workunit.client.0.smithi081.stdout:1/29: write d6/fb [22058,121898] 0 2024-03-22T07:08:32.072 INFO:tasks.workunit.client.0.smithi081.stdout:1/30: chown l5 29573 1 2024-03-22T07:08:32.073 INFO:tasks.workunit.client.0.smithi081.stdout:9/27: link d2/d5/f6 d2/d5/fa 0 2024-03-22T07:08:32.074 INFO:tasks.workunit.client.0.smithi081.stdout:9/28: write d2/f4 [502243,62638] 0 2024-03-22T07:08:32.075 INFO:tasks.workunit.client.0.smithi081.stdout:9/29: dread d2/d5/f7 [0,4194304] 0 2024-03-22T07:08:32.075 INFO:tasks.workunit.client.0.smithi081.stdout:9/30: truncate d2/d5/f6 851979 0 2024-03-22T07:08:32.075 INFO:tasks.workunit.client.0.smithi081.stdout:6/24: stat l1 0 2024-03-22T07:08:32.075 INFO:tasks.workunit.client.0.smithi081.stdout:6/25: fdatasync - no filename 2024-03-22T07:08:32.077 INFO:tasks.workunit.client.0.smithi081.stdout:5/42: symlink d4/lb 0 2024-03-22T07:08:32.084 INFO:tasks.workunit.client.0.smithi081.stdout:0/52: fsync d3/dc/fe 0 2024-03-22T07:08:32.084 INFO:tasks.workunit.client.0.smithi081.stdout:0/53: chown d3/dc/fd 137607760 1 2024-03-22T07:08:32.084 INFO:tasks.workunit.client.0.smithi081.stdout:0/54: dread - d3/dc/fd zero size 2024-03-22T07:08:32.084 INFO:tasks.workunit.client.0.smithi081.stdout:0/55: fsync d3/f8 0 2024-03-22T07:08:32.085 INFO:tasks.workunit.client.0.smithi081.stdout:4/102: mkdir da/d18/d1b 0 2024-03-22T07:08:32.088 INFO:tasks.workunit.client.0.smithi081.stdout:1/31: truncate d6/fd 3451186 0 2024-03-22T07:08:32.089 INFO:tasks.workunit.client.0.smithi081.stdout:1/32: dread - d6/f9 zero size 2024-03-22T07:08:32.089 INFO:tasks.workunit.client.0.smithi081.stdout:1/33: chown d6/cf 11 1 2024-03-22T07:08:32.089 INFO:tasks.workunit.client.0.smithi081.stdout:1/34: chown d6/fd 4629 1 2024-03-22T07:08:32.091 INFO:tasks.workunit.client.0.smithi081.stdout:9/31: dwrite d2/f8 [0,4194304] 0 2024-03-22T07:08:32.092 INFO:tasks.workunit.client.0.smithi081.stdout:9/32: read d2/f4 [420051,110306] 0 2024-03-22T07:08:32.093 INFO:tasks.workunit.client.0.smithi081.stdout:5/43: creat d4/fc x:0 0 0 2024-03-22T07:08:32.093 INFO:tasks.workunit.client.0.smithi081.stdout:5/44: read f2 [844044,98996] 0 2024-03-22T07:08:32.093 INFO:tasks.workunit.client.0.smithi081.stdout:5/45: write f3 [4390170,69560] 0 2024-03-22T07:08:32.094 INFO:tasks.workunit.client.0.smithi081.stdout:4/103: mkdir da/dd/d1c 0 2024-03-22T07:08:32.096 INFO:tasks.workunit.client.0.smithi081.stdout:2/29: sync 2024-03-22T07:08:32.096 INFO:tasks.workunit.client.0.smithi081.stdout:8/16: sync 2024-03-22T07:08:32.096 INFO:tasks.workunit.client.0.smithi081.stdout:1/35: rename d6/fa to d6/f10 0 2024-03-22T07:08:32.098 INFO:tasks.workunit.client.0.smithi081.stdout:0/56: dwrite d3/f8 [4194304,4194304] 0 2024-03-22T07:08:32.099 INFO:tasks.workunit.client.0.smithi081.stdout:6/26: mkdir d3/d4 0 2024-03-22T07:08:32.099 INFO:tasks.workunit.client.0.smithi081.stdout:6/27: write - no filename 2024-03-22T07:08:32.099 INFO:tasks.workunit.client.0.smithi081.stdout:6/28: truncate - no filename 2024-03-22T07:08:32.099 INFO:tasks.workunit.client.0.smithi081.stdout:6/29: write - no filename 2024-03-22T07:08:32.100 INFO:tasks.workunit.client.0.smithi081.stdout:3/39: sync 2024-03-22T07:08:32.100 INFO:tasks.workunit.client.0.smithi081.stdout:9/33: mknod d2/d5/cb 0 2024-03-22T07:08:32.101 INFO:tasks.workunit.client.0.smithi081.stdout:5/46: mkdir d4/dd 0 2024-03-22T07:08:32.103 INFO:tasks.workunit.client.0.smithi081.stdout:4/104: creat da/f1d x:0 0 0 2024-03-22T07:08:32.106 INFO:tasks.workunit.client.0.smithi081.stdout:8/17: symlink d2/d3/l7 0 2024-03-22T07:08:32.112 INFO:tasks.workunit.client.0.smithi081.stdout:0/57: creat d3/f10 x:0 0 0 2024-03-22T07:08:32.112 INFO:tasks.workunit.client.0.smithi081.stdout:0/58: creat d3/dc/f11 x:0 0 0 2024-03-22T07:08:32.116 INFO:tasks.workunit.client.0.smithi081.stdout:5/47: dwrite d4/fc [0,4194304] 0 2024-03-22T07:08:32.116 INFO:tasks.workunit.client.0.smithi081.stdout:5/48: chown d4/dd 140 1 2024-03-22T07:08:32.118 INFO:tasks.workunit.client.0.smithi081.stdout:8/18: read d2/f6 [2554918,36285] 0 2024-03-22T07:08:32.118 INFO:tasks.workunit.client.0.smithi081.stdout:8/19: chown d2/d3 2720050 1 2024-03-22T07:08:32.119 INFO:tasks.workunit.client.0.smithi081.stdout:8/20: dread d2/d3/f4 [0,4194304] 0 2024-03-22T07:08:32.119 INFO:tasks.workunit.client.0.smithi081.stdout:8/21: write d2/f6 [435955,86815] 0 2024-03-22T07:08:32.120 INFO:tasks.workunit.client.0.smithi081.stdout:5/49: dread f2 [0,4194304] 0 2024-03-22T07:08:32.120 INFO:tasks.workunit.client.0.smithi081.stdout:5/50: creat d4/fe x:0 0 0 2024-03-22T07:08:32.125 INFO:tasks.workunit.client.0.smithi081.stdout:3/40: mkdir db/de 0 2024-03-22T07:08:32.125 INFO:tasks.workunit.client.0.smithi081.stdout:3/41: chown db/fd 19738249 1 2024-03-22T07:08:32.126 INFO:tasks.workunit.client.0.smithi081.stdout:9/34: symlink d2/lc 0 2024-03-22T07:08:32.126 INFO:tasks.workunit.client.0.smithi081.stdout:9/35: write d2/d5/fa [1872153,106743] 0 2024-03-22T07:08:32.128 INFO:tasks.workunit.client.0.smithi081.stdout:4/105: mkdir da/d18/d1b/d1e 0 2024-03-22T07:08:32.128 INFO:tasks.workunit.client.0.smithi081.stdout:4/106: write da/fb [1550052,114636] 0 2024-03-22T07:08:32.137 INFO:tasks.workunit.client.0.smithi081.stdout:0/59: link d3/f4 d3/dc/f12 0 2024-03-22T07:08:32.138 INFO:tasks.workunit.client.0.smithi081.stdout:1/36: dread d6/fd [0,4194304] 0 2024-03-22T07:08:32.142 INFO:tasks.workunit.client.0.smithi081.stdout:8/22: symlink d2/d3/l8 0 2024-03-22T07:08:32.146 INFO:tasks.workunit.client.0.smithi081.stdout:5/51: rmdir d4/dd 0 2024-03-22T07:08:32.158 INFO:tasks.workunit.client.0.smithi081.stdout:0/60: rename d3/dc/f12 to d3/f13 0 2024-03-22T07:08:32.159 INFO:tasks.workunit.client.0.smithi081.stdout:0/61: write d3/f10 [93285,80937] 0 2024-03-22T07:08:32.160 INFO:tasks.workunit.client.0.smithi081.stdout:1/37: creat d6/f11 x:0 0 0 2024-03-22T07:08:32.162 INFO:tasks.workunit.client.0.smithi081.stdout:5/52: truncate f1 1290755 0 2024-03-22T07:08:32.164 INFO:tasks.workunit.client.0.smithi081.stdout:3/42: getdents db 0 2024-03-22T07:08:32.166 INFO:tasks.workunit.client.0.smithi081.stdout:8/23: dwrite f0 [0,4194304] 0 2024-03-22T07:08:32.166 INFO:tasks.workunit.client.0.smithi081.stdout:9/36: rmdir d2/d5 39 2024-03-22T07:08:32.166 INFO:tasks.workunit.client.0.smithi081.stdout:9/37: readlink d2/l9 0 2024-03-22T07:08:32.169 INFO:tasks.workunit.client.0.smithi081.stdout:0/62: read d3/f8 [1973230,22024] 0 2024-03-22T07:08:32.169 INFO:tasks.workunit.client.0.smithi081.stdout:8/24: dread d2/d3/f4 [0,4194304] 0 2024-03-22T07:08:32.169 INFO:tasks.workunit.client.0.smithi081.stdout:8/25: chown d2/c5 266857 1 2024-03-22T07:08:32.170 INFO:tasks.workunit.client.0.smithi081.stdout:4/107: getdents da 0 2024-03-22T07:08:32.170 INFO:tasks.workunit.client.0.smithi081.stdout:4/108: dread - da/fc zero size 2024-03-22T07:08:32.171 INFO:tasks.workunit.client.0.smithi081.stdout:8/26: dread d2/d3/f4 [0,4194304] 0 2024-03-22T07:08:32.172 INFO:tasks.workunit.client.0.smithi081.stdout:5/53: mkdir d4/df 0 2024-03-22T07:08:32.174 INFO:tasks.workunit.client.0.smithi081.stdout:9/38: write d2/f8 [1136887,31457] 0 2024-03-22T07:08:32.175 INFO:tasks.workunit.client.0.smithi081.stdout:3/43: mknod db/cf 0 2024-03-22T07:08:32.177 INFO:tasks.workunit.client.0.smithi081.stdout:4/109: getdents da/d18/d1b 0 2024-03-22T07:08:32.177 INFO:tasks.workunit.client.0.smithi081.stdout:4/110: chown f0 31 1 2024-03-22T07:08:32.177 INFO:tasks.workunit.client.0.smithi081.stdout:4/111: chown da/df/c12 8633 1 2024-03-22T07:08:32.179 INFO:tasks.workunit.client.0.smithi081.stdout:8/27: read f1 [114476,28037] 0 2024-03-22T07:08:32.182 INFO:tasks.workunit.client.0.smithi081.stdout:5/54: mkdir d4/d10 0 2024-03-22T07:08:32.182 INFO:tasks.workunit.client.0.smithi081.stdout:1/38: dwrite f1 [0,4194304] 0 2024-03-22T07:08:32.182 INFO:tasks.workunit.client.0.smithi081.stdout:1/39: fdatasync d6/fb 0 2024-03-22T07:08:32.187 INFO:tasks.workunit.client.0.smithi081.stdout:4/112: dread da/fb [0,4194304] 0 2024-03-22T07:08:32.188 INFO:tasks.workunit.client.0.smithi081.stdout:4/113: truncate f9 1987211 0 2024-03-22T07:08:32.197 INFO:tasks.workunit.client.0.smithi081.stdout:0/63: dwrite d3/f7 [0,4194304] 0 2024-03-22T07:08:32.197 INFO:tasks.workunit.client.0.smithi081.stdout:1/40: link d6/fd d6/f12 0 2024-03-22T07:08:32.199 INFO:tasks.workunit.client.0.smithi081.stdout:4/114: mkdir da/dd/d1c/d1f 0 2024-03-22T07:08:32.199 INFO:tasks.workunit.client.0.smithi081.stdout:9/39: dwrite d2/f8 [0,4194304] 0 2024-03-22T07:08:32.204 INFO:tasks.workunit.client.0.smithi081.stdout:5/55: link d4/fe d4/d9/f11 0 2024-03-22T07:08:32.205 INFO:tasks.workunit.client.0.smithi081.stdout:5/56: stat d4/fe 0 2024-03-22T07:08:32.206 INFO:tasks.workunit.client.0.smithi081.stdout:3/44: getdents db 0 2024-03-22T07:08:32.210 INFO:tasks.workunit.client.0.smithi081.stdout:0/64: symlink d3/l14 0 2024-03-22T07:08:32.210 INFO:tasks.workunit.client.0.smithi081.stdout:0/65: creat d3/dc/f15 x:0 0 0 2024-03-22T07:08:32.210 INFO:tasks.workunit.client.0.smithi081.stdout:0/66: chown d3/f10 13 1 2024-03-22T07:08:32.212 INFO:tasks.workunit.client.0.smithi081.stdout:1/41: unlink d6/cf 0 2024-03-22T07:08:32.212 INFO:tasks.workunit.client.0.smithi081.stdout:1/42: truncate d6/f10 1175530 0 2024-03-22T07:08:32.214 INFO:tasks.workunit.client.0.smithi081.stdout:4/115: creat da/dd/f20 x:0 0 0 2024-03-22T07:08:32.214 INFO:tasks.workunit.client.0.smithi081.stdout:4/116: chown da/df/l17 0 1 2024-03-22T07:08:32.217 INFO:tasks.workunit.client.0.smithi081.stdout:9/40: creat d2/d5/fd x:0 0 0 2024-03-22T07:08:32.217 INFO:tasks.workunit.client.0.smithi081.stdout:9/41: chown d2/l9 4973635 1 2024-03-22T07:08:32.217 INFO:tasks.workunit.client.0.smithi081.stdout:9/42: write d2/d5/fa [2478746,31225] 0 2024-03-22T07:08:32.218 INFO:tasks.workunit.client.0.smithi081.stdout:3/45: symlink db/de/l10 0 2024-03-22T07:08:32.220 INFO:tasks.workunit.client.0.smithi081.stdout:1/43: write f1 [4564167,44985] 0 2024-03-22T07:08:32.220 INFO:tasks.workunit.client.0.smithi081.stdout:1/44: fdatasync d6/fd 0 2024-03-22T07:08:32.220 INFO:tasks.workunit.client.0.smithi081.stdout:5/57: dwrite d4/fe [0,4194304] 0 2024-03-22T07:08:32.224 INFO:tasks.workunit.client.0.smithi081.stdout:4/117: creat da/d18/f21 x:0 0 0 2024-03-22T07:08:32.224 INFO:tasks.workunit.client.0.smithi081.stdout:4/118: truncate da/dd/f20 169047 0 2024-03-22T07:08:32.225 INFO:tasks.workunit.client.0.smithi081.stdout:5/58: dread f3 [0,4194304] 0 2024-03-22T07:08:32.225 INFO:tasks.workunit.client.0.smithi081.stdout:4/119: write da/fe [210899,91725] 0 2024-03-22T07:08:32.230 INFO:tasks.workunit.client.0.smithi081.stdout:3/46: unlink ca 0 2024-03-22T07:08:32.240 INFO:tasks.workunit.client.0.smithi081.stdout:5/59: dread d4/d9/f11 [0,4194304] 0 2024-03-22T07:08:32.240 INFO:tasks.workunit.client.0.smithi081.stdout:9/43: getdents d2 0 2024-03-22T07:08:32.240 INFO:tasks.workunit.client.0.smithi081.stdout:0/67: dwrite d3/f7 [4194304,4194304] 0 2024-03-22T07:08:32.240 INFO:tasks.workunit.client.0.smithi081.stdout:0/68: dread - d3/dc/fd zero size 2024-03-22T07:08:32.242 INFO:tasks.workunit.client.0.smithi081.stdout:8/28: dwrite f1 [0,4194304] 0 2024-03-22T07:08:32.242 INFO:tasks.workunit.client.0.smithi081.stdout:8/29: chown d2/d3/l7 58 1 2024-03-22T07:08:32.243 INFO:tasks.workunit.client.0.smithi081.stdout:5/60: dread d4/fe [0,4194304] 0 2024-03-22T07:08:32.256 INFO:tasks.workunit.client.0.smithi081.stdout:0/69: write d3/f7 [2703872,86598] 0 2024-03-22T07:08:32.256 INFO:tasks.workunit.client.0.smithi081.stdout:3/47: dwrite db/fd [0,4194304] 0 2024-03-22T07:08:32.257 INFO:tasks.workunit.client.0.smithi081.stdout:5/61: dread d4/fc [0,4194304] 0 2024-03-22T07:08:32.258 INFO:tasks.workunit.client.0.smithi081.stdout:1/45: dwrite f1 [4194304,4194304] 0 2024-03-22T07:08:32.258 INFO:tasks.workunit.client.0.smithi081.stdout:1/46: fdatasync d6/f9 0 2024-03-22T07:08:32.258 INFO:tasks.workunit.client.0.smithi081.stdout:1/47: dread - d6/f9 zero size 2024-03-22T07:08:32.258 INFO:tasks.workunit.client.0.smithi081.stdout:1/48: fsync d6/f10 0 2024-03-22T07:08:32.261 INFO:tasks.workunit.client.0.smithi081.stdout:0/70: symlink d3/dc/l16 0 2024-03-22T07:08:32.261 INFO:tasks.workunit.client.0.smithi081.stdout:0/71: creat d3/dc/f17 x:0 0 0 2024-03-22T07:08:32.261 INFO:tasks.workunit.client.0.smithi081.stdout:5/62: dread f3 [0,4194304] 0 2024-03-22T07:08:32.261 INFO:tasks.workunit.client.0.smithi081.stdout:5/63: chown d4/fe 101 1 2024-03-22T07:08:32.263 INFO:tasks.workunit.client.0.smithi081.stdout:4/120: dwrite da/fe [0,4194304] 0 2024-03-22T07:08:32.263 INFO:tasks.workunit.client.0.smithi081.stdout:9/44: dread d2/d5/fa [0,4194304] 0 2024-03-22T07:08:32.264 INFO:tasks.workunit.client.0.smithi081.stdout:4/121: truncate f3 1136006 0 2024-03-22T07:08:32.264 INFO:tasks.workunit.client.0.smithi081.stdout:9/45: write d2/f8 [4522522,72168] 0 2024-03-22T07:08:32.264 INFO:tasks.workunit.client.0.smithi081.stdout:9/46: chown d2/d5/f7 1223 1 2024-03-22T07:08:32.264 INFO:tasks.workunit.client.0.smithi081.stdout:9/47: fsync d2/f4 0 2024-03-22T07:08:32.264 INFO:tasks.workunit.client.0.smithi081.stdout:4/122: write f3 [983765,17940] 0 2024-03-22T07:08:32.264 INFO:tasks.workunit.client.0.smithi081.stdout:9/48: dread d2/d5/f7 [0,4194304] 0 2024-03-22T07:08:32.266 INFO:tasks.workunit.client.0.smithi081.stdout:4/123: dread f9 [0,4194304] 0 2024-03-22T07:08:32.270 INFO:tasks.workunit.client.0.smithi081.stdout:3/48: symlink db/l11 0 2024-03-22T07:08:32.275 INFO:tasks.workunit.client.0.smithi081.stdout:0/72: write d3/f4 [3927507,19877] 0 2024-03-22T07:08:32.275 INFO:tasks.workunit.client.0.smithi081.stdout:0/73: chown d3/f4 175 1 2024-03-22T07:08:32.275 INFO:tasks.workunit.client.0.smithi081.stdout:0/74: creat d3/dc/f18 x:0 0 0 2024-03-22T07:08:32.275 INFO:tasks.workunit.client.0.smithi081.stdout:0/75: dread - d3/dc/ff zero size 2024-03-22T07:08:32.275 INFO:tasks.workunit.client.0.smithi081.stdout:0/76: fsync d3/dc/f11 0 2024-03-22T07:08:32.275 INFO:tasks.workunit.client.0.smithi081.stdout:0/77: chown d3/l14 37929180 1 2024-03-22T07:08:32.275 INFO:tasks.workunit.client.0.smithi081.stdout:1/49: mknod d6/c13 0 2024-03-22T07:08:32.278 INFO:tasks.workunit.client.0.smithi081.stdout:5/64: link d4/lb d4/d10/l12 0 2024-03-22T07:08:32.282 INFO:tasks.workunit.client.0.smithi081.stdout:3/49: rename f4 to db/f12 0 2024-03-22T07:08:32.284 INFO:tasks.workunit.client.0.smithi081.stdout:8/30: truncate f0 3760915 0 2024-03-22T07:08:32.290 INFO:tasks.workunit.client.0.smithi081.stdout:1/50: unlink d6/f11 0 2024-03-22T07:08:32.295 INFO:tasks.workunit.client.0.smithi081.stdout:1/51: dread d6/fb [0,4194304] 0 2024-03-22T07:08:32.297 INFO:tasks.workunit.client.0.smithi081.stdout:3/50: creat db/f13 x:0 0 0 2024-03-22T07:08:32.297 INFO:tasks.workunit.client.0.smithi081.stdout:3/51: stat l0 0 2024-03-22T07:08:32.306 INFO:tasks.workunit.client.0.smithi081.stdout:1/52: read d6/fd [631591,108563] 0 2024-03-22T07:08:32.307 INFO:tasks.workunit.client.0.smithi081.stdout:3/52: symlink db/de/l14 0 2024-03-22T07:08:32.308 INFO:tasks.workunit.client.0.smithi081.stdout:9/49: dwrite d2/d5/f7 [0,4194304] 0 2024-03-22T07:08:32.308 INFO:tasks.workunit.client.0.smithi081.stdout:4/124: dwrite da/fc [0,4194304] 0 2024-03-22T07:08:32.308 INFO:tasks.workunit.client.0.smithi081.stdout:5/65: dwrite d4/fc [4194304,4194304] 0 2024-03-22T07:08:32.309 INFO:tasks.workunit.client.0.smithi081.stdout:8/31: rmdir d2 39 2024-03-22T07:08:32.311 INFO:tasks.workunit.client.0.smithi081.stdout:1/53: symlink d6/l14 0 2024-03-22T07:08:32.311 INFO:tasks.workunit.client.0.smithi081.stdout:1/54: fdatasync f1 0 2024-03-22T07:08:32.311 INFO:tasks.workunit.client.0.smithi081.stdout:1/55: fsync d6/fb 0 2024-03-22T07:08:32.311 INFO:tasks.workunit.client.0.smithi081.stdout:1/56: stat d6/c13 0 2024-03-22T07:08:32.311 INFO:tasks.workunit.client.0.smithi081.stdout:3/53: write f1 [3760698,98899] 0 2024-03-22T07:08:32.317 INFO:tasks.workunit.client.0.smithi081.stdout:0/78: getdents d3/dc 0 2024-03-22T07:08:32.317 INFO:tasks.workunit.client.0.smithi081.stdout:0/79: write d3/f13 [2961527,17266] 0 2024-03-22T07:08:32.324 INFO:tasks.workunit.client.0.smithi081.stdout:9/50: creat d2/fe x:0 0 0 2024-03-22T07:08:32.326 INFO:tasks.workunit.client.0.smithi081.stdout:4/125: getdents da/dd/d1c 0 2024-03-22T07:08:32.329 INFO:tasks.workunit.client.0.smithi081.stdout:5/66: rename d4/c7 to d4/d10/c13 0 2024-03-22T07:08:32.331 INFO:tasks.workunit.client.0.smithi081.stdout:8/32: truncate f1 3282317 0 2024-03-22T07:08:32.332 INFO:tasks.workunit.client.0.smithi081.stdout:8/33: fdatasync f0 0 2024-03-22T07:08:32.332 INFO:tasks.workunit.client.0.smithi081.stdout:8/34: write f0 [3776723,7836] 0 2024-03-22T07:08:32.333 INFO:tasks.workunit.client.0.smithi081.stdout:1/57: mknod d6/c15 0 2024-03-22T07:08:32.333 INFO:tasks.workunit.client.0.smithi081.stdout:1/58: fsync d6/f12 0 2024-03-22T07:08:32.335 INFO:tasks.workunit.client.0.smithi081.stdout:3/54: symlink db/de/l15 0 2024-03-22T07:08:32.337 INFO:tasks.workunit.client.0.smithi081.stdout:0/80: dwrite d3/dc/ff [0,4194304] 0 2024-03-22T07:08:32.339 INFO:tasks.workunit.client.0.smithi081.stdout:3/55: dread db/f12 [0,4194304] 0 2024-03-22T07:08:32.344 INFO:tasks.workunit.client.0.smithi081.stdout:9/51: mknod d2/d5/cf 0 2024-03-22T07:08:32.347 INFO:tasks.workunit.client.0.smithi081.stdout:3/56: dread f1 [0,4194304] 0 2024-03-22T07:08:32.347 INFO:tasks.workunit.client.0.smithi081.stdout:4/126: rename l5 to da/d18/l22 0 2024-03-22T07:08:32.353 INFO:tasks.workunit.client.0.smithi081.stdout:5/67: write d4/fc [5029327,127703] 0 2024-03-22T07:08:32.358 INFO:tasks.workunit.client.0.smithi081.stdout:0/81: rename d3/dc/f11 to d3/dc/f19 0 2024-03-22T07:08:32.359 INFO:tasks.workunit.client.0.smithi081.stdout:0/82: fsync d3/dc/f18 0 2024-03-22T07:08:32.359 INFO:tasks.workunit.client.0.smithi081.stdout:9/52: mkdir d2/d10 0 2024-03-22T07:08:32.359 INFO:tasks.workunit.client.0.smithi081.stdout:7/135: sync 2024-03-22T07:08:32.360 INFO:tasks.workunit.client.0.smithi081.stdout:1/59: dwrite d6/f12 [0,4194304] 0 2024-03-22T07:08:32.361 INFO:tasks.workunit.client.0.smithi081.stdout:7/136: write d9/dc/f1f [1279253,78006] 0 2024-03-22T07:08:32.361 INFO:tasks.workunit.client.0.smithi081.stdout:7/137: write f4 [1190061,112844] 0 2024-03-22T07:08:32.361 INFO:tasks.workunit.client.0.smithi081.stdout:7/138: creat d9/dc/f38 x:0 0 0 2024-03-22T07:08:32.361 INFO:tasks.workunit.client.0.smithi081.stdout:3/57: mknod db/c16 0 2024-03-22T07:08:32.362 INFO:tasks.workunit.client.0.smithi081.stdout:4/127: symlink da/l23 0 2024-03-22T07:08:32.363 INFO:tasks.workunit.client.0.smithi081.stdout:4/128: dread da/dd/f20 [0,4194304] 0 2024-03-22T07:08:32.363 INFO:tasks.workunit.client.0.smithi081.stdout:4/129: truncate da/dd/f20 494429 0 2024-03-22T07:08:32.364 INFO:tasks.workunit.client.0.smithi081.stdout:5/68: symlink d4/d10/l14 0 2024-03-22T07:08:32.368 INFO:tasks.workunit.client.0.smithi081.stdout:9/53: link d2/f8 d2/f11 0 2024-03-22T07:08:32.368 INFO:tasks.workunit.client.0.smithi081.stdout:9/54: creat d2/f12 x:0 0 0 2024-03-22T07:08:32.368 INFO:tasks.workunit.client.0.smithi081.stdout:9/55: creat d2/f13 x:0 0 0 2024-03-22T07:08:32.369 INFO:tasks.workunit.client.0.smithi081.stdout:3/58: read db/fd [641969,68435] 0 2024-03-22T07:08:32.370 INFO:tasks.workunit.client.0.smithi081.stdout:1/60: link d6/l7 d6/l16 0 2024-03-22T07:08:32.373 INFO:tasks.workunit.client.0.smithi081.stdout:4/130: write da/fc [2811102,81385] 0 2024-03-22T07:08:32.374 INFO:tasks.workunit.client.0.smithi081.stdout:7/139: truncate f7 4337022 0 2024-03-22T07:08:32.375 INFO:tasks.workunit.client.0.smithi081.stdout:5/69: creat d4/d9/f15 x:0 0 0 2024-03-22T07:08:32.379 INFO:tasks.workunit.client.0.smithi081.stdout:6/30: sync 2024-03-22T07:08:32.379 INFO:tasks.workunit.client.0.smithi081.stdout:2/30: sync 2024-03-22T07:08:32.379 INFO:tasks.workunit.client.0.smithi081.stdout:6/31: dwrite - no filename 2024-03-22T07:08:32.379 INFO:tasks.workunit.client.0.smithi081.stdout:6/32: write - no filename 2024-03-22T07:08:32.382 INFO:tasks.workunit.client.0.smithi081.stdout:9/56: link d2/d5/fd d2/d10/f14 0 2024-03-22T07:08:32.384 INFO:tasks.workunit.client.0.smithi081.stdout:8/35: link f1 d2/f9 0 2024-03-22T07:08:32.388 INFO:tasks.workunit.client.0.smithi081.stdout:3/59: rename f6 to db/de/f17 0 2024-03-22T07:08:32.388 INFO:tasks.workunit.client.0.smithi081.stdout:3/60: fsync db/fd 0 2024-03-22T07:08:32.390 INFO:tasks.workunit.client.0.smithi081.stdout:9/57: dread d2/f11 [0,4194304] 0 2024-03-22T07:08:32.390 INFO:tasks.workunit.client.0.smithi081.stdout:9/58: truncate d2/f12 422633 0 2024-03-22T07:08:32.390 INFO:tasks.workunit.client.0.smithi081.stdout:9/59: creat d2/f15 x:0 0 0 2024-03-22T07:08:32.393 INFO:tasks.workunit.client.0.smithi081.stdout:6/33: unlink l1 0 2024-03-22T07:08:32.397 INFO:tasks.workunit.client.0.smithi081.stdout:8/36: read f0 [75657,39476] 0 2024-03-22T07:08:32.399 INFO:tasks.workunit.client.0.smithi081.stdout:9/60: write d2/d5/fa [938116,40859] 0 2024-03-22T07:08:32.399 INFO:tasks.workunit.client.0.smithi081.stdout:4/131: dwrite da/fe [0,4194304] 0 2024-03-22T07:08:32.399 INFO:tasks.workunit.client.0.smithi081.stdout:4/132: chown da/l23 395710 1 2024-03-22T07:08:32.401 INFO:tasks.workunit.client.0.smithi081.stdout:0/83: getdents d3/dc 0 2024-03-22T07:08:32.402 INFO:tasks.workunit.client.0.smithi081.stdout:2/31: dread f4 [0,4194304] 0 2024-03-22T07:08:32.405 INFO:tasks.workunit.client.0.smithi081.stdout:7/140: rename d9/le to d9/d10/d16/d1e/l39 0 2024-03-22T07:08:32.405 INFO:tasks.workunit.client.0.smithi081.stdout:7/141: creat d9/d10/f3a x:0 0 0 2024-03-22T07:08:32.406 INFO:tasks.workunit.client.0.smithi081.stdout:3/61: rmdir db/de 39 2024-03-22T07:08:32.413 INFO:tasks.workunit.client.0.smithi081.stdout:1/61: rmdir d6 39 2024-03-22T07:08:32.417 INFO:tasks.workunit.client.0.smithi081.stdout:8/37: symlink d2/d3/la 0 2024-03-22T07:08:32.418 INFO:tasks.workunit.client.0.smithi081.stdout:9/61: mkdir d2/d5/d16 0 2024-03-22T07:08:32.419 INFO:tasks.workunit.client.0.smithi081.stdout:9/62: write d2/f4 [908915,128516] 0 2024-03-22T07:08:32.419 INFO:tasks.workunit.client.0.smithi081.stdout:9/63: read - d2/fe zero size 2024-03-22T07:08:32.421 INFO:tasks.workunit.client.0.smithi081.stdout:2/32: mkdir da/db 0 2024-03-22T07:08:32.423 INFO:tasks.workunit.client.0.smithi081.stdout:4/133: rename da/fc to da/d18/d1b/f24 0 2024-03-22T07:08:32.427 INFO:tasks.workunit.client.0.smithi081.stdout:0/84: dwrite d3/dc/f19 [0,4194304] 0 2024-03-22T07:08:32.427 INFO:tasks.workunit.client.0.smithi081.stdout:3/62: write db/de/f17 [987845,123619] 0 2024-03-22T07:08:32.427 INFO:tasks.workunit.client.0.smithi081.stdout:4/134: read da/dd/f20 [189659,87429] 0 2024-03-22T07:08:32.427 INFO:tasks.workunit.client.0.smithi081.stdout:3/63: creat db/f18 x:0 0 0 2024-03-22T07:08:32.427 INFO:tasks.workunit.client.0.smithi081.stdout:3/64: chown c9 55619014 1 2024-03-22T07:08:32.427 INFO:tasks.workunit.client.0.smithi081.stdout:4/135: creat da/dd/f25 x:0 0 0 2024-03-22T07:08:32.427 INFO:tasks.workunit.client.0.smithi081.stdout:4/136: write da/dd/f20 [564633,96993] 0 2024-03-22T07:08:32.427 INFO:tasks.workunit.client.0.smithi081.stdout:7/142: mknod d9/d14/c3b 0 2024-03-22T07:08:32.429 INFO:tasks.workunit.client.0.smithi081.stdout:1/62: creat d6/f17 x:0 0 0 2024-03-22T07:08:32.429 INFO:tasks.workunit.client.0.smithi081.stdout:1/63: dread - d6/f17 zero size 2024-03-22T07:08:32.429 INFO:tasks.workunit.client.0.smithi081.stdout:7/143: dread d9/dc/f33 [0,4194304] 0 2024-03-22T07:08:32.431 INFO:tasks.workunit.client.0.smithi081.stdout:6/34: unlink c2 0 2024-03-22T07:08:32.435 INFO:tasks.workunit.client.0.smithi081.stdout:8/38: mknod d2/d3/cb 0 2024-03-22T07:08:32.447 INFO:tasks.workunit.client.0.smithi081.stdout:0/85: rename d3/c6 to d3/c1a 0 2024-03-22T07:08:32.447 INFO:tasks.workunit.client.0.smithi081.stdout:0/86: getdents d3/dc 0 2024-03-22T07:08:32.447 INFO:tasks.workunit.client.0.smithi081.stdout:0/87: stat d3/dc/f19 0 2024-03-22T07:08:32.456 INFO:tasks.workunit.client.0.smithi081.stdout:7/144: dwrite d9/dc/f1f [0,4194304] 0 2024-03-22T07:08:32.457 INFO:tasks.workunit.client.0.smithi081.stdout:4/137: unlink da/c10 0 2024-03-22T07:08:32.457 INFO:tasks.workunit.client.0.smithi081.stdout:1/64: dwrite d6/f12 [0,4194304] 0 2024-03-22T07:08:32.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/35: rmdir d3 39 2024-03-22T07:08:32.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/36: fsync - no filename 2024-03-22T07:08:32.463 INFO:tasks.workunit.client.0.smithi081.stdout:4/138: dread f9 [0,4194304] 0 2024-03-22T07:08:32.468 INFO:tasks.workunit.client.0.smithi081.stdout:8/39: dwrite d2/d3/f4 [0,4194304] 0 2024-03-22T07:08:32.468 INFO:tasks.workunit.client.0.smithi081.stdout:0/88: dwrite d3/dc/ff [4194304,4194304] 0 2024-03-22T07:08:32.469 INFO:tasks.workunit.client.0.smithi081.stdout:6/37: symlink d3/d4/l5 0 2024-03-22T07:08:32.471 INFO:tasks.workunit.client.0.smithi081.stdout:4/139: unlink da/l19 0 2024-03-22T07:08:32.474 INFO:tasks.workunit.client.0.smithi081.stdout:7/145: rmdir d9/dc 39 2024-03-22T07:08:32.479 INFO:tasks.workunit.client.0.smithi081.stdout:8/40: mkdir d2/d3/dc 0 2024-03-22T07:08:32.483 INFO:tasks.workunit.client.0.smithi081.stdout:6/38: creat d3/f6 x:0 0 0 2024-03-22T07:08:32.485 INFO:tasks.workunit.client.0.smithi081.stdout:7/146: creat d9/f3c x:0 0 0 2024-03-22T07:08:32.487 INFO:tasks.workunit.client.0.smithi081.stdout:8/41: symlink d2/ld 0 2024-03-22T07:08:32.487 INFO:tasks.workunit.client.0.smithi081.stdout:1/65: dwrite d6/f17 [0,4194304] 0 2024-03-22T07:08:32.489 INFO:tasks.workunit.client.0.smithi081.stdout:6/39: creat d3/d4/f7 x:0 0 0 2024-03-22T07:08:32.489 INFO:tasks.workunit.client.0.smithi081.stdout:6/40: creat d3/d4/f8 x:0 0 0 2024-03-22T07:08:32.489 INFO:tasks.workunit.client.0.smithi081.stdout:6/41: dread - d3/d4/f8 zero size 2024-03-22T07:08:32.489 INFO:tasks.workunit.client.0.smithi081.stdout:0/89: dwrite d3/dc/f17 [0,4194304] 0 2024-03-22T07:08:32.491 INFO:tasks.workunit.client.0.smithi081.stdout:7/147: mknod d9/c3d 0 2024-03-22T07:08:32.493 INFO:tasks.workunit.client.0.smithi081.stdout:8/42: mknod d2/d3/ce 0 2024-03-22T07:08:32.494 INFO:tasks.workunit.client.0.smithi081.stdout:7/148: dread f2 [4194304,4194304] 0 2024-03-22T07:08:32.494 INFO:tasks.workunit.client.0.smithi081.stdout:7/149: readlink d9/d10/d16/d1e/l23 0 2024-03-22T07:08:32.494 INFO:tasks.workunit.client.0.smithi081.stdout:7/150: chown d9/dc 1109652 1 2024-03-22T07:08:32.494 INFO:tasks.workunit.client.0.smithi081.stdout:7/151: write d9/dc/f33 [1419216,118810] 0 2024-03-22T07:08:32.501 INFO:tasks.workunit.client.0.smithi081.stdout:6/42: link d3/d4/f7 d3/f9 0 2024-03-22T07:08:32.502 INFO:tasks.workunit.client.0.smithi081.stdout:6/43: read - d3/f6 zero size 2024-03-22T07:08:32.502 INFO:tasks.workunit.client.0.smithi081.stdout:6/44: write d3/d4/f8 [510598,67087] 0 2024-03-22T07:08:32.502 INFO:tasks.workunit.client.0.smithi081.stdout:1/66: write d6/f17 [839124,99169] 0 2024-03-22T07:08:32.506 INFO:tasks.workunit.client.0.smithi081.stdout:4/140: dwrite da/dd/f20 [0,4194304] 0 2024-03-22T07:08:32.508 INFO:tasks.workunit.client.0.smithi081.stdout:0/90: write d3/dc/f17 [2347601,109730] 0 2024-03-22T07:08:32.508 INFO:tasks.workunit.client.0.smithi081.stdout:0/91: fdatasync d3/dc/ff 0 2024-03-22T07:08:32.508 INFO:tasks.workunit.client.0.smithi081.stdout:0/92: fdatasync d3/dc/fe 0 2024-03-22T07:08:32.539 INFO:tasks.workunit.client.0.smithi081.stdout:8/43: rename d2/ld to d2/d3/lf 0 2024-03-22T07:08:32.542 INFO:tasks.workunit.client.0.smithi081.stdout:7/152: symlink d9/d10/d16/d1e/d27/l3e 0 2024-03-22T07:08:32.546 INFO:tasks.workunit.client.0.smithi081.stdout:1/67: mkdir d6/d18 0 2024-03-22T07:08:32.556 INFO:tasks.workunit.client.0.smithi081.stdout:4/141: mkdir da/d18/d1b/d26 0 2024-03-22T07:08:32.557 INFO:tasks.workunit.client.0.smithi081.stdout:7/153: write d9/dc/f1a [2680084,17324] 0 2024-03-22T07:08:32.560 INFO:tasks.workunit.client.0.smithi081.stdout:0/93: truncate d3/f8 1581449 0 2024-03-22T07:08:32.560 INFO:tasks.workunit.client.0.smithi081.stdout:0/94: chown f2 11 1 2024-03-22T07:08:32.560 INFO:tasks.workunit.client.0.smithi081.stdout:0/95: dread - d3/dc/f15 zero size 2024-03-22T07:08:32.575 INFO:tasks.workunit.client.0.smithi081.stdout:8/44: mkdir d2/d3/d10 0 2024-03-22T07:08:32.579 INFO:tasks.workunit.client.0.smithi081.stdout:1/68: dwrite d6/fd [0,4194304] 0 2024-03-22T07:08:32.584 INFO:tasks.workunit.client.0.smithi081.stdout:4/142: mkdir da/dd/d1c/d1f/d27 0 2024-03-22T07:08:32.585 INFO:tasks.workunit.client.0.smithi081.stdout:7/154: dwrite d9/d10/d16/d1e/f18 [0,4194304] 0 2024-03-22T07:08:32.586 INFO:tasks.workunit.client.0.smithi081.stdout:0/96: dwrite d3/dc/fd [0,4194304] 0 2024-03-22T07:08:32.591 INFO:tasks.workunit.client.0.smithi081.stdout:8/45: rename d2/f6 to d2/f11 0 2024-03-22T07:08:32.591 INFO:tasks.workunit.client.0.smithi081.stdout:8/46: fdatasync f1 0 2024-03-22T07:08:32.596 INFO:tasks.workunit.client.0.smithi081.stdout:1/69: mknod d6/c19 0 2024-03-22T07:08:32.597 INFO:tasks.workunit.client.0.smithi081.stdout:4/143: dread da/fe [0,4194304] 0 2024-03-22T07:08:32.597 INFO:tasks.workunit.client.0.smithi081.stdout:4/144: fdatasync da/fb 0 2024-03-22T07:08:32.601 INFO:tasks.workunit.client.0.smithi081.stdout:7/155: dwrite d9/d10/f3a [0,4194304] 0 2024-03-22T07:08:32.611 INFO:tasks.workunit.client.0.smithi081.stdout:0/97: rename d3/dc/fd to d3/dc/f1b 0 2024-03-22T07:08:32.613 INFO:tasks.workunit.client.0.smithi081.stdout:8/47: mknod d2/d3/dc/c12 0 2024-03-22T07:08:32.616 INFO:tasks.workunit.client.0.smithi081.stdout:1/70: mknod d6/c1a 0 2024-03-22T07:08:32.616 INFO:tasks.workunit.client.0.smithi081.stdout:1/71: fdatasync d6/fc 0 2024-03-22T07:08:32.618 INFO:tasks.workunit.client.0.smithi081.stdout:4/145: creat da/dd/d1c/f28 x:0 0 0 2024-03-22T07:08:32.627 INFO:tasks.workunit.client.0.smithi081.stdout:8/48: truncate d2/f9 552594 0 2024-03-22T07:08:32.629 INFO:tasks.workunit.client.0.smithi081.stdout:1/72: mknod d6/d18/c1b 0 2024-03-22T07:08:32.634 INFO:tasks.workunit.client.0.smithi081.stdout:0/98: dwrite d3/dc/f18 [0,4194304] 0 2024-03-22T07:08:32.645 INFO:tasks.workunit.client.0.smithi081.stdout:4/146: dwrite da/d18/d1b/f24 [0,4194304] 0 2024-03-22T07:08:32.655 INFO:tasks.workunit.client.0.smithi081.stdout:8/49: mkdir d2/d3/d10/d13 0 2024-03-22T07:08:32.739 INFO:tasks.workunit.client.0.smithi081.stdout:8/50: dwrite d2/f9 [0,4194304] 0 2024-03-22T07:08:32.749 INFO:tasks.workunit.client.0.smithi081.stdout:8/51: dread f1 [0,4194304] 0 2024-03-22T07:08:32.751 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:32 smithi149 ceph-mon[36068]: pgmap v239: 65 pgs: 65 active+clean; 188 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 735 op/s 2024-03-22T07:08:32.756 INFO:tasks.workunit.client.0.smithi081.stdout:8/52: dread d2/f9 [0,4194304] 0 2024-03-22T07:08:32.763 INFO:tasks.workunit.client.0.smithi081.stdout:8/53: dread d2/f9 [0,4194304] 0 2024-03-22T07:08:32.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:32 smithi081 ceph-mon[29245]: pgmap v239: 65 pgs: 65 active+clean; 188 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 735 op/s 2024-03-22T07:08:34.668 INFO:tasks.workunit.client.0.smithi081.stdout:8/54: mknod d2/d3/d10/c14 0 2024-03-22T07:08:34.668 INFO:tasks.workunit.client.0.smithi081.stdout:1/73: symlink d6/l1c 0 2024-03-22T07:08:34.669 INFO:tasks.workunit.client.0.smithi081.stdout:0/99: creat d3/dc/f1c x:0 0 0 2024-03-22T07:08:34.669 INFO:tasks.workunit.client.0.smithi081.stdout:0/100: creat d3/f1d x:0 0 0 2024-03-22T07:08:34.670 INFO:tasks.workunit.client.0.smithi081.stdout:0/101: symlink d3/dc/l1e 0 2024-03-22T07:08:34.671 INFO:tasks.workunit.client.0.smithi081.stdout:0/102: write d3/f4 [5158235,87671] 0 2024-03-22T07:08:34.673 INFO:tasks.workunit.client.0.smithi081.stdout:0/103: rename d3/dc/f18 to d3/dc/f1f 0 2024-03-22T07:08:34.676 INFO:tasks.workunit.client.0.smithi081.stdout:1/74: creat d6/d18/f1d x:0 0 0 2024-03-22T07:08:34.677 INFO:tasks.workunit.client.0.smithi081.stdout:0/104: dread f2 [0,4194304] 0 2024-03-22T07:08:34.680 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:34 smithi149 ceph-mon[36068]: pgmap v240: 65 pgs: 65 active+clean; 240 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 10 MiB/s rd, 23 MiB/s wr, 758 op/s 2024-03-22T07:08:34.681 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:34 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:34.681 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:34 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:34.681 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:34 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:34.681 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:34 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:34.689 INFO:tasks.workunit.client.0.smithi081.stdout:8/55: dwrite d2/f9 [0,4194304] 0 2024-03-22T07:08:34.691 INFO:tasks.workunit.client.0.smithi081.stdout:8/56: creat d2/f15 x:0 0 0 2024-03-22T07:08:34.702 INFO:tasks.workunit.client.0.smithi081.stdout:8/57: dread f1 [0,4194304] 0 2024-03-22T07:08:34.704 INFO:tasks.workunit.client.0.smithi081.stdout:8/58: creat d2/d3/f16 x:0 0 0 2024-03-22T07:08:34.707 INFO:tasks.workunit.client.0.smithi081.stdout:8/59: symlink d2/d3/d10/l17 0 2024-03-22T07:08:34.708 INFO:tasks.workunit.client.0.smithi081.stdout:0/105: dwrite d3/dc/fe [0,4194304] 0 2024-03-22T07:08:34.708 INFO:tasks.workunit.client.0.smithi081.stdout:0/106: stat d3/f10 0 2024-03-22T07:08:34.708 INFO:tasks.workunit.client.0.smithi081.stdout:0/107: chown d3/dc/f17 7 1 2024-03-22T07:08:34.717 INFO:tasks.workunit.client.0.smithi081.stdout:0/108: rename d3/dc/f17 to d3/f20 0 2024-03-22T07:08:34.722 INFO:tasks.workunit.client.0.smithi081.stdout:0/109: getdents d3 0 2024-03-22T07:08:34.729 INFO:tasks.workunit.client.0.smithi081.stdout:0/110: symlink d3/l21 0 2024-03-22T07:08:34.748 INFO:tasks.workunit.client.0.smithi081.stdout:0/111: dwrite d3/f7 [4194304,4194304] 0 2024-03-22T07:08:34.748 INFO:tasks.workunit.client.0.smithi081.stdout:0/112: read d3/f7 [2442084,21531] 0 2024-03-22T07:08:34.750 INFO:tasks.workunit.client.0.smithi081.stdout:0/113: creat d3/f22 x:0 0 0 2024-03-22T07:08:34.761 INFO:tasks.workunit.client.0.smithi081.stdout:0/114: unlink d3/f20 0 2024-03-22T07:08:34.775 INFO:tasks.workunit.client.0.smithi081.stdout:0/115: write d3/f4 [2369945,6427] 0 2024-03-22T07:08:34.781 INFO:tasks.workunit.client.0.smithi081.stdout:0/116: truncate d3/dc/fe 3671183 0 2024-03-22T07:08:34.803 INFO:tasks.workunit.client.0.smithi081.stdout:0/117: dwrite d3/f13 [4194304,4194304] 0 2024-03-22T07:08:34.804 INFO:tasks.workunit.client.0.smithi081.stdout:0/118: chown d3/l9 1129 1 2024-03-22T07:08:34.809 INFO:tasks.workunit.client.0.smithi081.stdout:0/119: link d3/l9 d3/l23 0 2024-03-22T07:08:34.809 INFO:tasks.workunit.client.0.smithi081.stdout:0/120: creat d3/f24 x:0 0 0 2024-03-22T07:08:34.815 INFO:tasks.workunit.client.0.smithi081.stdout:0/121: dread d3/f8 [0,4194304] 0 2024-03-22T07:08:34.817 INFO:tasks.workunit.client.0.smithi081.stdout:0/122: rename d3/c1a to d3/c25 0 2024-03-22T07:08:34.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:34 smithi081 ceph-mon[29245]: pgmap v240: 65 pgs: 65 active+clean; 240 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 10 MiB/s rd, 23 MiB/s wr, 758 op/s 2024-03-22T07:08:34.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:34 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:34.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:34 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:34.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:34 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:34.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:34 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:34.825 INFO:tasks.workunit.client.0.smithi081.stdout:0/123: dread d3/f4 [0,4194304] 0 2024-03-22T07:08:34.826 INFO:tasks.workunit.client.0.smithi081.stdout:0/124: fsync d3/dc/f1f 0 2024-03-22T07:08:34.831 INFO:tasks.workunit.client.0.smithi081.stdout:0/125: dread d3/f13 [0,4194304] 0 2024-03-22T07:08:34.831 INFO:tasks.workunit.client.0.smithi081.stdout:0/126: write d3/dc/f1c [992044,41959] 0 2024-03-22T07:08:34.871 INFO:tasks.workunit.client.0.smithi081.stdout:1/75: getdents d6/d18 0 2024-03-22T07:08:34.871 INFO:tasks.workunit.client.0.smithi081.stdout:1/76: truncate d6/f9 753981 0 2024-03-22T07:08:34.872 INFO:tasks.workunit.client.0.smithi081.stdout:1/77: truncate d6/f12 2840162 0 2024-03-22T07:08:34.890 INFO:tasks.workunit.client.0.smithi081.stdout:1/78: dwrite d6/fb [0,4194304] 0 2024-03-22T07:08:34.933 INFO:tasks.workunit.client.0.smithi081.stdout:1/79: dread f1 [4194304,4194304] 0 2024-03-22T07:08:34.933 INFO:tasks.workunit.client.0.smithi081.stdout:1/80: readlink d6/l7 0 2024-03-22T07:08:35.015 INFO:tasks.workunit.client.0.smithi081.stdout:8/60: link d2/d3/d10/c14 d2/d3/c18 0 2024-03-22T07:08:35.015 INFO:tasks.workunit.client.0.smithi081.stdout:8/61: chown f0 24681813 1 2024-03-22T07:08:35.015 INFO:tasks.workunit.client.0.smithi081.stdout:8/62: read - d2/f15 zero size 2024-03-22T07:08:35.015 INFO:tasks.workunit.client.0.smithi081.stdout:8/63: fdatasync f1 0 2024-03-22T07:08:35.015 INFO:tasks.workunit.client.0.smithi081.stdout:8/64: chown d2/d3 5221 1 2024-03-22T07:08:35.016 INFO:tasks.workunit.client.0.smithi081.stdout:8/65: mkdir d2/d3/d19 0 2024-03-22T07:08:35.016 INFO:tasks.workunit.client.0.smithi081.stdout:8/66: chown d2/c5 27652216 1 2024-03-22T07:08:35.017 INFO:tasks.workunit.client.0.smithi081.stdout:4/147: link l4 da/d18/d1b/d26/l29 0 2024-03-22T07:08:35.019 INFO:tasks.workunit.client.0.smithi081.stdout:4/148: creat da/d18/d1b/d26/f2a x:0 0 0 2024-03-22T07:08:35.023 INFO:tasks.workunit.client.0.smithi081.stdout:4/149: dread f0 [0,4194304] 0 2024-03-22T07:08:35.097 INFO:tasks.workunit.client.0.smithi081.stdout:1/81: rename d6/c13 to d6/d18/c1e 0 2024-03-22T07:08:35.097 INFO:tasks.workunit.client.0.smithi081.stdout:1/82: chown d6/fd 17 1 2024-03-22T07:08:35.097 INFO:tasks.workunit.client.0.smithi081.stdout:4/150: rename da to da/d18/d1b/d26/d2b 22 2024-03-22T07:08:35.098 INFO:tasks.workunit.client.0.smithi081.stdout:4/151: truncate da/d18/d1b/f24 377183 0 2024-03-22T07:08:35.099 INFO:tasks.workunit.client.0.smithi081.stdout:4/152: creat da/f2c x:0 0 0 2024-03-22T07:08:35.100 INFO:tasks.workunit.client.0.smithi081.stdout:4/153: write da/d18/f21 [694113,70693] 0 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:6/45: sync 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:5/70: sync 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:9/64: sync 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:2/33: sync 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:3/65: sync 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:9/65: chown d2/d5/fa 205839314 1 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:5/71: dread - d4/d9/f15 zero size 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:5/72: chown d4/c5 17410097 1 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:5/73: chown d4/c5 11 1 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:7/156: sync 2024-03-22T07:08:35.101 INFO:tasks.workunit.client.0.smithi081.stdout:1/83: mkdir d6/d18/d1f 0 2024-03-22T07:08:35.102 INFO:tasks.workunit.client.0.smithi081.stdout:1/84: dread d6/f9 [0,4194304] 0 2024-03-22T07:08:35.104 INFO:tasks.workunit.client.0.smithi081.stdout:3/66: getdents db/de 0 2024-03-22T07:08:35.104 INFO:tasks.workunit.client.0.smithi081.stdout:3/67: chown l8 154 1 2024-03-22T07:08:35.104 INFO:tasks.workunit.client.0.smithi081.stdout:9/66: dread d2/f11 [0,4194304] 0 2024-03-22T07:08:35.108 INFO:tasks.workunit.client.0.smithi081.stdout:3/68: dread db/fd [0,4194304] 0 2024-03-22T07:08:35.109 INFO:tasks.workunit.client.0.smithi081.stdout:9/67: rename d2/d5/fa to d2/d10/f17 0 2024-03-22T07:08:35.109 INFO:tasks.workunit.client.0.smithi081.stdout:2/34: dread f3 [0,4194304] 0 2024-03-22T07:08:35.115 INFO:tasks.workunit.client.0.smithi081.stdout:1/85: dread d6/f17 [0,4194304] 0 2024-03-22T07:08:35.117 INFO:tasks.workunit.client.0.smithi081.stdout:7/157: unlink d9/c3d 0 2024-03-22T07:08:35.117 INFO:tasks.workunit.client.0.smithi081.stdout:7/158: chown d9/c34 1921 1 2024-03-22T07:08:35.118 INFO:tasks.workunit.client.0.smithi081.stdout:1/86: rename d6/c1a to d6/c20 0 2024-03-22T07:08:35.119 INFO:tasks.workunit.client.0.smithi081.stdout:1/87: chown d6/l14 134564 1 2024-03-22T07:08:35.120 INFO:tasks.workunit.client.0.smithi081.stdout:6/46: rmdir d3 39 2024-03-22T07:08:35.120 INFO:tasks.workunit.client.0.smithi081.stdout:6/47: truncate d3/d4/f7 466253 0 2024-03-22T07:08:35.121 INFO:tasks.workunit.client.0.smithi081.stdout:7/159: mkdir d9/d3f 0 2024-03-22T07:08:35.121 INFO:tasks.workunit.client.0.smithi081.stdout:5/74: rmdir d4 39 2024-03-22T07:08:35.123 INFO:tasks.workunit.client.0.smithi081.stdout:7/160: symlink d9/d10/d2a/l40 0 2024-03-22T07:08:35.126 INFO:tasks.workunit.client.0.smithi081.stdout:7/161: mkdir d9/dc/d41 0 2024-03-22T07:08:35.127 INFO:tasks.workunit.client.0.smithi081.stdout:7/162: creat d9/d10/d16/f42 x:0 0 0 2024-03-22T07:08:35.127 INFO:tasks.workunit.client.0.smithi081.stdout:7/163: write d9/dc/f38 [337606,6037] 0 2024-03-22T07:08:35.128 INFO:tasks.workunit.client.0.smithi081.stdout:7/164: truncate f4 2412634 0 2024-03-22T07:08:35.131 INFO:tasks.workunit.client.0.smithi081.stdout:6/48: write d3/d4/f8 [122445,55351] 0 2024-03-22T07:08:35.132 INFO:tasks.workunit.client.0.smithi081.stdout:4/154: dwrite da/fe [0,4194304] 0 2024-03-22T07:08:35.132 INFO:tasks.workunit.client.0.smithi081.stdout:6/49: mkdir d3/da 0 2024-03-22T07:08:35.133 INFO:tasks.workunit.client.0.smithi081.stdout:4/155: mknod da/c2d 0 2024-03-22T07:08:35.134 INFO:tasks.workunit.client.0.smithi081.stdout:6/50: creat d3/da/fb x:0 0 0 2024-03-22T07:08:35.135 INFO:tasks.workunit.client.0.smithi081.stdout:4/156: symlink da/l2e 0 2024-03-22T07:08:35.136 INFO:tasks.workunit.client.0.smithi081.stdout:6/51: creat d3/d4/fc x:0 0 0 2024-03-22T07:08:35.136 INFO:tasks.workunit.client.0.smithi081.stdout:4/157: truncate f9 1957944 0 2024-03-22T07:08:35.137 INFO:tasks.workunit.client.0.smithi081.stdout:2/35: dwrite f3 [0,4194304] 0 2024-03-22T07:08:35.138 INFO:tasks.workunit.client.0.smithi081.stdout:4/158: symlink da/dd/l2f 0 2024-03-22T07:08:35.138 INFO:tasks.workunit.client.0.smithi081.stdout:4/159: fdatasync da/dd/f25 0 2024-03-22T07:08:35.139 INFO:tasks.workunit.client.0.smithi081.stdout:4/160: mknod da/c30 0 2024-03-22T07:08:35.140 INFO:tasks.workunit.client.0.smithi081.stdout:4/161: creat da/d18/f31 x:0 0 0 2024-03-22T07:08:35.141 INFO:tasks.workunit.client.0.smithi081.stdout:4/162: chown da/df/l16 1397 1 2024-03-22T07:08:35.141 INFO:tasks.workunit.client.0.smithi081.stdout:4/163: dread - da/dd/d1c/f28 zero size 2024-03-22T07:08:35.149 INFO:tasks.workunit.client.0.smithi081.stdout:7/165: read f4 [748560,64548] 0 2024-03-22T07:08:35.149 INFO:tasks.workunit.client.0.smithi081.stdout:2/36: write f3 [3210085,57707] 0 2024-03-22T07:08:35.150 INFO:tasks.workunit.client.0.smithi081.stdout:6/52: getdents d3 0 2024-03-22T07:08:35.150 INFO:tasks.workunit.client.0.smithi081.stdout:6/53: creat d3/da/fd x:0 0 0 2024-03-22T07:08:35.150 INFO:tasks.workunit.client.0.smithi081.stdout:7/166: read d9/dc/f33 [620971,51572] 0 2024-03-22T07:08:35.153 INFO:tasks.workunit.client.0.smithi081.stdout:6/54: mkdir d3/da/de 0 2024-03-22T07:08:35.153 INFO:tasks.workunit.client.0.smithi081.stdout:6/55: chown d3 1 1 2024-03-22T07:08:35.154 INFO:tasks.workunit.client.0.smithi081.stdout:9/68: dwrite d2/d5/f6 [0,4194304] 0 2024-03-22T07:08:35.154 INFO:tasks.workunit.client.0.smithi081.stdout:9/69: chown d2/d5/fd 364831 1 2024-03-22T07:08:35.155 INFO:tasks.workunit.client.0.smithi081.stdout:7/167: symlink d9/d10/d2a/l43 0 2024-03-22T07:08:35.155 INFO:tasks.workunit.client.0.smithi081.stdout:3/69: dwrite db/f12 [0,4194304] 0 2024-03-22T07:08:35.155 INFO:tasks.workunit.client.0.smithi081.stdout:7/168: chown d9/d14/c30 1 1 2024-03-22T07:08:35.155 INFO:tasks.workunit.client.0.smithi081.stdout:3/70: stat db/de/l15 0 2024-03-22T07:08:35.156 INFO:tasks.workunit.client.0.smithi081.stdout:6/56: mkdir d3/da/de/df 0 2024-03-22T07:08:35.159 INFO:tasks.workunit.client.0.smithi081.stdout:9/70: link d2/d5/f6 d2/d5/d16/f18 0 2024-03-22T07:08:35.160 INFO:tasks.workunit.client.0.smithi081.stdout:9/71: dread d2/f12 [0,4194304] 0 2024-03-22T07:08:35.161 INFO:tasks.workunit.client.0.smithi081.stdout:9/72: dread d2/f3 [0,4194304] 0 2024-03-22T07:08:35.170 INFO:tasks.workunit.client.0.smithi081.stdout:6/57: symlink d3/d4/l10 0 2024-03-22T07:08:35.170 INFO:tasks.workunit.client.0.smithi081.stdout:6/58: chown d3/da 41011968 1 2024-03-22T07:08:35.170 INFO:tasks.workunit.client.0.smithi081.stdout:6/59: write d3/d4/f8 [1058127,45289] 0 2024-03-22T07:08:35.170 INFO:tasks.workunit.client.0.smithi081.stdout:6/60: dread - d3/f6 zero size 2024-03-22T07:08:35.172 INFO:tasks.workunit.client.0.smithi081.stdout:6/61: fdatasync d3/d4/f8 0 2024-03-22T07:08:35.173 INFO:tasks.workunit.client.0.smithi081.stdout:9/73: rmdir d2/d5/d16 39 2024-03-22T07:08:35.173 INFO:tasks.workunit.client.0.smithi081.stdout:7/169: rmdir d9/d10/d16/d1e 39 2024-03-22T07:08:35.176 INFO:tasks.workunit.client.0.smithi081.stdout:9/74: symlink d2/d10/l19 0 2024-03-22T07:08:35.176 INFO:tasks.workunit.client.0.smithi081.stdout:9/75: write d2/d5/fd [580921,49229] 0 2024-03-22T07:08:35.178 INFO:tasks.workunit.client.0.smithi081.stdout:7/170: link d9/d10/d16/l2b d9/l44 0 2024-03-22T07:08:35.178 INFO:tasks.workunit.client.0.smithi081.stdout:7/171: fdatasync d9/dc/f38 0 2024-03-22T07:08:35.180 INFO:tasks.workunit.client.0.smithi081.stdout:9/76: symlink d2/d5/d16/l1a 0 2024-03-22T07:08:35.182 INFO:tasks.workunit.client.0.smithi081.stdout:7/172: symlink d9/d10/d16/l45 0 2024-03-22T07:08:35.182 INFO:tasks.workunit.client.0.smithi081.stdout:7/173: readlink d9/d10/d2a/l36 0 2024-03-22T07:08:35.184 INFO:tasks.workunit.client.0.smithi081.stdout:6/62: truncate d3/d4/f8 1052832 0 2024-03-22T07:08:35.184 INFO:tasks.workunit.client.0.smithi081.stdout:6/63: write d3/da/fd [718947,22981] 0 2024-03-22T07:08:35.186 INFO:tasks.workunit.client.0.smithi081.stdout:6/64: creat d3/da/de/f11 x:0 0 0 2024-03-22T07:08:35.189 INFO:tasks.workunit.client.0.smithi081.stdout:6/65: mkdir d3/d12 0 2024-03-22T07:08:35.189 INFO:tasks.workunit.client.0.smithi081.stdout:6/66: truncate d3/d4/fc 576548 0 2024-03-22T07:08:35.189 INFO:tasks.workunit.client.0.smithi081.stdout:6/67: fdatasync d3/d4/fc 0 2024-03-22T07:08:35.189 INFO:tasks.workunit.client.0.smithi081.stdout:6/68: chown d3/da/de/f11 2118119 1 2024-03-22T07:08:35.189 INFO:tasks.workunit.client.0.smithi081.stdout:6/69: chown d3/da/de 37895248 1 2024-03-22T07:08:35.189 INFO:tasks.workunit.client.0.smithi081.stdout:6/70: write d3/d4/fc [1086208,4521] 0 2024-03-22T07:08:35.191 INFO:tasks.workunit.client.0.smithi081.stdout:6/71: link d3/d4/l5 d3/da/l13 0 2024-03-22T07:08:35.191 INFO:tasks.workunit.client.0.smithi081.stdout:6/72: creat d3/da/f14 x:0 0 0 2024-03-22T07:08:35.210 INFO:tasks.workunit.client.0.smithi081.stdout:9/77: dwrite d2/d10/f17 [0,4194304] 0 2024-03-22T07:08:35.220 INFO:tasks.workunit.client.0.smithi081.stdout:9/78: write d2/d10/f17 [2235540,32562] 0 2024-03-22T07:08:35.240 INFO:tasks.workunit.client.0.smithi081.stdout:9/79: dwrite d2/f3 [0,4194304] 0 2024-03-22T07:08:35.240 INFO:tasks.workunit.client.0.smithi081.stdout:9/80: truncate d2/f15 996416 0 2024-03-22T07:08:35.278 INFO:tasks.workunit.client.0.smithi081.stdout:8/67: link d2/d3/c18 d2/d3/d19/c1a 0 2024-03-22T07:08:35.284 INFO:tasks.workunit.client.0.smithi081.stdout:1/88: rename d6/f9 to d6/d18/f21 0 2024-03-22T07:08:35.286 INFO:tasks.workunit.client.0.smithi081.stdout:5/75: rename d4/d10/c13 to d4/df/c16 0 2024-03-22T07:08:35.287 INFO:tasks.workunit.client.0.smithi081.stdout:8/68: dread f0 [0,4194304] 0 2024-03-22T07:08:35.287 INFO:tasks.workunit.client.0.smithi081.stdout:8/69: creat d2/d3/f1b x:0 0 0 2024-03-22T07:08:35.287 INFO:tasks.workunit.client.0.smithi081.stdout:8/70: readlink d2/d3/l7 0 2024-03-22T07:08:35.287 INFO:tasks.workunit.client.0.smithi081.stdout:8/71: fdatasync d2/f9 0 2024-03-22T07:08:35.288 INFO:tasks.workunit.client.0.smithi081.stdout:5/76: symlink d4/d9/l17 0 2024-03-22T07:08:35.288 INFO:tasks.workunit.client.0.smithi081.stdout:5/77: read - d4/d9/f15 zero size 2024-03-22T07:08:35.291 INFO:tasks.workunit.client.0.smithi081.stdout:5/78: link d4/d10/l14 d4/df/l18 0 2024-03-22T07:08:35.292 INFO:tasks.workunit.client.0.smithi081.stdout:5/79: readlink d4/d10/l14 0 2024-03-22T07:08:35.294 INFO:tasks.workunit.client.0.smithi081.stdout:4/164: rename da/dd/d1c/d1f/d27 to da/d18/d1b/d26/d32 0 2024-03-22T07:08:35.295 INFO:tasks.workunit.client.0.smithi081.stdout:7/174: rename d9 to d9/dc/d41/d46 22 2024-03-22T07:08:35.295 INFO:tasks.workunit.client.0.smithi081.stdout:5/80: mkdir d4/df/d19 0 2024-03-22T07:08:35.298 INFO:tasks.workunit.client.0.smithi081.stdout:4/165: symlink da/dd/d1c/d1f/l33 0 2024-03-22T07:08:35.298 INFO:tasks.workunit.client.0.smithi081.stdout:4/166: chown l4 72120007 1 2024-03-22T07:08:35.298 INFO:tasks.workunit.client.0.smithi081.stdout:4/167: chown l2 1602 1 2024-03-22T07:08:35.299 INFO:tasks.workunit.client.0.smithi081.stdout:5/81: truncate d4/fe 1084523 0 2024-03-22T07:08:35.301 INFO:tasks.workunit.client.0.smithi081.stdout:5/82: link f3 d4/df/f1a 0 2024-03-22T07:08:35.302 INFO:tasks.workunit.client.0.smithi081.stdout:7/175: rmdir d9/d10/d16/d1e 39 2024-03-22T07:08:35.302 INFO:tasks.workunit.client.0.smithi081.stdout:7/176: chown f7 85 1 2024-03-22T07:08:35.304 INFO:tasks.workunit.client.0.smithi081.stdout:7/177: creat d9/d10/d2a/f47 x:0 0 0 2024-03-22T07:08:35.304 INFO:tasks.workunit.client.0.smithi081.stdout:5/83: dread f1 [0,4194304] 0 2024-03-22T07:08:35.305 INFO:tasks.workunit.client.0.smithi081.stdout:5/84: creat d4/d9/f1b x:0 0 0 2024-03-22T07:08:35.306 INFO:tasks.workunit.client.0.smithi081.stdout:1/89: dwrite f1 [0,4194304] 0 2024-03-22T07:08:35.306 INFO:tasks.workunit.client.0.smithi081.stdout:5/85: link d4/d9/l17 d4/d9/l1c 0 2024-03-22T07:08:35.307 INFO:tasks.workunit.client.0.smithi081.stdout:6/73: rename d3/d4/f8 to d3/da/de/df/f15 0 2024-03-22T07:08:35.307 INFO:tasks.workunit.client.0.smithi081.stdout:7/178: dread f2 [0,4194304] 0 2024-03-22T07:08:35.307 INFO:tasks.workunit.client.0.smithi081.stdout:8/72: dwrite d2/f9 [4194304,4194304] 0 2024-03-22T07:08:35.308 INFO:tasks.workunit.client.0.smithi081.stdout:5/86: creat d4/d10/f1d x:0 0 0 2024-03-22T07:08:35.309 INFO:tasks.workunit.client.0.smithi081.stdout:1/90: mkdir d6/d18/d1f/d22 0 2024-03-22T07:08:35.309 INFO:tasks.workunit.client.0.smithi081.stdout:1/91: chown d6/d18 842544376 1 2024-03-22T07:08:35.310 INFO:tasks.workunit.client.0.smithi081.stdout:6/74: mkdir d3/d16 0 2024-03-22T07:08:35.310 INFO:tasks.workunit.client.0.smithi081.stdout:6/75: creat d3/da/f17 x:0 0 0 2024-03-22T07:08:35.311 INFO:tasks.workunit.client.0.smithi081.stdout:1/92: mkdir d6/d18/d1f/d23 0 2024-03-22T07:08:35.312 INFO:tasks.workunit.client.0.smithi081.stdout:1/93: fdatasync d6/d18/f1d 0 2024-03-22T07:08:35.317 INFO:tasks.workunit.client.0.smithi081.stdout:1/94: mkdir d6/d18/d1f/d24 0 2024-03-22T07:08:35.321 INFO:tasks.workunit.client.0.smithi081.stdout:5/87: write d4/d9/f11 [174121,76749] 0 2024-03-22T07:08:35.321 INFO:tasks.workunit.client.0.smithi081.stdout:5/88: fdatasync d4/d10/f1d 0 2024-03-22T07:08:35.321 INFO:tasks.workunit.client.0.smithi081.stdout:1/95: creat d6/d18/d1f/f25 x:0 0 0 2024-03-22T07:08:35.323 INFO:tasks.workunit.client.0.smithi081.stdout:1/96: creat d6/d18/d1f/d24/f26 x:0 0 0 2024-03-22T07:08:35.328 INFO:tasks.workunit.client.0.smithi081.stdout:7/179: dwrite d9/f3c [0,4194304] 0 2024-03-22T07:08:35.333 INFO:tasks.workunit.client.0.smithi081.stdout:7/180: getdents d9/d10/d2a 0 2024-03-22T07:08:35.336 INFO:tasks.workunit.client.0.smithi081.stdout:1/97: dread d6/f17 [0,4194304] 0 2024-03-22T07:08:35.340 INFO:tasks.workunit.client.0.smithi081.stdout:4/168: rename da/dd/d1c/d1f to da/d18/d1b/d34 0 2024-03-22T07:08:35.342 INFO:tasks.workunit.client.0.smithi081.stdout:5/89: rmdir d4/d9 39 2024-03-22T07:08:35.342 INFO:tasks.workunit.client.0.smithi081.stdout:5/90: write d4/fe [1360464,15624] 0 2024-03-22T07:08:35.342 INFO:tasks.workunit.client.0.smithi081.stdout:5/91: write f3 [3388240,14579] 0 2024-03-22T07:08:35.342 INFO:tasks.workunit.client.0.smithi081.stdout:5/92: read f2 [325645,37727] 0 2024-03-22T07:08:35.343 INFO:tasks.workunit.client.0.smithi081.stdout:6/76: dwrite d3/da/de/df/f15 [0,4194304] 0 2024-03-22T07:08:35.344 INFO:tasks.workunit.client.0.smithi081.stdout:4/169: truncate f9 1445248 0 2024-03-22T07:08:35.345 INFO:tasks.workunit.client.0.smithi081.stdout:5/93: symlink d4/d10/l1e 0 2024-03-22T07:08:35.347 INFO:tasks.workunit.client.0.smithi081.stdout:4/170: symlink da/df/l35 0 2024-03-22T07:08:35.351 INFO:tasks.workunit.client.0.smithi081.stdout:5/94: fdatasync d4/df/f1a 0 2024-03-22T07:08:35.352 INFO:tasks.workunit.client.0.smithi081.stdout:8/73: rename d2/f9 to d2/d3/f1c 0 2024-03-22T07:08:35.354 INFO:tasks.workunit.client.0.smithi081.stdout:5/95: mkdir d4/df/d19/d1f 0 2024-03-22T07:08:35.356 INFO:tasks.workunit.client.0.smithi081.stdout:7/181: dwrite d9/dc/f38 [0,4194304] 0 2024-03-22T07:08:35.357 INFO:tasks.workunit.client.0.smithi081.stdout:7/182: read d9/dc/f33 [1396892,71131] 0 2024-03-22T07:08:35.361 INFO:tasks.workunit.client.0.smithi081.stdout:7/183: symlink d9/d10/d16/l48 0 2024-03-22T07:08:35.366 INFO:tasks.workunit.client.0.smithi081.stdout:7/184: mknod d9/dc/d41/c49 0 2024-03-22T07:08:35.367 INFO:tasks.workunit.client.0.smithi081.stdout:4/171: dread da/dd/f20 [0,4194304] 0 2024-03-22T07:08:35.371 INFO:tasks.workunit.client.0.smithi081.stdout:7/185: creat d9/dc/f4a x:0 0 0 2024-03-22T07:08:35.375 INFO:tasks.workunit.client.0.smithi081.stdout:4/172: creat da/dd/f36 x:0 0 0 2024-03-22T07:08:35.375 INFO:tasks.workunit.client.0.smithi081.stdout:5/96: rmdir d4 39 2024-03-22T07:08:35.376 INFO:tasks.workunit.client.0.smithi081.stdout:7/186: mkdir d9/d10/d4b 0 2024-03-22T07:08:35.378 INFO:tasks.workunit.client.0.smithi081.stdout:4/173: truncate da/dd/f20 3785558 0 2024-03-22T07:08:35.381 INFO:tasks.workunit.client.0.smithi081.stdout:5/97: rmdir d4 39 2024-03-22T07:08:35.381 INFO:tasks.workunit.client.0.smithi081.stdout:8/74: dwrite d2/f15 [0,4194304] 0 2024-03-22T07:08:35.383 INFO:tasks.workunit.client.0.smithi081.stdout:6/77: dwrite d3/da/de/df/f15 [4194304,4194304] 0 2024-03-22T07:08:35.383 INFO:tasks.workunit.client.0.smithi081.stdout:6/78: read - d3/f6 zero size 2024-03-22T07:08:35.384 INFO:tasks.workunit.client.0.smithi081.stdout:5/98: rename d4/df/d19 to d4/d10/d20 0 2024-03-22T07:08:35.384 INFO:tasks.workunit.client.0.smithi081.stdout:5/99: chown f2 115149 1 2024-03-22T07:08:35.384 INFO:tasks.workunit.client.0.smithi081.stdout:5/100: chown d4/lb 130107 1 2024-03-22T07:08:35.387 INFO:tasks.workunit.client.0.smithi081.stdout:7/187: write d9/dc/f1f [2259370,78740] 0 2024-03-22T07:08:35.389 INFO:tasks.workunit.client.0.smithi081.stdout:5/101: dread f2 [0,4194304] 0 2024-03-22T07:08:35.393 INFO:tasks.workunit.client.0.smithi081.stdout:6/79: unlink d3/da/fb 0 2024-03-22T07:08:35.395 INFO:tasks.workunit.client.0.smithi081.stdout:5/102: rename d4/d9/f11 to d4/d10/f21 0 2024-03-22T07:08:35.396 INFO:tasks.workunit.client.0.smithi081.stdout:6/80: mknod d3/da/de/c18 0 2024-03-22T07:08:35.398 INFO:tasks.workunit.client.0.smithi081.stdout:7/188: rmdir d9/d10 39 2024-03-22T07:08:35.398 INFO:tasks.workunit.client.0.smithi081.stdout:7/189: fsync d9/d10/d2a/f47 0 2024-03-22T07:08:35.398 INFO:tasks.workunit.client.0.smithi081.stdout:7/190: write d9/d10/d16/d1e/f28 [280871,18875] 0 2024-03-22T07:08:35.398 INFO:tasks.workunit.client.0.smithi081.stdout:7/191: write d9/d10/d16/d1e/f35 [3016,27023] 0 2024-03-22T07:08:35.398 INFO:tasks.workunit.client.0.smithi081.stdout:8/75: truncate f1 2578948 0 2024-03-22T07:08:35.399 INFO:tasks.workunit.client.0.smithi081.stdout:6/81: rename d3/d4/l5 to d3/da/de/df/l19 0 2024-03-22T07:08:35.402 INFO:tasks.workunit.client.0.smithi081.stdout:7/192: symlink d9/d10/l4c 0 2024-03-22T07:08:35.403 INFO:tasks.workunit.client.0.smithi081.stdout:8/76: rename d2/d3/l7 to d2/l1d 0 2024-03-22T07:08:35.404 INFO:tasks.workunit.client.0.smithi081.stdout:6/82: truncate d3/da/fd 346819 0 2024-03-22T07:08:35.404 INFO:tasks.workunit.client.0.smithi081.stdout:6/83: dread - d3/da/f17 zero size 2024-03-22T07:08:35.405 INFO:tasks.workunit.client.0.smithi081.stdout:7/193: dread f4 [0,4194304] 0 2024-03-22T07:08:35.405 INFO:tasks.workunit.client.0.smithi081.stdout:7/194: chown d9/dc/f38 75501 1 2024-03-22T07:08:35.405 INFO:tasks.workunit.client.0.smithi081.stdout:7/195: read d9/dc/f33 [84270,36660] 0 2024-03-22T07:08:35.406 INFO:tasks.workunit.client.0.smithi081.stdout:7/196: stat d9/lb 0 2024-03-22T07:08:35.409 INFO:tasks.workunit.client.0.smithi081.stdout:5/103: truncate d4/df/f1a 1906047 0 2024-03-22T07:08:35.409 INFO:tasks.workunit.client.0.smithi081.stdout:5/104: truncate d4/d9/f1b 21360 0 2024-03-22T07:08:35.409 INFO:tasks.workunit.client.0.smithi081.stdout:7/197: creat d9/d10/d2a/f4d x:0 0 0 2024-03-22T07:08:35.411 INFO:tasks.workunit.client.0.smithi081.stdout:6/84: unlink d3/d4/fc 0 2024-03-22T07:08:35.412 INFO:tasks.workunit.client.0.smithi081.stdout:5/105: mkdir d4/df/d22 0 2024-03-22T07:08:35.414 INFO:tasks.workunit.client.0.smithi081.stdout:7/198: truncate d9/d10/d16/d1e/f32 3569932 0 2024-03-22T07:08:35.417 INFO:tasks.workunit.client.0.smithi081.stdout:5/106: symlink d4/d10/l23 0 2024-03-22T07:08:35.424 INFO:tasks.workunit.client.0.smithi081.stdout:5/107: read f1 [1031960,43981] 0 2024-03-22T07:08:35.424 INFO:tasks.workunit.client.0.smithi081.stdout:5/108: creat d4/d10/f24 x:0 0 0 2024-03-22T07:08:35.426 INFO:tasks.workunit.client.0.smithi081.stdout:5/109: dread f3 [0,4194304] 0 2024-03-22T07:08:35.438 INFO:tasks.workunit.client.0.smithi081.stdout:6/85: dwrite d3/da/f17 [0,4194304] 0 2024-03-22T07:08:35.444 INFO:tasks.workunit.client.0.smithi081.stdout:5/110: dwrite d4/d10/f1d [0,4194304] 0 2024-03-22T07:08:35.444 INFO:tasks.workunit.client.0.smithi081.stdout:7/199: dwrite d9/d10/d2a/f47 [0,4194304] 0 2024-03-22T07:08:35.448 INFO:tasks.workunit.client.0.smithi081.stdout:5/111: mkdir d4/d10/d20/d25 0 2024-03-22T07:08:35.449 INFO:tasks.workunit.client.0.smithi081.stdout:5/112: dread d4/d9/f1b [0,4194304] 0 2024-03-22T07:08:35.449 INFO:tasks.workunit.client.0.smithi081.stdout:5/113: fsync d4/d10/f21 0 2024-03-22T07:08:35.449 INFO:tasks.workunit.client.0.smithi081.stdout:5/114: creat d4/d10/f26 x:0 0 0 2024-03-22T07:08:35.450 INFO:tasks.workunit.client.0.smithi081.stdout:7/200: creat d9/d10/d16/d1e/f4e x:0 0 0 2024-03-22T07:08:35.451 INFO:tasks.workunit.client.0.smithi081.stdout:7/201: symlink d9/d10/d4b/l4f 0 2024-03-22T07:08:35.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/86: dwrite d3/da/de/f11 [0,4194304] 0 2024-03-22T07:08:35.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/87: write d3/da/f14 [636283,70693] 0 2024-03-22T07:08:35.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/88: creat d3/da/de/f1a x:0 0 0 2024-03-22T07:08:35.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/89: fsync d3/da/de/f11 0 2024-03-22T07:08:35.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/90: dread - d3/f6 zero size 2024-03-22T07:08:35.553 INFO:tasks.workunit.client.0.smithi081.stdout:8/77: rename d2/d3/d10 to d2/d3/d19/d1e 0 2024-03-22T07:08:35.555 INFO:tasks.workunit.client.0.smithi081.stdout:5/115: rename d4/fc to d4/d10/d20/d1f/f27 0 2024-03-22T07:08:35.557 INFO:tasks.workunit.client.0.smithi081.stdout:7/202: rename d9/c29 to d9/dc/c50 0 2024-03-22T07:08:35.557 INFO:tasks.workunit.client.0.smithi081.stdout:6/91: rename d3/d12 to d3/d12/d1b 22 2024-03-22T07:08:35.557 INFO:tasks.workunit.client.0.smithi081.stdout:6/92: chown d3 9334 1 2024-03-22T07:08:35.557 INFO:tasks.workunit.client.0.smithi081.stdout:7/203: write d9/d10/f2e [415423,50104] 0 2024-03-22T07:08:35.558 INFO:tasks.workunit.client.0.smithi081.stdout:7/204: mknod d9/dc/d41/c51 0 2024-03-22T07:08:35.559 INFO:tasks.workunit.client.0.smithi081.stdout:8/78: write d2/d3/f1c [974129,66731] 0 2024-03-22T07:08:35.559 INFO:tasks.workunit.client.0.smithi081.stdout:8/79: creat d2/d3/f1f x:0 0 0 2024-03-22T07:08:35.563 INFO:tasks.workunit.client.0.smithi081.stdout:5/116: dread d4/d10/f1d [0,4194304] 0 2024-03-22T07:08:35.565 INFO:tasks.workunit.client.0.smithi081.stdout:5/117: creat d4/d10/d20/d1f/f28 x:0 0 0 2024-03-22T07:08:35.565 INFO:tasks.workunit.client.0.smithi081.stdout:5/118: write d4/df/f1a [1314792,94763] 0 2024-03-22T07:08:35.566 INFO:tasks.workunit.client.0.smithi081.stdout:5/119: symlink d4/d10/d20/d25/l29 0 2024-03-22T07:08:35.567 INFO:tasks.workunit.client.0.smithi081.stdout:5/120: mkdir d4/d10/d20/d1f/d2a 0 2024-03-22T07:08:35.567 INFO:tasks.workunit.client.0.smithi081.stdout:7/205: read d9/d10/d16/d1e/f28 [161189,58644] 0 2024-03-22T07:08:35.567 INFO:tasks.workunit.client.0.smithi081.stdout:6/93: dread d3/da/de/df/f15 [0,4194304] 0 2024-03-22T07:08:35.568 INFO:tasks.workunit.client.0.smithi081.stdout:5/121: symlink d4/d10/d20/l2b 0 2024-03-22T07:08:35.574 INFO:tasks.workunit.client.0.smithi081.stdout:8/80: write d2/f15 [943004,30818] 0 2024-03-22T07:08:35.594 INFO:tasks.workunit.client.0.smithi081.stdout:6/94: dwrite d3/da/de/f1a [0,4194304] 0 2024-03-22T07:08:35.594 INFO:tasks.workunit.client.0.smithi081.stdout:6/95: write d3/d4/f7 [595251,14927] 0 2024-03-22T07:08:35.594 INFO:tasks.workunit.client.0.smithi081.stdout:6/96: chown d3/da 762138226 1 2024-03-22T07:08:35.596 INFO:tasks.workunit.client.0.smithi081.stdout:5/122: dwrite d4/d10/f1d [0,4194304] 0 2024-03-22T07:08:35.596 INFO:tasks.workunit.client.0.smithi081.stdout:6/97: creat d3/d12/f1c x:0 0 0 2024-03-22T07:08:35.597 INFO:tasks.workunit.client.0.smithi081.stdout:5/123: creat d4/d9/f2c x:0 0 0 2024-03-22T07:08:35.600 INFO:tasks.workunit.client.0.smithi081.stdout:8/81: dread d2/f15 [0,4194304] 0 2024-03-22T07:08:35.601 INFO:tasks.workunit.client.0.smithi081.stdout:8/82: rename d2/d3/l8 to d2/d3/l20 0 2024-03-22T07:08:35.604 INFO:tasks.workunit.client.0.smithi081.stdout:7/206: dwrite d9/d10/d16/d1e/f28 [0,4194304] 0 2024-03-22T07:08:35.607 INFO:tasks.workunit.client.0.smithi081.stdout:7/207: mkdir d9/d3f/d52 0 2024-03-22T07:08:35.609 INFO:tasks.workunit.client.0.smithi081.stdout:7/208: mkdir d9/d10/d53 0 2024-03-22T07:08:35.611 INFO:tasks.workunit.client.0.smithi081.stdout:7/209: truncate d9/d10/d16/d1e/f32 562579 0 2024-03-22T07:08:35.613 INFO:tasks.workunit.client.0.smithi081.stdout:7/210: creat d9/d14/f54 x:0 0 0 2024-03-22T07:08:35.616 INFO:tasks.workunit.client.0.smithi081.stdout:7/211: link d9/d10/d16/d1e/l39 d9/d10/d4b/l55 0 2024-03-22T07:08:35.616 INFO:tasks.workunit.client.0.smithi081.stdout:7/212: chown d9/d10/d2a/f4d 929646 1 2024-03-22T07:08:35.619 INFO:tasks.workunit.client.0.smithi081.stdout:5/124: truncate f3 549324 0 2024-03-22T07:08:35.621 INFO:tasks.workunit.client.0.smithi081.stdout:5/125: symlink d4/d10/l2d 0 2024-03-22T07:08:35.625 INFO:tasks.workunit.client.0.smithi081.stdout:8/83: dwrite f1 [0,4194304] 0 2024-03-22T07:08:35.625 INFO:tasks.workunit.client.0.smithi081.stdout:8/84: fdatasync f1 0 2024-03-22T07:08:35.630 INFO:tasks.workunit.client.0.smithi081.stdout:8/85: getdents d2/d3 0 2024-03-22T07:08:35.635 INFO:tasks.workunit.client.0.smithi081.stdout:7/213: write d9/dc/f38 [2711929,120892] 0 2024-03-22T07:08:35.638 INFO:tasks.workunit.client.0.smithi081.stdout:6/98: dwrite d3/da/de/f1a [4194304,4194304] 0 2024-03-22T07:08:35.638 INFO:tasks.workunit.client.0.smithi081.stdout:7/214: link d9/d10/c1c d9/d10/d16/d1e/c56 0 2024-03-22T07:08:35.644 INFO:tasks.workunit.client.0.smithi081.stdout:8/86: write f1 [3326125,3197] 0 2024-03-22T07:08:35.645 INFO:tasks.workunit.client.0.smithi081.stdout:7/215: unlink d9/d14/c30 0 2024-03-22T07:08:35.649 INFO:tasks.workunit.client.0.smithi081.stdout:7/216: fsync d9/dc/f1f 0 2024-03-22T07:08:35.649 INFO:tasks.workunit.client.0.smithi081.stdout:7/217: write d9/d14/f24 [973230,123388] 0 2024-03-22T07:08:35.649 INFO:tasks.workunit.client.0.smithi081.stdout:7/218: write f4 [1264446,48722] 0 2024-03-22T07:08:35.649 INFO:tasks.workunit.client.0.smithi081.stdout:7/219: chown d9/d10/l4c 9069891 1 2024-03-22T07:08:35.650 INFO:tasks.workunit.client.0.smithi081.stdout:5/126: truncate f3 206180 0 2024-03-22T07:08:35.650 INFO:tasks.workunit.client.0.smithi081.stdout:5/127: read - d4/d10/f24 zero size 2024-03-22T07:08:35.653 INFO:tasks.workunit.client.0.smithi081.stdout:5/128: mkdir d4/d10/d20/d25/d2e 0 2024-03-22T07:08:35.654 INFO:tasks.workunit.client.0.smithi081.stdout:5/129: chown d4/df/d22 30043 1 2024-03-22T07:08:35.656 INFO:tasks.workunit.client.0.smithi081.stdout:5/130: mkdir d4/d10/d20/d1f/d2f 0 2024-03-22T07:08:35.658 INFO:tasks.workunit.client.0.smithi081.stdout:5/131: symlink d4/df/d22/l30 0 2024-03-22T07:08:35.662 INFO:tasks.workunit.client.0.smithi081.stdout:5/132: creat d4/d10/d20/d25/d2e/f31 x:0 0 0 2024-03-22T07:08:35.662 INFO:tasks.workunit.client.0.smithi081.stdout:5/133: fsync d4/d10/d20/d1f/f28 0 2024-03-22T07:08:35.664 INFO:tasks.workunit.client.0.smithi081.stdout:5/134: symlink d4/d10/d20/l32 0 2024-03-22T07:08:35.669 INFO:tasks.workunit.client.0.smithi081.stdout:6/99: dwrite d3/da/f14 [0,4194304] 0 2024-03-22T07:08:35.671 INFO:tasks.workunit.client.0.smithi081.stdout:8/87: dwrite d2/f11 [0,4194304] 0 2024-03-22T07:08:35.672 INFO:tasks.workunit.client.0.smithi081.stdout:7/220: dwrite d9/dc/f1f [0,4194304] 0 2024-03-22T07:08:35.673 INFO:tasks.workunit.client.0.smithi081.stdout:8/88: symlink d2/d3/d19/d1e/l21 0 2024-03-22T07:08:35.673 INFO:tasks.workunit.client.0.smithi081.stdout:8/89: readlink d2/d3/lf 0 2024-03-22T07:08:35.675 INFO:tasks.workunit.client.0.smithi081.stdout:7/221: creat d9/d10/d16/d1e/d27/f57 x:0 0 0 2024-03-22T07:08:35.676 INFO:tasks.workunit.client.0.smithi081.stdout:7/222: creat d9/d10/d16/d1e/f58 x:0 0 0 2024-03-22T07:08:35.677 INFO:tasks.workunit.client.0.smithi081.stdout:6/100: getdents d3/d4 0 2024-03-22T07:08:35.677 INFO:tasks.workunit.client.0.smithi081.stdout:6/101: creat d3/d12/f1d x:0 0 0 2024-03-22T07:08:35.677 INFO:tasks.workunit.client.0.smithi081.stdout:6/102: readlink d3/da/de/df/l19 0 2024-03-22T07:08:35.680 INFO:tasks.workunit.client.0.smithi081.stdout:8/90: mkdir d2/d3/d19/d1e/d13/d22 0 2024-03-22T07:08:35.680 INFO:tasks.workunit.client.0.smithi081.stdout:6/103: creat d3/da/de/df/f1e x:0 0 0 2024-03-22T07:08:35.681 INFO:tasks.workunit.client.0.smithi081.stdout:6/104: readlink d3/da/de/df/l19 0 2024-03-22T07:08:35.684 INFO:tasks.workunit.client.0.smithi081.stdout:5/135: dwrite d4/d9/f2c [0,4194304] 0 2024-03-22T07:08:35.684 INFO:tasks.workunit.client.0.smithi081.stdout:5/136: readlink d4/d10/l12 0 2024-03-22T07:08:35.686 INFO:tasks.workunit.client.0.smithi081.stdout:5/137: truncate d4/d10/f21 979471 0 2024-03-22T07:08:35.686 INFO:tasks.workunit.client.0.smithi081.stdout:5/138: readlink d4/d9/l17 0 2024-03-22T07:08:35.687 INFO:tasks.workunit.client.0.smithi081.stdout:5/139: getdents d4/df/d22 0 2024-03-22T07:08:35.690 INFO:tasks.workunit.client.0.smithi081.stdout:5/140: link d4/d10/d20/d25/d2e/f31 d4/f33 0 2024-03-22T07:08:35.691 INFO:tasks.workunit.client.0.smithi081.stdout:5/141: mkdir d4/d10/d20/d25/d2e/d34 0 2024-03-22T07:08:35.700 INFO:tasks.workunit.client.0.smithi081.stdout:5/142: link d4/d10/l1e d4/l35 0 2024-03-22T07:08:35.700 INFO:tasks.workunit.client.0.smithi081.stdout:5/143: readlink d4/df/l18 0 2024-03-22T07:08:35.702 INFO:tasks.workunit.client.0.smithi081.stdout:5/144: truncate d4/d10/d20/d1f/f27 960348 0 2024-03-22T07:08:35.702 INFO:tasks.workunit.client.0.smithi081.stdout:5/145: chown d4/d10/d20/d25/d2e/f31 0 1 2024-03-22T07:08:35.704 INFO:tasks.workunit.client.0.smithi081.stdout:5/146: symlink d4/d10/d20/l36 0 2024-03-22T07:08:35.705 INFO:tasks.workunit.client.0.smithi081.stdout:7/223: dwrite d9/f3c [0,4194304] 0 2024-03-22T07:08:35.706 INFO:tasks.workunit.client.0.smithi081.stdout:5/147: creat d4/d10/d20/d1f/d2f/f37 x:0 0 0 2024-03-22T07:08:35.706 INFO:tasks.workunit.client.0.smithi081.stdout:5/148: fdatasync f1 0 2024-03-22T07:08:35.707 INFO:tasks.workunit.client.0.smithi081.stdout:7/224: mkdir d9/dc/d41/d59 0 2024-03-22T07:08:35.717 INFO:tasks.workunit.client.0.smithi081.stdout:8/91: dwrite d2/d3/f16 [0,4194304] 0 2024-03-22T07:08:35.717 INFO:tasks.workunit.client.0.smithi081.stdout:6/105: dwrite d3/da/de/df/f15 [8388608,4194304] 0 2024-03-22T07:08:35.717 INFO:tasks.workunit.client.0.smithi081.stdout:6/106: creat d3/da/de/f1f x:0 0 0 2024-03-22T07:08:35.721 INFO:tasks.workunit.client.0.smithi081.stdout:8/92: truncate f0 3185540 0 2024-03-22T07:08:35.722 INFO:tasks.workunit.client.0.smithi081.stdout:8/93: read - d2/d3/f1f zero size 2024-03-22T07:08:35.724 INFO:tasks.workunit.client.0.smithi081.stdout:6/107: symlink d3/da/de/l20 0 2024-03-22T07:08:35.729 INFO:tasks.workunit.client.0.smithi081.stdout:5/149: dwrite d4/d10/f21 [0,4194304] 0 2024-03-22T07:08:35.732 INFO:tasks.workunit.client.0.smithi081.stdout:5/150: symlink d4/d10/d20/l38 0 2024-03-22T07:08:35.733 INFO:tasks.workunit.client.0.smithi081.stdout:7/225: dwrite d9/d14/f24 [0,4194304] 0 2024-03-22T07:08:35.733 INFO:tasks.workunit.client.0.smithi081.stdout:7/226: fsync d9/d10/f3a 0 2024-03-22T07:08:35.733 INFO:tasks.workunit.client.0.smithi081.stdout:7/227: creat d9/d10/d16/d1e/d27/f5a x:0 0 0 2024-03-22T07:08:35.733 INFO:tasks.workunit.client.0.smithi081.stdout:7/228: truncate d9/dc/f33 1904031 0 2024-03-22T07:08:35.733 INFO:tasks.workunit.client.0.smithi081.stdout:7/229: write d9/d10/d16/d1e/d27/f57 [471995,44133] 0 2024-03-22T07:08:35.736 INFO:tasks.workunit.client.0.smithi081.stdout:8/94: rename d2/d3/dc to d2/d3/d23 0 2024-03-22T07:08:35.736 INFO:tasks.workunit.client.0.smithi081.stdout:8/95: readlink d2/d3/l20 0 2024-03-22T07:08:35.736 INFO:tasks.workunit.client.0.smithi081.stdout:8/96: write d2/f11 [5026614,68818] 0 2024-03-22T07:08:35.740 INFO:tasks.workunit.client.0.smithi081.stdout:5/151: rmdir d4/d10/d20/d25/d2e 39 2024-03-22T07:08:35.746 INFO:tasks.workunit.client.0.smithi081.stdout:6/108: dwrite d3/da/f14 [0,4194304] 0 2024-03-22T07:08:35.751 INFO:tasks.workunit.client.0.smithi081.stdout:7/230: rmdir d9 39 2024-03-22T07:08:35.754 INFO:tasks.workunit.client.0.smithi081.stdout:6/109: truncate d3/d4/f7 91242 0 2024-03-22T07:08:35.754 INFO:tasks.workunit.client.0.smithi081.stdout:6/110: write d3/d12/f1c [290225,113583] 0 2024-03-22T07:08:35.755 INFO:tasks.workunit.client.0.smithi081.stdout:6/111: dread - d3/da/de/f1f zero size 2024-03-22T07:08:35.755 INFO:tasks.workunit.client.0.smithi081.stdout:6/112: readlink d3/da/de/df/l19 0 2024-03-22T07:08:35.755 INFO:tasks.workunit.client.0.smithi081.stdout:6/113: stat d3/d16 0 2024-03-22T07:08:35.755 INFO:tasks.workunit.client.0.smithi081.stdout:6/114: chown d3/d4/l10 911 1 2024-03-22T07:08:35.756 INFO:tasks.workunit.client.0.smithi081.stdout:7/231: mknod d9/dc/c5b 0 2024-03-22T07:08:35.758 INFO:tasks.workunit.client.0.smithi081.stdout:7/232: link d9/d10/c1d d9/d10/d4b/c5c 0 2024-03-22T07:08:35.759 INFO:tasks.workunit.client.0.smithi081.stdout:7/233: symlink d9/d3f/l5d 0 2024-03-22T07:08:35.759 INFO:tasks.workunit.client.0.smithi081.stdout:7/234: write d9/d10/d16/f42 [296041,15764] 0 2024-03-22T07:08:35.764 INFO:tasks.workunit.client.0.smithi081.stdout:7/235: dread d9/d10/f2e [0,4194304] 0 2024-03-22T07:08:35.764 INFO:tasks.workunit.client.0.smithi081.stdout:5/152: dwrite d4/d9/f2c [0,4194304] 0 2024-03-22T07:08:35.766 INFO:tasks.workunit.client.0.smithi081.stdout:5/153: link d4/df/l18 d4/d9/l39 0 2024-03-22T07:08:35.766 INFO:tasks.workunit.client.0.smithi081.stdout:5/154: fdatasync d4/d10/d20/d1f/f28 0 2024-03-22T07:08:35.766 INFO:tasks.workunit.client.0.smithi081.stdout:5/155: chown d4/df/d22/l30 789 1 2024-03-22T07:08:35.767 INFO:tasks.workunit.client.0.smithi081.stdout:7/236: symlink d9/d14/l5e 0 2024-03-22T07:08:35.768 INFO:tasks.workunit.client.0.smithi081.stdout:5/156: mknod d4/df/c3a 0 2024-03-22T07:08:35.768 INFO:tasks.workunit.client.0.smithi081.stdout:5/157: readlink d4/d10/l1e 0 2024-03-22T07:08:35.768 INFO:tasks.workunit.client.0.smithi081.stdout:7/237: mknod d9/d10/c5f 0 2024-03-22T07:08:35.769 INFO:tasks.workunit.client.0.smithi081.stdout:5/158: rmdir d4/df/d22 39 2024-03-22T07:08:35.771 INFO:tasks.workunit.client.0.smithi081.stdout:7/238: rename d9/dc/c5b to d9/d3f/d52/c60 0 2024-03-22T07:08:35.771 INFO:tasks.workunit.client.0.smithi081.stdout:7/239: chown d9/d3f 5 1 2024-03-22T07:08:35.772 INFO:tasks.workunit.client.0.smithi081.stdout:5/159: creat d4/d10/d20/d25/d2e/f3b x:0 0 0 2024-03-22T07:08:35.772 INFO:tasks.workunit.client.0.smithi081.stdout:5/160: chown d4/d10/d20/d1f/f28 53478 1 2024-03-22T07:08:35.772 INFO:tasks.workunit.client.0.smithi081.stdout:5/161: read f1 [1116575,25858] 0 2024-03-22T07:08:35.772 INFO:tasks.workunit.client.0.smithi081.stdout:5/162: creat d4/d10/f3c x:0 0 0 2024-03-22T07:08:35.773 INFO:tasks.workunit.client.0.smithi081.stdout:5/163: mknod d4/d10/d20/d25/d2e/c3d 0 2024-03-22T07:08:35.774 INFO:tasks.workunit.client.0.smithi081.stdout:6/115: dwrite d3/f6 [0,4194304] 0 2024-03-22T07:08:35.775 INFO:tasks.workunit.client.0.smithi081.stdout:5/164: creat d4/f3e x:0 0 0 2024-03-22T07:08:35.775 INFO:tasks.workunit.client.0.smithi081.stdout:5/165: readlink d4/d10/l14 0 2024-03-22T07:08:35.775 INFO:tasks.workunit.client.0.smithi081.stdout:5/166: fsync d4/d10/d20/d1f/f27 0 2024-03-22T07:08:35.775 INFO:tasks.workunit.client.0.smithi081.stdout:5/167: write f2 [967652,130781] 0 2024-03-22T07:08:35.780 INFO:tasks.workunit.client.0.smithi081.stdout:6/116: rmdir d3/d16 0 2024-03-22T07:08:35.781 INFO:tasks.workunit.client.0.smithi081.stdout:7/240: write d9/d10/f3a [77155,96860] 0 2024-03-22T07:08:35.781 INFO:tasks.workunit.client.0.smithi081.stdout:7/241: fsync d9/d10/d16/d1e/f4e 0 2024-03-22T07:08:35.783 INFO:tasks.workunit.client.0.smithi081.stdout:7/242: mkdir d9/d10/d4b/d61 0 2024-03-22T07:08:35.786 INFO:tasks.workunit.client.0.smithi081.stdout:7/243: mknod d9/dc/c62 0 2024-03-22T07:08:35.789 INFO:tasks.workunit.client.0.smithi081.stdout:6/117: dread d3/da/de/df/f15 [0,4194304] 0 2024-03-22T07:08:35.792 INFO:tasks.workunit.client.0.smithi081.stdout:5/168: dwrite d4/d10/d20/d1f/f27 [0,4194304] 0 2024-03-22T07:08:35.804 INFO:tasks.workunit.client.0.smithi081.stdout:5/169: dread d4/d10/f21 [0,4194304] 0 2024-03-22T07:08:35.804 INFO:tasks.workunit.client.0.smithi081.stdout:5/170: dread - d4/d10/d20/d25/d2e/f3b zero size 2024-03-22T07:08:35.807 INFO:tasks.workunit.client.0.smithi081.stdout:5/171: rmdir d4/d10/d20/d25/d2e/d34 0 2024-03-22T07:08:35.807 INFO:tasks.workunit.client.0.smithi081.stdout:5/172: fsync d4/d10/f26 0 2024-03-22T07:08:35.807 INFO:tasks.workunit.client.0.smithi081.stdout:5/173: dread - d4/d9/f15 zero size 2024-03-22T07:08:35.808 INFO:tasks.workunit.client.0.smithi081.stdout:5/174: stat d4/d10/d20/l2b 0 2024-03-22T07:08:35.808 INFO:tasks.workunit.client.0.smithi081.stdout:7/244: dwrite d9/d10/d16/d1e/f4e [0,4194304] 0 2024-03-22T07:08:35.810 INFO:tasks.workunit.client.0.smithi081.stdout:7/245: creat d9/d10/d16/d1e/d27/f63 x:0 0 0 2024-03-22T07:08:35.810 INFO:tasks.workunit.client.0.smithi081.stdout:7/246: readlink d9/d10/d2a/l40 0 2024-03-22T07:08:35.810 INFO:tasks.workunit.client.0.smithi081.stdout:7/247: read - d9/dc/f4a zero size 2024-03-22T07:08:35.811 INFO:tasks.workunit.client.0.smithi081.stdout:7/248: getdents d9/d10/d2a 0 2024-03-22T07:08:35.812 INFO:tasks.workunit.client.0.smithi081.stdout:5/175: dread f2 [0,4194304] 0 2024-03-22T07:08:35.813 INFO:tasks.workunit.client.0.smithi081.stdout:7/249: rename d9/d10/d4b/c5c to d9/d10/d4b/d61/c64 0 2024-03-22T07:08:35.816 INFO:tasks.workunit.client.0.smithi081.stdout:5/176: unlink d4/d10/d20/d1f/f27 0 2024-03-22T07:08:35.820 INFO:tasks.workunit.client.0.smithi081.stdout:5/177: symlink d4/d10/d20/d1f/d2f/l3f 0 2024-03-22T07:08:35.820 INFO:tasks.workunit.client.0.smithi081.stdout:5/178: dread - d4/d10/f3c zero size 2024-03-22T07:08:35.821 INFO:tasks.workunit.client.0.smithi081.stdout:5/179: symlink d4/d10/d20/l40 0 2024-03-22T07:08:35.821 INFO:tasks.workunit.client.0.smithi081.stdout:5/180: chown f3 2656 1 2024-03-22T07:08:35.821 INFO:tasks.workunit.client.0.smithi081.stdout:7/250: rmdir d9/d10/d16 39 2024-03-22T07:08:35.828 INFO:tasks.workunit.client.0.smithi081.stdout:7/251: dread d9/d10/d16/d1e/f4e [0,4194304] 0 2024-03-22T07:08:35.831 INFO:tasks.workunit.client.0.smithi081.stdout:7/252: dread d9/d10/d16/d1e/f32 [0,4194304] 0 2024-03-22T07:08:35.831 INFO:tasks.workunit.client.0.smithi081.stdout:7/253: write f4 [1661205,118608] 0 2024-03-22T07:08:35.837 INFO:tasks.workunit.client.0.smithi081.stdout:7/254: dread d9/f3c [0,4194304] 0 2024-03-22T07:08:35.839 INFO:tasks.workunit.client.0.smithi081.stdout:7/255: creat d9/d3f/f65 x:0 0 0 2024-03-22T07:08:35.839 INFO:tasks.workunit.client.0.smithi081.stdout:7/256: fsync d9/d14/f54 0 2024-03-22T07:08:35.840 INFO:tasks.workunit.client.0.smithi081.stdout:7/257: fsync d9/d10/d16/d1e/f58 0 2024-03-22T07:08:35.840 INFO:tasks.workunit.client.0.smithi081.stdout:5/181: dwrite d4/d10/d20/d25/d2e/f31 [0,4194304] 0 2024-03-22T07:08:35.840 INFO:tasks.workunit.client.0.smithi081.stdout:7/258: mknod d9/dc/c66 0 2024-03-22T07:08:35.843 INFO:tasks.workunit.client.0.smithi081.stdout:7/259: mknod d9/c67 0 2024-03-22T07:08:35.843 INFO:tasks.workunit.client.0.smithi081.stdout:7/260: write d9/f3c [5072386,50129] 0 2024-03-22T07:08:35.858 INFO:tasks.workunit.client.0.smithi081.stdout:5/182: dwrite d4/d9/f2c [0,4194304] 0 2024-03-22T07:08:35.860 INFO:tasks.workunit.client.0.smithi081.stdout:5/183: mknod d4/d10/d20/c41 0 2024-03-22T07:08:35.860 INFO:tasks.workunit.client.0.smithi081.stdout:5/184: readlink d4/d10/d20/d25/l29 0 2024-03-22T07:08:35.861 INFO:tasks.workunit.client.0.smithi081.stdout:5/185: link d4/df/l18 d4/d10/d20/d25/l42 0 2024-03-22T07:08:35.862 INFO:tasks.workunit.client.0.smithi081.stdout:5/186: chown d4/df/l18 79 1 2024-03-22T07:08:35.862 INFO:tasks.workunit.client.0.smithi081.stdout:5/187: read - d4/d10/f3c zero size 2024-03-22T07:08:35.862 INFO:tasks.workunit.client.0.smithi081.stdout:5/188: read d4/df/f1a [42562,43687] 0 2024-03-22T07:08:35.862 INFO:tasks.workunit.client.0.smithi081.stdout:5/189: write d4/df/f1a [867601,15607] 0 2024-03-22T07:08:35.864 INFO:tasks.workunit.client.0.smithi081.stdout:5/190: write d4/d10/f21 [2281324,65370] 0 2024-03-22T07:08:35.864 INFO:tasks.workunit.client.0.smithi081.stdout:5/191: chown d4/d10/l2d 107566623 1 2024-03-22T07:08:35.866 INFO:tasks.workunit.client.0.smithi081.stdout:5/192: creat d4/d10/d20/d25/f43 x:0 0 0 2024-03-22T07:08:35.866 INFO:tasks.workunit.client.0.smithi081.stdout:5/193: chown d4/d10/f24 104 1 2024-03-22T07:08:35.866 INFO:tasks.workunit.client.0.smithi081.stdout:5/194: readlink d4/d10/d20/l40 0 2024-03-22T07:08:35.867 INFO:tasks.workunit.client.0.smithi081.stdout:5/195: link d4/d10/d20/l40 d4/d10/d20/d25/d2e/l44 0 2024-03-22T07:08:35.867 INFO:tasks.workunit.client.0.smithi081.stdout:5/196: mkdir d4/d10/d20/d1f/d2f/d45 0 2024-03-22T07:08:35.882 INFO:tasks.workunit.client.0.smithi081.stdout:5/197: dwrite d4/d10/f26 [0,4194304] 0 2024-03-22T07:08:35.882 INFO:tasks.workunit.client.0.smithi081.stdout:5/198: dread - d4/d10/d20/d1f/f28 zero size 2024-03-22T07:08:35.888 INFO:tasks.workunit.client.0.smithi081.stdout:5/199: dread d4/d10/f26 [0,4194304] 0 2024-03-22T07:08:35.890 INFO:tasks.workunit.client.0.smithi081.stdout:5/200: creat d4/df/f46 x:0 0 0 2024-03-22T07:08:35.892 INFO:tasks.workunit.client.0.smithi081.stdout:5/201: symlink d4/l47 0 2024-03-22T07:08:36.542 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:36 smithi149 ceph-mon[36068]: pgmap v241: 65 pgs: 65 active+clean; 319 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 30 MiB/s wr, 699 op/s 2024-03-22T07:08:36.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:36 smithi081 ceph-mon[29245]: pgmap v241: 65 pgs: 65 active+clean; 319 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 30 MiB/s wr, 699 op/s 2024-03-22T07:08:36.872 INFO:tasks.workunit.client.0.smithi081.stdout:2/37: sync 2024-03-22T07:08:36.872 INFO:tasks.workunit.client.0.smithi081.stdout:1/98: sync 2024-03-22T07:08:36.872 INFO:tasks.workunit.client.0.smithi081.stdout:0/127: sync 2024-03-22T07:08:36.872 INFO:tasks.workunit.client.0.smithi081.stdout:3/71: sync 2024-03-22T07:08:36.872 INFO:tasks.workunit.client.0.smithi081.stdout:9/81: sync 2024-03-22T07:08:36.874 INFO:tasks.workunit.client.0.smithi081.stdout:3/72: getdents db 0 2024-03-22T07:08:36.875 INFO:tasks.workunit.client.0.smithi081.stdout:9/82: mkdir d2/d10/d1b 0 2024-03-22T07:08:36.875 INFO:tasks.workunit.client.0.smithi081.stdout:9/83: write d2/f15 [661713,118964] 0 2024-03-22T07:08:36.876 INFO:tasks.workunit.client.0.smithi081.stdout:2/38: mknod da/db/cc 0 2024-03-22T07:08:36.879 INFO:tasks.workunit.client.0.smithi081.stdout:0/128: link d3/l14 d3/dc/l26 0 2024-03-22T07:08:36.879 INFO:tasks.workunit.client.0.smithi081.stdout:9/84: truncate d2/d10/f17 2434819 0 2024-03-22T07:08:36.880 INFO:tasks.workunit.client.0.smithi081.stdout:2/39: mknod da/db/cd 0 2024-03-22T07:08:36.880 INFO:tasks.workunit.client.0.smithi081.stdout:2/40: write f6 [87659,79811] 0 2024-03-22T07:08:36.881 INFO:tasks.workunit.client.0.smithi081.stdout:2/41: mknod da/ce 0 2024-03-22T07:08:36.881 INFO:tasks.workunit.client.0.smithi081.stdout:2/42: truncate f6 721663 0 2024-03-22T07:08:36.883 INFO:tasks.workunit.client.0.smithi081.stdout:1/99: read d6/fd [1159757,12068] 0 2024-03-22T07:08:36.883 INFO:tasks.workunit.client.0.smithi081.stdout:9/85: unlink d2/f13 0 2024-03-22T07:08:36.884 INFO:tasks.workunit.client.0.smithi081.stdout:0/129: rename d3/la to d3/l27 0 2024-03-22T07:08:36.885 INFO:tasks.workunit.client.0.smithi081.stdout:1/100: unlink d6/f10 0 2024-03-22T07:08:36.885 INFO:tasks.workunit.client.0.smithi081.stdout:1/101: read - d6/d18/d1f/d24/f26 zero size 2024-03-22T07:08:36.887 INFO:tasks.workunit.client.0.smithi081.stdout:0/130: truncate d3/dc/f19 3593597 0 2024-03-22T07:08:36.887 INFO:tasks.workunit.client.0.smithi081.stdout:0/131: creat d3/dc/f28 x:0 0 0 2024-03-22T07:08:36.887 INFO:tasks.workunit.client.0.smithi081.stdout:2/43: dread f3 [0,4194304] 0 2024-03-22T07:08:36.891 INFO:tasks.workunit.client.0.smithi081.stdout:0/132: symlink d3/dc/l29 0 2024-03-22T07:08:36.891 INFO:tasks.workunit.client.0.smithi081.stdout:0/133: fdatasync d3/dc/f28 0 2024-03-22T07:08:36.892 INFO:tasks.workunit.client.0.smithi081.stdout:2/44: symlink da/db/lf 0 2024-03-22T07:08:36.894 INFO:tasks.workunit.client.0.smithi081.stdout:2/45: symlink da/db/l10 0 2024-03-22T07:08:36.895 INFO:tasks.workunit.client.0.smithi081.stdout:0/134: dread d3/f8 [0,4194304] 0 2024-03-22T07:08:36.895 INFO:tasks.workunit.client.0.smithi081.stdout:0/135: creat d3/dc/f2a x:0 0 0 2024-03-22T07:08:36.895 INFO:tasks.workunit.client.0.smithi081.stdout:9/86: read d2/d5/f7 [4176347,116448] 0 2024-03-22T07:08:36.895 INFO:tasks.workunit.client.0.smithi081.stdout:9/87: dread - d2/fe zero size 2024-03-22T07:08:36.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/73: dwrite f1 [0,4194304] 0 2024-03-22T07:08:36.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/74: readlink db/de/l10 0 2024-03-22T07:08:36.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/75: readlink db/de/l10 0 2024-03-22T07:08:36.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/76: getdents db 0 2024-03-22T07:08:36.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/77: chown db/f18 52566 1 2024-03-22T07:08:36.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/78: chown db/de/l15 447158127 1 2024-03-22T07:08:36.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/79: read - db/f18 zero size 2024-03-22T07:08:36.896 INFO:tasks.workunit.client.0.smithi081.stdout:2/46: mkdir da/d11 0 2024-03-22T07:08:36.896 INFO:tasks.workunit.client.0.smithi081.stdout:2/47: read f3 [3861811,113975] 0 2024-03-22T07:08:36.898 INFO:tasks.workunit.client.0.smithi081.stdout:0/136: mkdir d3/dc/d2b 0 2024-03-22T07:08:36.899 INFO:tasks.workunit.client.0.smithi081.stdout:9/88: symlink d2/d5/d16/l1c 0 2024-03-22T07:08:36.901 INFO:tasks.workunit.client.0.smithi081.stdout:3/80: unlink db/de/l15 0 2024-03-22T07:08:36.907 INFO:tasks.workunit.client.0.smithi081.stdout:1/102: dwrite d6/fc [0,4194304] 0 2024-03-22T07:08:36.910 INFO:tasks.workunit.client.0.smithi081.stdout:9/89: dread d2/f11 [0,4194304] 0 2024-03-22T07:08:36.917 INFO:tasks.workunit.client.0.smithi081.stdout:3/81: read f7 [2277552,23017] 0 2024-03-22T07:08:36.920 INFO:tasks.workunit.client.0.smithi081.stdout:3/82: unlink f7 0 2024-03-22T07:08:36.925 INFO:tasks.workunit.client.0.smithi081.stdout:3/83: mknod db/c19 0 2024-03-22T07:08:36.935 INFO:tasks.workunit.client.0.smithi081.stdout:2/48: dwrite f4 [4194304,4194304] 0 2024-03-22T07:08:36.935 INFO:tasks.workunit.client.0.smithi081.stdout:0/137: dread d3/f4 [4194304,4194304] 0 2024-03-22T07:08:36.938 INFO:tasks.workunit.client.0.smithi081.stdout:2/49: unlink f3 0 2024-03-22T07:08:36.938 INFO:tasks.workunit.client.0.smithi081.stdout:2/50: fdatasync f5 0 2024-03-22T07:08:36.941 INFO:tasks.workunit.client.0.smithi081.stdout:2/51: mknod da/d11/c12 0 2024-03-22T07:08:36.941 INFO:tasks.workunit.client.0.smithi081.stdout:2/52: chown da/db/l10 6778823 1 2024-03-22T07:08:36.941 INFO:tasks.workunit.client.0.smithi081.stdout:2/53: readlink l1 0 2024-03-22T07:08:36.942 INFO:tasks.workunit.client.0.smithi081.stdout:2/54: rmdir da 39 2024-03-22T07:08:36.944 INFO:tasks.workunit.client.0.smithi081.stdout:2/55: link f6 da/f13 0 2024-03-22T07:08:36.949 INFO:tasks.workunit.client.0.smithi081.stdout:3/84: dwrite db/f13 [0,4194304] 0 2024-03-22T07:08:36.951 INFO:tasks.workunit.client.0.smithi081.stdout:9/90: dwrite d2/d5/fd [0,4194304] 0 2024-03-22T07:08:36.951 INFO:tasks.workunit.client.0.smithi081.stdout:0/138: dread d3/dc/f1f [0,4194304] 0 2024-03-22T07:08:36.951 INFO:tasks.workunit.client.0.smithi081.stdout:0/139: fsync f2 0 2024-03-22T07:08:36.951 INFO:tasks.workunit.client.0.smithi081.stdout:0/140: chown d3/dc/l26 2 1 2024-03-22T07:08:36.951 INFO:tasks.workunit.client.0.smithi081.stdout:0/141: dread - d3/dc/f28 zero size 2024-03-22T07:08:36.952 INFO:tasks.workunit.client.0.smithi081.stdout:3/85: creat db/de/f1a x:0 0 0 2024-03-22T07:08:36.952 INFO:tasks.workunit.client.0.smithi081.stdout:3/86: chown db/c16 895154287 1 2024-03-22T07:08:36.952 INFO:tasks.workunit.client.0.smithi081.stdout:3/87: chown c9 488991919 1 2024-03-22T07:08:36.952 INFO:tasks.workunit.client.0.smithi081.stdout:3/88: readlink l8 0 2024-03-22T07:08:36.952 INFO:tasks.workunit.client.0.smithi081.stdout:3/89: creat db/de/f1b x:0 0 0 2024-03-22T07:08:36.954 INFO:tasks.workunit.client.0.smithi081.stdout:9/91: symlink d2/d5/d16/l1d 0 2024-03-22T07:08:36.960 INFO:tasks.workunit.client.0.smithi081.stdout:2/56: dread f4 [0,4194304] 0 2024-03-22T07:08:36.960 INFO:tasks.workunit.client.0.smithi081.stdout:2/57: truncate f6 1026872 0 2024-03-22T07:08:36.967 INFO:tasks.workunit.client.0.smithi081.stdout:2/58: fsync da/f13 0 2024-03-22T07:08:36.967 INFO:tasks.workunit.client.0.smithi081.stdout:0/142: dwrite d3/f22 [0,4194304] 0 2024-03-22T07:08:36.971 INFO:tasks.workunit.client.0.smithi081.stdout:0/143: mknod d3/dc/d2b/c2c 0 2024-03-22T07:08:36.972 INFO:tasks.workunit.client.0.smithi081.stdout:0/144: truncate d3/f8 768421 0 2024-03-22T07:08:36.973 INFO:tasks.workunit.client.0.smithi081.stdout:0/145: write d3/f1d [915536,19241] 0 2024-03-22T07:08:36.977 INFO:tasks.workunit.client.0.smithi081.stdout:9/92: dwrite d2/f4 [0,4194304] 0 2024-03-22T07:08:36.983 INFO:tasks.workunit.client.0.smithi081.stdout:9/93: creat d2/f1e x:0 0 0 2024-03-22T07:08:36.984 INFO:tasks.workunit.client.0.smithi081.stdout:2/59: dread f4 [4194304,4194304] 0 2024-03-22T07:08:36.984 INFO:tasks.workunit.client.0.smithi081.stdout:2/60: chown c9 62157 1 2024-03-22T07:08:36.987 INFO:tasks.workunit.client.0.smithi081.stdout:9/94: fdatasync d2/f15 0 2024-03-22T07:08:36.987 INFO:tasks.workunit.client.0.smithi081.stdout:9/95: readlink d2/lc 0 2024-03-22T07:08:36.987 INFO:tasks.workunit.client.0.smithi081.stdout:2/61: truncate f4 1591355 0 2024-03-22T07:08:36.987 INFO:tasks.workunit.client.0.smithi081.stdout:2/62: dread - f5 zero size 2024-03-22T07:08:36.987 INFO:tasks.workunit.client.0.smithi081.stdout:2/63: chown c7 579562888 1 2024-03-22T07:08:36.988 INFO:tasks.workunit.client.0.smithi081.stdout:2/64: write da/f13 [270090,124641] 0 2024-03-22T07:08:36.988 INFO:tasks.workunit.client.0.smithi081.stdout:9/96: symlink d2/d5/d16/l1f 0 2024-03-22T07:08:37.008 INFO:tasks.workunit.client.0.smithi081.stdout:9/97: write d2/d10/f14 [1624020,35132] 0 2024-03-22T07:08:37.008 INFO:tasks.workunit.client.0.smithi081.stdout:9/98: chown d2/d5/f6 66023803 1 2024-03-22T07:08:37.009 INFO:tasks.workunit.client.0.smithi081.stdout:9/99: chown d2/d10/f17 403 1 2024-03-22T07:08:37.009 INFO:tasks.workunit.client.0.smithi081.stdout:9/100: stat d2/d10 0 2024-03-22T07:08:37.011 INFO:tasks.workunit.client.0.smithi081.stdout:0/146: dwrite d3/f8 [0,4194304] 0 2024-03-22T07:08:37.012 INFO:tasks.workunit.client.0.smithi081.stdout:0/147: fdatasync d3/f22 0 2024-03-22T07:08:37.013 INFO:tasks.workunit.client.0.smithi081.stdout:0/148: getdents d3/dc 0 2024-03-22T07:08:37.013 INFO:tasks.workunit.client.0.smithi081.stdout:0/149: fdatasync d3/dc/f15 0 2024-03-22T07:08:37.014 INFO:tasks.workunit.client.0.smithi081.stdout:0/150: creat d3/f2d x:0 0 0 2024-03-22T07:08:37.020 INFO:tasks.workunit.client.0.smithi081.stdout:2/65: dwrite f6 [0,4194304] 0 2024-03-22T07:08:37.027 INFO:tasks.workunit.client.0.smithi081.stdout:9/101: dread d2/f4 [0,4194304] 0 2024-03-22T07:08:37.030 INFO:tasks.workunit.client.0.smithi081.stdout:9/102: link d2/f12 d2/f20 0 2024-03-22T07:08:37.037 INFO:tasks.workunit.client.0.smithi081.stdout:2/66: write da/f13 [2679965,92367] 0 2024-03-22T07:08:37.042 INFO:tasks.workunit.client.0.smithi081.stdout:2/67: fsync da/f13 0 2024-03-22T07:08:37.057 INFO:tasks.workunit.client.0.smithi081.stdout:9/103: dwrite d2/fe [0,4194304] 0 2024-03-22T07:08:37.057 INFO:tasks.workunit.client.0.smithi081.stdout:2/68: dwrite da/f13 [4194304,4194304] 0 2024-03-22T07:08:37.057 INFO:tasks.workunit.client.0.smithi081.stdout:2/69: chown da/db/cc 3 1 2024-03-22T07:08:37.059 INFO:tasks.workunit.client.0.smithi081.stdout:9/104: creat d2/d10/d1b/f21 x:0 0 0 2024-03-22T07:08:37.059 INFO:tasks.workunit.client.0.smithi081.stdout:9/105: fsync d2/f1e 0 2024-03-22T07:08:37.061 INFO:tasks.workunit.client.0.smithi081.stdout:9/106: symlink d2/d10/d1b/l22 0 2024-03-22T07:08:37.062 INFO:tasks.workunit.client.0.smithi081.stdout:9/107: mknod d2/d5/c23 0 2024-03-22T07:08:37.062 INFO:tasks.workunit.client.0.smithi081.stdout:9/108: chown d2/d5/d16 0 1 2024-03-22T07:08:37.063 INFO:tasks.workunit.client.0.smithi081.stdout:9/109: mknod d2/d5/d16/c24 0 2024-03-22T07:08:37.066 INFO:tasks.workunit.client.0.smithi081.stdout:2/70: dread f6 [4194304,4194304] 0 2024-03-22T07:08:37.068 INFO:tasks.workunit.client.0.smithi081.stdout:2/71: unlink c9 0 2024-03-22T07:08:37.072 INFO:tasks.workunit.client.0.smithi081.stdout:2/72: dread da/f13 [4194304,4194304] 0 2024-03-22T07:08:37.072 INFO:tasks.workunit.client.0.smithi081.stdout:1/103: rename d6/fb to d6/d18/f27 0 2024-03-22T07:08:37.073 INFO:tasks.workunit.client.0.smithi081.stdout:2/73: mkdir da/db/d14 0 2024-03-22T07:08:37.074 INFO:tasks.workunit.client.0.smithi081.stdout:6/118: sync 2024-03-22T07:08:37.074 INFO:tasks.workunit.client.0.smithi081.stdout:5/202: sync 2024-03-22T07:08:37.074 INFO:tasks.workunit.client.0.smithi081.stdout:7/261: sync 2024-03-22T07:08:37.074 INFO:tasks.workunit.client.0.smithi081.stdout:8/97: sync 2024-03-22T07:08:37.074 INFO:tasks.workunit.client.0.smithi081.stdout:5/203: dread - d4/d10/d20/d25/f43 zero size 2024-03-22T07:08:37.074 INFO:tasks.workunit.client.0.smithi081.stdout:4/174: sync 2024-03-22T07:08:37.075 INFO:tasks.workunit.client.0.smithi081.stdout:1/104: creat d6/d18/f28 x:0 0 0 2024-03-22T07:08:37.077 INFO:tasks.workunit.client.0.smithi081.stdout:3/90: rename db/l11 to db/de/l1c 0 2024-03-22T07:08:37.085 INFO:tasks.workunit.client.0.smithi081.stdout:0/151: rename f2 to d3/f2e 0 2024-03-22T07:08:37.089 INFO:tasks.workunit.client.0.smithi081.stdout:3/91: write db/f13 [1381399,108769] 0 2024-03-22T07:08:37.089 INFO:tasks.workunit.client.0.smithi081.stdout:3/92: dread - db/de/f1a zero size 2024-03-22T07:08:37.089 INFO:tasks.workunit.client.0.smithi081.stdout:3/93: chown c9 7158 1 2024-03-22T07:08:37.089 INFO:tasks.workunit.client.0.smithi081.stdout:6/119: unlink d3/da/de/df/l19 0 2024-03-22T07:08:37.089 INFO:tasks.workunit.client.0.smithi081.stdout:6/120: readlink d3/da/de/l20 0 2024-03-22T07:08:37.095 INFO:tasks.workunit.client.0.smithi081.stdout:3/94: mknod db/c1d 0 2024-03-22T07:08:37.096 INFO:tasks.workunit.client.0.smithi081.stdout:9/110: dwrite d2/f3 [0,4194304] 0 2024-03-22T07:08:37.096 INFO:tasks.workunit.client.0.smithi081.stdout:9/111: chown d2/d5 31 1 2024-03-22T07:08:37.097 INFO:tasks.workunit.client.0.smithi081.stdout:7/262: dread d9/d10/d16/d1e/f28 [0,4194304] 0 2024-03-22T07:08:37.097 INFO:tasks.workunit.client.0.smithi081.stdout:7/263: stat f4 0 2024-03-22T07:08:37.103 INFO:tasks.workunit.client.0.smithi081.stdout:7/264: symlink d9/d10/d16/d1e/d27/l68 0 2024-03-22T07:08:37.103 INFO:tasks.workunit.client.0.smithi081.stdout:7/265: read d9/d10/f3a [1256976,101517] 0 2024-03-22T07:08:37.103 INFO:tasks.workunit.client.0.smithi081.stdout:7/266: read d9/f11 [214773,66278] 0 2024-03-22T07:08:37.103 INFO:tasks.workunit.client.0.smithi081.stdout:7/267: truncate d9/d10/d16/d1e/d27/f5a 631100 0 2024-03-22T07:08:37.104 INFO:tasks.workunit.client.0.smithi081.stdout:1/105: dwrite d6/d18/d1f/f25 [0,4194304] 0 2024-03-22T07:08:37.105 INFO:tasks.workunit.client.0.smithi081.stdout:7/268: getdents d9/d10/d4b/d61 0 2024-03-22T07:08:37.114 INFO:tasks.workunit.client.0.smithi081.stdout:9/112: write d2/fe [3771988,54481] 0 2024-03-22T07:08:37.114 INFO:tasks.workunit.client.0.smithi081.stdout:9/113: creat d2/f25 x:0 0 0 2024-03-22T07:08:37.117 INFO:tasks.workunit.client.0.smithi081.stdout:7/269: dread d9/d10/d16/d1e/f28 [0,4194304] 0 2024-03-22T07:08:37.117 INFO:tasks.workunit.client.0.smithi081.stdout:7/270: fdatasync d9/d3f/f65 0 2024-03-22T07:08:37.118 INFO:tasks.workunit.client.0.smithi081.stdout:7/271: dread - d9/dc/f4a zero size 2024-03-22T07:08:37.118 INFO:tasks.workunit.client.0.smithi081.stdout:1/106: write d6/d18/d1f/f25 [2253363,19717] 0 2024-03-22T07:08:37.119 INFO:tasks.workunit.client.0.smithi081.stdout:9/114: dread d2/f15 [0,4194304] 0 2024-03-22T07:08:37.120 INFO:tasks.workunit.client.0.smithi081.stdout:9/115: write d2/f25 [20371,88839] 0 2024-03-22T07:08:37.120 INFO:tasks.workunit.client.0.smithi081.stdout:9/116: creat d2/d10/d1b/f26 x:0 0 0 2024-03-22T07:08:37.121 INFO:tasks.workunit.client.0.smithi081.stdout:6/121: dwrite d3/d4/f7 [0,4194304] 0 2024-03-22T07:08:37.121 INFO:tasks.workunit.client.0.smithi081.stdout:4/175: dwrite da/d18/d1b/d26/f2a [0,4194304] 0 2024-03-22T07:08:37.121 INFO:tasks.workunit.client.0.smithi081.stdout:4/176: chown da/d18/f21 140521 1 2024-03-22T07:08:37.125 INFO:tasks.workunit.client.0.smithi081.stdout:3/95: dread db/f13 [0,4194304] 0 2024-03-22T07:08:37.126 INFO:tasks.workunit.client.0.smithi081.stdout:2/74: dwrite f4 [0,4194304] 0 2024-03-22T07:08:37.126 INFO:tasks.workunit.client.0.smithi081.stdout:8/98: dwrite f1 [0,4194304] 0 2024-03-22T07:08:37.128 INFO:tasks.workunit.client.0.smithi081.stdout:3/96: creat db/f1e x:0 0 0 2024-03-22T07:08:37.128 INFO:tasks.workunit.client.0.smithi081.stdout:9/117: dread d2/d5/f7 [0,4194304] 0 2024-03-22T07:08:37.131 INFO:tasks.workunit.client.0.smithi081.stdout:6/122: dread d3/d4/f7 [0,4194304] 0 2024-03-22T07:08:37.133 INFO:tasks.workunit.client.0.smithi081.stdout:6/123: mknod d3/d12/c21 0 2024-03-22T07:08:37.134 INFO:tasks.workunit.client.0.smithi081.stdout:2/75: write f6 [6509652,90080] 0 2024-03-22T07:08:37.135 INFO:tasks.workunit.client.0.smithi081.stdout:2/76: read da/f13 [237670,48180] 0 2024-03-22T07:08:37.136 INFO:tasks.workunit.client.0.smithi081.stdout:0/152: rmdir d3 39 2024-03-22T07:08:37.138 INFO:tasks.workunit.client.0.smithi081.stdout:1/107: dwrite d6/d18/f1d [0,4194304] 0 2024-03-22T07:08:37.138 INFO:tasks.workunit.client.0.smithi081.stdout:1/108: write d6/d18/d1f/d24/f26 [852837,32249] 0 2024-03-22T07:08:37.138 INFO:tasks.workunit.client.0.smithi081.stdout:8/99: dread d2/f15 [0,4194304] 0 2024-03-22T07:08:37.138 INFO:tasks.workunit.client.0.smithi081.stdout:8/100: dread - d2/d3/f1f zero size 2024-03-22T07:08:37.138 INFO:tasks.workunit.client.0.smithi081.stdout:8/101: read - d2/d3/f1b zero size 2024-03-22T07:08:37.138 INFO:tasks.workunit.client.0.smithi081.stdout:8/102: dread - d2/d3/f1b zero size 2024-03-22T07:08:37.139 INFO:tasks.workunit.client.0.smithi081.stdout:8/103: creat d2/f24 x:0 0 0 2024-03-22T07:08:37.145 INFO:tasks.workunit.client.0.smithi081.stdout:3/97: write db/f12 [1558906,87884] 0 2024-03-22T07:08:37.147 INFO:tasks.workunit.client.0.smithi081.stdout:7/272: dread d9/dc/f1a [0,4194304] 0 2024-03-22T07:08:37.147 INFO:tasks.workunit.client.0.smithi081.stdout:7/273: chown d9/l44 822826 1 2024-03-22T07:08:37.147 INFO:tasks.workunit.client.0.smithi081.stdout:7/274: fsync f2 0 2024-03-22T07:08:37.147 INFO:tasks.workunit.client.0.smithi081.stdout:9/118: dread d2/d10/f17 [0,4194304] 0 2024-03-22T07:08:37.148 INFO:tasks.workunit.client.0.smithi081.stdout:1/109: read d6/d18/f1d [1153684,70964] 0 2024-03-22T07:08:37.152 INFO:tasks.workunit.client.0.smithi081.stdout:3/98: mknod db/c1f 0 2024-03-22T07:08:37.152 INFO:tasks.workunit.client.0.smithi081.stdout:3/99: write db/de/f17 [1668560,179] 0 2024-03-22T07:08:37.152 INFO:tasks.workunit.client.0.smithi081.stdout:3/100: write db/f12 [3967306,121149] 0 2024-03-22T07:08:37.152 INFO:tasks.workunit.client.0.smithi081.stdout:3/101: read - db/f1e zero size 2024-03-22T07:08:37.152 INFO:tasks.workunit.client.0.smithi081.stdout:3/102: creat db/de/f20 x:0 0 0 2024-03-22T07:08:37.152 INFO:tasks.workunit.client.0.smithi081.stdout:3/103: fdatasync db/f1e 0 2024-03-22T07:08:37.153 INFO:tasks.workunit.client.0.smithi081.stdout:8/104: write f1 [3030262,3065] 0 2024-03-22T07:08:37.153 INFO:tasks.workunit.client.0.smithi081.stdout:8/105: truncate d2/f24 635042 0 2024-03-22T07:08:37.153 INFO:tasks.workunit.client.0.smithi081.stdout:9/119: mknod d2/d5/d16/c27 0 2024-03-22T07:08:37.153 INFO:tasks.workunit.client.0.smithi081.stdout:9/120: readlink d2/d5/d16/l1c 0 2024-03-22T07:08:37.153 INFO:tasks.workunit.client.0.smithi081.stdout:9/121: readlink d2/d5/d16/l1a 0 2024-03-22T07:08:37.156 INFO:tasks.workunit.client.0.smithi081.stdout:6/124: write d3/f6 [1002456,125381] 0 2024-03-22T07:08:37.156 INFO:tasks.workunit.client.0.smithi081.stdout:0/153: dwrite d3/f22 [0,4194304] 0 2024-03-22T07:08:37.157 INFO:tasks.workunit.client.0.smithi081.stdout:3/104: symlink db/l21 0 2024-03-22T07:08:37.157 INFO:tasks.workunit.client.0.smithi081.stdout:3/105: dread - db/f1e zero size 2024-03-22T07:08:37.158 INFO:tasks.workunit.client.0.smithi081.stdout:7/275: write d9/d10/d16/d1e/f28 [793248,32151] 0 2024-03-22T07:08:37.162 INFO:tasks.workunit.client.0.smithi081.stdout:8/106: unlink d2/d3/f16 0 2024-03-22T07:08:37.164 INFO:tasks.workunit.client.0.smithi081.stdout:9/122: mknod d2/d10/c28 0 2024-03-22T07:08:37.164 INFO:tasks.workunit.client.0.smithi081.stdout:9/123: write d2/f15 [1213480,109445] 0 2024-03-22T07:08:37.164 INFO:tasks.workunit.client.0.smithi081.stdout:9/124: chown d2/d10 0 1 2024-03-22T07:08:37.164 INFO:tasks.workunit.client.0.smithi081.stdout:2/77: dwrite f6 [4194304,4194304] 0 2024-03-22T07:08:37.166 INFO:tasks.workunit.client.0.smithi081.stdout:6/125: creat d3/f22 x:0 0 0 2024-03-22T07:08:37.169 INFO:tasks.workunit.client.0.smithi081.stdout:0/154: creat d3/dc/d2b/f2f x:0 0 0 2024-03-22T07:08:37.169 INFO:tasks.workunit.client.0.smithi081.stdout:9/125: dread d2/d5/d16/f18 [0,4194304] 0 2024-03-22T07:08:37.170 INFO:tasks.workunit.client.0.smithi081.stdout:4/177: rename da/dd to da/d18/d1b/d1e/d37 0 2024-03-22T07:08:37.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/78: creat da/d11/f15 x:0 0 0 2024-03-22T07:08:37.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/79: fsync da/f13 0 2024-03-22T07:08:37.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/80: truncate f5 865675 0 2024-03-22T07:08:37.190 INFO:tasks.workunit.client.0.smithi081.stdout:6/126: symlink d3/da/l23 0 2024-03-22T07:08:37.193 INFO:tasks.workunit.client.0.smithi081.stdout:2/81: unlink c8 0 2024-03-22T07:08:37.193 INFO:tasks.workunit.client.0.smithi081.stdout:4/178: write da/fe [302805,41363] 0 2024-03-22T07:08:37.193 INFO:tasks.workunit.client.0.smithi081.stdout:4/179: readlink da/df/l13 0 2024-03-22T07:08:37.194 INFO:tasks.workunit.client.0.smithi081.stdout:3/106: rmdir db 39 2024-03-22T07:08:37.194 INFO:tasks.workunit.client.0.smithi081.stdout:3/107: write db/f13 [1568687,127892] 0 2024-03-22T07:08:37.194 INFO:tasks.workunit.client.0.smithi081.stdout:3/108: fdatasync db/f18 0 2024-03-22T07:08:37.196 INFO:tasks.workunit.client.0.smithi081.stdout:6/127: chown d3/da/de/df/f15 2320 1 2024-03-22T07:08:37.196 INFO:tasks.workunit.client.0.smithi081.stdout:6/128: write d3/da/de/f1f [197377,95865] 0 2024-03-22T07:08:37.196 INFO:tasks.workunit.client.0.smithi081.stdout:6/129: stat d3 0 2024-03-22T07:08:37.196 INFO:tasks.workunit.client.0.smithi081.stdout:6/130: chown d3/d12/f1c 99346 1 2024-03-22T07:08:37.198 INFO:tasks.workunit.client.0.smithi081.stdout:2/82: rmdir da/db/d14 0 2024-03-22T07:08:37.199 INFO:tasks.workunit.client.0.smithi081.stdout:2/83: chown l1 70488744 1 2024-03-22T07:08:37.204 INFO:tasks.workunit.client.0.smithi081.stdout:0/155: dwrite d3/f22 [4194304,4194304] 0 2024-03-22T07:08:37.207 INFO:tasks.workunit.client.0.smithi081.stdout:4/180: write da/d18/d1b/d26/f2a [1665936,62607] 0 2024-03-22T07:08:37.207 INFO:tasks.workunit.client.0.smithi081.stdout:4/181: chown da/df/l17 615 1 2024-03-22T07:08:37.207 INFO:tasks.workunit.client.0.smithi081.stdout:7/276: dwrite d9/dc/f1a [0,4194304] 0 2024-03-22T07:08:37.208 INFO:tasks.workunit.client.0.smithi081.stdout:4/182: mkdir da/df/d38 0 2024-03-22T07:08:37.210 INFO:tasks.workunit.client.0.smithi081.stdout:2/84: write f6 [4604224,9989] 0 2024-03-22T07:08:37.210 INFO:tasks.workunit.client.0.smithi081.stdout:3/109: rmdir db 39 2024-03-22T07:08:37.210 INFO:tasks.workunit.client.0.smithi081.stdout:3/110: dread - db/fc zero size 2024-03-22T07:08:37.212 INFO:tasks.workunit.client.0.smithi081.stdout:7/277: symlink d9/d3f/d52/l69 0 2024-03-22T07:08:37.214 INFO:tasks.workunit.client.0.smithi081.stdout:7/278: dread d9/d10/d16/f42 [0,4194304] 0 2024-03-22T07:08:37.218 INFO:tasks.workunit.client.0.smithi081.stdout:2/85: fdatasync da/f13 0 2024-03-22T07:08:37.218 INFO:tasks.workunit.client.0.smithi081.stdout:8/107: dwrite d2/d3/f1c [0,4194304] 0 2024-03-22T07:08:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:3/111: dread db/f13 [0,4194304] 0 2024-03-22T07:08:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:0/156: dread d3/f8 [0,4194304] 0 2024-03-22T07:08:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:9/126: dwrite d2/d10/f14 [0,4194304] 0 2024-03-22T07:08:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:0/157: read - d3/f2d zero size 2024-03-22T07:08:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:0/158: fsync d3/f2e 0 2024-03-22T07:08:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:0/159: chown d3/l9 11093176 1 2024-03-22T07:08:37.223 INFO:tasks.workunit.client.0.smithi081.stdout:7/279: truncate d9/dc/f33 675347 0 2024-03-22T07:08:37.228 INFO:tasks.workunit.client.0.smithi081.stdout:2/86: mknod da/db/c16 0 2024-03-22T07:08:37.229 INFO:tasks.workunit.client.0.smithi081.stdout:2/87: write da/f13 [3171311,63700] 0 2024-03-22T07:08:37.229 INFO:tasks.workunit.client.0.smithi081.stdout:8/108: creat d2/d3/d19/f25 x:0 0 0 2024-03-22T07:08:37.231 INFO:tasks.workunit.client.0.smithi081.stdout:4/183: dwrite da/d18/d1b/d1e/d37/f25 [0,4194304] 0 2024-03-22T07:08:37.232 INFO:tasks.workunit.client.0.smithi081.stdout:9/127: creat d2/d10/f29 x:0 0 0 2024-03-22T07:08:37.234 INFO:tasks.workunit.client.0.smithi081.stdout:0/160: mknod d3/dc/c30 0 2024-03-22T07:08:37.234 INFO:tasks.workunit.client.0.smithi081.stdout:0/161: creat d3/dc/d2b/f31 x:0 0 0 2024-03-22T07:08:37.235 INFO:tasks.workunit.client.0.smithi081.stdout:7/280: symlink d9/d10/d16/l6a 0 2024-03-22T07:08:37.244 INFO:tasks.workunit.client.0.smithi081.stdout:3/112: truncate db/de/f17 3353074 0 2024-03-22T07:08:37.244 INFO:tasks.workunit.client.0.smithi081.stdout:3/113: read - db/f1e zero size 2024-03-22T07:08:37.252 INFO:tasks.workunit.client.0.smithi081.stdout:9/128: dwrite d2/d10/f29 [0,4194304] 0 2024-03-22T07:08:37.252 INFO:tasks.workunit.client.0.smithi081.stdout:9/129: stat d2/d10/d1b/f21 0 2024-03-22T07:08:37.255 INFO:tasks.workunit.client.0.smithi081.stdout:9/130: dread d2/d5/f6 [0,4194304] 0 2024-03-22T07:08:37.255 INFO:tasks.workunit.client.0.smithi081.stdout:9/131: write d2/d5/d16/f18 [2135557,68343] 0 2024-03-22T07:08:37.265 INFO:tasks.workunit.client.0.smithi081.stdout:4/184: link da/d18/d1b/d1e/d37/f20 da/d18/d1b/d26/d32/f39 0 2024-03-22T07:08:37.267 INFO:tasks.workunit.client.0.smithi081.stdout:3/114: unlink db/c16 0 2024-03-22T07:08:37.268 INFO:tasks.workunit.client.0.smithi081.stdout:9/132: mknod d2/d10/d1b/c2a 0 2024-03-22T07:08:37.269 INFO:tasks.workunit.client.0.smithi081.stdout:0/162: mkdir d3/d32 0 2024-03-22T07:08:37.270 INFO:tasks.workunit.client.0.smithi081.stdout:0/163: write d3/dc/d2b/f2f [465910,72419] 0 2024-03-22T07:08:37.270 INFO:tasks.workunit.client.0.smithi081.stdout:0/164: stat d3 0 2024-03-22T07:08:37.270 INFO:tasks.workunit.client.0.smithi081.stdout:3/115: unlink db/cf 0 2024-03-22T07:08:37.272 INFO:tasks.workunit.client.0.smithi081.stdout:0/165: mkdir d3/dc/d33 0 2024-03-22T07:08:37.272 INFO:tasks.workunit.client.0.smithi081.stdout:0/166: write d3/dc/f1b [5054093,129309] 0 2024-03-22T07:08:37.273 INFO:tasks.workunit.client.0.smithi081.stdout:3/116: symlink db/l22 0 2024-03-22T07:08:37.273 INFO:tasks.workunit.client.0.smithi081.stdout:3/117: dread - db/de/f20 zero size 2024-03-22T07:08:37.276 INFO:tasks.workunit.client.0.smithi081.stdout:0/167: symlink d3/dc/l34 0 2024-03-22T07:08:37.276 INFO:tasks.workunit.client.0.smithi081.stdout:0/168: chown d3/dc/l1e 12 1 2024-03-22T07:08:37.277 INFO:tasks.workunit.client.0.smithi081.stdout:3/118: mkdir db/d23 0 2024-03-22T07:08:37.279 INFO:tasks.workunit.client.0.smithi081.stdout:3/119: stat db/de/l1c 0 2024-03-22T07:08:37.280 INFO:tasks.workunit.client.0.smithi081.stdout:3/120: creat db/de/f24 x:0 0 0 2024-03-22T07:08:37.282 INFO:tasks.workunit.client.0.smithi081.stdout:4/185: dwrite f0 [0,4194304] 0 2024-03-22T07:08:37.288 INFO:tasks.workunit.client.0.smithi081.stdout:0/169: dread d3/f22 [4194304,4194304] 0 2024-03-22T07:08:37.290 INFO:tasks.workunit.client.0.smithi081.stdout:0/170: mkdir d3/d32/d35 0 2024-03-22T07:08:37.291 INFO:tasks.workunit.client.0.smithi081.stdout:0/171: chown d3/f8 15 1 2024-03-22T07:08:37.291 INFO:tasks.workunit.client.0.smithi081.stdout:0/172: write d3/dc/f1c [2010745,129417] 0 2024-03-22T07:08:37.296 INFO:tasks.workunit.client.0.smithi081.stdout:3/121: dwrite db/de/f24 [0,4194304] 0 2024-03-22T07:08:37.296 INFO:tasks.workunit.client.0.smithi081.stdout:3/122: fsync db/de/f20 0 2024-03-22T07:08:37.302 INFO:tasks.workunit.client.0.smithi081.stdout:4/186: dwrite da/f1d [0,4194304] 0 2024-03-22T07:08:37.302 INFO:tasks.workunit.client.0.smithi081.stdout:4/187: readlink da/l15 0 2024-03-22T07:08:37.302 INFO:tasks.workunit.client.0.smithi081.stdout:0/173: rename d3/dc/ff to d3/dc/f36 0 2024-03-22T07:08:37.305 INFO:tasks.workunit.client.0.smithi081.stdout:4/188: symlink da/d18/d1b/d1e/d37/l3a 0 2024-03-22T07:08:37.316 INFO:tasks.workunit.client.0.smithi081.stdout:1/110: rename d6/ce to d6/d18/d1f/d24/c29 0 2024-03-22T07:08:37.316 INFO:tasks.workunit.client.0.smithi081.stdout:1/111: fsync f1 0 2024-03-22T07:08:37.317 INFO:tasks.workunit.client.0.smithi081.stdout:3/123: dwrite db/de/f24 [0,4194304] 0 2024-03-22T07:08:37.317 INFO:tasks.workunit.client.0.smithi081.stdout:3/124: chown l0 53181 1 2024-03-22T07:08:37.317 INFO:tasks.workunit.client.0.smithi081.stdout:3/125: write db/f18 [351377,16075] 0 2024-03-22T07:08:37.318 INFO:tasks.workunit.client.0.smithi081.stdout:1/112: creat d6/f2a x:0 0 0 2024-03-22T07:08:37.318 INFO:tasks.workunit.client.0.smithi081.stdout:1/113: fsync d6/d18/d1f/f25 0 2024-03-22T07:08:37.318 INFO:tasks.workunit.client.0.smithi081.stdout:1/114: write d6/d18/f21 [1629673,20611] 0 2024-03-22T07:08:37.318 INFO:tasks.workunit.client.0.smithi081.stdout:1/115: creat d6/d18/d1f/d24/f2b x:0 0 0 2024-03-22T07:08:37.322 INFO:tasks.workunit.client.0.smithi081.stdout:9/133: rename d2/d10/d1b to d2/d2b 0 2024-03-22T07:08:37.322 INFO:tasks.workunit.client.0.smithi081.stdout:0/174: rename d3/dc to d3/dc/d37 22 2024-03-22T07:08:37.322 INFO:tasks.workunit.client.0.smithi081.stdout:9/134: chown d2/l9 1077393364 1 2024-03-22T07:08:37.325 INFO:tasks.workunit.client.0.smithi081.stdout:4/189: truncate f0 3980210 0 2024-03-22T07:08:37.325 INFO:tasks.workunit.client.0.smithi081.stdout:1/116: creat d6/d18/f2c x:0 0 0 2024-03-22T07:08:37.326 INFO:tasks.workunit.client.0.smithi081.stdout:8/109: link d2/d3/d19/c1a d2/d3/d23/c26 0 2024-03-22T07:08:37.329 INFO:tasks.workunit.client.0.smithi081.stdout:1/117: getdents d6/d18/d1f 0 2024-03-22T07:08:37.330 INFO:tasks.workunit.client.0.smithi081.stdout:1/118: write d6/f12 [1406385,32584] 0 2024-03-22T07:08:37.330 INFO:tasks.workunit.client.0.smithi081.stdout:9/135: symlink d2/d10/l2c 0 2024-03-22T07:08:37.332 INFO:tasks.workunit.client.0.smithi081.stdout:9/136: rename d2/d5/f6 to d2/d2b/f2d 0 2024-03-22T07:08:37.332 INFO:tasks.workunit.client.0.smithi081.stdout:1/119: mknod d6/d18/d1f/d24/c2d 0 2024-03-22T07:08:37.334 INFO:tasks.workunit.client.0.smithi081.stdout:9/137: creat d2/d5/d16/f2e x:0 0 0 2024-03-22T07:08:37.334 INFO:tasks.workunit.client.0.smithi081.stdout:9/138: read - d2/f1e zero size 2024-03-22T07:08:37.336 INFO:tasks.workunit.client.0.smithi081.stdout:4/190: dread da/d18/d1b/d1e/d37/f25 [0,4194304] 0 2024-03-22T07:08:37.339 INFO:tasks.workunit.client.0.smithi081.stdout:8/110: symlink d2/d3/d19/d1e/l27 0 2024-03-22T07:08:37.339 INFO:tasks.workunit.client.0.smithi081.stdout:8/111: readlink d2/d3/lf 0 2024-03-22T07:08:37.340 INFO:tasks.workunit.client.0.smithi081.stdout:4/191: truncate da/d18/d1b/f24 359741 0 2024-03-22T07:08:37.343 INFO:tasks.workunit.client.0.smithi081.stdout:1/120: rmdir d6/d18/d1f/d22 0 2024-03-22T07:08:37.343 INFO:tasks.workunit.client.0.smithi081.stdout:1/121: dread - d6/d18/f2c zero size 2024-03-22T07:08:37.345 INFO:tasks.workunit.client.0.smithi081.stdout:0/175: dwrite d3/dc/f1c [0,4194304] 0 2024-03-22T07:08:37.348 INFO:tasks.workunit.client.0.smithi081.stdout:9/139: write d2/f4 [4153097,7388] 0 2024-03-22T07:08:37.348 INFO:tasks.workunit.client.0.smithi081.stdout:9/140: fsync d2/d2b/f26 0 2024-03-22T07:08:37.348 INFO:tasks.workunit.client.0.smithi081.stdout:9/141: truncate d2/d2b/f26 702044 0 2024-03-22T07:08:37.348 INFO:tasks.workunit.client.0.smithi081.stdout:9/142: creat d2/d5/d16/f2f x:0 0 0 2024-03-22T07:08:37.349 INFO:tasks.workunit.client.0.smithi081.stdout:1/122: rmdir d6/d18/d1f/d23 0 2024-03-22T07:08:37.352 INFO:tasks.workunit.client.0.smithi081.stdout:0/176: creat d3/d32/d35/f38 x:0 0 0 2024-03-22T07:08:37.354 INFO:tasks.workunit.client.0.smithi081.stdout:9/143: mkdir d2/d10/d30 0 2024-03-22T07:08:37.355 INFO:tasks.workunit.client.0.smithi081.stdout:4/192: link da/d18/d1b/d1e/d37/f20 da/d18/d1b/d1e/f3b 0 2024-03-22T07:08:37.357 INFO:tasks.workunit.client.0.smithi081.stdout:1/123: mkdir d6/d2e 0 2024-03-22T07:08:37.358 INFO:tasks.workunit.client.0.smithi081.stdout:0/177: mkdir d3/dc/d2b/d39 0 2024-03-22T07:08:37.359 INFO:tasks.workunit.client.0.smithi081.stdout:8/112: dwrite d2/f15 [0,4194304] 0 2024-03-22T07:08:37.359 INFO:tasks.workunit.client.0.smithi081.stdout:8/113: dread - d2/d3/d19/f25 zero size 2024-03-22T07:08:37.360 INFO:tasks.workunit.client.0.smithi081.stdout:8/114: chown d2/f24 23587108 1 2024-03-22T07:08:37.360 INFO:tasks.workunit.client.0.smithi081.stdout:8/115: write d2/d3/d19/f25 [11301,105692] 0 2024-03-22T07:08:37.360 INFO:tasks.workunit.client.0.smithi081.stdout:8/116: stat f1 0 2024-03-22T07:08:37.360 INFO:tasks.workunit.client.0.smithi081.stdout:8/117: chown f1 46 1 2024-03-22T07:08:37.360 INFO:tasks.workunit.client.0.smithi081.stdout:9/144: creat d2/d5/f31 x:0 0 0 2024-03-22T07:08:37.360 INFO:tasks.workunit.client.0.smithi081.stdout:4/193: symlink da/df/l3c 0 2024-03-22T07:08:37.361 INFO:tasks.workunit.client.0.smithi081.stdout:1/124: truncate d6/fc 1937192 0 2024-03-22T07:08:37.361 INFO:tasks.workunit.client.0.smithi081.stdout:1/125: chown d6/c15 6155880 1 2024-03-22T07:08:37.362 INFO:tasks.workunit.client.0.smithi081.stdout:9/145: mknod d2/d2b/c32 0 2024-03-22T07:08:37.365 INFO:tasks.workunit.client.0.smithi081.stdout:4/194: mkdir da/d3d 0 2024-03-22T07:08:37.367 INFO:tasks.workunit.client.0.smithi081.stdout:9/146: rename d2/d2b/l22 to d2/d10/d30/l33 0 2024-03-22T07:08:37.369 INFO:tasks.workunit.client.0.smithi081.stdout:9/147: dread d2/d2b/f26 [0,4194304] 0 2024-03-22T07:08:37.369 INFO:tasks.workunit.client.0.smithi081.stdout:9/148: stat d2/d5/d16 0 2024-03-22T07:08:37.374 INFO:tasks.workunit.client.0.smithi081.stdout:9/149: mknod d2/c34 0 2024-03-22T07:08:37.374 INFO:tasks.workunit.client.0.smithi081.stdout:9/150: write d2/f20 [1139478,126100] 0 2024-03-22T07:08:37.379 INFO:tasks.workunit.client.0.smithi081.stdout:4/195: getdents da 0 2024-03-22T07:08:37.380 INFO:tasks.workunit.client.0.smithi081.stdout:4/196: write da/d18/d1b/d1e/d37/d1c/f28 [550254,15153] 0 2024-03-22T07:08:37.380 INFO:tasks.workunit.client.0.smithi081.stdout:4/197: stat da/c30 0 2024-03-22T07:08:37.380 INFO:tasks.workunit.client.0.smithi081.stdout:4/198: fsync f3 0 2024-03-22T07:08:37.381 INFO:tasks.workunit.client.0.smithi081.stdout:8/118: dread d2/d3/f1c [0,4194304] 0 2024-03-22T07:08:37.381 INFO:tasks.workunit.client.0.smithi081.stdout:8/119: stat d2/d3/f1b 0 2024-03-22T07:08:37.382 INFO:tasks.workunit.client.0.smithi081.stdout:8/120: readlink d2/d3/lf 0 2024-03-22T07:08:37.382 INFO:tasks.workunit.client.0.smithi081.stdout:8/121: read - d2/d3/f1b zero size 2024-03-22T07:08:37.384 INFO:tasks.workunit.client.0.smithi081.stdout:0/178: dread d3/f4 [4194304,4194304] 0 2024-03-22T07:08:37.384 INFO:tasks.workunit.client.0.smithi081.stdout:4/199: dread da/d18/d1b/f24 [0,4194304] 0 2024-03-22T07:08:37.384 INFO:tasks.workunit.client.0.smithi081.stdout:4/200: fdatasync da/d18/d1b/d26/f2a 0 2024-03-22T07:08:37.384 INFO:tasks.workunit.client.0.smithi081.stdout:0/179: truncate d3/f2d 532582 0 2024-03-22T07:08:37.384 INFO:tasks.workunit.client.0.smithi081.stdout:8/122: mkdir d2/d3/d28 0 2024-03-22T07:08:37.388 INFO:tasks.workunit.client.0.smithi081.stdout:4/201: dread da/d18/d1b/d1e/d37/f25 [0,4194304] 0 2024-03-22T07:08:37.393 INFO:tasks.workunit.client.0.smithi081.stdout:4/202: mkdir da/d18/d1b/d26/d3e 0 2024-03-22T07:08:37.398 INFO:tasks.workunit.client.0.smithi081.stdout:1/126: dwrite d6/f12 [0,4194304] 0 2024-03-22T07:08:37.408 INFO:tasks.workunit.client.0.smithi081.stdout:4/203: write da/d18/d1b/d26/d32/f39 [1730261,35143] 0 2024-03-22T07:08:37.412 INFO:tasks.workunit.client.0.smithi081.stdout:5/204: sync 2024-03-22T07:08:37.420 INFO:tasks.workunit.client.0.smithi081.stdout:4/204: dread da/f1d [0,4194304] 0 2024-03-22T07:08:37.421 INFO:tasks.workunit.client.0.smithi081.stdout:4/205: write da/d18/d1b/f24 [282190,66928] 0 2024-03-22T07:08:37.421 INFO:tasks.workunit.client.0.smithi081.stdout:4/206: write da/f2c [155510,63062] 0 2024-03-22T07:08:37.421 INFO:tasks.workunit.client.0.smithi081.stdout:4/207: chown da/d18/d1b/d1e/d37/l3a 47420145 1 2024-03-22T07:08:37.423 INFO:tasks.workunit.client.0.smithi081.stdout:1/127: dwrite d6/fc [0,4194304] 0 2024-03-22T07:08:37.423 INFO:tasks.workunit.client.0.smithi081.stdout:9/151: dwrite d2/d2b/f2d [0,4194304] 0 2024-03-22T07:08:37.423 INFO:tasks.workunit.client.0.smithi081.stdout:9/152: fsync d2/f8 0 2024-03-22T07:08:37.423 INFO:tasks.workunit.client.0.smithi081.stdout:4/208: creat da/d18/d1b/d1e/f3f x:0 0 0 2024-03-22T07:08:37.423 INFO:tasks.workunit.client.0.smithi081.stdout:0/180: dwrite d3/f2d [0,4194304] 0 2024-03-22T07:08:37.425 INFO:tasks.workunit.client.0.smithi081.stdout:1/128: rename d6/c8 to d6/c2f 0 2024-03-22T07:08:37.426 INFO:tasks.workunit.client.0.smithi081.stdout:4/209: creat da/d18/d1b/f40 x:0 0 0 2024-03-22T07:08:37.429 INFO:tasks.workunit.client.0.smithi081.stdout:5/205: write d4/d9/f2c [1818238,34093] 0 2024-03-22T07:08:37.437 INFO:tasks.workunit.client.0.smithi081.stdout:4/210: getdents da/d18/d1b/d1e/d37 0 2024-03-22T07:08:37.442 INFO:tasks.workunit.client.0.smithi081.stdout:5/206: link d4/df/c3a d4/d10/c48 0 2024-03-22T07:08:37.444 INFO:tasks.workunit.client.0.smithi081.stdout:5/207: mkdir d4/df/d22/d49 0 2024-03-22T07:08:37.447 INFO:tasks.workunit.client.0.smithi081.stdout:5/208: creat d4/df/d22/d49/f4a x:0 0 0 2024-03-22T07:08:37.447 INFO:tasks.workunit.client.0.smithi081.stdout:5/209: write d4/d9/f15 [422844,121059] 0 2024-03-22T07:08:37.454 INFO:tasks.workunit.client.0.smithi081.stdout:5/210: dread f2 [0,4194304] 0 2024-03-22T07:08:37.455 INFO:tasks.workunit.client.0.smithi081.stdout:5/211: symlink d4/d10/d20/d1f/d2a/l4b 0 2024-03-22T07:08:37.456 INFO:tasks.workunit.client.0.smithi081.stdout:5/212: dread - d4/f3e zero size 2024-03-22T07:08:37.458 INFO:tasks.workunit.client.0.smithi081.stdout:5/213: link d4/df/f46 d4/d10/d20/d1f/d2f/d45/f4c 0 2024-03-22T07:08:37.458 INFO:tasks.workunit.client.0.smithi081.stdout:5/214: truncate d4/d10/d20/d25/d2e/f31 4664038 0 2024-03-22T07:08:37.458 INFO:tasks.workunit.client.0.smithi081.stdout:5/215: stat d4/d10/d20/d25/d2e/f31 0 2024-03-22T07:08:37.458 INFO:tasks.workunit.client.0.smithi081.stdout:1/129: write d6/fd [389905,22304] 0 2024-03-22T07:08:37.458 INFO:tasks.workunit.client.0.smithi081.stdout:9/153: dwrite d2/d5/d16/f18 [0,4194304] 0 2024-03-22T07:08:37.459 INFO:tasks.workunit.client.0.smithi081.stdout:9/154: fdatasync d2/f4 0 2024-03-22T07:08:37.459 INFO:tasks.workunit.client.0.smithi081.stdout:9/155: creat d2/d10/f35 x:0 0 0 2024-03-22T07:08:37.460 INFO:tasks.workunit.client.0.smithi081.stdout:9/156: mkdir d2/d5/d36 0 2024-03-22T07:08:37.460 INFO:tasks.workunit.client.0.smithi081.stdout:9/157: read - d2/d10/f35 zero size 2024-03-22T07:08:37.461 INFO:tasks.workunit.client.0.smithi081.stdout:9/158: rmdir d2/d10/d30 39 2024-03-22T07:08:37.462 INFO:tasks.workunit.client.0.smithi081.stdout:1/130: unlink d6/d18/c1b 0 2024-03-22T07:08:37.464 INFO:tasks.workunit.client.0.smithi081.stdout:1/131: mkdir d6/d2e/d30 0 2024-03-22T07:08:37.466 INFO:tasks.workunit.client.0.smithi081.stdout:1/132: truncate d6/fc 2768986 0 2024-03-22T07:08:37.469 INFO:tasks.workunit.client.0.smithi081.stdout:0/181: dwrite d3/f1d [0,4194304] 0 2024-03-22T07:08:37.469 INFO:tasks.workunit.client.0.smithi081.stdout:4/211: dwrite da/d18/d1b/d1e/d37/f36 [0,4194304] 0 2024-03-22T07:08:37.469 INFO:tasks.workunit.client.0.smithi081.stdout:4/212: fdatasync da/d18/d1b/d26/f2a 0 2024-03-22T07:08:37.469 INFO:tasks.workunit.client.0.smithi081.stdout:0/182: creat d3/dc/d2b/f3a x:0 0 0 2024-03-22T07:08:37.469 INFO:tasks.workunit.client.0.smithi081.stdout:0/183: chown d3/dc/d2b/f3a 32261734 1 2024-03-22T07:08:37.478 INFO:tasks.workunit.client.0.smithi081.stdout:4/213: dread da/d18/d1b/d1e/d37/d1c/f28 [0,4194304] 0 2024-03-22T07:08:37.486 INFO:tasks.workunit.client.0.smithi081.stdout:0/184: dread d3/dc/f36 [4194304,4194304] 0 2024-03-22T07:08:37.487 INFO:tasks.workunit.client.0.smithi081.stdout:0/185: readlink d3/dc/l29 0 2024-03-22T07:08:37.487 INFO:tasks.workunit.client.0.smithi081.stdout:5/216: dwrite d4/f33 [0,4194304] 0 2024-03-22T07:08:37.487 INFO:tasks.workunit.client.0.smithi081.stdout:4/214: dread da/d18/f21 [0,4194304] 0 2024-03-22T07:08:37.488 INFO:tasks.workunit.client.0.smithi081.stdout:4/215: fdatasync da/d18/d1b/d1e/d37/f36 0 2024-03-22T07:08:37.488 INFO:tasks.workunit.client.0.smithi081.stdout:0/186: truncate d3/f4 3166793 0 2024-03-22T07:08:37.488 INFO:tasks.workunit.client.0.smithi081.stdout:1/133: dwrite d6/d18/f21 [0,4194304] 0 2024-03-22T07:08:37.489 INFO:tasks.workunit.client.0.smithi081.stdout:5/217: mkdir d4/df/d22/d4d 0 2024-03-22T07:08:37.490 INFO:tasks.workunit.client.0.smithi081.stdout:4/216: creat da/d18/d1b/d1e/f41 x:0 0 0 2024-03-22T07:08:37.493 INFO:tasks.workunit.client.0.smithi081.stdout:1/134: creat d6/d18/d1f/d24/f31 x:0 0 0 2024-03-22T07:08:37.493 INFO:tasks.workunit.client.0.smithi081.stdout:4/217: mkdir da/d18/d42 0 2024-03-22T07:08:37.493 INFO:tasks.workunit.client.0.smithi081.stdout:1/135: write d6/d18/d1f/d24/f31 [957908,77751] 0 2024-03-22T07:08:37.493 INFO:tasks.workunit.client.0.smithi081.stdout:1/136: truncate d6/d18/f2c 737748 0 2024-03-22T07:08:37.493 INFO:tasks.workunit.client.0.smithi081.stdout:1/137: truncate d6/d18/f2c 861096 0 2024-03-22T07:08:37.495 INFO:tasks.workunit.client.0.smithi081.stdout:0/187: mknod d3/c3b 0 2024-03-22T07:08:37.497 INFO:tasks.workunit.client.0.smithi081.stdout:0/188: creat d3/dc/d2b/d39/f3c x:0 0 0 2024-03-22T07:08:37.497 INFO:tasks.workunit.client.0.smithi081.stdout:0/189: stat d3/f1d 0 2024-03-22T07:08:37.498 INFO:tasks.workunit.client.0.smithi081.stdout:8/123: rename d2/d3/c18 to d2/d3/d19/d1e/d13/c29 0 2024-03-22T07:08:37.501 INFO:tasks.workunit.client.0.smithi081.stdout:1/138: mkdir d6/d18/d1f/d32 0 2024-03-22T07:08:37.507 INFO:tasks.workunit.client.0.smithi081.stdout:4/218: rename da/d18/d1b/d1e/d37/d1c/f28 to da/d18/d1b/d26/f43 0 2024-03-22T07:08:37.511 INFO:tasks.workunit.client.0.smithi081.stdout:5/218: dwrite d4/d10/d20/d1f/f28 [0,4194304] 0 2024-03-22T07:08:37.514 INFO:tasks.workunit.client.0.smithi081.stdout:5/219: mknod d4/df/d22/d4d/c4e 0 2024-03-22T07:08:37.514 INFO:tasks.workunit.client.0.smithi081.stdout:5/220: truncate d4/d10/d20/d1f/d2f/d45/f4c 733105 0 2024-03-22T07:08:37.517 INFO:tasks.workunit.client.0.smithi081.stdout:5/221: dread f1 [0,4194304] 0 2024-03-22T07:08:37.517 INFO:tasks.workunit.client.0.smithi081.stdout:5/222: fsync d4/f33 0 2024-03-22T07:08:37.518 INFO:tasks.workunit.client.0.smithi081.stdout:5/223: read d4/df/f1a [315278,42193] 0 2024-03-22T07:08:37.520 INFO:tasks.workunit.client.0.smithi081.stdout:5/224: dread d4/d9/f15 [0,4194304] 0 2024-03-22T07:08:37.520 INFO:tasks.workunit.client.0.smithi081.stdout:5/225: write f2 [452448,38141] 0 2024-03-22T07:08:37.520 INFO:tasks.workunit.client.0.smithi081.stdout:5/226: stat d4/fe 0 2024-03-22T07:08:37.521 INFO:tasks.workunit.client.0.smithi081.stdout:5/227: write d4/fe [1381092,94364] 0 2024-03-22T07:08:37.522 INFO:tasks.workunit.client.0.smithi081.stdout:0/190: dwrite d3/f4 [0,4194304] 0 2024-03-22T07:08:37.522 INFO:tasks.workunit.client.0.smithi081.stdout:0/191: chown d3/dc/f1c 120 1 2024-03-22T07:08:37.526 INFO:tasks.workunit.client.0.smithi081.stdout:0/192: truncate d3/f22 2747785 0 2024-03-22T07:08:37.526 INFO:tasks.workunit.client.0.smithi081.stdout:0/193: readlink d3/dc/l29 0 2024-03-22T07:08:37.527 INFO:tasks.workunit.client.0.smithi081.stdout:0/194: truncate d3/dc/f2a 484097 0 2024-03-22T07:08:37.527 INFO:tasks.workunit.client.0.smithi081.stdout:5/228: getdents d4/d9 0 2024-03-22T07:08:37.529 INFO:tasks.workunit.client.0.smithi081.stdout:5/229: dread d4/d9/f15 [0,4194304] 0 2024-03-22T07:08:37.529 INFO:tasks.workunit.client.0.smithi081.stdout:5/230: fdatasync d4/d10/d20/d25/d2e/f3b 0 2024-03-22T07:08:37.530 INFO:tasks.workunit.client.0.smithi081.stdout:4/219: unlink da/d18/d1b/d1e/d37/l11 0 2024-03-22T07:08:37.531 INFO:tasks.workunit.client.0.smithi081.stdout:5/231: mkdir d4/d10/d20/d4f 0 2024-03-22T07:08:37.532 INFO:tasks.workunit.client.0.smithi081.stdout:4/220: creat da/d18/d1b/d34/f44 x:0 0 0 2024-03-22T07:08:37.532 INFO:tasks.workunit.client.0.smithi081.stdout:4/221: write f1 [199686,57474] 0 2024-03-22T07:08:37.532 INFO:tasks.workunit.client.0.smithi081.stdout:4/222: fsync da/d18/d1b/d1e/d37/f25 0 2024-03-22T07:08:37.535 INFO:tasks.workunit.client.0.smithi081.stdout:5/232: link d4/l35 d4/d10/d20/d1f/d2a/l50 0 2024-03-22T07:08:37.537 INFO:tasks.workunit.client.0.smithi081.stdout:5/233: stat d4/d9/l17 0 2024-03-22T07:08:37.541 INFO:tasks.workunit.client.0.smithi081.stdout:5/234: mknod d4/df/d22/d4d/c51 0 2024-03-22T07:08:37.548 INFO:tasks.workunit.client.0.smithi081.stdout:6/131: sync 2024-03-22T07:08:37.548 INFO:tasks.workunit.client.0.smithi081.stdout:2/88: sync 2024-03-22T07:08:37.548 INFO:tasks.workunit.client.0.smithi081.stdout:7/281: sync 2024-03-22T07:08:37.548 INFO:tasks.workunit.client.0.smithi081.stdout:7/282: chown l8 157 1 2024-03-22T07:08:37.549 INFO:tasks.workunit.client.0.smithi081.stdout:0/195: rename d3/f7 to d3/f3d 0 2024-03-22T07:08:37.567 INFO:tasks.workunit.client.0.smithi081.stdout:5/235: dwrite d4/df/f1a [0,4194304] 0 2024-03-22T07:08:37.569 INFO:tasks.workunit.client.0.smithi081.stdout:5/236: mknod d4/df/d22/d4d/c52 0 2024-03-22T07:08:37.570 INFO:tasks.workunit.client.0.smithi081.stdout:7/283: dread d9/d10/d2a/f47 [0,4194304] 0 2024-03-22T07:08:37.570 INFO:tasks.workunit.client.0.smithi081.stdout:7/284: chown d9/d3f/f65 11749125 1 2024-03-22T07:08:37.571 INFO:tasks.workunit.client.0.smithi081.stdout:2/89: dread f4 [0,4194304] 0 2024-03-22T07:08:37.572 INFO:tasks.workunit.client.0.smithi081.stdout:7/285: truncate f1 1010629 0 2024-03-22T07:08:37.584 INFO:tasks.workunit.client.0.smithi081.stdout:2/90: dread f4 [0,4194304] 0 2024-03-22T07:08:37.584 INFO:tasks.workunit.client.0.smithi081.stdout:2/91: symlink da/d11/l17 0 2024-03-22T07:08:37.595 INFO:tasks.workunit.client.0.smithi081.stdout:0/196: dwrite d3/dc/f15 [0,4194304] 0 2024-03-22T07:08:37.595 INFO:tasks.workunit.client.0.smithi081.stdout:5/237: dwrite d4/d10/d20/d25/d2e/f31 [4194304,4194304] 0 2024-03-22T07:08:37.595 INFO:tasks.workunit.client.0.smithi081.stdout:5/238: truncate d4/d10/d20/d25/f43 646473 0 2024-03-22T07:08:37.595 INFO:tasks.workunit.client.0.smithi081.stdout:5/239: chown d4/d10/d20/l32 38918 1 2024-03-22T07:08:37.597 INFO:tasks.workunit.client.0.smithi081.stdout:7/286: dwrite d9/dc/f38 [4194304,4194304] 0 2024-03-22T07:08:37.597 INFO:tasks.workunit.client.0.smithi081.stdout:7/287: read d9/d10/d16/d1e/d27/f5a [507281,62350] 0 2024-03-22T07:08:37.597 INFO:tasks.workunit.client.0.smithi081.stdout:0/197: creat d3/f3e x:0 0 0 2024-03-22T07:08:37.599 INFO:tasks.workunit.client.0.smithi081.stdout:5/240: truncate d4/d9/f1b 7951 0 2024-03-22T07:08:37.602 INFO:tasks.workunit.client.0.smithi081.stdout:7/288: creat d9/d10/d16/f6b x:0 0 0 2024-03-22T07:08:37.603 INFO:tasks.workunit.client.0.smithi081.stdout:0/198: creat d3/dc/f3f x:0 0 0 2024-03-22T07:08:37.603 INFO:tasks.workunit.client.0.smithi081.stdout:0/199: fdatasync d3/f1d 0 2024-03-22T07:08:37.603 INFO:tasks.workunit.client.0.smithi081.stdout:0/200: write d3/dc/f28 [685253,32124] 0 2024-03-22T07:08:37.604 INFO:tasks.workunit.client.0.smithi081.stdout:7/289: getdents d9/d14 0 2024-03-22T07:08:37.606 INFO:tasks.workunit.client.0.smithi081.stdout:7/290: mknod d9/dc/d41/c6c 0 2024-03-22T07:08:37.606 INFO:tasks.workunit.client.0.smithi081.stdout:7/291: dread - d9/d3f/f65 zero size 2024-03-22T07:08:37.607 INFO:tasks.workunit.client.0.smithi081.stdout:5/241: dread f3 [0,4194304] 0 2024-03-22T07:08:37.607 INFO:tasks.workunit.client.0.smithi081.stdout:7/292: mkdir d9/d10/d16/d1e/d27/d6d 0 2024-03-22T07:08:37.607 INFO:tasks.workunit.client.0.smithi081.stdout:5/242: write d4/d10/f3c [494443,71433] 0 2024-03-22T07:08:37.607 INFO:tasks.workunit.client.0.smithi081.stdout:5/243: write d4/d10/d20/d25/f43 [128440,42671] 0 2024-03-22T07:08:37.608 INFO:tasks.workunit.client.0.smithi081.stdout:5/244: fdatasync d4/d10/f1d 0 2024-03-22T07:08:37.608 INFO:tasks.workunit.client.0.smithi081.stdout:8/124: rmdir d2/d3/d19/d1e 39 2024-03-22T07:08:37.608 INFO:tasks.workunit.client.0.smithi081.stdout:3/126: sync 2024-03-22T07:08:37.610 INFO:tasks.workunit.client.0.smithi081.stdout:7/293: link d9/d10/d16/l45 d9/d10/d16/d1e/d27/d6d/l6e 0 2024-03-22T07:08:37.611 INFO:tasks.workunit.client.0.smithi081.stdout:5/245: mknod d4/d10/d20/d1f/d2f/c53 0 2024-03-22T07:08:37.613 INFO:tasks.workunit.client.0.smithi081.stdout:8/125: mknod d2/c2a 0 2024-03-22T07:08:37.614 INFO:tasks.workunit.client.0.smithi081.stdout:0/201: dread d3/dc/f15 [0,4194304] 0 2024-03-22T07:08:37.615 INFO:tasks.workunit.client.0.smithi081.stdout:5/246: dread d4/d10/d20/d1f/d2f/d45/f4c [0,4194304] 0 2024-03-22T07:08:37.615 INFO:tasks.workunit.client.0.smithi081.stdout:3/127: symlink db/de/l25 0 2024-03-22T07:08:37.615 INFO:tasks.workunit.client.0.smithi081.stdout:3/128: chown db/f1e 3638 1 2024-03-22T07:08:37.615 INFO:tasks.workunit.client.0.smithi081.stdout:3/129: creat db/de/f26 x:0 0 0 2024-03-22T07:08:37.615 INFO:tasks.workunit.client.0.smithi081.stdout:5/247: dread d4/d9/f15 [0,4194304] 0 2024-03-22T07:08:37.617 INFO:tasks.workunit.client.0.smithi081.stdout:7/294: mkdir d9/d10/d16/d1e/d6f 0 2024-03-22T07:08:37.620 INFO:tasks.workunit.client.0.smithi081.stdout:8/126: mkdir d2/d3/d28/d2b 0 2024-03-22T07:08:37.620 INFO:tasks.workunit.client.0.smithi081.stdout:0/202: mkdir d3/dc/d40 0 2024-03-22T07:08:37.620 INFO:tasks.workunit.client.0.smithi081.stdout:8/127: fsync d2/d3/f4 0 2024-03-22T07:08:37.627 INFO:tasks.workunit.client.0.smithi081.stdout:5/248: rmdir d4/d10/d20/d25 39 2024-03-22T07:08:37.627 INFO:tasks.workunit.client.0.smithi081.stdout:5/249: stat d4/d10/f21 0 2024-03-22T07:08:37.627 INFO:tasks.workunit.client.0.smithi081.stdout:5/250: fsync d4/d10/f26 0 2024-03-22T07:08:37.627 INFO:tasks.workunit.client.0.smithi081.stdout:5/251: dread - d4/df/d22/d49/f4a zero size 2024-03-22T07:08:37.630 INFO:tasks.workunit.client.0.smithi081.stdout:5/252: unlink d4/d9/l17 0 2024-03-22T07:08:37.632 INFO:tasks.workunit.client.0.smithi081.stdout:8/128: rmdir d2/d3/d19/d1e 39 2024-03-22T07:08:37.634 INFO:tasks.workunit.client.0.smithi081.stdout:8/129: truncate f0 901254 0 2024-03-22T07:08:37.644 INFO:tasks.workunit.client.0.smithi081.stdout:5/253: dread d4/d10/f3c [0,4194304] 0 2024-03-22T07:08:37.646 INFO:tasks.workunit.client.0.smithi081.stdout:5/254: creat d4/d10/d20/d25/d2e/f54 x:0 0 0 2024-03-22T07:08:37.648 INFO:tasks.workunit.client.0.smithi081.stdout:5/255: symlink d4/df/d22/d4d/l55 0 2024-03-22T07:08:37.649 INFO:tasks.workunit.client.0.smithi081.stdout:3/130: rmdir db/de 39 2024-03-22T07:08:37.651 INFO:tasks.workunit.client.0.smithi081.stdout:3/131: symlink db/l27 0 2024-03-22T07:08:37.654 INFO:tasks.workunit.client.0.smithi081.stdout:5/256: dread d4/fe [0,4194304] 0 2024-03-22T07:08:37.655 INFO:tasks.workunit.client.0.smithi081.stdout:5/257: mknod d4/d10/c56 0 2024-03-22T07:08:37.655 INFO:tasks.workunit.client.0.smithi081.stdout:5/258: getdents d4/d10/d20/d4f 0 2024-03-22T07:08:37.656 INFO:tasks.workunit.client.0.smithi081.stdout:5/259: creat d4/df/d22/f57 x:0 0 0 2024-03-22T07:08:37.657 INFO:tasks.workunit.client.0.smithi081.stdout:7/295: dwrite d9/d10/d16/d1e/d27/f5a [0,4194304] 0 2024-03-22T07:08:37.657 INFO:tasks.workunit.client.0.smithi081.stdout:8/130: dwrite f1 [4194304,4194304] 0 2024-03-22T07:08:37.657 INFO:tasks.workunit.client.0.smithi081.stdout:0/203: dwrite d3/f4 [0,4194304] 0 2024-03-22T07:08:37.664 INFO:tasks.workunit.client.0.smithi081.stdout:3/132: read f1 [1682218,11673] 0 2024-03-22T07:08:37.664 INFO:tasks.workunit.client.0.smithi081.stdout:3/133: dread - db/fc zero size 2024-03-22T07:08:37.664 INFO:tasks.workunit.client.0.smithi081.stdout:3/134: dread - db/de/f20 zero size 2024-03-22T07:08:37.664 INFO:tasks.workunit.client.0.smithi081.stdout:3/135: write db/de/f1a [844707,45070] 0 2024-03-22T07:08:37.664 INFO:tasks.workunit.client.0.smithi081.stdout:3/136: readlink db/de/l1c 0 2024-03-22T07:08:37.664 INFO:tasks.workunit.client.0.smithi081.stdout:3/137: stat db/c1d 0 2024-03-22T07:08:37.666 INFO:tasks.workunit.client.0.smithi081.stdout:8/131: creat d2/d3/f2c x:0 0 0 2024-03-22T07:08:37.666 INFO:tasks.workunit.client.0.smithi081.stdout:5/260: write f3 [1262651,20221] 0 2024-03-22T07:08:37.666 INFO:tasks.workunit.client.0.smithi081.stdout:0/204: symlink d3/dc/d33/l41 0 2024-03-22T07:08:37.669 INFO:tasks.workunit.client.0.smithi081.stdout:0/205: creat d3/dc/d40/f42 x:0 0 0 2024-03-22T07:08:37.669 INFO:tasks.workunit.client.0.smithi081.stdout:0/206: read - d3/f3e zero size 2024-03-22T07:08:37.669 INFO:tasks.workunit.client.0.smithi081.stdout:8/132: creat d2/d3/d19/d1e/d13/f2d x:0 0 0 2024-03-22T07:08:37.669 INFO:tasks.workunit.client.0.smithi081.stdout:8/133: truncate d2/d3/f2c 674600 0 2024-03-22T07:08:37.670 INFO:tasks.workunit.client.0.smithi081.stdout:8/134: creat d2/d3/d28/d2b/f2e x:0 0 0 2024-03-22T07:08:37.672 INFO:tasks.workunit.client.0.smithi081.stdout:8/135: mkdir d2/d3/d19/d1e/d13/d22/d2f 0 2024-03-22T07:08:37.672 INFO:tasks.workunit.client.0.smithi081.stdout:1/139: rename d6/c15 to d6/d18/d1f/c33 0 2024-03-22T07:08:37.673 INFO:tasks.workunit.client.0.smithi081.stdout:8/136: creat d2/d3/d28/f30 x:0 0 0 2024-03-22T07:08:37.675 INFO:tasks.workunit.client.0.smithi081.stdout:8/137: symlink d2/d3/d19/d1e/d13/d22/d2f/l31 0 2024-03-22T07:08:37.678 INFO:tasks.workunit.client.0.smithi081.stdout:7/296: dwrite d9/d10/d16/d1e/f35 [0,4194304] 0 2024-03-22T07:08:37.678 INFO:tasks.workunit.client.0.smithi081.stdout:7/297: chown d9/d10/d16/d1e/d27/l3e 180307 1 2024-03-22T07:08:37.682 INFO:tasks.workunit.client.0.smithi081.stdout:0/207: dread d3/f4 [0,4194304] 0 2024-03-22T07:08:37.682 INFO:tasks.workunit.client.0.smithi081.stdout:0/208: dread - d3/dc/d2b/f3a zero size 2024-03-22T07:08:37.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/209: creat d3/dc/f43 x:0 0 0 2024-03-22T07:08:37.688 INFO:tasks.workunit.client.0.smithi081.stdout:1/140: dread f1 [0,4194304] 0 2024-03-22T07:08:37.689 INFO:tasks.workunit.client.0.smithi081.stdout:7/298: mkdir d9/d10/d16/d1e/d27/d70 0 2024-03-22T07:08:37.692 INFO:tasks.workunit.client.0.smithi081.stdout:0/210: creat d3/f44 x:0 0 0 2024-03-22T07:08:37.692 INFO:tasks.workunit.client.0.smithi081.stdout:0/211: fdatasync d3/f13 0 2024-03-22T07:08:37.693 INFO:tasks.workunit.client.0.smithi081.stdout:1/141: creat d6/d2e/d30/f34 x:0 0 0 2024-03-22T07:08:37.694 INFO:tasks.workunit.client.0.smithi081.stdout:0/212: mknod d3/dc/d2b/c45 0 2024-03-22T07:08:37.696 INFO:tasks.workunit.client.0.smithi081.stdout:0/213: mknod d3/dc/d2b/c46 0 2024-03-22T07:08:37.696 INFO:tasks.workunit.client.0.smithi081.stdout:6/132: rename d3/da/de/df/f15 to d3/da/de/f24 0 2024-03-22T07:08:37.696 INFO:tasks.workunit.client.0.smithi081.stdout:6/133: creat d3/d12/f25 x:0 0 0 2024-03-22T07:08:37.696 INFO:tasks.workunit.client.0.smithi081.stdout:6/134: write d3/f6 [120484,83486] 0 2024-03-22T07:08:37.697 INFO:tasks.workunit.client.0.smithi081.stdout:0/214: creat d3/dc/d33/f47 x:0 0 0 2024-03-22T07:08:37.697 INFO:tasks.workunit.client.0.smithi081.stdout:0/215: truncate d3/dc/f43 874154 0 2024-03-22T07:08:37.698 INFO:tasks.workunit.client.0.smithi081.stdout:7/299: write d9/d10/d2a/f47 [4091608,53304] 0 2024-03-22T07:08:37.699 INFO:tasks.workunit.client.0.smithi081.stdout:2/92: rename f4 to da/f18 0 2024-03-22T07:08:37.704 INFO:tasks.workunit.client.0.smithi081.stdout:7/300: truncate d9/d10/d16/d1e/f35 2903378 0 2024-03-22T07:08:37.704 INFO:tasks.workunit.client.0.smithi081.stdout:1/142: dread d6/f17 [0,4194304] 0 2024-03-22T07:08:37.704 INFO:tasks.workunit.client.0.smithi081.stdout:1/143: write d6/d2e/d30/f34 [818575,15860] 0 2024-03-22T07:08:37.704 INFO:tasks.workunit.client.0.smithi081.stdout:2/93: rmdir da/d11 39 2024-03-22T07:08:37.706 INFO:tasks.workunit.client.0.smithi081.stdout:7/301: symlink d9/d10/d16/d1e/d6f/l71 0 2024-03-22T07:08:37.708 INFO:tasks.workunit.client.0.smithi081.stdout:7/302: rmdir d9/dc/d41 39 2024-03-22T07:08:37.711 INFO:tasks.workunit.client.0.smithi081.stdout:0/216: write d3/f13 [3916759,62083] 0 2024-03-22T07:08:37.711 INFO:tasks.workunit.client.0.smithi081.stdout:0/217: truncate d3/dc/d2b/f3a 798913 0 2024-03-22T07:08:37.721 INFO:tasks.workunit.client.0.smithi081.stdout:7/303: dread d9/d10/d16/d1e/f28 [0,4194304] 0 2024-03-22T07:08:37.726 INFO:tasks.workunit.client.0.smithi081.stdout:7/304: write d9/d10/f2e [309089,111124] 0 2024-03-22T07:08:37.727 INFO:tasks.workunit.client.0.smithi081.stdout:7/305: write d9/d10/d16/f42 [24520,125091] 0 2024-03-22T07:08:37.727 INFO:tasks.workunit.client.0.smithi081.stdout:7/306: write d9/d10/d16/d1e/d27/f63 [344965,73747] 0 2024-03-22T07:08:37.727 INFO:tasks.workunit.client.0.smithi081.stdout:7/307: readlink d9/d10/d16/l6a 0 2024-03-22T07:08:37.733 INFO:tasks.workunit.client.0.smithi081.stdout:7/308: dread d9/d10/f3a [0,4194304] 0 2024-03-22T07:08:37.734 INFO:tasks.workunit.client.0.smithi081.stdout:7/309: truncate d9/d10/f3a 4077079 0 2024-03-22T07:08:37.736 INFO:tasks.workunit.client.0.smithi081.stdout:7/310: link d9/d10/c1c d9/d10/d16/d1e/d27/d70/c72 0 2024-03-22T07:08:37.738 INFO:tasks.workunit.client.0.smithi081.stdout:7/311: dread d9/d10/d16/d1e/f32 [0,4194304] 0 2024-03-22T07:08:37.738 INFO:tasks.workunit.client.0.smithi081.stdout:7/312: chown d9/lb 3 1 2024-03-22T07:08:37.740 INFO:tasks.workunit.client.0.smithi081.stdout:1/144: dwrite d6/d18/d1f/d24/f2b [0,4194304] 0 2024-03-22T07:08:37.741 INFO:tasks.workunit.client.0.smithi081.stdout:0/218: dwrite d3/f8 [0,4194304] 0 2024-03-22T07:08:37.742 INFO:tasks.workunit.client.0.smithi081.stdout:1/145: creat d6/f35 x:0 0 0 2024-03-22T07:08:37.744 INFO:tasks.workunit.client.0.smithi081.stdout:1/146: getdents d6/d2e/d30 0 2024-03-22T07:08:37.744 INFO:tasks.workunit.client.0.smithi081.stdout:1/147: chown d6/d18/f2c 3382 1 2024-03-22T07:08:37.747 INFO:tasks.workunit.client.0.smithi081.stdout:1/148: symlink d6/d18/l36 0 2024-03-22T07:08:37.758 INFO:tasks.workunit.client.0.smithi081.stdout:9/159: sync 2024-03-22T07:08:37.759 INFO:tasks.workunit.client.0.smithi081.stdout:9/160: truncate d2/d2b/f2d 2445191 0 2024-03-22T07:08:37.760 INFO:tasks.workunit.client.0.smithi081.stdout:0/219: dwrite d3/dc/f1b [0,4194304] 0 2024-03-22T07:08:37.761 INFO:tasks.workunit.client.0.smithi081.stdout:9/161: mkdir d2/d5/d16/d37 0 2024-03-22T07:08:37.762 INFO:tasks.workunit.client.0.smithi081.stdout:9/162: symlink d2/d2b/l38 0 2024-03-22T07:08:37.765 INFO:tasks.workunit.client.0.smithi081.stdout:9/163: mknod d2/d2b/c39 0 2024-03-22T07:08:37.768 INFO:tasks.workunit.client.0.smithi081.stdout:9/164: mknod d2/d5/c3a 0 2024-03-22T07:08:37.773 INFO:tasks.workunit.client.0.smithi081.stdout:9/165: unlink d2/d5/d16/c27 0 2024-03-22T07:08:37.773 INFO:tasks.workunit.client.0.smithi081.stdout:7/313: dwrite d9/f3c [0,4194304] 0 2024-03-22T07:08:37.773 INFO:tasks.workunit.client.0.smithi081.stdout:9/166: readlink d2/d5/d16/l1f 0 2024-03-22T07:08:37.773 INFO:tasks.workunit.client.0.smithi081.stdout:0/220: dread d3/dc/f1f [0,4194304] 0 2024-03-22T07:08:37.773 INFO:tasks.workunit.client.0.smithi081.stdout:0/221: creat d3/dc/d2b/f48 x:0 0 0 2024-03-22T07:08:37.775 INFO:tasks.workunit.client.0.smithi081.stdout:7/314: mkdir d9/d10/d2a/d73 0 2024-03-22T07:08:37.790 INFO:tasks.workunit.client.0.smithi081.stdout:8/138: rename d2/d3/lf to d2/d3/d19/d1e/d13/d22/l32 0 2024-03-22T07:08:37.796 INFO:tasks.workunit.client.0.smithi081.stdout:0/222: dwrite d3/dc/f28 [0,4194304] 0 2024-03-22T07:08:37.796 INFO:tasks.workunit.client.0.smithi081.stdout:6/135: rename d3/d12 to d3/da/de/df/d26 0 2024-03-22T07:08:37.797 INFO:tasks.workunit.client.0.smithi081.stdout:6/136: creat d3/da/de/f27 x:0 0 0 2024-03-22T07:08:37.797 INFO:tasks.workunit.client.0.smithi081.stdout:6/137: creat d3/da/de/f28 x:0 0 0 2024-03-22T07:08:37.798 INFO:tasks.workunit.client.0.smithi081.stdout:7/315: unlink d9/d10/d16/d1e/d27/f63 0 2024-03-22T07:08:37.798 INFO:tasks.workunit.client.0.smithi081.stdout:1/149: link d6/fd d6/d18/d1f/f37 0 2024-03-22T07:08:37.802 INFO:tasks.workunit.client.0.smithi081.stdout:0/223: stat d3/c25 0 2024-03-22T07:08:37.810 INFO:tasks.workunit.client.0.smithi081.stdout:6/138: creat d3/d4/f29 x:0 0 0 2024-03-22T07:08:37.814 INFO:tasks.workunit.client.0.smithi081.stdout:6/139: truncate d3/f6 2047010 0 2024-03-22T07:08:37.817 INFO:tasks.workunit.client.0.smithi081.stdout:0/224: read d3/dc/f19 [2480920,116425] 0 2024-03-22T07:08:37.827 INFO:tasks.workunit.client.0.smithi081.stdout:7/316: dwrite d9/d10/f3a [0,4194304] 0 2024-03-22T07:08:37.829 INFO:tasks.workunit.client.0.smithi081.stdout:8/139: dwrite f0 [0,4194304] 0 2024-03-22T07:08:37.829 INFO:tasks.workunit.client.0.smithi081.stdout:8/140: chown d2/f11 61862889 1 2024-03-22T07:08:37.830 INFO:tasks.workunit.client.0.smithi081.stdout:8/141: stat d2/d3/d19/d1e/l21 0 2024-03-22T07:08:37.833 INFO:tasks.workunit.client.0.smithi081.stdout:1/150: dwrite d6/d18/f28 [0,4194304] 0 2024-03-22T07:08:37.834 INFO:tasks.workunit.client.0.smithi081.stdout:1/151: chown d6/d2e/d30/f34 507 1 2024-03-22T07:08:37.834 INFO:tasks.workunit.client.0.smithi081.stdout:8/142: read d2/f24 [176948,89706] 0 2024-03-22T07:08:37.834 INFO:tasks.workunit.client.0.smithi081.stdout:6/140: write d3/da/de/f11 [1984663,48136] 0 2024-03-22T07:08:37.835 INFO:tasks.workunit.client.0.smithi081.stdout:1/152: mknod d6/c38 0 2024-03-22T07:08:37.838 INFO:tasks.workunit.client.0.smithi081.stdout:0/225: write d3/dc/f15 [1133216,103865] 0 2024-03-22T07:08:37.839 INFO:tasks.workunit.client.0.smithi081.stdout:0/226: write d3/dc/d2b/d39/f3c [695307,69395] 0 2024-03-22T07:08:37.839 INFO:tasks.workunit.client.0.smithi081.stdout:0/227: stat d3/l21 0 2024-03-22T07:08:37.840 INFO:tasks.workunit.client.0.smithi081.stdout:8/143: unlink d2/d3/f2c 0 2024-03-22T07:08:37.855 INFO:tasks.workunit.client.0.smithi081.stdout:7/317: dwrite f4 [0,4194304] 0 2024-03-22T07:08:37.855 INFO:tasks.workunit.client.0.smithi081.stdout:7/318: chown d9/dc/f4a 43 1 2024-03-22T07:08:37.855 INFO:tasks.workunit.client.0.smithi081.stdout:7/319: creat d9/d10/d16/f74 x:0 0 0 2024-03-22T07:08:37.856 INFO:tasks.workunit.client.0.smithi081.stdout:6/141: dread d3/f6 [0,4194304] 0 2024-03-22T07:08:37.864 INFO:tasks.workunit.client.0.smithi081.stdout:0/228: dwrite d3/dc/d2b/d39/f3c [0,4194304] 0 2024-03-22T07:08:37.864 INFO:tasks.workunit.client.0.smithi081.stdout:0/229: chown d3/dc/f1b 1350014 1 2024-03-22T07:08:37.865 INFO:tasks.workunit.client.0.smithi081.stdout:0/230: truncate d3/dc/d2b/f31 778604 0 2024-03-22T07:08:37.865 INFO:tasks.workunit.client.0.smithi081.stdout:0/231: chown d3/dc/f15 60 1 2024-03-22T07:08:37.870 INFO:tasks.workunit.client.0.smithi081.stdout:6/142: getdents d3/da 0 2024-03-22T07:08:37.879 INFO:tasks.workunit.client.0.smithi081.stdout:0/232: dread d3/f1d [0,4194304] 0 2024-03-22T07:08:37.879 INFO:tasks.workunit.client.0.smithi081.stdout:7/320: dwrite d9/d10/d16/d1e/f18 [0,4194304] 0 2024-03-22T07:08:37.879 INFO:tasks.workunit.client.0.smithi081.stdout:0/233: fdatasync d3/f4 0 2024-03-22T07:08:37.879 INFO:tasks.workunit.client.0.smithi081.stdout:6/143: creat d3/da/f2a x:0 0 0 2024-03-22T07:08:37.883 INFO:tasks.workunit.client.0.smithi081.stdout:0/234: write d3/f2e [4056185,110548] 0 2024-03-22T07:08:37.883 INFO:tasks.workunit.client.0.smithi081.stdout:6/144: mknod d3/da/de/df/c2b 0 2024-03-22T07:08:37.887 INFO:tasks.workunit.client.0.smithi081.stdout:6/145: unlink d3/da/l13 0 2024-03-22T07:08:37.890 INFO:tasks.workunit.client.0.smithi081.stdout:6/146: truncate d3/da/de/f24 4249295 0 2024-03-22T07:08:37.890 INFO:tasks.workunit.client.0.smithi081.stdout:6/147: creat d3/da/de/df/f2c x:0 0 0 2024-03-22T07:08:37.890 INFO:tasks.workunit.client.0.smithi081.stdout:6/148: dread - d3/da/de/df/d26/f25 zero size 2024-03-22T07:08:37.895 INFO:tasks.workunit.client.0.smithi081.stdout:6/149: stat d3/da/fd 0 2024-03-22T07:08:37.896 INFO:tasks.workunit.client.0.smithi081.stdout:6/150: mknod d3/d4/c2d 0 2024-03-22T07:08:37.897 INFO:tasks.workunit.client.0.smithi081.stdout:6/151: chown d3/da/de/df/d26/f1d 13072604 1 2024-03-22T07:08:37.898 INFO:tasks.workunit.client.0.smithi081.stdout:6/152: rename d3/da/de/f1a to d3/da/de/f2e 0 2024-03-22T07:08:37.899 INFO:tasks.workunit.client.0.smithi081.stdout:6/153: readlink d3/da/de/l20 0 2024-03-22T07:08:37.900 INFO:tasks.workunit.client.0.smithi081.stdout:6/154: mknod d3/d4/c2f 0 2024-03-22T07:08:37.902 INFO:tasks.workunit.client.0.smithi081.stdout:0/235: dwrite d3/dc/f2a [0,4194304] 0 2024-03-22T07:08:37.902 INFO:tasks.workunit.client.0.smithi081.stdout:0/236: write d3/dc/f3f [171998,58259] 0 2024-03-22T07:08:37.902 INFO:tasks.workunit.client.0.smithi081.stdout:0/237: creat d3/dc/d2b/f49 x:0 0 0 2024-03-22T07:08:37.909 INFO:tasks.workunit.client.0.smithi081.stdout:6/155: dread d3/da/de/f11 [0,4194304] 0 2024-03-22T07:08:37.912 INFO:tasks.workunit.client.0.smithi081.stdout:6/156: link d3/da/l23 d3/l30 0 2024-03-22T07:08:37.912 INFO:tasks.workunit.client.0.smithi081.stdout:6/157: chown d3/da/de/f11 320621399 1 2024-03-22T07:08:37.914 INFO:tasks.workunit.client.0.smithi081.stdout:6/158: mkdir d3/da/de/d31 0 2024-03-22T07:08:37.914 INFO:tasks.workunit.client.0.smithi081.stdout:4/223: sync 2024-03-22T07:08:37.914 INFO:tasks.workunit.client.0.smithi081.stdout:6/159: write d3/da/de/f11 [4517163,127115] 0 2024-03-22T07:08:37.914 INFO:tasks.workunit.client.0.smithi081.stdout:6/160: fdatasync d3/da/de/f24 0 2024-03-22T07:08:37.918 INFO:tasks.workunit.client.0.smithi081.stdout:6/161: symlink d3/da/l32 0 2024-03-22T07:08:37.919 INFO:tasks.workunit.client.0.smithi081.stdout:6/162: fsync d3/f22 0 2024-03-22T07:08:37.919 INFO:tasks.workunit.client.0.smithi081.stdout:6/163: dread - d3/da/de/df/d26/f1d zero size 2024-03-22T07:08:37.922 INFO:tasks.workunit.client.0.smithi081.stdout:6/164: creat d3/da/de/df/d26/f33 x:0 0 0 2024-03-22T07:08:37.930 INFO:tasks.workunit.client.0.smithi081.stdout:4/224: dwrite f0 [0,4194304] 0 2024-03-22T07:08:37.930 INFO:tasks.workunit.client.0.smithi081.stdout:4/225: write da/d18/d1b/d34/f44 [974310,6277] 0 2024-03-22T07:08:37.930 INFO:tasks.workunit.client.0.smithi081.stdout:6/165: write d3/d4/f7 [2703069,9727] 0 2024-03-22T07:08:37.930 INFO:tasks.workunit.client.0.smithi081.stdout:6/166: write d3/da/de/df/d26/f25 [893449,41452] 0 2024-03-22T07:08:37.932 INFO:tasks.workunit.client.0.smithi081.stdout:6/167: rename d3/da/l23 to d3/da/l34 0 2024-03-22T07:08:37.932 INFO:tasks.workunit.client.0.smithi081.stdout:6/168: write d3/da/fd [1198036,90418] 0 2024-03-22T07:08:37.937 INFO:tasks.workunit.client.0.smithi081.stdout:6/169: dread d3/da/de/df/d26/f25 [0,4194304] 0 2024-03-22T07:08:37.945 INFO:tasks.workunit.client.0.smithi081.stdout:6/170: dread d3/d4/f7 [0,4194304] 0 2024-03-22T07:08:37.952 INFO:tasks.workunit.client.0.smithi081.stdout:6/171: write d3/da/f17 [3577455,31503] 0 2024-03-22T07:08:37.953 INFO:tasks.workunit.client.0.smithi081.stdout:6/172: mknod d3/da/de/df/c35 0 2024-03-22T07:08:37.955 INFO:tasks.workunit.client.0.smithi081.stdout:6/173: fsync d3/da/fd 0 2024-03-22T07:08:37.955 INFO:tasks.workunit.client.0.smithi081.stdout:6/174: chown d3/da/de/df/d26/f33 404096 1 2024-03-22T07:08:37.955 INFO:tasks.workunit.client.0.smithi081.stdout:6/175: creat d3/f36 x:0 0 0 2024-03-22T07:08:37.955 INFO:tasks.workunit.client.0.smithi081.stdout:6/176: creat d3/da/de/df/f37 x:0 0 0 2024-03-22T07:08:37.958 INFO:tasks.workunit.client.0.smithi081.stdout:6/177: mknod d3/da/de/df/c38 0 2024-03-22T07:08:37.960 INFO:tasks.workunit.client.0.smithi081.stdout:1/153: link f1 d6/d18/d1f/d24/f39 0 2024-03-22T07:08:37.961 INFO:tasks.workunit.client.0.smithi081.stdout:1/154: stat d6/d18/d1f/d24/c29 0 2024-03-22T07:08:37.961 INFO:tasks.workunit.client.0.smithi081.stdout:1/155: stat d6/d18/d1f/d24/f31 0 2024-03-22T07:08:37.966 INFO:tasks.workunit.client.0.smithi081.stdout:6/178: truncate d3/d4/f7 1750627 0 2024-03-22T07:08:37.966 INFO:tasks.workunit.client.0.smithi081.stdout:6/179: fsync d3/f36 0 2024-03-22T07:08:37.966 INFO:tasks.workunit.client.0.smithi081.stdout:6/180: dread - d3/f36 zero size 2024-03-22T07:08:37.974 INFO:tasks.workunit.client.0.smithi081.stdout:1/156: write d6/d18/f27 [3590822,107100] 0 2024-03-22T07:08:37.976 INFO:tasks.workunit.client.0.smithi081.stdout:8/144: rename d2/c2a to d2/d3/d19/d1e/c33 0 2024-03-22T07:08:37.977 INFO:tasks.workunit.client.0.smithi081.stdout:8/145: creat d2/d3/d19/d1e/d13/d22/f34 x:0 0 0 2024-03-22T07:08:37.978 INFO:tasks.workunit.client.0.smithi081.stdout:0/238: rename d3/dc/f1f to d3/dc/d40/f4a 0 2024-03-22T07:08:37.979 INFO:tasks.workunit.client.0.smithi081.stdout:6/181: unlink d3/da/de/f24 0 2024-03-22T07:08:37.980 INFO:tasks.workunit.client.0.smithi081.stdout:4/226: rename da/c2d to da/df/c45 0 2024-03-22T07:08:37.981 INFO:tasks.workunit.client.0.smithi081.stdout:4/227: dread - da/d18/d1b/d1e/f41 zero size 2024-03-22T07:08:37.981 INFO:tasks.workunit.client.0.smithi081.stdout:4/228: chown da/fe 1025843451 1 2024-03-22T07:08:37.983 INFO:tasks.workunit.client.0.smithi081.stdout:0/239: rename d3/dc/l34 to d3/d32/l4b 0 2024-03-22T07:08:37.984 INFO:tasks.workunit.client.0.smithi081.stdout:4/229: creat da/df/d38/f46 x:0 0 0 2024-03-22T07:08:37.984 INFO:tasks.workunit.client.0.smithi081.stdout:4/230: chown da/d18/d1b/d1e 2 1 2024-03-22T07:08:37.984 INFO:tasks.workunit.client.0.smithi081.stdout:4/231: readlink da/d18/l22 0 2024-03-22T07:08:37.985 INFO:tasks.workunit.client.0.smithi081.stdout:6/182: rmdir d3/da/de/df/d26 39 2024-03-22T07:08:37.985 INFO:tasks.workunit.client.0.smithi081.stdout:6/183: fdatasync d3/da/de/df/f37 0 2024-03-22T07:08:37.987 INFO:tasks.workunit.client.0.smithi081.stdout:6/184: mknod d3/da/de/d31/c39 0 2024-03-22T07:08:37.987 INFO:tasks.workunit.client.0.smithi081.stdout:6/185: dread - d3/f22 zero size 2024-03-22T07:08:37.987 INFO:tasks.workunit.client.0.smithi081.stdout:6/186: write d3/d4/f29 [524263,50793] 0 2024-03-22T07:08:37.987 INFO:tasks.workunit.client.0.smithi081.stdout:6/187: write d3/f22 [839004,51709] 0 2024-03-22T07:08:37.987 INFO:tasks.workunit.client.0.smithi081.stdout:6/188: truncate d3/da/f2a 507498 0 2024-03-22T07:08:37.987 INFO:tasks.workunit.client.0.smithi081.stdout:6/189: write d3/da/de/df/d26/f33 [19107,107942] 0 2024-03-22T07:08:37.988 INFO:tasks.workunit.client.0.smithi081.stdout:6/190: link d3/da/de/c18 d3/da/c3a 0 2024-03-22T07:08:37.988 INFO:tasks.workunit.client.0.smithi081.stdout:6/191: readlink d3/da/de/l20 0 2024-03-22T07:08:37.988 INFO:tasks.workunit.client.0.smithi081.stdout:6/192: read d3/d4/f29 [55245,44053] 0 2024-03-22T07:08:37.989 INFO:tasks.workunit.client.0.smithi081.stdout:6/193: write d3/da/de/df/f37 [303206,7837] 0 2024-03-22T07:08:37.989 INFO:tasks.workunit.client.0.smithi081.stdout:6/194: creat d3/da/de/df/f3b x:0 0 0 2024-03-22T07:08:37.989 INFO:tasks.workunit.client.0.smithi081.stdout:6/195: chown d3/da/de/df/d26 491895 1 2024-03-22T07:08:37.991 INFO:tasks.workunit.client.0.smithi081.stdout:4/232: read f0 [2103028,53647] 0 2024-03-22T07:08:37.991 INFO:tasks.workunit.client.0.smithi081.stdout:6/196: write d3/f22 [636865,56089] 0 2024-03-22T07:08:37.991 INFO:tasks.workunit.client.0.smithi081.stdout:6/197: write d3/d4/f7 [588727,71993] 0 2024-03-22T07:08:37.991 INFO:tasks.workunit.client.0.smithi081.stdout:6/198: creat d3/f3c x:0 0 0 2024-03-22T07:08:37.991 INFO:tasks.workunit.client.0.smithi081.stdout:4/233: mkdir da/d18/d1b/d47 0 2024-03-22T07:08:37.993 INFO:tasks.workunit.client.0.smithi081.stdout:6/199: rename d3/d4/c2d to d3/da/de/d31/c3d 0 2024-03-22T07:08:37.993 INFO:tasks.workunit.client.0.smithi081.stdout:6/200: write d3/f9 [1590626,49296] 0 2024-03-22T07:08:37.994 INFO:tasks.workunit.client.0.smithi081.stdout:4/234: rename da/d18/d1b/d26/f2a to da/f48 0 2024-03-22T07:08:37.994 INFO:tasks.workunit.client.0.smithi081.stdout:4/235: fdatasync da/df/d38/f46 0 2024-03-22T07:08:37.995 INFO:tasks.workunit.client.0.smithi081.stdout:4/236: chown da/f2c 9488102 1 2024-03-22T07:08:37.995 INFO:tasks.workunit.client.0.smithi081.stdout:4/237: write da/d18/d1b/f40 [671271,26968] 0 2024-03-22T07:08:37.995 INFO:tasks.workunit.client.0.smithi081.stdout:6/201: symlink d3/d4/l3e 0 2024-03-22T07:08:38.011 INFO:tasks.workunit.client.0.smithi081.stdout:4/238: dwrite da/d18/d1b/f24 [0,4194304] 0 2024-03-22T07:08:38.015 INFO:tasks.workunit.client.0.smithi081.stdout:4/239: mknod da/d18/d1b/d26/d3e/c49 0 2024-03-22T07:08:38.016 INFO:tasks.workunit.client.0.smithi081.stdout:4/240: truncate da/fb 364070 0 2024-03-22T07:08:38.016 INFO:tasks.workunit.client.0.smithi081.stdout:4/241: chown da/d18/d1b/d47 4781082 1 2024-03-22T07:08:38.035 INFO:tasks.workunit.client.0.smithi081.stdout:4/242: dwrite da/d18/d1b/d26/d32/f39 [0,4194304] 0 2024-03-22T07:08:38.035 INFO:tasks.workunit.client.0.smithi081.stdout:4/243: stat da/d18 0 2024-03-22T07:08:38.035 INFO:tasks.workunit.client.0.smithi081.stdout:4/244: stat da/d18/d1b/d1e/f41 0 2024-03-22T07:08:38.035 INFO:tasks.workunit.client.0.smithi081.stdout:4/245: write da/df/d38/f46 [266648,1857] 0 2024-03-22T07:08:38.035 INFO:tasks.workunit.client.0.smithi081.stdout:4/246: chown da/l2e 2015453767 1 2024-03-22T07:08:38.035 INFO:tasks.workunit.client.0.smithi081.stdout:4/247: write da/d18/d1b/d1e/f41 [229095,63984] 0 2024-03-22T07:08:38.035 INFO:tasks.workunit.client.0.smithi081.stdout:4/248: truncate da/d18/f21 1111498 0 2024-03-22T07:08:38.035 INFO:tasks.workunit.client.0.smithi081.stdout:4/249: mkdir da/d18/d42/d4a 0 2024-03-22T07:08:38.036 INFO:tasks.workunit.client.0.smithi081.stdout:4/250: mknod da/d3d/c4b 0 2024-03-22T07:08:38.036 INFO:tasks.workunit.client.0.smithi081.stdout:4/251: creat da/d18/d1b/d34/f4c x:0 0 0 2024-03-22T07:08:38.038 INFO:tasks.workunit.client.0.smithi081.stdout:4/252: mknod da/d18/d42/d4a/c4d 0 2024-03-22T07:08:38.039 INFO:tasks.workunit.client.0.smithi081.stdout:4/253: rmdir da/d18/d1b/d26/d3e 39 2024-03-22T07:08:38.062 INFO:tasks.workunit.client.0.smithi081.stdout:4/254: dwrite da/d18/d1b/d34/f4c [0,4194304] 0 2024-03-22T07:08:38.062 INFO:tasks.workunit.client.0.smithi081.stdout:4/255: stat da/d18/d1b/d34/f44 0 2024-03-22T07:08:38.065 INFO:tasks.workunit.client.0.smithi081.stdout:4/256: symlink da/d18/d1b/l4e 0 2024-03-22T07:08:38.067 INFO:tasks.workunit.client.0.smithi081.stdout:4/257: dread da/d18/f21 [0,4194304] 0 2024-03-22T07:08:38.068 INFO:tasks.workunit.client.0.smithi081.stdout:4/258: mkdir da/d18/d42/d4f 0 2024-03-22T07:08:38.068 INFO:tasks.workunit.client.0.smithi081.stdout:4/259: readlink da/l1a 0 2024-03-22T07:08:38.068 INFO:tasks.workunit.client.0.smithi081.stdout:4/260: fdatasync da/f48 0 2024-03-22T07:08:38.071 INFO:tasks.workunit.client.0.smithi081.stdout:4/261: dread da/f48 [0,4194304] 0 2024-03-22T07:08:38.071 INFO:tasks.workunit.client.0.smithi081.stdout:4/262: truncate f3 1589293 0 2024-03-22T07:08:38.114 INFO:tasks.workunit.client.0.smithi081.stdout:4/263: rmdir da/d18/d1b/d1e/d37 39 2024-03-22T07:08:38.114 INFO:tasks.workunit.client.0.smithi081.stdout:4/264: fdatasync da/f48 0 2024-03-22T07:08:38.116 INFO:tasks.workunit.client.0.smithi081.stdout:4/265: write f9 [929123,51962] 0 2024-03-22T07:08:38.116 INFO:tasks.workunit.client.0.smithi081.stdout:4/266: write f9 [1794923,113508] 0 2024-03-22T07:08:38.117 INFO:tasks.workunit.client.0.smithi081.stdout:4/267: mknod da/d18/d42/c50 0 2024-03-22T07:08:38.117 INFO:tasks.workunit.client.0.smithi081.stdout:4/268: dread - da/d18/d1b/d1e/f3f zero size 2024-03-22T07:08:38.118 INFO:tasks.workunit.client.0.smithi081.stdout:4/269: write f3 [945804,81528] 0 2024-03-22T07:08:38.118 INFO:tasks.workunit.client.0.smithi081.stdout:4/270: chown f9 1113296 1 2024-03-22T07:08:38.119 INFO:tasks.workunit.client.0.smithi081.stdout:4/271: unlink da/l1a 0 2024-03-22T07:08:38.119 INFO:tasks.workunit.client.0.smithi081.stdout:4/272: read f9 [1007397,126148] 0 2024-03-22T07:08:38.120 INFO:tasks.workunit.client.0.smithi081.stdout:6/202: rename d3/d4 to d3/da/de/df/d26/d3f 0 2024-03-22T07:08:38.120 INFO:tasks.workunit.client.0.smithi081.stdout:6/203: readlink d3/l30 0 2024-03-22T07:08:38.120 INFO:tasks.workunit.client.0.smithi081.stdout:6/204: dread - d3/da/de/df/d26/f1d zero size 2024-03-22T07:08:38.121 INFO:tasks.workunit.client.0.smithi081.stdout:6/205: mkdir d3/da/de/d31/d40 0 2024-03-22T07:08:38.122 INFO:tasks.workunit.client.0.smithi081.stdout:6/206: link d3/da/de/df/d26/f33 d3/da/de/f41 0 2024-03-22T07:08:38.128 INFO:tasks.workunit.client.0.smithi081.stdout:6/207: write d3/da/de/df/d26/f25 [852958,84545] 0 2024-03-22T07:08:38.134 INFO:tasks.workunit.client.0.smithi081.stdout:6/208: write d3/da/de/f2e [1836707,52467] 0 2024-03-22T07:08:38.137 INFO:tasks.workunit.client.0.smithi081.stdout:4/273: dwrite da/d18/d1b/d26/f43 [0,4194304] 0 2024-03-22T07:08:38.147 INFO:tasks.workunit.client.0.smithi081.stdout:1/157: link f1 d6/d18/d1f/d24/f3a 0 2024-03-22T07:08:38.147 INFO:tasks.workunit.client.0.smithi081.stdout:4/274: dread da/d18/d1b/f40 [0,4194304] 0 2024-03-22T07:08:38.148 INFO:tasks.workunit.client.0.smithi081.stdout:3/138: sync 2024-03-22T07:08:38.149 INFO:tasks.workunit.client.0.smithi081.stdout:1/158: rename d6/d18/f2c to d6/d18/d1f/d24/f3b 0 2024-03-22T07:08:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:5/261: sync 2024-03-22T07:08:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:2/94: sync 2024-03-22T07:08:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:2/95: chown da 0 1 2024-03-22T07:08:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:5/262: creat d4/df/d22/f58 x:0 0 0 2024-03-22T07:08:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:9/167: sync 2024-03-22T07:08:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:7/321: sync 2024-03-22T07:08:38.152 INFO:tasks.workunit.client.0.smithi081.stdout:3/139: creat db/d23/f28 x:0 0 0 2024-03-22T07:08:38.154 INFO:tasks.workunit.client.0.smithi081.stdout:1/159: creat d6/d2e/f3c x:0 0 0 2024-03-22T07:08:38.154 INFO:tasks.workunit.client.0.smithi081.stdout:1/160: chown d6/d18/d1f/d24/f31 11148618 1 2024-03-22T07:08:38.158 INFO:tasks.workunit.client.0.smithi081.stdout:9/168: creat d2/d2b/f3b x:0 0 0 2024-03-22T07:08:38.158 INFO:tasks.workunit.client.0.smithi081.stdout:9/169: readlink d2/d5/d16/l1a 0 2024-03-22T07:08:38.160 INFO:tasks.workunit.client.0.smithi081.stdout:4/275: unlink da/f48 0 2024-03-22T07:08:38.160 INFO:tasks.workunit.client.0.smithi081.stdout:4/276: readlink da/d18/d1b/d1e/d37/l14 0 2024-03-22T07:08:38.161 INFO:tasks.workunit.client.0.smithi081.stdout:6/209: dwrite d3/da/de/df/d26/f25 [0,4194304] 0 2024-03-22T07:08:38.161 INFO:tasks.workunit.client.0.smithi081.stdout:6/210: dread - d3/da/de/df/f1e zero size 2024-03-22T07:08:38.162 INFO:tasks.workunit.client.0.smithi081.stdout:2/96: dread f6 [0,4194304] 0 2024-03-22T07:08:38.163 INFO:tasks.workunit.client.0.smithi081.stdout:9/170: rename d2/d5/d16/l1f to d2/l3c 0 2024-03-22T07:08:38.164 INFO:tasks.workunit.client.0.smithi081.stdout:1/161: write d6/fc [1175726,71232] 0 2024-03-22T07:08:38.165 INFO:tasks.workunit.client.0.smithi081.stdout:2/97: dread f5 [0,4194304] 0 2024-03-22T07:08:38.165 INFO:tasks.workunit.client.0.smithi081.stdout:2/98: chown da/f18 264095 1 2024-03-22T07:08:38.165 INFO:tasks.workunit.client.0.smithi081.stdout:2/99: fsync da/f13 0 2024-03-22T07:08:38.165 INFO:tasks.workunit.client.0.smithi081.stdout:3/140: read db/de/f24 [650977,12497] 0 2024-03-22T07:08:38.165 INFO:tasks.workunit.client.0.smithi081.stdout:3/141: dread - db/d23/f28 zero size 2024-03-22T07:08:38.165 INFO:tasks.workunit.client.0.smithi081.stdout:3/142: chown db/d23 5521807 1 2024-03-22T07:08:38.167 INFO:tasks.workunit.client.0.smithi081.stdout:6/211: symlink d3/da/de/df/l42 0 2024-03-22T07:08:38.167 INFO:tasks.workunit.client.0.smithi081.stdout:6/212: chown d3/da/de 109931859 1 2024-03-22T07:08:38.167 INFO:tasks.workunit.client.0.smithi081.stdout:9/171: symlink d2/l3d 0 2024-03-22T07:08:38.170 INFO:tasks.workunit.client.0.smithi081.stdout:9/172: mknod d2/d5/d16/c3e 0 2024-03-22T07:08:38.172 INFO:tasks.workunit.client.0.smithi081.stdout:3/143: write db/de/f1a [133000,53420] 0 2024-03-22T07:08:38.173 INFO:tasks.workunit.client.0.smithi081.stdout:9/173: unlink d2/d2b/f21 0 2024-03-22T07:08:38.173 INFO:tasks.workunit.client.0.smithi081.stdout:9/174: fdatasync d2/f25 0 2024-03-22T07:08:38.177 INFO:tasks.workunit.client.0.smithi081.stdout:5/263: dwrite d4/d9/f2c [0,4194304] 0 2024-03-22T07:08:38.179 INFO:tasks.workunit.client.0.smithi081.stdout:5/264: mknod d4/df/d22/d49/c59 0 2024-03-22T07:08:38.182 INFO:tasks.workunit.client.0.smithi081.stdout:5/265: symlink d4/l5a 0 2024-03-22T07:08:38.182 INFO:tasks.workunit.client.0.smithi081.stdout:5/266: fdatasync d4/df/d22/d49/f4a 0 2024-03-22T07:08:38.197 INFO:tasks.workunit.client.0.smithi081.stdout:6/213: dwrite d3/f9 [0,4194304] 0 2024-03-22T07:08:38.197 INFO:tasks.workunit.client.0.smithi081.stdout:6/214: truncate d3/da/de/df/d26/f1d 933173 0 2024-03-22T07:08:38.201 INFO:tasks.workunit.client.0.smithi081.stdout:4/277: dwrite da/d18/d1b/d1e/d37/f36 [0,4194304] 0 2024-03-22T07:08:38.206 INFO:tasks.workunit.client.0.smithi081.stdout:2/100: dwrite da/d11/f15 [0,4194304] 0 2024-03-22T07:08:38.212 INFO:tasks.workunit.client.0.smithi081.stdout:8/146: sync 2024-03-22T07:08:38.213 INFO:tasks.workunit.client.0.smithi081.stdout:0/240: sync 2024-03-22T07:08:38.213 INFO:tasks.workunit.client.0.smithi081.stdout:8/147: fsync d2/f11 0 2024-03-22T07:08:38.215 INFO:tasks.workunit.client.0.smithi081.stdout:0/241: mknod d3/dc/d40/c4c 0 2024-03-22T07:08:38.215 INFO:tasks.workunit.client.0.smithi081.stdout:0/242: chown d3/f1d 19376 1 2024-03-22T07:08:38.217 INFO:tasks.workunit.client.0.smithi081.stdout:0/243: mkdir d3/dc/d2b/d4d 0 2024-03-22T07:08:38.217 INFO:tasks.workunit.client.0.smithi081.stdout:4/278: write da/f1d [3896550,110004] 0 2024-03-22T07:08:38.218 INFO:tasks.workunit.client.0.smithi081.stdout:4/279: chown da/df/l16 51099991 1 2024-03-22T07:08:38.218 INFO:tasks.workunit.client.0.smithi081.stdout:9/175: dwrite d2/d10/f14 [4194304,4194304] 0 2024-03-22T07:08:38.218 INFO:tasks.workunit.client.0.smithi081.stdout:9/176: fdatasync d2/f3 0 2024-03-22T07:08:38.221 INFO:tasks.workunit.client.0.smithi081.stdout:0/244: getdents d3 0 2024-03-22T07:08:38.229 INFO:tasks.workunit.client.0.smithi081.stdout:3/144: dwrite db/f13 [0,4194304] 0 2024-03-22T07:08:38.230 INFO:tasks.workunit.client.0.smithi081.stdout:3/145: link f1 db/de/f29 0 2024-03-22T07:08:38.231 INFO:tasks.workunit.client.0.smithi081.stdout:3/146: rmdir db/d23 39 2024-03-22T07:08:38.241 INFO:tasks.workunit.client.0.smithi081.stdout:5/267: dwrite d4/df/f46 [0,4194304] 0 2024-03-22T07:08:38.242 INFO:tasks.workunit.client.0.smithi081.stdout:6/215: dwrite d3/da/de/df/f3b [0,4194304] 0 2024-03-22T07:08:38.242 INFO:tasks.workunit.client.0.smithi081.stdout:6/216: chown d3/da/de/df/d26/f25 441056680 1 2024-03-22T07:08:38.244 INFO:tasks.workunit.client.0.smithi081.stdout:5/268: unlink d4/df/l18 0 2024-03-22T07:08:38.244 INFO:tasks.workunit.client.0.smithi081.stdout:5/269: fdatasync d4/d10/d20/d25/d2e/f31 0 2024-03-22T07:08:38.246 INFO:tasks.workunit.client.0.smithi081.stdout:8/148: dwrite f0 [4194304,4194304] 0 2024-03-22T07:08:38.247 INFO:tasks.workunit.client.0.smithi081.stdout:5/270: creat d4/d9/f5b x:0 0 0 2024-03-22T07:08:38.249 INFO:tasks.workunit.client.0.smithi081.stdout:5/271: mknod d4/d10/d20/d25/d2e/c5c 0 2024-03-22T07:08:38.251 INFO:tasks.workunit.client.0.smithi081.stdout:5/272: mknod d4/d10/d20/c5d 0 2024-03-22T07:08:38.252 INFO:tasks.workunit.client.0.smithi081.stdout:3/147: dwrite db/de/f26 [0,4194304] 0 2024-03-22T07:08:38.254 INFO:tasks.workunit.client.0.smithi081.stdout:3/148: rename db/de/l10 to db/d23/l2a 0 2024-03-22T07:08:38.255 INFO:tasks.workunit.client.0.smithi081.stdout:3/149: link l8 db/d23/l2b 0 2024-03-22T07:08:38.256 INFO:tasks.workunit.client.0.smithi081.stdout:3/150: link db/l22 db/de/l2c 0 2024-03-22T07:08:38.257 INFO:tasks.workunit.client.0.smithi081.stdout:3/151: chown db/d23/l2a 430 1 2024-03-22T07:08:38.257 INFO:tasks.workunit.client.0.smithi081.stdout:3/152: mknod db/d23/c2d 0 2024-03-22T07:08:38.258 INFO:tasks.workunit.client.0.smithi081.stdout:3/153: mkdir db/d2e 0 2024-03-22T07:08:38.258 INFO:tasks.workunit.client.0.smithi081.stdout:3/154: readlink db/d23/l2a 0 2024-03-22T07:08:38.261 INFO:tasks.workunit.client.0.smithi081.stdout:6/217: dwrite d3/da/de/df/d26/f1d [0,4194304] 0 2024-03-22T07:08:38.261 INFO:tasks.workunit.client.0.smithi081.stdout:6/218: fdatasync d3/f9 0 2024-03-22T07:08:38.279 INFO:tasks.workunit.client.0.smithi081.stdout:8/149: mknod d2/d3/d19/d1e/c35 0 2024-03-22T07:08:38.280 INFO:tasks.workunit.client.0.smithi081.stdout:8/150: rename d2/d3/d19/d1e/l17 to d2/d3/d19/d1e/d13/l36 0 2024-03-22T07:08:38.280 INFO:tasks.workunit.client.0.smithi081.stdout:0/245: mkdir d3/d4e 0 2024-03-22T07:08:38.283 INFO:tasks.workunit.client.0.smithi081.stdout:0/246: unlink d3/c25 0 2024-03-22T07:08:38.284 INFO:tasks.workunit.client.0.smithi081.stdout:0/247: write d3/f3d [3731078,58931] 0 2024-03-22T07:08:38.284 INFO:tasks.workunit.client.0.smithi081.stdout:0/248: getdents d3/dc/d2b/d4d 0 2024-03-22T07:08:38.284 INFO:tasks.workunit.client.0.smithi081.stdout:0/249: truncate d3/dc/d2b/f31 1051352 0 2024-03-22T07:08:38.285 INFO:tasks.workunit.client.0.smithi081.stdout:0/250: read - d3/f24 zero size 2024-03-22T07:08:38.297 INFO:tasks.workunit.client.0.smithi081.stdout:6/219: dwrite d3/da/de/f27 [0,4194304] 0 2024-03-22T07:08:38.297 INFO:tasks.workunit.client.0.smithi081.stdout:5/273: dwrite d4/d10/d20/d25/d2e/f31 [4194304,4194304] 0 2024-03-22T07:08:38.299 INFO:tasks.workunit.client.0.smithi081.stdout:3/155: dwrite db/fd [4194304,4194304] 0 2024-03-22T07:08:38.299 INFO:tasks.workunit.client.0.smithi081.stdout:0/251: dread d3/f2d [0,4194304] 0 2024-03-22T07:08:38.299 INFO:tasks.workunit.client.0.smithi081.stdout:5/274: rename d4/d10/d20/d25/l42 to d4/df/d22/d49/l5e 0 2024-03-22T07:08:38.301 INFO:tasks.workunit.client.0.smithi081.stdout:3/156: creat db/f2f x:0 0 0 2024-03-22T07:08:38.301 INFO:tasks.workunit.client.0.smithi081.stdout:3/157: write db/d23/f28 [266051,101293] 0 2024-03-22T07:08:38.306 INFO:tasks.workunit.client.0.smithi081.stdout:6/220: dread d3/da/f17 [0,4194304] 0 2024-03-22T07:08:38.308 INFO:tasks.workunit.client.0.smithi081.stdout:6/221: mknod d3/da/de/d31/c43 0 2024-03-22T07:08:38.313 INFO:tasks.workunit.client.0.smithi081.stdout:0/252: dread d3/f1d [0,4194304] 0 2024-03-22T07:08:38.314 INFO:tasks.workunit.client.0.smithi081.stdout:0/253: truncate d3/f3e 206184 0 2024-03-22T07:08:38.314 INFO:tasks.workunit.client.0.smithi081.stdout:0/254: write d3/dc/d33/f47 [786399,70256] 0 2024-03-22T07:08:38.314 INFO:tasks.workunit.client.0.smithi081.stdout:0/255: getdents d3/dc/d2b/d4d 0 2024-03-22T07:08:38.318 INFO:tasks.workunit.client.0.smithi081.stdout:8/151: mknod d2/d3/d19/d1e/c37 0 2024-03-22T07:08:38.319 INFO:tasks.workunit.client.0.smithi081.stdout:4/280: link da/d18/d1b/d1e/f3b da/d18/d1b/d1e/d37/d1c/f51 0 2024-03-22T07:08:38.319 INFO:tasks.workunit.client.0.smithi081.stdout:4/281: truncate da/fb 2652337 0 2024-03-22T07:08:38.320 INFO:tasks.workunit.client.0.smithi081.stdout:4/282: getdents da/d18/d1b/d1e 0 2024-03-22T07:08:38.320 INFO:tasks.workunit.client.0.smithi081.stdout:4/283: chown da/d18/d42/c50 1449377 1 2024-03-22T07:08:38.321 INFO:tasks.workunit.client.0.smithi081.stdout:8/152: mkdir d2/d3/d19/d1e/d13/d22/d38 0 2024-03-22T07:08:38.321 INFO:tasks.workunit.client.0.smithi081.stdout:8/153: fdatasync d2/d3/f1b 0 2024-03-22T07:08:38.321 INFO:tasks.workunit.client.0.smithi081.stdout:8/154: chown d2/d3/d19/d1e/c37 9 1 2024-03-22T07:08:38.321 INFO:tasks.workunit.client.0.smithi081.stdout:4/284: rename da/d3d/c4b to da/d18/d1b/d26/d3e/c52 0 2024-03-22T07:08:38.322 INFO:tasks.workunit.client.0.smithi081.stdout:8/155: getdents d2/d3/d23 0 2024-03-22T07:08:38.322 INFO:tasks.workunit.client.0.smithi081.stdout:6/222: read d3/f9 [3007850,54527] 0 2024-03-22T07:08:38.326 INFO:tasks.workunit.client.0.smithi081.stdout:4/285: getdents da/d18/d1b 0 2024-03-22T07:08:38.329 INFO:tasks.workunit.client.0.smithi081.stdout:3/158: dwrite db/de/f1b [0,4194304] 0 2024-03-22T07:08:38.329 INFO:tasks.workunit.client.0.smithi081.stdout:5/275: dwrite d4/d10/f21 [0,4194304] 0 2024-03-22T07:08:38.329 INFO:tasks.workunit.client.0.smithi081.stdout:0/256: truncate d3/dc/d33/f47 118770 0 2024-03-22T07:08:38.329 INFO:tasks.workunit.client.0.smithi081.stdout:6/223: rename d3/da/f14 to d3/da/de/df/d26/f44 0 2024-03-22T07:08:38.334 INFO:tasks.workunit.client.0.smithi081.stdout:3/159: symlink db/l30 0 2024-03-22T07:08:38.334 INFO:tasks.workunit.client.0.smithi081.stdout:3/160: write db/f2f [698818,80157] 0 2024-03-22T07:08:38.336 INFO:tasks.workunit.client.0.smithi081.stdout:8/156: write f0 [4705999,101639] 0 2024-03-22T07:08:38.336 INFO:tasks.workunit.client.0.smithi081.stdout:8/157: creat d2/d3/d19/d1e/d13/f39 x:0 0 0 2024-03-22T07:08:38.337 INFO:tasks.workunit.client.0.smithi081.stdout:8/158: write d2/d3/f1f [348236,67902] 0 2024-03-22T07:08:38.337 INFO:tasks.workunit.client.0.smithi081.stdout:8/159: write d2/d3/d19/d1e/d13/f39 [1037938,40306] 0 2024-03-22T07:08:38.337 INFO:tasks.workunit.client.0.smithi081.stdout:0/257: write d3/dc/f1c [3414104,76467] 0 2024-03-22T07:08:38.339 INFO:tasks.workunit.client.0.smithi081.stdout:8/160: fdatasync f0 0 2024-03-22T07:08:38.339 INFO:tasks.workunit.client.0.smithi081.stdout:8/161: creat d2/d3/d19/d1e/d13/d22/f3a x:0 0 0 2024-03-22T07:08:38.345 INFO:tasks.workunit.client.0.smithi081.stdout:5/276: creat d4/d10/d20/d4f/f5f x:0 0 0 2024-03-22T07:08:38.347 INFO:tasks.workunit.client.0.smithi081.stdout:4/286: rename da/c30 to da/d18/d1b/d1e/d37/c53 0 2024-03-22T07:08:38.347 INFO:tasks.workunit.client.0.smithi081.stdout:4/287: creat da/df/d38/f54 x:0 0 0 2024-03-22T07:08:38.348 INFO:tasks.workunit.client.0.smithi081.stdout:6/224: getdents d3/da/de/d31 0 2024-03-22T07:08:38.350 INFO:tasks.workunit.client.0.smithi081.stdout:0/258: write d3/d32/d35/f38 [792228,105554] 0 2024-03-22T07:08:38.351 INFO:tasks.workunit.client.0.smithi081.stdout:8/162: creat d2/d3/d23/f3b x:0 0 0 2024-03-22T07:08:38.352 INFO:tasks.workunit.client.0.smithi081.stdout:8/163: truncate d2/d3/f1b 670310 0 2024-03-22T07:08:38.352 INFO:tasks.workunit.client.0.smithi081.stdout:8/164: truncate d2/d3/d19/f25 679321 0 2024-03-22T07:08:38.353 INFO:tasks.workunit.client.0.smithi081.stdout:5/277: link d4/d10/d20/d1f/d2f/f37 d4/df/d22/d49/f60 0 2024-03-22T07:08:38.355 INFO:tasks.workunit.client.0.smithi081.stdout:8/165: creat d2/d3/d19/d1e/f3c x:0 0 0 2024-03-22T07:08:38.355 INFO:tasks.workunit.client.0.smithi081.stdout:8/166: chown d2/d3/l20 5 1 2024-03-22T07:08:38.355 INFO:tasks.workunit.client.0.smithi081.stdout:3/161: getdents db 0 2024-03-22T07:08:38.356 INFO:tasks.workunit.client.0.smithi081.stdout:5/278: symlink d4/d10/d20/d4f/l61 0 2024-03-22T07:08:38.365 INFO:tasks.workunit.client.0.smithi081.stdout:0/259: rmdir d3 39 2024-03-22T07:08:38.365 INFO:tasks.workunit.client.0.smithi081.stdout:0/260: chown d3/dc/d40/f42 101760 1 2024-03-22T07:08:38.367 INFO:tasks.workunit.client.0.smithi081.stdout:0/261: unlink d3/f10 0 2024-03-22T07:08:38.371 INFO:tasks.workunit.client.0.smithi081.stdout:3/162: write db/f13 [2853849,66439] 0 2024-03-22T07:08:38.371 INFO:tasks.workunit.client.0.smithi081.stdout:7/322: sync 2024-03-22T07:08:38.372 INFO:tasks.workunit.client.0.smithi081.stdout:1/162: sync 2024-03-22T07:08:38.372 INFO:tasks.workunit.client.0.smithi081.stdout:8/167: read d2/d3/f1c [7715467,103661] 0 2024-03-22T07:08:38.376 INFO:tasks.workunit.client.0.smithi081.stdout:8/168: rmdir d2/d3/d19/d1e/d13/d22/d2f 39 2024-03-22T07:08:38.392 INFO:tasks.workunit.client.0.smithi081.stdout:6/225: dwrite d3/da/de/df/d26/f33 [0,4194304] 0 2024-03-22T07:08:38.392 INFO:tasks.workunit.client.0.smithi081.stdout:6/226: fsync d3/da/de/f1f 0 2024-03-22T07:08:38.392 INFO:tasks.workunit.client.0.smithi081.stdout:6/227: write d3/da/de/df/f1e [353788,37959] 0 2024-03-22T07:08:38.393 INFO:tasks.workunit.client.0.smithi081.stdout:4/288: dwrite da/d18/f31 [0,4194304] 0 2024-03-22T07:08:38.394 INFO:tasks.workunit.client.0.smithi081.stdout:5/279: dwrite f2 [0,4194304] 0 2024-03-22T07:08:38.395 INFO:tasks.workunit.client.0.smithi081.stdout:0/262: dwrite d3/dc/d2b/f31 [0,4194304] 0 2024-03-22T07:08:38.397 INFO:tasks.workunit.client.0.smithi081.stdout:3/163: dread db/f2f [0,4194304] 0 2024-03-22T07:08:38.397 INFO:tasks.workunit.client.0.smithi081.stdout:4/289: creat da/d18/d1b/d26/d3e/f55 x:0 0 0 2024-03-22T07:08:38.397 INFO:tasks.workunit.client.0.smithi081.stdout:4/290: dread - da/df/d38/f54 zero size 2024-03-22T07:08:38.399 INFO:tasks.workunit.client.0.smithi081.stdout:5/280: creat d4/d10/d20/d1f/d2f/d45/f62 x:0 0 0 2024-03-22T07:08:38.400 INFO:tasks.workunit.client.0.smithi081.stdout:0/263: getdents d3/dc/d40 0 2024-03-22T07:08:38.410 INFO:tasks.workunit.client.0.smithi081.stdout:8/169: dwrite f0 [4194304,4194304] 0 2024-03-22T07:08:38.412 INFO:tasks.workunit.client.0.smithi081.stdout:8/170: getdents d2/d3 0 2024-03-22T07:08:38.412 INFO:tasks.workunit.client.0.smithi081.stdout:8/171: creat d2/d3/d28/d2b/f3d x:0 0 0 2024-03-22T07:08:38.413 INFO:tasks.workunit.client.0.smithi081.stdout:0/264: dread d3/f2d [0,4194304] 0 2024-03-22T07:08:38.416 INFO:tasks.workunit.client.0.smithi081.stdout:5/281: dread d4/f33 [0,4194304] 0 2024-03-22T07:08:38.417 INFO:tasks.workunit.client.0.smithi081.stdout:5/282: fsync d4/d10/f21 0 2024-03-22T07:08:38.418 INFO:tasks.workunit.client.0.smithi081.stdout:5/283: creat d4/df/d22/d4d/f63 x:0 0 0 2024-03-22T07:08:38.421 INFO:tasks.workunit.client.0.smithi081.stdout:4/291: dwrite da/d18/d1b/d34/f4c [0,4194304] 0 2024-03-22T07:08:38.422 INFO:tasks.workunit.client.0.smithi081.stdout:4/292: mknod da/d18/d1b/d26/d3e/c56 0 2024-03-22T07:08:38.422 INFO:tasks.workunit.client.0.smithi081.stdout:4/293: readlink da/d18/d1b/d34/l33 0 2024-03-22T07:08:38.424 INFO:tasks.workunit.client.0.smithi081.stdout:3/164: dread db/f12 [0,4194304] 0 2024-03-22T07:08:38.424 INFO:tasks.workunit.client.0.smithi081.stdout:7/323: dwrite d9/d10/d16/d1e/f28 [0,4194304] 0 2024-03-22T07:08:38.424 INFO:tasks.workunit.client.0.smithi081.stdout:7/324: fdatasync d9/f3c 0 2024-03-22T07:08:38.424 INFO:tasks.workunit.client.0.smithi081.stdout:4/294: mkdir da/d57 0 2024-03-22T07:08:38.428 INFO:tasks.workunit.client.0.smithi081.stdout:7/325: dread d9/d10/d16/d1e/d27/f57 [0,4194304] 0 2024-03-22T07:08:38.428 INFO:tasks.workunit.client.0.smithi081.stdout:7/326: stat d9/d10/d2a/d73 0 2024-03-22T07:08:38.434 INFO:tasks.workunit.client.0.smithi081.stdout:7/327: creat d9/f75 x:0 0 0 2024-03-22T07:08:38.438 INFO:tasks.workunit.client.0.smithi081.stdout:7/328: mknod d9/d10/c76 0 2024-03-22T07:08:38.454 INFO:tasks.workunit.client.0.smithi081.stdout:1/163: rename d6/d18/d1f/d24/f3a to d6/d2e/d30/f3d 0 2024-03-22T07:08:38.454 INFO:tasks.workunit.client.0.smithi081.stdout:1/164: creat d6/d2e/f3e x:0 0 0 2024-03-22T07:08:38.454 INFO:tasks.workunit.client.0.smithi081.stdout:1/165: creat d6/d18/d1f/d24/f3f x:0 0 0 2024-03-22T07:08:38.455 INFO:tasks.workunit.client.0.smithi081.stdout:5/284: dwrite d4/f33 [0,4194304] 0 2024-03-22T07:08:38.457 INFO:tasks.workunit.client.0.smithi081.stdout:6/228: rename d3/f36 to d3/da/f45 0 2024-03-22T07:08:38.458 INFO:tasks.workunit.client.0.smithi081.stdout:1/166: mkdir d6/d18/d1f/d32/d40 0 2024-03-22T07:08:38.461 INFO:tasks.workunit.client.0.smithi081.stdout:1/167: truncate d6/d18/d1f/d24/f26 214526 0 2024-03-22T07:08:38.464 INFO:tasks.workunit.client.0.smithi081.stdout:9/177: sync 2024-03-22T07:08:38.464 INFO:tasks.workunit.client.0.smithi081.stdout:2/101: sync 2024-03-22T07:08:38.464 INFO:tasks.workunit.client.0.smithi081.stdout:5/285: dread f3 [0,4194304] 0 2024-03-22T07:08:38.464 INFO:tasks.workunit.client.0.smithi081.stdout:5/286: write d4/d10/d20/d25/f43 [496321,42793] 0 2024-03-22T07:08:38.465 INFO:tasks.workunit.client.0.smithi081.stdout:5/287: fsync d4/d10/d20/d4f/f5f 0 2024-03-22T07:08:38.466 INFO:tasks.workunit.client.0.smithi081.stdout:9/178: link d2/d2b/f2d d2/d5/f3f 0 2024-03-22T07:08:38.474 INFO:tasks.workunit.client.0.smithi081.stdout:5/288: dread d4/d10/f1d [0,4194304] 0 2024-03-22T07:08:38.476 INFO:tasks.workunit.client.0.smithi081.stdout:9/179: dread d2/f15 [0,4194304] 0 2024-03-22T07:08:38.476 INFO:tasks.workunit.client.0.smithi081.stdout:9/180: truncate d2/d5/f3f 2592560 0 2024-03-22T07:08:38.476 INFO:tasks.workunit.client.0.smithi081.stdout:9/181: write d2/d2b/f26 [1576156,47987] 0 2024-03-22T07:08:38.476 INFO:tasks.workunit.client.0.smithi081.stdout:9/182: creat d2/f40 x:0 0 0 2024-03-22T07:08:38.479 INFO:tasks.workunit.client.0.smithi081.stdout:6/229: dread d3/da/de/df/d26/f25 [0,4194304] 0 2024-03-22T07:08:38.481 INFO:tasks.workunit.client.0.smithi081.stdout:5/289: dread d4/d10/f3c [0,4194304] 0 2024-03-22T07:08:38.481 INFO:tasks.workunit.client.0.smithi081.stdout:6/230: write d3/da/de/df/d26/f1c [279653,60249] 0 2024-03-22T07:08:38.481 INFO:tasks.workunit.client.0.smithi081.stdout:6/231: readlink d3/da/l32 0 2024-03-22T07:08:38.481 INFO:tasks.workunit.client.0.smithi081.stdout:6/232: creat d3/da/de/f46 x:0 0 0 2024-03-22T07:08:38.481 INFO:tasks.workunit.client.0.smithi081.stdout:5/290: symlink d4/df/d22/d49/l64 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:5/291: mknod d4/d10/d20/d1f/d2f/d45/c65 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:7/329: dwrite d9/d10/d16/d1e/f58 [0,4194304] 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:7/330: chown d9/d3f 14212 1 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:7/331: write d9/dc/f1a [5079013,8471] 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:3/165: dwrite db/de/f26 [0,4194304] 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:2/102: write da/f18 [3533208,91565] 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:4/295: dwrite da/f1d [0,4194304] 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:4/296: fsync da/d18/d1b/d1e/f3b 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:4/297: fsync da/df/d38/f54 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:4/298: write da/fe [513948,101782] 0 2024-03-22T07:08:38.485 INFO:tasks.workunit.client.0.smithi081.stdout:4/299: truncate da/d18/d1b/d1e/f3f 644408 0 2024-03-22T07:08:38.486 INFO:tasks.workunit.client.0.smithi081.stdout:6/233: mknod d3/da/c47 0 2024-03-22T07:08:38.486 INFO:tasks.workunit.client.0.smithi081.stdout:1/168: dwrite d6/f35 [0,4194304] 0 2024-03-22T07:08:38.488 INFO:tasks.workunit.client.0.smithi081.stdout:7/332: mknod d9/d10/d16/d1e/d6f/c77 0 2024-03-22T07:08:38.489 INFO:tasks.workunit.client.0.smithi081.stdout:3/166: link db/c1d db/d2e/c31 0 2024-03-22T07:08:38.489 INFO:tasks.workunit.client.0.smithi081.stdout:3/167: truncate db/f2f 1574816 0 2024-03-22T07:08:38.489 INFO:tasks.workunit.client.0.smithi081.stdout:2/103: mknod da/c19 0 2024-03-22T07:08:38.490 INFO:tasks.workunit.client.0.smithi081.stdout:1/169: mkdir d6/d18/d41 0 2024-03-22T07:08:38.491 INFO:tasks.workunit.client.0.smithi081.stdout:7/333: dread d9/d10/f2e [0,4194304] 0 2024-03-22T07:08:38.492 INFO:tasks.workunit.client.0.smithi081.stdout:9/183: dwrite d2/d2b/f3b [0,4194304] 0 2024-03-22T07:08:38.493 INFO:tasks.workunit.client.0.smithi081.stdout:5/292: rmdir d4/d10/d20/d1f 39 2024-03-22T07:08:38.493 INFO:tasks.workunit.client.0.smithi081.stdout:1/170: mkdir d6/d18/d1f/d24/d42 0 2024-03-22T07:08:38.493 INFO:tasks.workunit.client.0.smithi081.stdout:1/171: chown d6/f35 12043 1 2024-03-22T07:08:38.493 INFO:tasks.workunit.client.0.smithi081.stdout:1/172: fdatasync d6/fd 0 2024-03-22T07:08:38.504 INFO:tasks.workunit.client.0.smithi081.stdout:3/168: creat db/d2e/f32 x:0 0 0 2024-03-22T07:08:38.505 INFO:tasks.workunit.client.0.smithi081.stdout:2/104: creat da/d11/f1a x:0 0 0 2024-03-22T07:08:38.505 INFO:tasks.workunit.client.0.smithi081.stdout:2/105: creat da/d11/f1b x:0 0 0 2024-03-22T07:08:38.505 INFO:tasks.workunit.client.0.smithi081.stdout:7/334: creat d9/d10/d2a/f78 x:0 0 0 2024-03-22T07:08:38.511 INFO:tasks.workunit.client.0.smithi081.stdout:6/234: mkdir d3/da/d48 0 2024-03-22T07:08:38.512 INFO:tasks.workunit.client.0.smithi081.stdout:3/169: unlink db/d2e/c31 0 2024-03-22T07:08:38.513 INFO:tasks.workunit.client.0.smithi081.stdout:3/170: write db/fc [680185,60866] 0 2024-03-22T07:08:38.514 INFO:tasks.workunit.client.0.smithi081.stdout:2/106: symlink da/d11/l1c 0 2024-03-22T07:08:38.514 INFO:tasks.workunit.client.0.smithi081.stdout:2/107: fdatasync da/d11/f1a 0 2024-03-22T07:08:38.515 INFO:tasks.workunit.client.0.smithi081.stdout:9/184: getdents d2/d5 0 2024-03-22T07:08:38.517 INFO:tasks.workunit.client.0.smithi081.stdout:3/171: mknod db/c33 0 2024-03-22T07:08:38.517 INFO:tasks.workunit.client.0.smithi081.stdout:3/172: write db/de/f29 [2393829,22558] 0 2024-03-22T07:08:38.517 INFO:tasks.workunit.client.0.smithi081.stdout:3/173: read db/f2f [1387596,129126] 0 2024-03-22T07:08:38.518 INFO:tasks.workunit.client.0.smithi081.stdout:2/108: link l1 da/l1d 0 2024-03-22T07:08:38.519 INFO:tasks.workunit.client.0.smithi081.stdout:9/185: link d2/f3 d2/d2b/f41 0 2024-03-22T07:08:38.519 INFO:tasks.workunit.client.0.smithi081.stdout:9/186: write d2/f1e [16555,21707] 0 2024-03-22T07:08:38.519 INFO:tasks.workunit.client.0.smithi081.stdout:9/187: truncate d2/d5/f31 718859 0 2024-03-22T07:08:38.522 INFO:tasks.workunit.client.0.smithi081.stdout:6/235: getdents d3/da 0 2024-03-22T07:08:38.524 INFO:tasks.workunit.client.0.smithi081.stdout:6/236: symlink d3/da/de/l49 0 2024-03-22T07:08:38.525 INFO:tasks.workunit.client.0.smithi081.stdout:3/174: dread db/f12 [0,4194304] 0 2024-03-22T07:08:38.529 INFO:tasks.workunit.client.0.smithi081.stdout:7/335: rmdir d9 39 2024-03-22T07:08:38.530 INFO:tasks.workunit.client.0.smithi081.stdout:7/336: fdatasync d9/dc/f38 0 2024-03-22T07:08:38.530 INFO:tasks.workunit.client.0.smithi081.stdout:6/237: symlink d3/da/l4a 0 2024-03-22T07:08:38.531 INFO:tasks.workunit.client.0.smithi081.stdout:3/175: mknod db/de/c34 0 2024-03-22T07:08:38.532 INFO:tasks.workunit.client.0.smithi081.stdout:3/176: symlink db/d2e/l35 0 2024-03-22T07:08:38.532 INFO:tasks.workunit.client.0.smithi081.stdout:3/177: read db/f12 [1402600,46979] 0 2024-03-22T07:08:38.534 INFO:tasks.workunit.client.0.smithi081.stdout:6/238: unlink d3/da/de/df/d26/f25 0 2024-03-22T07:08:38.534 INFO:tasks.workunit.client.0.smithi081.stdout:6/239: write d3/da/de/f28 [69907,25069] 0 2024-03-22T07:08:38.534 INFO:tasks.workunit.client.0.smithi081.stdout:6/240: write d3/da/de/df/f37 [1347972,46620] 0 2024-03-22T07:08:38.534 INFO:tasks.workunit.client.0.smithi081.stdout:3/178: symlink db/d2e/l36 0 2024-03-22T07:08:38.534 INFO:tasks.workunit.client.0.smithi081.stdout:3/179: stat db/l27 0 2024-03-22T07:08:38.535 INFO:tasks.workunit.client.0.smithi081.stdout:6/241: readlink d3/da/l32 0 2024-03-22T07:08:38.535 INFO:tasks.workunit.client.0.smithi081.stdout:3/180: symlink db/d2e/l37 0 2024-03-22T07:08:38.539 INFO:tasks.workunit.client.0.smithi081.stdout:6/242: dread d3/da/de/df/d26/d3f/f29 [0,4194304] 0 2024-03-22T07:08:38.540 INFO:tasks.workunit.client.0.smithi081.stdout:6/243: chown d3/da/d48 19 1 2024-03-22T07:08:38.540 INFO:tasks.workunit.client.0.smithi081.stdout:6/244: write d3/da/de/f2e [1035149,17713] 0 2024-03-22T07:08:38.540 INFO:tasks.workunit.client.0.smithi081.stdout:6/245: creat d3/da/de/f4b x:0 0 0 2024-03-22T07:08:38.541 INFO:tasks.workunit.client.0.smithi081.stdout:7/337: write d9/f3c [1991635,5148] 0 2024-03-22T07:08:38.541 INFO:tasks.workunit.client.0.smithi081.stdout:7/338: fsync d9/d10/d16/d1e/d27/f5a 0 2024-03-22T07:08:38.543 INFO:tasks.workunit.client.0.smithi081.stdout:6/246: dread d3/da/de/df/d26/d3f/f7 [0,4194304] 0 2024-03-22T07:08:38.546 INFO:tasks.workunit.client.0.smithi081.stdout:3/181: dread db/de/f17 [0,4194304] 0 2024-03-22T07:08:38.546 INFO:tasks.workunit.client.0.smithi081.stdout:3/182: stat db/de/l1c 0 2024-03-22T07:08:38.546 INFO:tasks.workunit.client.0.smithi081.stdout:3/183: mkdir db/d38 0 2024-03-22T07:08:38.548 INFO:tasks.workunit.client.0.smithi081.stdout:9/188: dwrite d2/fe [4194304,4194304] 0 2024-03-22T07:08:38.548 INFO:tasks.workunit.client.0.smithi081.stdout:9/189: write d2/f4 [4043151,124348] 0 2024-03-22T07:08:38.552 INFO:tasks.workunit.client.0.smithi081.stdout:2/109: dwrite f6 [4194304,4194304] 0 2024-03-22T07:08:38.552 INFO:tasks.workunit.client.0.smithi081.stdout:2/110: write da/d11/f1a [940272,54685] 0 2024-03-22T07:08:38.553 INFO:tasks.workunit.client.0.smithi081.stdout:2/111: write da/d11/f1b [96058,125339] 0 2024-03-22T07:08:38.553 INFO:tasks.workunit.client.0.smithi081.stdout:2/112: dread f5 [0,4194304] 0 2024-03-22T07:08:38.556 INFO:tasks.workunit.client.0.smithi081.stdout:2/113: mknod da/c1e 0 2024-03-22T07:08:38.556 INFO:tasks.workunit.client.0.smithi081.stdout:7/339: dread d9/dc/f1f [0,4194304] 0 2024-03-22T07:08:38.559 INFO:tasks.workunit.client.0.smithi081.stdout:7/340: creat d9/d3f/f79 x:0 0 0 2024-03-22T07:08:38.560 INFO:tasks.workunit.client.0.smithi081.stdout:3/184: write db/de/f26 [1528374,128895] 0 2024-03-22T07:08:38.563 INFO:tasks.workunit.client.0.smithi081.stdout:8/172: rename d2/d3/d28/d2b/f3d to d2/d3/d19/d1e/f3e 0 2024-03-22T07:08:38.566 INFO:tasks.workunit.client.0.smithi081.stdout:3/185: truncate db/f18 360007 0 2024-03-22T07:08:38.567 INFO:tasks.workunit.client.0.smithi081.stdout:3/186: creat db/d2e/f39 x:0 0 0 2024-03-22T07:08:38.567 INFO:tasks.workunit.client.0.smithi081.stdout:3/187: readlink l5 0 2024-03-22T07:08:38.567 INFO:tasks.workunit.client.0.smithi081.stdout:3/188: chown db/l30 2047965 1 2024-03-22T07:08:38.567 INFO:tasks.workunit.client.0.smithi081.stdout:8/173: mknod d2/d3/d19/d1e/c3f 0 2024-03-22T07:08:38.568 INFO:tasks.workunit.client.0.smithi081.stdout:8/174: creat d2/d3/d19/d1e/d13/d22/d2f/f40 x:0 0 0 2024-03-22T07:08:38.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:38 smithi081 ceph-mon[29245]: pgmap v242: 65 pgs: 65 active+clean; 363 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 22 MiB/s rd, 35 MiB/s wr, 535 op/s 2024-03-22T07:08:38.571 INFO:tasks.workunit.client.0.smithi081.stdout:3/189: mkdir db/d2e/d3a 0 2024-03-22T07:08:38.571 INFO:tasks.workunit.client.0.smithi081.stdout:3/190: readlink db/de/l1c 0 2024-03-22T07:08:38.576 INFO:tasks.workunit.client.0.smithi081.stdout:8/175: dread d2/d3/d19/d1e/d13/f39 [0,4194304] 0 2024-03-22T07:08:38.576 INFO:tasks.workunit.client.0.smithi081.stdout:7/341: dread d9/d10/d16/d1e/f18 [0,4194304] 0 2024-03-22T07:08:38.576 INFO:tasks.workunit.client.0.smithi081.stdout:7/342: write d9/d10/d16/f42 [346152,75586] 0 2024-03-22T07:08:38.578 INFO:tasks.workunit.client.0.smithi081.stdout:7/343: link d9/d10/d16/d1e/d27/f57 d9/d10/d16/f7a 0 2024-03-22T07:08:38.578 INFO:tasks.workunit.client.0.smithi081.stdout:7/344: write d9/d10/d16/f6b [1490,50839] 0 2024-03-22T07:08:38.578 INFO:tasks.workunit.client.0.smithi081.stdout:7/345: chown d9/d3f/d52/c60 3632820 1 2024-03-22T07:08:38.580 INFO:tasks.workunit.client.0.smithi081.stdout:7/346: mknod d9/d3f/d52/c7b 0 2024-03-22T07:08:38.580 INFO:tasks.workunit.client.0.smithi081.stdout:7/347: write d9/d10/d16/f7a [715898,40377] 0 2024-03-22T07:08:38.580 INFO:tasks.workunit.client.0.smithi081.stdout:7/348: write d9/d10/d2a/f78 [882492,98778] 0 2024-03-22T07:08:38.583 INFO:tasks.workunit.client.0.smithi081.stdout:7/349: dread d9/f11 [0,4194304] 0 2024-03-22T07:08:38.583 INFO:tasks.workunit.client.0.smithi081.stdout:7/350: chown d9/f11 0 1 2024-03-22T07:08:38.584 INFO:tasks.workunit.client.0.smithi081.stdout:6/247: dwrite d3/da/f17 [0,4194304] 0 2024-03-22T07:08:38.584 INFO:tasks.workunit.client.0.smithi081.stdout:9/190: dwrite d2/d2b/f26 [0,4194304] 0 2024-03-22T07:08:38.584 INFO:tasks.workunit.client.0.smithi081.stdout:9/191: dread - d2/d5/d16/f2e zero size 2024-03-22T07:08:38.585 INFO:tasks.workunit.client.0.smithi081.stdout:9/192: write d2/d2b/f2d [3366123,108683] 0 2024-03-22T07:08:38.585 INFO:tasks.workunit.client.0.smithi081.stdout:9/193: stat d2/d2b 0 2024-03-22T07:08:38.585 INFO:tasks.workunit.client.0.smithi081.stdout:9/194: write d2/d10/f35 [263097,53984] 0 2024-03-22T07:08:38.586 INFO:tasks.workunit.client.0.smithi081.stdout:2/114: dwrite da/d11/f15 [0,4194304] 0 2024-03-22T07:08:38.587 INFO:tasks.workunit.client.0.smithi081.stdout:2/115: creat da/d11/f1f x:0 0 0 2024-03-22T07:08:38.587 INFO:tasks.workunit.client.0.smithi081.stdout:2/116: chown da/db 405285 1 2024-03-22T07:08:38.588 INFO:tasks.workunit.client.0.smithi081.stdout:7/351: creat d9/f7c x:0 0 0 2024-03-22T07:08:38.588 INFO:tasks.workunit.client.0.smithi081.stdout:0/265: rename d3/dc/f1b to d3/d32/f4f 0 2024-03-22T07:08:38.588 INFO:tasks.workunit.client.0.smithi081.stdout:0/266: fdatasync d3/dc/d2b/f2f 0 2024-03-22T07:08:38.598 INFO:tasks.workunit.client.0.smithi081.stdout:2/117: symlink da/db/l20 0 2024-03-22T07:08:38.598 INFO:tasks.workunit.client.0.smithi081.stdout:0/267: mknod d3/d32/c50 0 2024-03-22T07:08:38.615 INFO:tasks.workunit.client.0.smithi081.stdout:4/300: rename da/d18/d1b/d26/d32/f39 to da/d18/d42/d4f/f58 0 2024-03-22T07:08:38.615 INFO:tasks.workunit.client.0.smithi081.stdout:4/301: write da/df/d38/f46 [530897,30731] 0 2024-03-22T07:08:38.615 INFO:tasks.workunit.client.0.smithi081.stdout:4/302: fsync da/d18/d1b/d1e/f41 0 2024-03-22T07:08:38.620 INFO:tasks.workunit.client.0.smithi081.stdout:2/118: rmdir da/db 39 2024-03-22T07:08:38.620 INFO:tasks.workunit.client.0.smithi081.stdout:2/119: readlink da/db/l20 0 2024-03-22T07:08:38.621 INFO:tasks.workunit.client.0.smithi081.stdout:4/303: symlink da/d18/l59 0 2024-03-22T07:08:38.622 INFO:tasks.workunit.client.0.smithi081.stdout:2/120: mkdir da/d11/d21 0 2024-03-22T07:08:38.622 INFO:tasks.workunit.client.0.smithi081.stdout:2/121: getdents da/d11/d21 0 2024-03-22T07:08:38.622 INFO:tasks.workunit.client.0.smithi081.stdout:2/122: write f6 [9029067,88777] 0 2024-03-22T07:08:38.623 INFO:tasks.workunit.client.0.smithi081.stdout:4/304: creat da/df/f5a x:0 0 0 2024-03-22T07:08:38.623 INFO:tasks.workunit.client.0.smithi081.stdout:4/305: fsync da/d18/d1b/d34/f44 0 2024-03-22T07:08:38.625 INFO:tasks.workunit.client.0.smithi081.stdout:4/306: creat da/d18/d42/d4f/f5b x:0 0 0 2024-03-22T07:08:38.626 INFO:tasks.workunit.client.0.smithi081.stdout:9/195: dwrite d2/f40 [0,4194304] 0 2024-03-22T07:08:38.628 INFO:tasks.workunit.client.0.smithi081.stdout:6/248: dwrite d3/da/de/df/f3b [0,4194304] 0 2024-03-22T07:08:38.628 INFO:tasks.workunit.client.0.smithi081.stdout:3/191: dwrite db/de/f29 [0,4194304] 0 2024-03-22T07:08:38.628 INFO:tasks.workunit.client.0.smithi081.stdout:3/192: write db/de/f26 [60556,109543] 0 2024-03-22T07:08:38.628 INFO:tasks.workunit.client.0.smithi081.stdout:3/193: getdents db/d2e/d3a 0 2024-03-22T07:08:38.629 INFO:tasks.workunit.client.0.smithi081.stdout:7/352: dwrite d9/f3c [4194304,4194304] 0 2024-03-22T07:08:38.629 INFO:tasks.workunit.client.0.smithi081.stdout:4/307: rename da/df/l16 to da/d18/d1b/d1e/l5c 0 2024-03-22T07:08:38.632 INFO:tasks.workunit.client.0.smithi081.stdout:4/308: dread da/d18/d1b/f40 [0,4194304] 0 2024-03-22T07:08:38.635 INFO:tasks.workunit.client.0.smithi081.stdout:6/249: mkdir d3/da/de/df/d26/d3f/d4c 0 2024-03-22T07:08:38.635 INFO:tasks.workunit.client.0.smithi081.stdout:6/250: chown d3/da/de/f4b 1245823646 1 2024-03-22T07:08:38.638 INFO:tasks.workunit.client.0.smithi081.stdout:7/353: creat d9/d10/d2a/f7d x:0 0 0 2024-03-22T07:08:38.639 INFO:tasks.workunit.client.0.smithi081.stdout:3/194: rename db/d23/l2a to db/d2e/l3b 0 2024-03-22T07:08:38.643 INFO:tasks.workunit.client.0.smithi081.stdout:9/196: rmdir d2/d5/d16 39 2024-03-22T07:08:38.645 INFO:tasks.workunit.client.0.smithi081.stdout:3/195: link db/c1d db/de/c3c 0 2024-03-22T07:08:38.645 INFO:tasks.workunit.client.0.smithi081.stdout:4/309: symlink da/d18/d1b/d26/l5d 0 2024-03-22T07:08:38.648 INFO:tasks.workunit.client.0.smithi081.stdout:4/310: creat da/d3d/f5e x:0 0 0 2024-03-22T07:08:38.648 INFO:tasks.workunit.client.0.smithi081.stdout:4/311: write da/f2c [1200798,62267] 0 2024-03-22T07:08:38.648 INFO:tasks.workunit.client.0.smithi081.stdout:7/354: rmdir d9/d10 39 2024-03-22T07:08:38.650 INFO:tasks.workunit.client.0.smithi081.stdout:7/355: mknod d9/d10/d16/d1e/d27/c7e 0 2024-03-22T07:08:38.650 INFO:tasks.workunit.client.0.smithi081.stdout:7/356: chown d9/d10/d16/d1e/d27/d70 1923166728 1 2024-03-22T07:08:38.651 INFO:tasks.workunit.client.0.smithi081.stdout:7/357: creat d9/d10/d4b/d61/f7f x:0 0 0 2024-03-22T07:08:38.652 INFO:tasks.workunit.client.0.smithi081.stdout:7/358: chown d9/d10/d2a/c2d 16617 1 2024-03-22T07:08:38.652 INFO:tasks.workunit.client.0.smithi081.stdout:7/359: truncate d9/d10/d16/f7a 876206 0 2024-03-22T07:08:38.652 INFO:tasks.workunit.client.0.smithi081.stdout:7/360: fsync d9/d10/d16/d1e/f4e 0 2024-03-22T07:08:38.653 INFO:tasks.workunit.client.0.smithi081.stdout:3/196: dread db/f13 [0,4194304] 0 2024-03-22T07:08:38.653 INFO:tasks.workunit.client.0.smithi081.stdout:3/197: stat db/fc 0 2024-03-22T07:08:38.653 INFO:tasks.workunit.client.0.smithi081.stdout:7/361: mkdir d9/d10/d16/d1e/d80 0 2024-03-22T07:08:38.655 INFO:tasks.workunit.client.0.smithi081.stdout:7/362: rename d9/dc/d41 to d9/d10/d53/d81 0 2024-03-22T07:08:38.656 INFO:tasks.workunit.client.0.smithi081.stdout:9/197: read d2/f8 [3704439,21070] 0 2024-03-22T07:08:38.657 INFO:tasks.workunit.client.0.smithi081.stdout:9/198: symlink d2/l42 0 2024-03-22T07:08:38.657 INFO:tasks.workunit.client.0.smithi081.stdout:9/199: chown d2/d5/d36 18812245 1 2024-03-22T07:08:38.657 INFO:tasks.workunit.client.0.smithi081.stdout:9/200: creat d2/d2b/f43 x:0 0 0 2024-03-22T07:08:38.657 INFO:tasks.workunit.client.0.smithi081.stdout:9/201: readlink d2/l3c 0 2024-03-22T07:08:38.657 INFO:tasks.workunit.client.0.smithi081.stdout:9/202: chown d2/d2b/f43 47496945 1 2024-03-22T07:08:38.657 INFO:tasks.workunit.client.0.smithi081.stdout:9/203: readlink d2/d5/d16/l1a 0 2024-03-22T07:08:38.661 INFO:tasks.workunit.client.0.smithi081.stdout:4/312: read da/d18/d1b/d26/f43 [2454368,79517] 0 2024-03-22T07:08:38.662 INFO:tasks.workunit.client.0.smithi081.stdout:2/123: dwrite da/d11/f1a [0,4194304] 0 2024-03-22T07:08:38.662 INFO:tasks.workunit.client.0.smithi081.stdout:2/124: fdatasync da/f18 0 2024-03-22T07:08:38.663 INFO:tasks.workunit.client.0.smithi081.stdout:4/313: mkdir da/d18/d1b/d1e/d5f 0 2024-03-22T07:08:38.663 INFO:tasks.workunit.client.0.smithi081.stdout:4/314: chown da/d18/d1b/d34/l33 39476086 1 2024-03-22T07:08:38.663 INFO:tasks.workunit.client.0.smithi081.stdout:6/251: dwrite d3/da/de/f27 [0,4194304] 0 2024-03-22T07:08:38.665 INFO:tasks.workunit.client.0.smithi081.stdout:2/125: creat da/f22 x:0 0 0 2024-03-22T07:08:38.666 INFO:tasks.workunit.client.0.smithi081.stdout:2/126: unlink c7 0 2024-03-22T07:08:38.667 INFO:tasks.workunit.client.0.smithi081.stdout:7/363: read d9/f3c [6119501,34886] 0 2024-03-22T07:08:38.668 INFO:tasks.workunit.client.0.smithi081.stdout:4/315: dread f3 [0,4194304] 0 2024-03-22T07:08:38.670 INFO:tasks.workunit.client.0.smithi081.stdout:3/198: getdents db/d23 0 2024-03-22T07:08:38.670 INFO:tasks.workunit.client.0.smithi081.stdout:3/199: write db/f12 [2834599,21407] 0 2024-03-22T07:08:38.672 INFO:tasks.workunit.client.0.smithi081.stdout:7/364: dread d9/d10/d16/d1e/d27/f57 [0,4194304] 0 2024-03-22T07:08:38.673 INFO:tasks.workunit.client.0.smithi081.stdout:2/127: unlink da/d11/f15 0 2024-03-22T07:08:38.673 INFO:tasks.workunit.client.0.smithi081.stdout:2/128: read - da/d11/f1f zero size 2024-03-22T07:08:38.678 INFO:tasks.workunit.client.0.smithi081.stdout:3/200: symlink db/l3d 0 2024-03-22T07:08:38.680 INFO:tasks.workunit.client.0.smithi081.stdout:7/365: creat d9/d10/d53/f82 x:0 0 0 2024-03-22T07:08:38.681 INFO:tasks.workunit.client.0.smithi081.stdout:4/316: write da/d18/d1b/d1e/d37/f20 [2142349,104431] 0 2024-03-22T07:08:38.681 INFO:tasks.workunit.client.0.smithi081.stdout:6/252: dwrite d3/f9 [4194304,4194304] 0 2024-03-22T07:08:38.682 INFO:tasks.workunit.client.0.smithi081.stdout:7/366: mkdir d9/d3f/d52/d83 0 2024-03-22T07:08:38.683 INFO:tasks.workunit.client.0.smithi081.stdout:9/204: fsync d2/d2b/f43 0 2024-03-22T07:08:38.683 INFO:tasks.workunit.client.0.smithi081.stdout:9/205: write d2/f12 [2199523,37602] 0 2024-03-22T07:08:38.683 INFO:tasks.workunit.client.0.smithi081.stdout:4/317: mknod da/d18/d42/d4a/c60 0 2024-03-22T07:08:38.683 INFO:tasks.workunit.client.0.smithi081.stdout:4/318: chown da/d18/d1b 1 1 2024-03-22T07:08:38.690 INFO:tasks.workunit.client.0.smithi081.stdout:4/319: link da/df/c45 da/d3d/c61 0 2024-03-22T07:08:38.690 INFO:tasks.workunit.client.0.smithi081.stdout:4/320: write f0 [3379219,49467] 0 2024-03-22T07:08:38.690 INFO:tasks.workunit.client.0.smithi081.stdout:4/321: chown da 469618 1 2024-03-22T07:08:38.690 INFO:tasks.workunit.client.0.smithi081.stdout:4/322: chown da/d18/d1b/d34/f4c 7052272 1 2024-03-22T07:08:38.690 INFO:tasks.workunit.client.0.smithi081.stdout:2/129: truncate f6 2668922 0 2024-03-22T07:08:38.692 INFO:tasks.workunit.client.0.smithi081.stdout:7/367: rmdir d9/d10/d2a 39 2024-03-22T07:08:38.693 INFO:tasks.workunit.client.0.smithi081.stdout:7/368: read d9/d10/d16/f7a [597053,124740] 0 2024-03-22T07:08:38.695 INFO:tasks.workunit.client.0.smithi081.stdout:4/323: creat da/d18/d1b/d1e/f62 x:0 0 0 2024-03-22T07:08:38.697 INFO:tasks.workunit.client.0.smithi081.stdout:2/130: symlink da/l23 0 2024-03-22T07:08:38.697 INFO:tasks.workunit.client.0.smithi081.stdout:4/324: truncate f3 94454 0 2024-03-22T07:08:38.697 INFO:tasks.workunit.client.0.smithi081.stdout:4/325: chown da/df/d38/f46 9089 1 2024-03-22T07:08:38.700 INFO:tasks.workunit.client.0.smithi081.stdout:6/253: rmdir d3/da/de/df/d26 39 2024-03-22T07:08:38.703 INFO:tasks.workunit.client.0.smithi081.stdout:6/254: getdents d3/da 0 2024-03-22T07:08:38.703 INFO:tasks.workunit.client.0.smithi081.stdout:6/255: chown d3/da/de/df/d26/d3f/d4c 9350 1 2024-03-22T07:08:38.706 INFO:tasks.workunit.client.0.smithi081.stdout:9/206: dwrite d2/d5/d16/f2e [0,4194304] 0 2024-03-22T07:08:38.720 INFO:tasks.workunit.client.0.smithi081.stdout:6/256: dread d3/da/de/df/d26/f1c [0,4194304] 0 2024-03-22T07:08:38.720 INFO:tasks.workunit.client.0.smithi081.stdout:6/257: chown d3/da/de/df/f3b 2 1 2024-03-22T07:08:38.720 INFO:tasks.workunit.client.0.smithi081.stdout:6/258: truncate d3/da/f45 1045196 0 2024-03-22T07:08:38.724 INFO:tasks.workunit.client.0.smithi081.stdout:9/207: write d2/d10/f17 [2283179,30947] 0 2024-03-22T07:08:38.727 INFO:tasks.workunit.client.0.smithi081.stdout:9/208: dread d2/f8 [0,4194304] 0 2024-03-22T07:08:38.728 INFO:tasks.workunit.client.0.smithi081.stdout:9/209: creat d2/d5/d16/d37/f44 x:0 0 0 2024-03-22T07:08:38.729 INFO:tasks.workunit.client.0.smithi081.stdout:9/210: getdents d2/d10 0 2024-03-22T07:08:38.737 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:38 smithi149 ceph-mon[36068]: pgmap v242: 65 pgs: 65 active+clean; 363 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 22 MiB/s rd, 35 MiB/s wr, 535 op/s 2024-03-22T07:08:38.738 INFO:tasks.workunit.client.0.smithi081.stdout:6/259: write d3/da/de/f2e [7860003,99368] 0 2024-03-22T07:08:38.738 INFO:tasks.workunit.client.0.smithi081.stdout:6/260: write d3/f6 [615243,13213] 0 2024-03-22T07:08:38.741 INFO:tasks.workunit.client.0.smithi081.stdout:2/131: dwrite da/d11/f1f [0,4194304] 0 2024-03-22T07:08:38.743 INFO:tasks.workunit.client.0.smithi081.stdout:9/211: dread d2/d2b/f26 [0,4194304] 0 2024-03-22T07:08:38.744 INFO:tasks.workunit.client.0.smithi081.stdout:7/369: dwrite d9/d10/f2e [0,4194304] 0 2024-03-22T07:08:38.744 INFO:tasks.workunit.client.0.smithi081.stdout:4/326: dwrite da/f1d [4194304,4194304] 0 2024-03-22T07:08:38.744 INFO:tasks.workunit.client.0.smithi081.stdout:9/212: write d2/f1e [966160,61340] 0 2024-03-22T07:08:38.744 INFO:tasks.workunit.client.0.smithi081.stdout:9/213: write d2/d5/d16/f2f [491246,64356] 0 2024-03-22T07:08:38.744 INFO:tasks.workunit.client.0.smithi081.stdout:9/214: truncate d2/f12 2902782 0 2024-03-22T07:08:38.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/132: dread da/d11/f1b [0,4194304] 0 2024-03-22T07:08:38.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/133: stat da/f18 0 2024-03-22T07:08:38.750 INFO:tasks.workunit.client.0.smithi081.stdout:6/261: write d3/da/de/df/d26/f33 [723521,36819] 0 2024-03-22T07:08:38.756 INFO:tasks.workunit.client.0.smithi081.stdout:7/370: mkdir d9/d10/d16/d84 0 2024-03-22T07:08:38.758 INFO:tasks.workunit.client.0.smithi081.stdout:4/327: link da/d18/d1b/d26/d3e/c56 da/d18/d1b/d47/c63 0 2024-03-22T07:08:38.758 INFO:tasks.workunit.client.0.smithi081.stdout:4/328: fdatasync da/d18/d42/d4f/f5b 0 2024-03-22T07:08:38.760 INFO:tasks.workunit.client.0.smithi081.stdout:9/215: creat d2/d10/d30/f45 x:0 0 0 2024-03-22T07:08:38.763 INFO:tasks.workunit.client.0.smithi081.stdout:6/262: rename d3/f22 to d3/da/de/df/d26/d3f/d4c/f4d 0 2024-03-22T07:08:38.769 INFO:tasks.workunit.client.0.smithi081.stdout:2/134: dwrite da/d11/f1f [0,4194304] 0 2024-03-22T07:08:38.771 INFO:tasks.workunit.client.0.smithi081.stdout:7/371: mknod d9/d10/d16/d1e/d80/c85 0 2024-03-22T07:08:38.774 INFO:tasks.workunit.client.0.smithi081.stdout:6/263: unlink d3/da/de/df/d26/c21 0 2024-03-22T07:08:38.779 INFO:tasks.workunit.client.0.smithi081.stdout:7/372: mknod d9/d10/d2a/c86 0 2024-03-22T07:08:38.779 INFO:tasks.workunit.client.0.smithi081.stdout:7/373: read d9/f11 [150892,69531] 0 2024-03-22T07:08:38.781 INFO:tasks.workunit.client.0.smithi081.stdout:7/374: getdents d9/d10/d4b 0 2024-03-22T07:08:38.783 INFO:tasks.workunit.client.0.smithi081.stdout:7/375: link d9/d10/d16/c2f d9/d10/d2a/d73/c87 0 2024-03-22T07:08:38.783 INFO:tasks.workunit.client.0.smithi081.stdout:7/376: chown d9/d3f/f79 2 1 2024-03-22T07:08:38.785 INFO:tasks.workunit.client.0.smithi081.stdout:7/377: rename d9/d10/l4c to d9/d10/d2a/d73/l88 0 2024-03-22T07:08:38.785 INFO:tasks.workunit.client.0.smithi081.stdout:4/329: dwrite f9 [0,4194304] 0 2024-03-22T07:08:38.785 INFO:tasks.workunit.client.0.smithi081.stdout:9/216: write d2/f40 [3197673,128994] 0 2024-03-22T07:08:38.785 INFO:tasks.workunit.client.0.smithi081.stdout:9/217: fdatasync d2/f11 0 2024-03-22T07:08:38.786 INFO:tasks.workunit.client.0.smithi081.stdout:9/218: unlink d2/d5/f31 0 2024-03-22T07:08:38.787 INFO:tasks.workunit.client.0.smithi081.stdout:9/219: stat d2/d5/cb 0 2024-03-22T07:08:38.797 INFO:tasks.workunit.client.0.smithi081.stdout:8/176: link d2/f24 d2/d3/d19/d1e/f41 0 2024-03-22T07:08:38.803 INFO:tasks.workunit.client.0.smithi081.stdout:1/173: sync 2024-03-22T07:08:38.803 INFO:tasks.workunit.client.0.smithi081.stdout:1/174: write d6/d18/f27 [1825042,57928] 0 2024-03-22T07:08:38.803 INFO:tasks.workunit.client.0.smithi081.stdout:5/293: sync 2024-03-22T07:08:38.807 INFO:tasks.workunit.client.0.smithi081.stdout:1/175: creat d6/d2e/d30/f43 x:0 0 0 2024-03-22T07:08:38.807 INFO:tasks.workunit.client.0.smithi081.stdout:5/294: rmdir d4/d10/d20/d1f/d2f 39 2024-03-22T07:08:38.808 INFO:tasks.workunit.client.0.smithi081.stdout:8/177: symlink d2/d3/d19/d1e/d13/d22/l42 0 2024-03-22T07:08:38.811 INFO:tasks.workunit.client.0.smithi081.stdout:8/178: creat d2/d3/d19/d1e/f43 x:0 0 0 2024-03-22T07:08:38.819 INFO:tasks.workunit.client.0.smithi081.stdout:1/176: write d6/d2e/d30/f3d [7408861,40692] 0 2024-03-22T07:08:38.896 INFO:tasks.workunit.client.0.smithi081.stdout:2/135: dwrite f5 [0,4194304] 0 2024-03-22T07:08:38.896 INFO:tasks.workunit.client.0.smithi081.stdout:2/136: mknod da/d11/d21/c24 0 2024-03-22T07:08:38.896 INFO:tasks.workunit.client.0.smithi081.stdout:7/378: dwrite d9/d14/f24 [0,4194304] 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:7/379: fdatasync f1 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:7/380: mkdir d9/d10/d53/d81/d59/d89 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:7/381: write d9/d3f/f65 [878151,105639] 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:6/264: dwrite d3/da/de/f28 [0,4194304] 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:6/265: dread - d3/da/de/f4b zero size 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:6/266: stat d3/da/de/f28 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:7/382: symlink d9/d10/d16/d84/l8a 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:4/330: dwrite da/d18/f31 [0,4194304] 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:6/267: unlink d3/da/f17 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:5/295: dwrite d4/df/f46 [0,4194304] 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:6/268: symlink d3/da/de/d31/l4e 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:9/220: dwrite d2/d10/f14 [4194304,4194304] 0 2024-03-22T07:08:38.897 INFO:tasks.workunit.client.0.smithi081.stdout:5/296: link d4/d10/d20/d1f/d2f/d45/c65 d4/d10/d20/d1f/d2a/c66 0 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:5/297: read - d4/df/d22/d49/f60 zero size 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:7/383: rmdir d9/d3f 39 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:9/221: creat d2/d5/d16/f46 x:0 0 0 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:7/384: link d9/d10/d16/d1e/d27/f5a d9/d10/d2a/d73/f8b 0 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:2/137: dwrite da/f22 [0,4194304] 0 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:1/177: dwrite d6/d2e/f3c [0,4194304] 0 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:1/178: creat d6/d2e/f44 x:0 0 0 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:5/298: dwrite d4/d10/d20/d1f/d2f/d45/f4c [4194304,4194304] 0 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:6/269: rmdir d3/da/de/df/d26 39 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:4/331: rename da/fe to da/d18/d1b/d34/f64 0 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:4/332: dread - da/d18/d42/d4f/f5b zero size 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:4/333: creat da/df/f65 x:0 0 0 2024-03-22T07:08:38.898 INFO:tasks.workunit.client.0.smithi081.stdout:4/334: write da/d18/d1b/d1e/d37/d1c/f51 [447447,46049] 0 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:4/335: readlink da/d18/l22 0 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:5/299: rename d4/df/d22/d49/c59 to d4/c67 0 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:6/270: rename d3/da/de/df/d26 to d3/da/de/df/d26/d4f 22 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:2/138: read da/f22 [2313681,45722] 0 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:7/385: dwrite d9/d10/d2a/f47 [0,4194304] 0 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:4/336: rename da/d18/d1b/f24 to da/d18/d1b/d26/f66 0 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:4/337: write da/d18/d1b/d34/f4c [4268368,74789] 0 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:4/338: symlink da/d57/l67 0 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:8/179: rmdir d2/d3/d19/d1e/d13/d22 39 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:8/180: dread - d2/d3/d19/d1e/d13/d22/d2f/f40 zero size 2024-03-22T07:08:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:8/181: creat d2/d3/d19/d1e/d13/d22/d38/f44 x:0 0 0 2024-03-22T07:08:38.900 INFO:tasks.workunit.client.0.smithi081.stdout:8/182: stat d2/d3 0 2024-03-22T07:08:38.900 INFO:tasks.workunit.client.0.smithi081.stdout:8/183: readlink d2/l1d 0 2024-03-22T07:08:38.900 INFO:tasks.workunit.client.0.smithi081.stdout:8/184: write d2/d3/f1b [1504833,49173] 0 2024-03-22T07:08:38.902 INFO:tasks.workunit.client.0.smithi081.stdout:7/386: dwrite d9/f7c [0,4194304] 0 2024-03-22T07:08:38.904 INFO:tasks.workunit.client.0.smithi081.stdout:8/185: unlink d2/d3/d19/d1e/d13/f2d 0 2024-03-22T07:08:38.904 INFO:tasks.workunit.client.0.smithi081.stdout:8/186: dread - d2/d3/d19/d1e/d13/d22/d38/f44 zero size 2024-03-22T07:08:38.918 INFO:tasks.workunit.client.0.smithi081.stdout:4/339: dwrite da/d18/d1b/d34/f4c [4194304,4194304] 0 2024-03-22T07:08:38.918 INFO:tasks.workunit.client.0.smithi081.stdout:4/340: creat da/df/d38/f68 x:0 0 0 2024-03-22T07:08:38.918 INFO:tasks.workunit.client.0.smithi081.stdout:4/341: dread - da/d18/d1b/d26/d3e/f55 zero size 2024-03-22T07:08:38.918 INFO:tasks.workunit.client.0.smithi081.stdout:7/387: mkdir d9/d10/d16/d1e/d27/d6d/d8c 0 2024-03-22T07:08:38.922 INFO:tasks.workunit.client.0.smithi081.stdout:2/139: dwrite da/d11/f1a [0,4194304] 0 2024-03-22T07:08:38.922 INFO:tasks.workunit.client.0.smithi081.stdout:2/140: creat da/f25 x:0 0 0 2024-03-22T07:08:38.922 INFO:tasks.workunit.client.0.smithi081.stdout:2/141: chown da/c19 2139103401 1 2024-03-22T07:08:38.925 INFO:tasks.workunit.client.0.smithi081.stdout:8/187: dwrite d2/d3/d28/f30 [0,4194304] 0 2024-03-22T07:08:38.925 INFO:tasks.workunit.client.0.smithi081.stdout:8/188: write d2/d3/d19/d1e/f3c [940095,2977] 0 2024-03-22T07:08:38.925 INFO:tasks.workunit.client.0.smithi081.stdout:0/268: sync 2024-03-22T07:08:38.928 INFO:tasks.workunit.client.0.smithi081.stdout:8/189: mknod d2/d3/d19/d1e/d13/d22/c45 0 2024-03-22T07:08:38.928 INFO:tasks.workunit.client.0.smithi081.stdout:8/190: chown d2/d3/d19/c1a 391078766 1 2024-03-22T07:08:38.937 INFO:tasks.workunit.client.0.smithi081.stdout:8/191: write d2/d3/d28/f30 [643438,38326] 0 2024-03-22T07:08:38.937 INFO:tasks.workunit.client.0.smithi081.stdout:8/192: truncate d2/d3/d19/d1e/f43 558204 0 2024-03-22T07:08:38.938 INFO:tasks.workunit.client.0.smithi081.stdout:3/201: sync 2024-03-22T07:08:38.938 INFO:tasks.workunit.client.0.smithi081.stdout:6/271: rmdir d3 39 2024-03-22T07:08:38.938 INFO:tasks.workunit.client.0.smithi081.stdout:4/342: mknod da/d3d/c69 0 2024-03-22T07:08:38.938 INFO:tasks.workunit.client.0.smithi081.stdout:3/202: creat db/d2e/f3e x:0 0 0 2024-03-22T07:08:38.944 INFO:tasks.workunit.client.0.smithi081.stdout:7/388: dwrite f4 [0,4194304] 0 2024-03-22T07:08:38.950 INFO:tasks.workunit.client.0.smithi081.stdout:2/142: rmdir da/d11/d21 39 2024-03-22T07:08:38.951 INFO:tasks.workunit.client.0.smithi081.stdout:0/269: mknod d3/d32/d35/c51 0 2024-03-22T07:08:38.951 INFO:tasks.workunit.client.0.smithi081.stdout:8/193: creat d2/d3/d28/d2b/f46 x:0 0 0 2024-03-22T07:08:38.951 INFO:tasks.workunit.client.0.smithi081.stdout:8/194: write d2/d3/d19/d1e/d13/d22/d38/f44 [673252,42256] 0 2024-03-22T07:08:38.951 INFO:tasks.workunit.client.0.smithi081.stdout:2/143: creat da/d11/f26 x:0 0 0 2024-03-22T07:08:38.951 INFO:tasks.workunit.client.0.smithi081.stdout:0/270: write d3/dc/d2b/f48 [255426,76463] 0 2024-03-22T07:08:38.951 INFO:tasks.workunit.client.0.smithi081.stdout:3/203: dread db/de/f26 [0,4194304] 0 2024-03-22T07:08:38.962 INFO:tasks.workunit.client.0.smithi081.stdout:4/343: truncate da/d18/d1b/d34/f4c 3027334 0 2024-03-22T07:08:38.963 INFO:tasks.workunit.client.0.smithi081.stdout:2/144: truncate f5 577726 0 2024-03-22T07:08:38.963 INFO:tasks.workunit.client.0.smithi081.stdout:0/271: mkdir d3/dc/d40/d52 0 2024-03-22T07:08:38.963 INFO:tasks.workunit.client.0.smithi081.stdout:7/389: creat d9/d10/d16/d1e/d27/d70/f8d x:0 0 0 2024-03-22T07:08:38.970 INFO:tasks.workunit.client.0.smithi081.stdout:3/204: dwrite db/fd [0,4194304] 0 2024-03-22T07:08:38.981 INFO:tasks.workunit.client.0.smithi081.stdout:3/205: dread db/de/f17 [0,4194304] 0 2024-03-22T07:08:38.981 INFO:tasks.workunit.client.0.smithi081.stdout:3/206: getdents db/d2e/d3a 0 2024-03-22T07:08:38.989 INFO:tasks.workunit.client.0.smithi081.stdout:2/145: rename da/f18 to da/db/f27 0 2024-03-22T07:08:38.989 INFO:tasks.workunit.client.0.smithi081.stdout:0/272: symlink d3/dc/d33/l53 0 2024-03-22T07:08:38.989 INFO:tasks.workunit.client.0.smithi081.stdout:2/146: creat da/f28 x:0 0 0 2024-03-22T07:08:38.997 INFO:tasks.workunit.client.0.smithi081.stdout:4/344: dread da/d18/d1b/d34/f4c [0,4194304] 0 2024-03-22T07:08:38.997 INFO:tasks.workunit.client.0.smithi081.stdout:4/345: dread - da/d3d/f5e zero size 2024-03-22T07:08:39.001 INFO:tasks.workunit.client.0.smithi081.stdout:0/273: creat d3/d4e/f54 x:0 0 0 2024-03-22T07:08:39.001 INFO:tasks.workunit.client.0.smithi081.stdout:2/147: creat da/db/f29 x:0 0 0 2024-03-22T07:08:39.001 INFO:tasks.workunit.client.0.smithi081.stdout:2/148: dread - da/d11/f26 zero size 2024-03-22T07:08:39.001 INFO:tasks.workunit.client.0.smithi081.stdout:2/149: dread - da/db/f29 zero size 2024-03-22T07:08:39.005 INFO:tasks.workunit.client.0.smithi081.stdout:2/150: rmdir da/d11/d21 39 2024-03-22T07:08:39.010 INFO:tasks.workunit.client.0.smithi081.stdout:0/274: read d3/dc/f36 [7897491,56639] 0 2024-03-22T07:08:39.010 INFO:tasks.workunit.client.0.smithi081.stdout:2/151: rename da/d11/l1c to da/d11/d21/l2a 0 2024-03-22T07:08:39.011 INFO:tasks.workunit.client.0.smithi081.stdout:2/152: write da/db/f29 [634933,17854] 0 2024-03-22T07:08:39.012 INFO:tasks.workunit.client.0.smithi081.stdout:0/275: rmdir d3/dc/d40/d52 0 2024-03-22T07:08:39.012 INFO:tasks.workunit.client.0.smithi081.stdout:0/276: chown d3/dc/d2b/d39/f3c 7039 1 2024-03-22T07:08:39.013 INFO:tasks.workunit.client.0.smithi081.stdout:2/153: mknod da/c2b 0 2024-03-22T07:08:39.014 INFO:tasks.workunit.client.0.smithi081.stdout:2/154: write da/d11/f1f [4964719,14921] 0 2024-03-22T07:08:39.015 INFO:tasks.workunit.client.0.smithi081.stdout:0/277: link d3/d32/l4b d3/dc/d2b/d39/l55 0 2024-03-22T07:08:39.015 INFO:tasks.workunit.client.0.smithi081.stdout:2/155: dread da/d11/f1b [0,4194304] 0 2024-03-22T07:08:39.016 INFO:tasks.workunit.client.0.smithi081.stdout:2/156: dread - da/f28 zero size 2024-03-22T07:08:39.016 INFO:tasks.workunit.client.0.smithi081.stdout:0/278: mknod d3/d4e/c56 0 2024-03-22T07:08:39.017 INFO:tasks.workunit.client.0.smithi081.stdout:4/346: dwrite da/d18/d1b/d1e/f62 [0,4194304] 0 2024-03-22T07:08:39.018 INFO:tasks.workunit.client.0.smithi081.stdout:2/157: mkdir da/d11/d21/d2c 0 2024-03-22T07:08:39.018 INFO:tasks.workunit.client.0.smithi081.stdout:2/158: fsync da/d11/f26 0 2024-03-22T07:08:39.018 INFO:tasks.workunit.client.0.smithi081.stdout:2/159: readlink da/db/l10 0 2024-03-22T07:08:39.018 INFO:tasks.workunit.client.0.smithi081.stdout:0/279: readlink d3/l27 0 2024-03-22T07:08:39.021 INFO:tasks.workunit.client.0.smithi081.stdout:2/160: creat da/d11/d21/f2d x:0 0 0 2024-03-22T07:08:39.022 INFO:tasks.workunit.client.0.smithi081.stdout:2/161: readlink da/l23 0 2024-03-22T07:08:39.022 INFO:tasks.workunit.client.0.smithi081.stdout:0/280: mknod d3/dc/c57 0 2024-03-22T07:08:39.022 INFO:tasks.workunit.client.0.smithi081.stdout:0/281: write d3/d32/d35/f38 [1435330,85] 0 2024-03-22T07:08:39.022 INFO:tasks.workunit.client.0.smithi081.stdout:0/282: chown d3/dc/l16 196859159 1 2024-03-22T07:08:39.023 INFO:tasks.workunit.client.0.smithi081.stdout:2/162: truncate da/d11/f1a 417645 0 2024-03-22T07:08:39.024 INFO:tasks.workunit.client.0.smithi081.stdout:0/283: truncate d3/dc/f3f 159215 0 2024-03-22T07:08:39.025 INFO:tasks.workunit.client.0.smithi081.stdout:2/163: creat da/d11/d21/d2c/f2e x:0 0 0 2024-03-22T07:08:39.027 INFO:tasks.workunit.client.0.smithi081.stdout:0/284: creat d3/dc/f58 x:0 0 0 2024-03-22T07:08:39.028 INFO:tasks.workunit.client.0.smithi081.stdout:2/164: symlink da/l2f 0 2024-03-22T07:08:39.029 INFO:tasks.workunit.client.0.smithi081.stdout:4/347: rmdir da/d18/d1b 39 2024-03-22T07:08:39.029 INFO:tasks.workunit.client.0.smithi081.stdout:4/348: chown da/fb 168941 1 2024-03-22T07:08:39.030 INFO:tasks.workunit.client.0.smithi081.stdout:0/285: creat d3/d32/f59 x:0 0 0 2024-03-22T07:08:39.039 INFO:tasks.workunit.client.0.smithi081.stdout:4/349: dread da/d18/d1b/d1e/d37/f25 [0,4194304] 0 2024-03-22T07:08:39.039 INFO:tasks.workunit.client.0.smithi081.stdout:4/350: creat da/d18/d42/d4f/f6a x:0 0 0 2024-03-22T07:08:39.039 INFO:tasks.workunit.client.0.smithi081.stdout:4/351: fdatasync da/d18/d1b/d34/f44 0 2024-03-22T07:08:39.039 INFO:tasks.workunit.client.0.smithi081.stdout:4/352: write da/d18/d1b/f40 [932841,110607] 0 2024-03-22T07:08:39.042 INFO:tasks.workunit.client.0.smithi081.stdout:8/195: link f1 d2/d3/d19/d1e/d13/f47 0 2024-03-22T07:08:39.042 INFO:tasks.workunit.client.0.smithi081.stdout:8/196: fsync f1 0 2024-03-22T07:08:39.042 INFO:tasks.workunit.client.0.smithi081.stdout:8/197: chown d2/d3/d19/d1e/d13 3320 1 2024-03-22T07:08:39.042 INFO:tasks.workunit.client.0.smithi081.stdout:8/198: fsync d2/d3/d28/f30 0 2024-03-22T07:08:39.044 INFO:tasks.workunit.client.0.smithi081.stdout:8/199: creat d2/d3/d19/f48 x:0 0 0 2024-03-22T07:08:39.044 INFO:tasks.workunit.client.0.smithi081.stdout:8/200: chown d2/d3/f1b 174 1 2024-03-22T07:08:39.044 INFO:tasks.workunit.client.0.smithi081.stdout:8/201: dread - d2/d3/d23/f3b zero size 2024-03-22T07:08:39.047 INFO:tasks.workunit.client.0.smithi081.stdout:8/202: creat d2/d3/d28/f49 x:0 0 0 2024-03-22T07:08:39.049 INFO:tasks.workunit.client.0.smithi081.stdout:8/203: getdents d2 0 2024-03-22T07:08:39.049 INFO:tasks.workunit.client.0.smithi081.stdout:8/204: creat d2/d3/d28/f4a x:0 0 0 2024-03-22T07:08:39.049 INFO:tasks.workunit.client.0.smithi081.stdout:8/205: dread - d2/d3/d19/f48 zero size 2024-03-22T07:08:39.049 INFO:tasks.workunit.client.0.smithi081.stdout:8/206: fsync d2/d3/d19/d1e/d13/d22/f3a 0 2024-03-22T07:08:39.055 INFO:tasks.workunit.client.0.smithi081.stdout:4/353: dread da/d18/d42/d4f/f58 [0,4194304] 0 2024-03-22T07:08:39.057 INFO:tasks.workunit.client.0.smithi081.stdout:0/286: dwrite d3/dc/d2b/f3a [0,4194304] 0 2024-03-22T07:08:39.058 INFO:tasks.workunit.client.0.smithi081.stdout:4/354: link da/d18/d1b/d1e/d37/l2f da/d18/d1b/l6b 0 2024-03-22T07:08:39.061 INFO:tasks.workunit.client.0.smithi081.stdout:4/355: creat da/d18/d1b/d1e/d37/f6c x:0 0 0 2024-03-22T07:08:39.062 INFO:tasks.workunit.client.0.smithi081.stdout:4/356: unlink da/l23 0 2024-03-22T07:08:39.072 INFO:tasks.workunit.client.0.smithi081.stdout:2/165: dwrite da/d11/f1a [0,4194304] 0 2024-03-22T07:08:39.074 INFO:tasks.workunit.client.0.smithi081.stdout:2/166: mknod da/c30 0 2024-03-22T07:08:39.076 INFO:tasks.workunit.client.0.smithi081.stdout:2/167: rename da/c1e to da/d11/d21/d2c/c31 0 2024-03-22T07:08:39.086 INFO:tasks.workunit.client.0.smithi081.stdout:4/357: dwrite da/d18/d1b/d34/f44 [0,4194304] 0 2024-03-22T07:08:39.086 INFO:tasks.workunit.client.0.smithi081.stdout:0/287: dwrite d3/dc/d2b/f31 [4194304,4194304] 0 2024-03-22T07:08:39.088 INFO:tasks.workunit.client.0.smithi081.stdout:4/358: unlink da/d18/d1b/d1e/d37/f25 0 2024-03-22T07:08:39.092 INFO:tasks.workunit.client.0.smithi081.stdout:2/168: truncate da/d11/f1a 3331026 0 2024-03-22T07:08:39.095 INFO:tasks.workunit.client.0.smithi081.stdout:0/288: link d3/dc/d40/f4a d3/dc/d33/f5a 0 2024-03-22T07:08:39.095 INFO:tasks.workunit.client.0.smithi081.stdout:4/359: link da/d18/d1b/d1e/d37/l3a da/d18/d1b/d26/d3e/l6d 0 2024-03-22T07:08:39.097 INFO:tasks.workunit.client.0.smithi081.stdout:0/289: rmdir d3/d4e 39 2024-03-22T07:08:39.097 INFO:tasks.workunit.client.0.smithi081.stdout:0/290: stat d3/l27 0 2024-03-22T07:08:39.133 INFO:tasks.workunit.client.0.smithi081.stdout:2/169: dwrite da/db/f27 [0,4194304] 0 2024-03-22T07:08:39.133 INFO:tasks.workunit.client.0.smithi081.stdout:0/291: dwrite d3/f3e [0,4194304] 0 2024-03-22T07:08:39.133 INFO:tasks.workunit.client.0.smithi081.stdout:4/360: dwrite da/d18/d1b/d34/f64 [0,4194304] 0 2024-03-22T07:08:39.133 INFO:tasks.workunit.client.0.smithi081.stdout:4/361: readlink da/d18/d1b/d26/d3e/l6d 0 2024-03-22T07:08:39.135 INFO:tasks.workunit.client.0.smithi081.stdout:0/292: truncate d3/f2d 3022965 0 2024-03-22T07:08:39.137 INFO:tasks.workunit.client.0.smithi081.stdout:4/362: mkdir da/d18/d6e 0 2024-03-22T07:08:39.138 INFO:tasks.workunit.client.0.smithi081.stdout:0/293: link d3/dc/d40/f4a d3/dc/d33/f5b 0 2024-03-22T07:08:39.150 INFO:tasks.workunit.client.0.smithi081.stdout:0/294: read d3/d32/f4f [1554168,52658] 0 2024-03-22T07:08:39.151 INFO:tasks.workunit.client.0.smithi081.stdout:2/170: dwrite da/f25 [0,4194304] 0 2024-03-22T07:08:39.152 INFO:tasks.workunit.client.0.smithi081.stdout:0/295: dread d3/dc/d2b/f2f [0,4194304] 0 2024-03-22T07:08:39.153 INFO:tasks.workunit.client.0.smithi081.stdout:0/296: read d3/dc/f15 [3852223,67385] 0 2024-03-22T07:08:39.153 INFO:tasks.workunit.client.0.smithi081.stdout:0/297: dread - d3/f24 zero size 2024-03-22T07:08:39.153 INFO:tasks.workunit.client.0.smithi081.stdout:2/171: mkdir da/d11/d32 0 2024-03-22T07:08:39.153 INFO:tasks.workunit.client.0.smithi081.stdout:2/172: creat da/d11/d21/f33 x:0 0 0 2024-03-22T07:08:39.205 INFO:tasks.workunit.client.0.smithi081.stdout:9/222: sync 2024-03-22T07:08:39.206 INFO:tasks.workunit.client.0.smithi081.stdout:1/179: sync 2024-03-22T07:08:39.206 INFO:tasks.workunit.client.0.smithi081.stdout:9/223: chown d2/d5/d36 22783 1 2024-03-22T07:08:39.206 INFO:tasks.workunit.client.0.smithi081.stdout:9/224: getdents d2/d5/d16 0 2024-03-22T07:08:39.208 INFO:tasks.workunit.client.0.smithi081.stdout:9/225: truncate d2/f11 171630 0 2024-03-22T07:08:39.208 INFO:tasks.workunit.client.0.smithi081.stdout:4/363: dread da/d18/d1b/d1e/f62 [0,4194304] 0 2024-03-22T07:08:39.208 INFO:tasks.workunit.client.0.smithi081.stdout:4/364: chown da/d18/d1b/d1e/l5c 105328176 1 2024-03-22T07:08:39.208 INFO:tasks.workunit.client.0.smithi081.stdout:4/365: dread - da/d18/d1b/d1e/d37/f6c zero size 2024-03-22T07:08:39.209 INFO:tasks.workunit.client.0.smithi081.stdout:9/226: mknod d2/d2b/c47 0 2024-03-22T07:08:39.209 INFO:tasks.workunit.client.0.smithi081.stdout:9/227: chown d2/fe 5 1 2024-03-22T07:08:39.213 INFO:tasks.workunit.client.0.smithi081.stdout:9/228: unlink d2/lc 0 2024-03-22T07:08:39.236 INFO:tasks.workunit.client.0.smithi081.stdout:9/229: read d2/d10/f29 [3384285,65941] 0 2024-03-22T07:08:39.236 INFO:tasks.workunit.client.0.smithi081.stdout:9/230: fdatasync d2/f12 0 2024-03-22T07:08:39.236 INFO:tasks.workunit.client.0.smithi081.stdout:9/231: symlink d2/d10/d30/l48 0 2024-03-22T07:08:39.239 INFO:tasks.workunit.client.0.smithi081.stdout:4/366: dwrite da/f1d [0,4194304] 0 2024-03-22T07:08:39.239 INFO:tasks.workunit.client.0.smithi081.stdout:4/367: creat da/d18/d1b/d26/d3e/f6f x:0 0 0 2024-03-22T07:08:39.240 INFO:tasks.workunit.client.0.smithi081.stdout:4/368: write da/d18/d1b/d26/d3e/f6f [601343,115317] 0 2024-03-22T07:08:39.240 INFO:tasks.workunit.client.0.smithi081.stdout:4/369: symlink da/d18/d1b/d47/l70 0 2024-03-22T07:08:39.291 INFO:tasks.workunit.client.0.smithi081.stdout:2/173: truncate da/f25 1390672 0 2024-03-22T07:08:39.292 INFO:tasks.workunit.client.0.smithi081.stdout:2/174: mknod da/d11/d32/c34 0 2024-03-22T07:08:39.294 INFO:tasks.workunit.client.0.smithi081.stdout:2/175: unlink da/d11/f1a 0 2024-03-22T07:08:39.294 INFO:tasks.workunit.client.0.smithi081.stdout:2/176: creat da/db/f35 x:0 0 0 2024-03-22T07:08:39.294 INFO:tasks.workunit.client.0.smithi081.stdout:2/177: dread - da/db/f35 zero size 2024-03-22T07:08:39.295 INFO:tasks.workunit.client.0.smithi081.stdout:2/178: link da/db/f35 da/d11/d21/d2c/f36 0 2024-03-22T07:08:39.363 INFO:tasks.workunit.client.0.smithi081.stdout:0/298: dwrite d3/dc/d2b/d39/f3c [0,4194304] 0 2024-03-22T07:08:39.363 INFO:tasks.workunit.client.0.smithi081.stdout:0/299: chown d3/dc/d2b/d39 120 1 2024-03-22T07:08:39.364 INFO:tasks.workunit.client.0.smithi081.stdout:0/300: truncate d3/f13 3724376 0 2024-03-22T07:08:39.366 INFO:tasks.workunit.client.0.smithi081.stdout:0/301: link d3/dc/d40/c4c d3/d4e/c5c 0 2024-03-22T07:08:39.366 INFO:tasks.workunit.client.0.smithi081.stdout:0/302: chown d3/dc/d2b/c2c 524 1 2024-03-22T07:08:39.366 INFO:tasks.workunit.client.0.smithi081.stdout:0/303: mknod d3/d4e/c5d 0 2024-03-22T07:08:39.367 INFO:tasks.workunit.client.0.smithi081.stdout:0/304: write d3/dc/f3f [62998,83263] 0 2024-03-22T07:08:39.368 INFO:tasks.workunit.client.0.smithi081.stdout:1/180: dwrite d6/d18/d1f/d24/f3f [0,4194304] 0 2024-03-22T07:08:39.368 INFO:tasks.workunit.client.0.smithi081.stdout:1/181: fdatasync d6/d18/f21 0 2024-03-22T07:08:39.368 INFO:tasks.workunit.client.0.smithi081.stdout:0/305: creat d3/dc/d2b/d4d/f5e x:0 0 0 2024-03-22T07:08:39.372 INFO:tasks.workunit.client.0.smithi081.stdout:4/370: rmdir da/d18 39 2024-03-22T07:08:39.373 INFO:tasks.workunit.client.0.smithi081.stdout:4/371: creat da/d18/d1b/d1e/f71 x:0 0 0 2024-03-22T07:08:39.375 INFO:tasks.workunit.client.0.smithi081.stdout:1/182: rmdir d6/d18/d1f/d32 39 2024-03-22T07:08:39.376 INFO:tasks.workunit.client.0.smithi081.stdout:2/179: getdents da/d11 0 2024-03-22T07:08:39.376 INFO:tasks.workunit.client.0.smithi081.stdout:2/180: dread - da/d11/d21/f2d zero size 2024-03-22T07:08:39.377 INFO:tasks.workunit.client.0.smithi081.stdout:1/183: mknod d6/c45 0 2024-03-22T07:08:39.379 INFO:tasks.workunit.client.0.smithi081.stdout:2/181: rmdir da/db 39 2024-03-22T07:08:39.380 INFO:tasks.workunit.client.0.smithi081.stdout:0/306: write d3/f22 [860184,35865] 0 2024-03-22T07:08:39.381 INFO:tasks.workunit.client.0.smithi081.stdout:1/184: creat d6/d18/d41/f46 x:0 0 0 2024-03-22T07:08:39.385 INFO:tasks.workunit.client.0.smithi081.stdout:1/185: getdents d6/d18/d1f 0 2024-03-22T07:08:39.388 INFO:tasks.workunit.client.0.smithi081.stdout:1/186: rmdir d6/d18/d1f/d32/d40 0 2024-03-22T07:08:39.389 INFO:tasks.workunit.client.0.smithi081.stdout:1/187: write d6/fd [2963269,27449] 0 2024-03-22T07:08:39.391 INFO:tasks.workunit.client.0.smithi081.stdout:2/182: dread da/f25 [0,4194304] 0 2024-03-22T07:08:39.402 INFO:tasks.workunit.client.0.smithi081.stdout:0/307: dwrite d3/f24 [0,4194304] 0 2024-03-22T07:08:39.404 INFO:tasks.workunit.client.0.smithi081.stdout:0/308: unlink d3/dc/d40/f42 0 2024-03-22T07:08:39.405 INFO:tasks.workunit.client.0.smithi081.stdout:0/309: truncate d3/d32/f4f 4870688 0 2024-03-22T07:08:39.405 INFO:tasks.workunit.client.0.smithi081.stdout:0/310: creat d3/d32/f5f x:0 0 0 2024-03-22T07:08:39.406 INFO:tasks.workunit.client.0.smithi081.stdout:0/311: chown d3/c3b 2897497 1 2024-03-22T07:08:39.407 INFO:tasks.workunit.client.0.smithi081.stdout:0/312: symlink d3/dc/d2b/l60 0 2024-03-22T07:08:39.410 INFO:tasks.workunit.client.0.smithi081.stdout:5/300: sync 2024-03-22T07:08:39.412 INFO:tasks.workunit.client.0.smithi081.stdout:5/301: link d4/d10/d20/c5d d4/d10/d20/d1f/d2f/d45/c68 0 2024-03-22T07:08:39.414 INFO:tasks.workunit.client.0.smithi081.stdout:5/302: symlink d4/d9/l69 0 2024-03-22T07:08:39.416 INFO:tasks.workunit.client.0.smithi081.stdout:5/303: dread d4/d9/f15 [0,4194304] 0 2024-03-22T07:08:39.416 INFO:tasks.workunit.client.0.smithi081.stdout:5/304: read - d4/df/d22/f58 zero size 2024-03-22T07:08:39.424 INFO:tasks.workunit.client.0.smithi081.stdout:0/313: dwrite d3/dc/f36 [0,4194304] 0 2024-03-22T07:08:39.426 INFO:tasks.workunit.client.0.smithi081.stdout:0/314: symlink d3/d4e/l61 0 2024-03-22T07:08:39.427 INFO:tasks.workunit.client.0.smithi081.stdout:5/305: dread d4/fe [0,4194304] 0 2024-03-22T07:08:39.427 INFO:tasks.workunit.client.0.smithi081.stdout:0/315: mkdir d3/dc/d33/d62 0 2024-03-22T07:08:39.429 INFO:tasks.workunit.client.0.smithi081.stdout:5/306: symlink d4/d10/d20/d1f/l6a 0 2024-03-22T07:08:39.430 INFO:tasks.workunit.client.0.smithi081.stdout:5/307: creat d4/d10/d20/d1f/d2f/f6b x:0 0 0 2024-03-22T07:08:39.431 INFO:tasks.workunit.client.0.smithi081.stdout:5/308: rmdir d4/d10/d20/d25/d2e 39 2024-03-22T07:08:39.432 INFO:tasks.workunit.client.0.smithi081.stdout:5/309: truncate d4/d10/d20/d25/d2e/f31 4826357 0 2024-03-22T07:08:39.433 INFO:tasks.workunit.client.0.smithi081.stdout:5/310: truncate d4/d9/f15 361054 0 2024-03-22T07:08:39.442 INFO:tasks.workunit.client.0.smithi081.stdout:0/316: dwrite d3/dc/f58 [0,4194304] 0 2024-03-22T07:08:39.445 INFO:tasks.workunit.client.0.smithi081.stdout:0/317: creat d3/dc/d33/d62/f63 x:0 0 0 2024-03-22T07:08:39.455 INFO:tasks.workunit.client.0.smithi081.stdout:0/318: dread d3/f13 [0,4194304] 0 2024-03-22T07:08:39.552 INFO:tasks.workunit.client.1.smithi149.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-22T07:08:39.552 INFO:tasks.workunit.client.1.smithi149.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T07:08:39.553 INFO:tasks.workunit.client.1.smithi149.stderr:+ make 2024-03-22T07:08:39.578 INFO:tasks.workunit.client.0.smithi081.stdout:7/390: sync 2024-03-22T07:08:39.578 INFO:tasks.workunit.client.0.smithi081.stdout:6/272: sync 2024-03-22T07:08:39.578 INFO:tasks.workunit.client.0.smithi081.stdout:3/207: sync 2024-03-22T07:08:39.578 INFO:tasks.workunit.client.0.smithi081.stdout:1/188: rename d6/d2e/d30/f43 to d6/d18/d1f/d32/f47 0 2024-03-22T07:08:39.579 INFO:tasks.workunit.client.0.smithi081.stdout:7/391: read - d9/d10/d2a/f7d zero size 2024-03-22T07:08:39.579 INFO:tasks.workunit.client.0.smithi081.stdout:7/392: chown d9/d10/d53/f82 1 1 2024-03-22T07:08:39.579 INFO:tasks.workunit.client.0.smithi081.stdout:1/189: write d6/d2e/f44 [772687,32131] 0 2024-03-22T07:08:39.579 INFO:tasks.workunit.client.0.smithi081.stdout:3/208: rename db/de/f1b to db/d23/f3f 0 2024-03-22T07:08:39.580 INFO:tasks.workunit.client.0.smithi081.stdout:1/190: mkdir d6/d18/d1f/d24/d48 0 2024-03-22T07:08:39.580 INFO:tasks.workunit.client.0.smithi081.stdout:1/191: chown d6/f35 218 1 2024-03-22T07:08:39.580 INFO:tasks.workunit.client.0.smithi081.stdout:1/192: fsync d6/d18/d1f/d24/f2b 0 2024-03-22T07:08:39.580 INFO:tasks.workunit.client.0.smithi081.stdout:6/273: mkdir d3/da/de/d31/d40/d50 0 2024-03-22T07:08:39.580 INFO:tasks.workunit.client.0.smithi081.stdout:1/193: fdatasync d6/d18/d1f/d32/f47 0 2024-03-22T07:08:39.580 INFO:tasks.workunit.client.0.smithi081.stdout:6/274: truncate d3/f3c 1007216 0 2024-03-22T07:08:39.581 INFO:tasks.workunit.client.0.smithi081.stdout:7/393: creat d9/d3f/d52/d83/f8e x:0 0 0 2024-03-22T07:08:39.581 INFO:tasks.workunit.client.0.smithi081.stdout:7/394: getdents d9/d10/d53/d81/d59/d89 0 2024-03-22T07:08:39.581 INFO:tasks.workunit.client.0.smithi081.stdout:7/395: write d9/d10/d2a/f4d [66710,56979] 0 2024-03-22T07:08:39.582 INFO:tasks.workunit.client.0.smithi081.stdout:1/194: mkdir d6/d18/d41/d49 0 2024-03-22T07:08:39.582 INFO:tasks.workunit.client.0.smithi081.stdout:1/195: creat d6/d2e/f4a x:0 0 0 2024-03-22T07:08:39.582 INFO:tasks.workunit.client.0.smithi081.stdout:1/196: dread - d6/d18/d1f/d32/f47 zero size 2024-03-22T07:08:39.582 INFO:tasks.workunit.client.0.smithi081.stdout:1/197: chown d6/d18/f21 242035 1 2024-03-22T07:08:39.582 INFO:tasks.workunit.client.0.smithi081.stdout:1/198: read - d6/d2e/f4a zero size 2024-03-22T07:08:39.583 INFO:tasks.workunit.client.0.smithi081.stdout:3/209: dread db/fc [0,4194304] 0 2024-03-22T07:08:39.583 INFO:tasks.workunit.client.0.smithi081.stdout:3/210: write db/f13 [4654146,118318] 0 2024-03-22T07:08:39.583 INFO:tasks.workunit.client.0.smithi081.stdout:2/183: rename da/d11/f26 to da/d11/d32/f37 0 2024-03-22T07:08:39.583 INFO:tasks.workunit.client.0.smithi081.stdout:6/275: symlink d3/da/de/d31/d40/l51 0 2024-03-22T07:08:39.584 INFO:tasks.workunit.client.0.smithi081.stdout:1/199: unlink d6/l16 0 2024-03-22T07:08:39.585 INFO:tasks.workunit.client.0.smithi081.stdout:3/211: rename db/d2e/f39 to db/d2e/f40 0 2024-03-22T07:08:39.585 INFO:tasks.workunit.client.0.smithi081.stdout:1/200: creat d6/d18/d1f/d32/f4b x:0 0 0 2024-03-22T07:08:39.586 INFO:tasks.workunit.client.0.smithi081.stdout:0/319: rename d3/dc/d2b/f3a to d3/dc/d40/f64 0 2024-03-22T07:08:39.587 INFO:tasks.workunit.client.0.smithi081.stdout:2/184: mkdir da/d11/d32/d38 0 2024-03-22T07:08:39.587 INFO:tasks.workunit.client.0.smithi081.stdout:2/185: write f6 [2658718,35544] 0 2024-03-22T07:08:39.587 INFO:tasks.workunit.client.0.smithi081.stdout:6/276: truncate d3/f9 5450753 0 2024-03-22T07:08:39.589 INFO:tasks.workunit.client.0.smithi081.stdout:3/212: getdents db 0 2024-03-22T07:08:39.595 INFO:tasks.workunit.client.0.smithi081.stdout:7/396: rename d9/d10/d16/l2b to d9/d10/d4b/d61/l8f 0 2024-03-22T07:08:39.595 INFO:tasks.workunit.client.0.smithi081.stdout:0/320: creat d3/dc/d2b/d39/f65 x:0 0 0 2024-03-22T07:08:39.595 INFO:tasks.workunit.client.0.smithi081.stdout:0/321: creat d3/dc/d2b/d39/f66 x:0 0 0 2024-03-22T07:08:39.595 INFO:tasks.workunit.client.0.smithi081.stdout:2/186: truncate da/db/f27 54412 0 2024-03-22T07:08:39.595 INFO:tasks.workunit.client.0.smithi081.stdout:0/322: write d3/dc/f19 [2119927,106844] 0 2024-03-22T07:08:39.598 INFO:tasks.workunit.client.0.smithi081.stdout:2/187: mkdir da/db/d39 0 2024-03-22T07:08:39.598 INFO:tasks.workunit.client.0.smithi081.stdout:2/188: chown da/db/c16 1222841 1 2024-03-22T07:08:39.598 INFO:tasks.workunit.client.0.smithi081.stdout:2/189: truncate da/d11/d21/f2d 757707 0 2024-03-22T07:08:39.601 INFO:tasks.workunit.client.0.smithi081.stdout:1/201: write d6/d18/d1f/d24/f3f [2949323,71845] 0 2024-03-22T07:08:39.603 INFO:tasks.workunit.client.0.smithi081.stdout:0/323: symlink d3/d32/d35/l67 0 2024-03-22T07:08:39.604 INFO:tasks.workunit.client.0.smithi081.stdout:1/202: mkdir d6/d18/d1f/d4c 0 2024-03-22T07:08:39.608 INFO:tasks.workunit.client.0.smithi081.stdout:2/190: mknod da/db/c3a 0 2024-03-22T07:08:39.610 INFO:tasks.workunit.client.0.smithi081.stdout:3/213: dread db/f12 [0,4194304] 0 2024-03-22T07:08:39.610 INFO:tasks.workunit.client.0.smithi081.stdout:3/214: chown db/d23/l2b 1198 1 2024-03-22T07:08:39.611 INFO:tasks.workunit.client.0.smithi081.stdout:3/215: truncate db/d2e/f3e 173295 0 2024-03-22T07:08:39.612 INFO:tasks.workunit.client.0.smithi081.stdout:2/191: rmdir da/db 39 2024-03-22T07:08:39.612 INFO:tasks.workunit.client.0.smithi081.stdout:2/192: dread - da/d11/d21/d2c/f2e zero size 2024-03-22T07:08:39.612 INFO:tasks.workunit.client.0.smithi081.stdout:2/193: dread - da/d11/d21/d2c/f36 zero size 2024-03-22T07:08:39.613 INFO:tasks.workunit.client.0.smithi081.stdout:2/194: mkdir da/d11/d32/d3b 0 2024-03-22T07:08:39.613 INFO:tasks.workunit.client.0.smithi081.stdout:2/195: truncate f5 1478962 0 2024-03-22T07:08:39.613 INFO:tasks.workunit.client.0.smithi081.stdout:2/196: chown da/d11/d32/f37 144945680 1 2024-03-22T07:08:39.621 INFO:tasks.workunit.client.0.smithi081.stdout:6/277: dwrite d3/da/de/f4b [0,4194304] 0 2024-03-22T07:08:39.622 INFO:tasks.workunit.client.0.smithi081.stdout:7/397: dwrite d9/dc/f33 [0,4194304] 0 2024-03-22T07:08:39.623 INFO:tasks.workunit.client.0.smithi081.stdout:1/203: write d6/d18/f28 [3518653,84660] 0 2024-03-22T07:08:39.625 INFO:tasks.workunit.client.0.smithi081.stdout:6/278: mkdir d3/da/de/df/d26/d52 0 2024-03-22T07:08:39.628 INFO:tasks.workunit.client.0.smithi081.stdout:3/216: dwrite db/f18 [0,4194304] 0 2024-03-22T07:08:39.640 INFO:tasks.workunit.client.0.smithi081.stdout:7/398: creat d9/dc/f90 x:0 0 0 2024-03-22T07:08:39.641 INFO:tasks.workunit.client.0.smithi081.stdout:7/399: chown d9/dc/f33 305 1 2024-03-22T07:08:39.643 INFO:tasks.workunit.client.0.smithi081.stdout:6/279: creat d3/da/f53 x:0 0 0 2024-03-22T07:08:39.647 INFO:tasks.workunit.client.0.smithi081.stdout:0/324: dread d3/dc/f19 [0,4194304] 0 2024-03-22T07:08:39.647 INFO:tasks.workunit.client.0.smithi081.stdout:3/217: creat db/de/f41 x:0 0 0 2024-03-22T07:08:39.647 INFO:tasks.workunit.client.0.smithi081.stdout:0/325: write d3/dc/d33/f47 [979138,73327] 0 2024-03-22T07:08:39.647 INFO:tasks.workunit.client.0.smithi081.stdout:0/326: creat d3/dc/d33/d62/f68 x:0 0 0 2024-03-22T07:08:39.647 INFO:tasks.workunit.client.0.smithi081.stdout:0/327: write d3/dc/f36 [6861337,43097] 0 2024-03-22T07:08:39.655 INFO:tasks.workunit.client.1.smithi149.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-22T07:08:39.659 INFO:tasks.workunit.client.0.smithi081.stdout:1/204: dwrite d6/d18/d1f/d32/f47 [0,4194304] 0 2024-03-22T07:08:39.659 INFO:tasks.workunit.client.0.smithi081.stdout:1/205: chown d6/l1c 62340691 1 2024-03-22T07:08:39.660 INFO:tasks.workunit.client.0.smithi081.stdout:8/207: sync 2024-03-22T07:08:39.660 INFO:tasks.workunit.client.0.smithi081.stdout:8/208: fdatasync d2/d3/d28/f30 0 2024-03-22T07:08:39.660 INFO:tasks.workunit.client.0.smithi081.stdout:8/209: chown f1 228218251 1 2024-03-22T07:08:39.660 INFO:tasks.workunit.client.0.smithi081.stdout:8/210: creat d2/d3/d28/d2b/f4b x:0 0 0 2024-03-22T07:08:39.660 INFO:tasks.workunit.client.0.smithi081.stdout:8/211: fsync d2/d3/d19/d1e/f41 0 2024-03-22T07:08:39.662 INFO:tasks.workunit.client.0.smithi081.stdout:5/311: sync 2024-03-22T07:08:39.662 INFO:tasks.workunit.client.0.smithi081.stdout:4/372: sync 2024-03-22T07:08:39.662 INFO:tasks.workunit.client.0.smithi081.stdout:5/312: creat d4/df/d22/d4d/f6c x:0 0 0 2024-03-22T07:08:39.664 INFO:tasks.workunit.client.0.smithi081.stdout:2/197: dwrite da/f13 [0,4194304] 0 2024-03-22T07:08:39.665 INFO:tasks.workunit.client.0.smithi081.stdout:9/232: sync 2024-03-22T07:08:39.672 INFO:tasks.workunit.client.0.smithi081.stdout:1/206: link d6/d18/d1f/c33 d6/d18/d1f/d32/c4d 0 2024-03-22T07:08:39.672 INFO:tasks.workunit.client.0.smithi081.stdout:5/313: rename d4/d10/d20/d1f/d2f/f6b to d4/df/f6d 0 2024-03-22T07:08:39.674 INFO:tasks.workunit.client.0.smithi081.stdout:7/400: dwrite d9/f11 [0,4194304] 0 2024-03-22T07:08:39.674 INFO:tasks.workunit.client.0.smithi081.stdout:7/401: fdatasync d9/d10/d16/d1e/f32 0 2024-03-22T07:08:39.675 INFO:tasks.workunit.client.0.smithi081.stdout:9/233: mkdir d2/d2b/d49 0 2024-03-22T07:08:39.675 INFO:tasks.workunit.client.0.smithi081.stdout:9/234: readlink d2/d10/l2c 0 2024-03-22T07:08:39.678 INFO:tasks.workunit.client.0.smithi081.stdout:0/328: write d3/f1d [1161886,100707] 0 2024-03-22T07:08:39.678 INFO:tasks.workunit.client.0.smithi081.stdout:0/329: readlink d3/dc/l1e 0 2024-03-22T07:08:39.678 INFO:tasks.workunit.client.0.smithi081.stdout:0/330: fsync d3/dc/d2b/d39/f66 0 2024-03-22T07:08:39.679 INFO:tasks.workunit.client.0.smithi081.stdout:9/235: creat d2/d5/d16/f4a x:0 0 0 2024-03-22T07:08:39.679 INFO:tasks.workunit.client.0.smithi081.stdout:9/236: chown d2/d2b/l38 156 1 2024-03-22T07:08:39.680 INFO:tasks.workunit.client.0.smithi081.stdout:9/237: read d2/f15 [1059239,37576] 0 2024-03-22T07:08:39.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/331: mkdir d3/dc/d40/d69 0 2024-03-22T07:08:39.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/332: chown d3/d32/f5f 254 1 2024-03-22T07:08:39.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/333: chown d3/d32/f59 135 1 2024-03-22T07:08:39.684 INFO:tasks.workunit.client.0.smithi081.stdout:4/373: dread da/d18/d1b/d1e/d37/d1c/f51 [0,4194304] 0 2024-03-22T07:08:39.684 INFO:tasks.workunit.client.0.smithi081.stdout:9/238: mkdir d2/d10/d4b 0 2024-03-22T07:08:39.687 INFO:tasks.workunit.client.0.smithi081.stdout:6/280: dwrite d3/da/de/df/d26/d3f/f7 [4194304,4194304] 0 2024-03-22T07:08:39.688 INFO:tasks.workunit.client.0.smithi081.stdout:7/402: unlink d9/d10/f3a 0 2024-03-22T07:08:39.689 INFO:tasks.workunit.client.0.smithi081.stdout:4/374: symlink da/d18/d1b/d26/d3e/l72 0 2024-03-22T07:08:39.689 INFO:tasks.workunit.client.0.smithi081.stdout:4/375: chown da/d18/d1b/d1e/d5f 6825634 1 2024-03-22T07:08:39.692 INFO:tasks.workunit.client.0.smithi081.stdout:2/198: dwrite da/f22 [4194304,4194304] 0 2024-03-22T07:08:39.692 INFO:tasks.workunit.client.0.smithi081.stdout:9/239: rename d2/d10/f17 to d2/d5/d16/f4c 0 2024-03-22T07:08:39.692 INFO:tasks.workunit.client.0.smithi081.stdout:8/212: dwrite d2/d3/d28/f4a [0,4194304] 0 2024-03-22T07:08:39.692 INFO:tasks.workunit.client.0.smithi081.stdout:8/213: write d2/d3/d19/d1e/d13/d22/d38/f44 [1027101,68903] 0 2024-03-22T07:08:39.692 INFO:tasks.workunit.client.0.smithi081.stdout:0/334: mkdir d3/d6a 0 2024-03-22T07:08:39.697 INFO:tasks.workunit.client.0.smithi081.stdout:7/403: symlink d9/dc/l91 0 2024-03-22T07:08:39.700 INFO:tasks.workunit.client.0.smithi081.stdout:1/207: dwrite d6/fc [0,4194304] 0 2024-03-22T07:08:39.700 INFO:tasks.workunit.client.0.smithi081.stdout:4/376: creat da/d18/d1b/d26/f73 x:0 0 0 2024-03-22T07:08:39.703 INFO:tasks.workunit.client.0.smithi081.stdout:0/335: mknod d3/dc/d2b/d4d/c6b 0 2024-03-22T07:08:39.705 INFO:tasks.workunit.client.0.smithi081.stdout:7/404: rename d9/d10/d4b/l55 to d9/d10/d53/d81/l92 0 2024-03-22T07:08:39.706 INFO:tasks.workunit.client.0.smithi081.stdout:1/208: stat l5 0 2024-03-22T07:08:39.706 INFO:tasks.workunit.client.0.smithi081.stdout:1/209: rename d6/d18/d41 to d6/d18/d41/d4e 22 2024-03-22T07:08:39.707 INFO:tasks.workunit.client.0.smithi081.stdout:1/210: creat d6/d18/f4f x:0 0 0 2024-03-22T07:08:39.707 INFO:tasks.workunit.client.0.smithi081.stdout:4/377: mkdir da/d18/d1b/d1e/d37/d1c/d74 0 2024-03-22T07:08:39.707 INFO:tasks.workunit.client.0.smithi081.stdout:4/378: read - da/d18/d1b/d1e/f71 zero size 2024-03-22T07:08:39.707 INFO:tasks.workunit.client.0.smithi081.stdout:1/211: mknod d6/d18/d1f/d24/d48/c50 0 2024-03-22T07:08:39.708 INFO:tasks.workunit.client.0.smithi081.stdout:2/199: getdents da 0 2024-03-22T07:08:39.708 INFO:tasks.workunit.client.0.smithi081.stdout:5/314: dwrite d4/df/d22/d49/f60 [0,4194304] 0 2024-03-22T07:08:39.708 INFO:tasks.workunit.client.0.smithi081.stdout:8/214: dread f0 [0,4194304] 0 2024-03-22T07:08:39.708 INFO:tasks.workunit.client.0.smithi081.stdout:8/215: truncate d2/d3/d19/d1e/d13/d22/f34 566367 0 2024-03-22T07:08:39.709 INFO:tasks.workunit.client.0.smithi081.stdout:8/216: rename d2 to d2/d3/d23/d4c 22 2024-03-22T07:08:39.709 INFO:tasks.workunit.client.0.smithi081.stdout:8/217: write d2/d3/d19/f25 [1263502,53763] 0 2024-03-22T07:08:39.713 INFO:tasks.workunit.client.0.smithi081.stdout:6/281: dwrite d3/da/de/f28 [0,4194304] 0 2024-03-22T07:08:39.713 INFO:tasks.workunit.client.0.smithi081.stdout:7/405: mknod d9/d3f/d52/c93 0 2024-03-22T07:08:39.715 INFO:tasks.workunit.client.0.smithi081.stdout:4/379: mknod da/d18/d1b/d26/d3e/c75 0 2024-03-22T07:08:39.715 INFO:tasks.workunit.client.0.smithi081.stdout:1/212: getdents d6/d18/d1f/d24/d48 0 2024-03-22T07:08:39.717 INFO:tasks.workunit.client.0.smithi081.stdout:1/213: mknod d6/d18/d41/d49/c51 0 2024-03-22T07:08:39.722 INFO:tasks.workunit.client.0.smithi081.stdout:0/336: read d3/dc/d2b/f31 [5709264,118057] 0 2024-03-22T07:08:39.723 INFO:tasks.workunit.client.0.smithi081.stdout:0/337: creat d3/dc/d2b/d4d/f6c x:0 0 0 2024-03-22T07:08:39.723 INFO:tasks.workunit.client.0.smithi081.stdout:5/315: truncate d4/d9/f2c 3423947 0 2024-03-22T07:08:39.723 INFO:tasks.workunit.client.0.smithi081.stdout:5/316: chown d4/df/c16 40554 1 2024-03-22T07:08:39.723 INFO:tasks.workunit.client.0.smithi081.stdout:5/317: chown d4/d10/d20 5951 1 2024-03-22T07:08:39.723 INFO:tasks.workunit.client.0.smithi081.stdout:4/380: dread da/d18/d1b/d34/f4c [0,4194304] 0 2024-03-22T07:08:39.725 INFO:tasks.workunit.client.0.smithi081.stdout:6/282: creat d3/da/de/df/f54 x:0 0 0 2024-03-22T07:08:39.726 INFO:tasks.workunit.client.0.smithi081.stdout:7/406: creat d9/d10/d16/d1e/d6f/f94 x:0 0 0 2024-03-22T07:08:39.734 INFO:tasks.workunit.client.0.smithi081.stdout:8/218: dwrite d2/d3/d19/f48 [0,4194304] 0 2024-03-22T07:08:39.741 INFO:tasks.workunit.client.0.smithi081.stdout:0/338: symlink d3/dc/d2b/l6d 0 2024-03-22T07:08:39.742 INFO:tasks.workunit.client.0.smithi081.stdout:5/318: rmdir d4/d10/d20/d1f 39 2024-03-22T07:08:39.748 INFO:tasks.workunit.client.0.smithi081.stdout:8/219: read f0 [7202285,52126] 0 2024-03-22T07:08:39.749 INFO:tasks.workunit.client.0.smithi081.stdout:8/220: chown d2/d3/l20 10 1 2024-03-22T07:08:39.749 INFO:tasks.workunit.client.0.smithi081.stdout:6/283: unlink d3/da/de/df/d26/d3f/l10 0 2024-03-22T07:08:39.750 INFO:tasks.workunit.client.0.smithi081.stdout:0/339: symlink d3/dc/d40/d69/l6e 0 2024-03-22T07:08:39.750 INFO:tasks.workunit.client.0.smithi081.stdout:0/340: fsync d3/dc/f36 0 2024-03-22T07:08:39.750 INFO:tasks.workunit.client.0.smithi081.stdout:0/341: read d3/dc/d2b/f31 [5265622,107444] 0 2024-03-22T07:08:39.751 INFO:tasks.workunit.client.0.smithi081.stdout:5/319: creat d4/d10/d20/d25/f6e x:0 0 0 2024-03-22T07:08:39.751 INFO:tasks.workunit.client.0.smithi081.stdout:1/214: rmdir d6/d2e 39 2024-03-22T07:08:39.753 INFO:tasks.workunit.client.0.smithi081.stdout:2/200: dwrite f5 [0,4194304] 0 2024-03-22T07:08:39.753 INFO:tasks.workunit.client.0.smithi081.stdout:1/215: mknod d6/d18/c52 0 2024-03-22T07:08:39.754 INFO:tasks.workunit.client.0.smithi081.stdout:5/320: creat d4/d10/d20/d25/d2e/f6f x:0 0 0 2024-03-22T07:08:39.756 INFO:tasks.workunit.client.0.smithi081.stdout:0/342: truncate d3/f4 2507807 0 2024-03-22T07:08:39.756 INFO:tasks.workunit.client.0.smithi081.stdout:4/381: dwrite da/df/f5a [0,4194304] 0 2024-03-22T07:08:39.756 INFO:tasks.workunit.client.0.smithi081.stdout:7/407: dwrite d9/d10/d2a/f47 [4194304,4194304] 0 2024-03-22T07:08:39.757 INFO:tasks.workunit.client.0.smithi081.stdout:2/201: truncate f6 311816 0 2024-03-22T07:08:39.757 INFO:tasks.workunit.client.0.smithi081.stdout:2/202: readlink da/d11/l17 0 2024-03-22T07:08:39.757 INFO:tasks.workunit.client.0.smithi081.stdout:2/203: chown da/c30 203 1 2024-03-22T07:08:39.757 INFO:tasks.workunit.client.0.smithi081.stdout:2/204: stat da/l23 0 2024-03-22T07:08:39.759 INFO:tasks.workunit.client.0.smithi081.stdout:0/343: unlink d3/dc/d33/l53 0 2024-03-22T07:08:39.759 INFO:tasks.workunit.client.0.smithi081.stdout:0/344: fdatasync d3/dc/d33/f5a 0 2024-03-22T07:08:39.760 INFO:tasks.workunit.client.0.smithi081.stdout:4/382: dread da/d18/d1b/f40 [0,4194304] 0 2024-03-22T07:08:39.760 INFO:tasks.workunit.client.0.smithi081.stdout:4/383: creat da/df/d38/f76 x:0 0 0 2024-03-22T07:08:39.761 INFO:tasks.workunit.client.0.smithi081.stdout:7/408: getdents d9/d10/d2a/d73 0 2024-03-22T07:08:39.761 INFO:tasks.workunit.client.0.smithi081.stdout:7/409: creat d9/d10/d53/f95 x:0 0 0 2024-03-22T07:08:39.764 INFO:tasks.workunit.client.0.smithi081.stdout:4/384: mknod da/d18/d42/d4a/c77 0 2024-03-22T07:08:39.765 INFO:tasks.workunit.client.0.smithi081.stdout:8/221: write d2/d3/d28/f4a [3214821,129593] 0 2024-03-22T07:08:39.766 INFO:tasks.workunit.client.0.smithi081.stdout:4/385: mknod da/d18/d6e/c78 0 2024-03-22T07:08:39.766 INFO:tasks.workunit.client.0.smithi081.stdout:7/410: write d9/d10/d16/d1e/d27/f57 [288561,127428] 0 2024-03-22T07:08:39.768 INFO:tasks.workunit.client.0.smithi081.stdout:8/222: dread d2/f24 [0,4194304] 0 2024-03-22T07:08:39.769 INFO:tasks.workunit.client.0.smithi081.stdout:1/216: dread d6/fc [0,4194304] 0 2024-03-22T07:08:39.769 INFO:tasks.workunit.client.0.smithi081.stdout:1/217: fdatasync d6/d18/f21 0 2024-03-22T07:08:39.769 INFO:tasks.workunit.client.0.smithi081.stdout:4/386: creat da/f79 x:0 0 0 2024-03-22T07:08:39.769 INFO:tasks.workunit.client.0.smithi081.stdout:4/387: truncate da/d18/d1b/d1e/f41 379675 0 2024-03-22T07:08:39.769 INFO:tasks.workunit.client.0.smithi081.stdout:7/411: dread d9/d10/d2a/f4d [0,4194304] 0 2024-03-22T07:08:39.770 INFO:tasks.workunit.client.0.smithi081.stdout:1/218: symlink d6/d18/l53 0 2024-03-22T07:08:39.772 INFO:tasks.workunit.client.0.smithi081.stdout:8/223: dread d2/d3/f1b [0,4194304] 0 2024-03-22T07:08:39.774 INFO:tasks.workunit.client.0.smithi081.stdout:2/205: write da/d11/f1f [601500,10640] 0 2024-03-22T07:08:39.774 INFO:tasks.workunit.client.0.smithi081.stdout:2/206: truncate da/f25 1925005 0 2024-03-22T07:08:39.774 INFO:tasks.workunit.client.0.smithi081.stdout:2/207: rename da/db/d39 to da/db/d39/d3c 22 2024-03-22T07:08:39.774 INFO:tasks.workunit.client.0.smithi081.stdout:8/224: mknod d2/c4d 0 2024-03-22T07:08:39.775 INFO:tasks.workunit.client.0.smithi081.stdout:8/225: truncate f0 9359678 0 2024-03-22T07:08:39.775 INFO:tasks.workunit.client.0.smithi081.stdout:6/284: dwrite d3/da/de/f1f [0,4194304] 0 2024-03-22T07:08:39.776 INFO:tasks.workunit.client.0.smithi081.stdout:0/345: dwrite d3/dc/d2b/d4d/f6c [0,4194304] 0 2024-03-22T07:08:39.777 INFO:tasks.workunit.client.0.smithi081.stdout:2/208: creat da/d11/d32/f3d x:0 0 0 2024-03-22T07:08:39.777 INFO:tasks.workunit.client.0.smithi081.stdout:2/209: readlink l1 0 2024-03-22T07:08:39.782 INFO:tasks.workunit.client.0.smithi081.stdout:8/226: mknod d2/c4e 0 2024-03-22T07:08:39.782 INFO:tasks.workunit.client.0.smithi081.stdout:8/227: write f0 [6090849,110808] 0 2024-03-22T07:08:39.783 INFO:tasks.workunit.client.0.smithi081.stdout:8/228: write d2/d3/d19/d1e/f3e [530010,2497] 0 2024-03-22T07:08:39.785 INFO:tasks.workunit.client.0.smithi081.stdout:4/388: dread da/d18/d42/d4f/f58 [0,4194304] 0 2024-03-22T07:08:39.785 INFO:tasks.workunit.client.0.smithi081.stdout:4/389: write f1 [805779,27875] 0 2024-03-22T07:08:39.789 INFO:tasks.workunit.client.0.smithi081.stdout:6/285: symlink d3/da/de/l55 0 2024-03-22T07:08:39.789 INFO:tasks.workunit.client.0.smithi081.stdout:7/412: read d9/d10/d16/d1e/f18 [2464982,110389] 0 2024-03-22T07:08:39.791 INFO:tasks.workunit.client.0.smithi081.stdout:0/346: fsync d3/f8 0 2024-03-22T07:08:39.793 INFO:tasks.workunit.client.0.smithi081.stdout:2/210: symlink da/l3e 0 2024-03-22T07:08:39.795 INFO:tasks.workunit.client.0.smithi081.stdout:8/229: symlink d2/d3/d28/d2b/l4f 0 2024-03-22T07:08:39.797 INFO:tasks.workunit.client.0.smithi081.stdout:1/219: dwrite d6/d18/f28 [0,4194304] 0 2024-03-22T07:08:39.797 INFO:tasks.workunit.client.0.smithi081.stdout:1/220: getdents d6/d18/d1f/d4c 0 2024-03-22T07:08:39.797 INFO:tasks.workunit.client.0.smithi081.stdout:1/221: write d6/d18/f4f [13332,47554] 0 2024-03-22T07:08:39.797 INFO:tasks.workunit.client.0.smithi081.stdout:8/230: symlink d2/d3/d19/d1e/d13/d22/d38/l50 0 2024-03-22T07:08:39.798 INFO:tasks.workunit.client.0.smithi081.stdout:8/231: dread - d2/d3/d23/f3b zero size 2024-03-22T07:08:39.798 INFO:tasks.workunit.client.0.smithi081.stdout:8/232: chown d2/d3/cb 37088 1 2024-03-22T07:08:39.798 INFO:tasks.workunit.client.0.smithi081.stdout:1/222: rmdir d6/d18/d41/d49 39 2024-03-22T07:08:39.798 INFO:tasks.workunit.client.0.smithi081.stdout:4/390: rmdir da/d18/d42/d4a 39 2024-03-22T07:08:39.798 INFO:tasks.workunit.client.0.smithi081.stdout:6/286: creat d3/da/de/f56 x:0 0 0 2024-03-22T07:08:39.798 INFO:tasks.workunit.client.0.smithi081.stdout:6/287: creat d3/da/f57 x:0 0 0 2024-03-22T07:08:39.800 INFO:tasks.workunit.client.0.smithi081.stdout:1/223: dread d6/fc [0,4194304] 0 2024-03-22T07:08:39.808 INFO:tasks.workunit.client.0.smithi081.stdout:6/288: rename d3/da/l32 to d3/da/de/df/d26/d3f/l58 0 2024-03-22T07:08:39.808 INFO:tasks.workunit.client.0.smithi081.stdout:6/289: truncate d3/da/f53 823916 0 2024-03-22T07:08:39.808 INFO:tasks.workunit.client.0.smithi081.stdout:6/290: creat d3/da/f59 x:0 0 0 2024-03-22T07:08:39.815 INFO:tasks.workunit.client.0.smithi081.stdout:0/347: link d3/cb d3/dc/d40/c6f 0 2024-03-22T07:08:39.817 INFO:tasks.workunit.client.0.smithi081.stdout:8/233: dread d2/d3/f1f [0,4194304] 0 2024-03-22T07:08:39.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr fail", "who": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: osdmap e46: 6 total, 6 up, 6 in 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "mgr fail", "who": "smithi081.bcxarw"}]': finished 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: mgrmap e20: smithi149.sgmtoa(active, starting, since 0.0183244s) 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: Active manager daemon smithi149.sgmtoa restarted 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: Activating manager daemon smithi149.sgmtoa 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: mgrmap e21: smithi149.sgmtoa(active, starting, since 0.006891s) 2024-03-22T07:08:39.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/crt"}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr metadata", "who": "smithi149.sgmtoa", "id": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:08:39.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/key"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: Manager daemon smithi149.sgmtoa is now available 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.sgmtoa/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.sgmtoa/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.sgmtoa/trash_purge_schedule"}]: dispatch 2024-03-22T07:08:39.822 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:39 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.sgmtoa/trash_purge_schedule"}]: dispatch 2024-03-22T07:08:39.823 INFO:tasks.workunit.client.0.smithi081.stdout:8/234: creat d2/d3/d19/d1e/d13/d22/f51 x:0 0 0 2024-03-22T07:08:39.823 INFO:tasks.workunit.client.0.smithi081.stdout:6/291: mkdir d3/da/de/df/d26/d3f/d5a 0 2024-03-22T07:08:39.823 INFO:tasks.workunit.client.0.smithi081.stdout:1/224: dread d6/f35 [0,4194304] 0 2024-03-22T07:08:39.823 INFO:tasks.workunit.client.0.smithi081.stdout:6/292: dread d3/da/de/df/d26/d3f/f29 [0,4194304] 0 2024-03-22T07:08:39.823 INFO:tasks.workunit.client.0.smithi081.stdout:6/293: creat d3/da/de/f5b x:0 0 0 2024-03-22T07:08:39.825 INFO:tasks.workunit.client.0.smithi081.stdout:6/294: mknod d3/da/de/df/c5c 0 2024-03-22T07:08:39.825 INFO:tasks.workunit.client.0.smithi081.stdout:6/295: fsync d3/da/de/f41 0 2024-03-22T07:08:39.828 INFO:tasks.workunit.client.0.smithi081.stdout:1/225: dread d6/d18/d1f/d24/f31 [0,4194304] 0 2024-03-22T07:08:39.828 INFO:tasks.workunit.client.0.smithi081.stdout:1/226: chown d6/d18/d1f/d32/c4d 62268461 1 2024-03-22T07:08:39.829 INFO:tasks.workunit.client.0.smithi081.stdout:1/227: mknod d6/d18/d1f/c54 0 2024-03-22T07:08:39.830 INFO:tasks.workunit.client.0.smithi081.stdout:7/413: dwrite d9/d10/d2a/d73/f8b [0,4194304] 0 2024-03-22T07:08:39.831 INFO:tasks.workunit.client.0.smithi081.stdout:0/348: dread d3/d32/f4f [0,4194304] 0 2024-03-22T07:08:39.831 INFO:tasks.workunit.client.0.smithi081.stdout:0/349: creat d3/d32/f70 x:0 0 0 2024-03-22T07:08:39.831 INFO:tasks.workunit.client.0.smithi081.stdout:0/350: write d3/dc/f3f [827070,105983] 0 2024-03-22T07:08:39.833 INFO:tasks.workunit.client.0.smithi081.stdout:2/211: dwrite da/db/f29 [0,4194304] 0 2024-03-22T07:08:39.833 INFO:tasks.workunit.client.0.smithi081.stdout:6/296: write d3/da/de/f1f [2691919,12837] 0 2024-03-22T07:08:39.833 INFO:tasks.workunit.client.0.smithi081.stdout:6/297: chown d3/da/de/l49 293514305 1 2024-03-22T07:08:39.834 INFO:tasks.workunit.client.0.smithi081.stdout:6/298: stat d3/da/de/d31/d40/d50 0 2024-03-22T07:08:39.834 INFO:tasks.workunit.client.0.smithi081.stdout:7/414: mknod d9/d10/d2a/c96 0 2024-03-22T07:08:39.836 INFO:tasks.workunit.client.0.smithi081.stdout:8/235: write d2/d3/f1c [385526,94355] 0 2024-03-22T07:08:39.836 INFO:tasks.workunit.client.0.smithi081.stdout:8/236: creat d2/d3/d23/f52 x:0 0 0 2024-03-22T07:08:39.843 INFO:tasks.workunit.client.0.smithi081.stdout:0/351: mknod d3/dc/d2b/d4d/c71 0 2024-03-22T07:08:39.844 INFO:tasks.workunit.client.0.smithi081.stdout:7/415: dread d9/f11 [0,4194304] 0 2024-03-22T07:08:39.844 INFO:tasks.workunit.client.0.smithi081.stdout:7/416: stat d9/d10/d16/d1e/l23 0 2024-03-22T07:08:39.845 INFO:tasks.workunit.client.0.smithi081.stdout:2/212: creat da/db/f3f x:0 0 0 2024-03-22T07:08:39.846 INFO:tasks.workunit.client.0.smithi081.stdout:6/299: creat d3/da/de/d31/d40/d50/f5d x:0 0 0 2024-03-22T07:08:39.847 INFO:tasks.workunit.client.0.smithi081.stdout:8/237: dread d2/d3/d19/f48 [0,4194304] 0 2024-03-22T07:08:39.847 INFO:tasks.workunit.client.0.smithi081.stdout:8/238: creat d2/d3/d23/f53 x:0 0 0 2024-03-22T07:08:39.849 INFO:tasks.workunit.client.0.smithi081.stdout:8/239: symlink d2/d3/d19/d1e/d13/d22/d2f/l54 0 2024-03-22T07:08:39.850 INFO:tasks.workunit.client.0.smithi081.stdout:6/300: mkdir d3/da/de/df/d5e 0 2024-03-22T07:08:39.852 INFO:tasks.workunit.client.0.smithi081.stdout:8/240: mknod d2/d3/d19/d1e/d13/c55 0 2024-03-22T07:08:39.853 INFO:tasks.workunit.client.0.smithi081.stdout:6/301: dread d3/da/de/f11 [4194304,4194304] 0 2024-03-22T07:08:39.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:39.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:39.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:08:39.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:08:39.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr fail", "who": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: osdmap e46: 6 total, 6 up, 6 in 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.14212 172.21.15.81:0/2311535750' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "mgr fail", "who": "smithi081.bcxarw"}]': finished 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: mgrmap e20: smithi149.sgmtoa(active, starting, since 0.0183244s) 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: Active manager daemon smithi149.sgmtoa restarted 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: Activating manager daemon smithi149.sgmtoa 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: mgrmap e21: smithi149.sgmtoa(active, starting, since 0.006891s) 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/crt"}]: dispatch 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:08:39.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr metadata", "who": "smithi149.sgmtoa", "id": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/key"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:08:39.855 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: Manager daemon smithi149.sgmtoa is now available 2024-03-22T07:08:39.856 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:39.856 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:39.856 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:08:39.856 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.sgmtoa/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:08:39.856 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.sgmtoa/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:08:39.856 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.sgmtoa/trash_purge_schedule"}]: dispatch 2024-03-22T07:08:39.856 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:39 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi149.sgmtoa/trash_purge_schedule"}]: dispatch 2024-03-22T07:08:39.856 INFO:tasks.workunit.client.0.smithi081.stdout:8/241: creat d2/d3/d19/d1e/f56 x:0 0 0 2024-03-22T07:08:39.857 INFO:tasks.workunit.client.0.smithi081.stdout:8/242: write d2/d3/d23/f3b [922922,98905] 0 2024-03-22T07:08:39.857 INFO:tasks.workunit.client.0.smithi081.stdout:8/243: symlink d2/d3/d19/d1e/d13/d22/l57 0 2024-03-22T07:08:39.860 INFO:tasks.workunit.client.0.smithi081.stdout:0/352: write d3/dc/d40/f4a [927715,107735] 0 2024-03-22T07:08:39.860 INFO:tasks.workunit.client.0.smithi081.stdout:6/302: creat d3/da/de/df/d26/d3f/d4c/f5f x:0 0 0 2024-03-22T07:08:39.864 INFO:tasks.workunit.client.0.smithi081.stdout:2/213: write da/db/f29 [1969418,30991] 0 2024-03-22T07:08:39.865 INFO:tasks.workunit.client.0.smithi081.stdout:1/228: dwrite d6/f2a [0,4194304] 0 2024-03-22T07:08:39.866 INFO:tasks.workunit.client.0.smithi081.stdout:6/303: mknod d3/da/de/df/d26/d52/c60 0 2024-03-22T07:08:39.866 INFO:tasks.workunit.client.0.smithi081.stdout:6/304: read - d3/da/de/f46 zero size 2024-03-22T07:08:39.866 INFO:tasks.workunit.client.0.smithi081.stdout:6/305: creat d3/da/de/f61 x:0 0 0 2024-03-22T07:08:39.872 INFO:tasks.workunit.client.0.smithi081.stdout:7/417: dwrite d9/d10/d16/d1e/f32 [0,4194304] 0 2024-03-22T07:08:39.872 INFO:tasks.workunit.client.0.smithi081.stdout:7/418: write d9/dc/f1f [4477762,109689] 0 2024-03-22T07:08:39.875 INFO:tasks.workunit.client.0.smithi081.stdout:8/244: unlink d2/d3/f1c 0 2024-03-22T07:08:39.875 INFO:tasks.workunit.client.0.smithi081.stdout:8/245: write d2/d3/d19/d1e/d13/d22/d2f/f40 [514539,99946] 0 2024-03-22T07:08:39.875 INFO:tasks.workunit.client.0.smithi081.stdout:8/246: truncate d2/d3/d28/f4a 4214255 0 2024-03-22T07:08:39.875 INFO:tasks.workunit.client.0.smithi081.stdout:8/247: creat d2/d3/d28/d2b/f58 x:0 0 0 2024-03-22T07:08:39.876 INFO:tasks.workunit.client.0.smithi081.stdout:0/353: dread d3/f1d [0,4194304] 0 2024-03-22T07:08:39.883 INFO:tasks.workunit.client.0.smithi081.stdout:2/214: creat da/d11/d32/d38/f40 x:0 0 0 2024-03-22T07:08:39.884 INFO:tasks.workunit.client.0.smithi081.stdout:1/229: creat d6/d2e/d30/f55 x:0 0 0 2024-03-22T07:08:39.884 INFO:tasks.workunit.client.0.smithi081.stdout:1/230: readlink d6/l14 0 2024-03-22T07:08:39.888 INFO:tasks.workunit.client.0.smithi081.stdout:6/306: unlink d3/da/de/df/d26/d3f/d4c/f4d 0 2024-03-22T07:08:39.891 INFO:tasks.workunit.client.0.smithi081.stdout:8/248: getdents d2 0 2024-03-22T07:08:39.891 INFO:tasks.workunit.client.0.smithi081.stdout:8/249: chown d2/d3/d19/d1e/d13/d22/d38 159142667 1 2024-03-22T07:08:39.893 INFO:tasks.workunit.client.0.smithi081.stdout:0/354: mkdir d3/dc/d72 0 2024-03-22T07:08:39.894 INFO:tasks.workunit.client.0.smithi081.stdout:2/215: mkdir da/d11/d32/d3b/d41 0 2024-03-22T07:08:39.900 INFO:tasks.workunit.client.0.smithi081.stdout:0/355: dread d3/dc/f1c [0,4194304] 0 2024-03-22T07:08:39.900 INFO:tasks.workunit.client.0.smithi081.stdout:0/356: creat d3/dc/f73 x:0 0 0 2024-03-22T07:08:39.902 INFO:tasks.workunit.client.0.smithi081.stdout:2/216: mkdir da/d11/d21/d2c/d42 0 2024-03-22T07:08:39.902 INFO:tasks.workunit.client.0.smithi081.stdout:2/217: creat da/d11/d32/f43 x:0 0 0 2024-03-22T07:08:39.902 INFO:tasks.workunit.client.0.smithi081.stdout:2/218: write da/d11/d32/d38/f40 [731356,62507] 0 2024-03-22T07:08:39.902 INFO:tasks.workunit.client.0.smithi081.stdout:2/219: creat da/d11/d21/d2c/f44 x:0 0 0 2024-03-22T07:08:39.907 INFO:tasks.workunit.client.0.smithi081.stdout:8/250: dread f0 [4194304,4194304] 0 2024-03-22T07:08:39.909 INFO:tasks.workunit.client.0.smithi081.stdout:8/251: link d2/d3/d19/d1e/d13/l36 d2/d3/d19/d1e/d13/l59 0 2024-03-22T07:08:39.912 INFO:tasks.workunit.client.0.smithi081.stdout:2/220: mkdir da/db/d39/d45 0 2024-03-22T07:08:39.917 INFO:tasks.workunit.client.0.smithi081.stdout:7/419: truncate d9/d10/d16/d1e/f32 1746690 0 2024-03-22T07:08:39.923 INFO:tasks.workunit.client.0.smithi081.stdout:6/307: dwrite d3/da/de/df/d26/f44 [0,4194304] 0 2024-03-22T07:08:39.923 INFO:tasks.workunit.client.0.smithi081.stdout:0/357: dwrite d3/dc/d2b/d4d/f5e [0,4194304] 0 2024-03-22T07:08:39.925 INFO:tasks.workunit.client.0.smithi081.stdout:6/308: mknod d3/da/d48/c62 0 2024-03-22T07:08:39.925 INFO:tasks.workunit.client.0.smithi081.stdout:6/309: chown d3/da/de/l55 779864 1 2024-03-22T07:08:39.929 INFO:tasks.workunit.client.0.smithi081.stdout:7/420: write d9/d10/d16/d1e/d27/f5a [2871403,100099] 0 2024-03-22T07:08:39.930 INFO:tasks.workunit.client.0.smithi081.stdout:0/358: getdents d3/dc/d2b 0 2024-03-22T07:08:39.939 INFO:tasks.workunit.client.0.smithi081.stdout:7/421: write d9/dc/f33 [2157871,15010] 0 2024-03-22T07:08:39.939 INFO:tasks.workunit.client.0.smithi081.stdout:7/422: write d9/d10/d16/f6b [231047,35629] 0 2024-03-22T07:08:39.942 INFO:tasks.workunit.client.0.smithi081.stdout:7/423: truncate d9/d14/f24 3251384 0 2024-03-22T07:08:39.944 INFO:tasks.workunit.client.0.smithi081.stdout:7/424: symlink d9/d10/d4b/l97 0 2024-03-22T07:08:39.944 INFO:tasks.workunit.client.0.smithi081.stdout:7/425: stat d9/d3f/l5d 0 2024-03-22T07:08:39.944 INFO:tasks.workunit.client.0.smithi081.stdout:7/426: creat d9/d10/d53/f98 x:0 0 0 2024-03-22T07:08:39.949 INFO:tasks.workunit.client.0.smithi081.stdout:7/427: dread d9/d10/d16/f7a [0,4194304] 0 2024-03-22T07:08:39.949 INFO:tasks.workunit.client.0.smithi081.stdout:7/428: write d9/dc/f33 [202342,11416] 0 2024-03-22T07:08:39.955 INFO:tasks.workunit.client.0.smithi081.stdout:0/359: dwrite d3/d32/f5f [0,4194304] 0 2024-03-22T07:08:39.955 INFO:tasks.workunit.client.0.smithi081.stdout:8/252: dwrite d2/d3/d19/d1e/f41 [0,4194304] 0 2024-03-22T07:08:39.955 INFO:tasks.workunit.client.0.smithi081.stdout:6/310: dwrite d3/f3c [0,4194304] 0 2024-03-22T07:08:39.961 INFO:tasks.workunit.client.0.smithi081.stdout:0/360: dread d3/dc/f3f [0,4194304] 0 2024-03-22T07:08:39.961 INFO:tasks.workunit.client.0.smithi081.stdout:0/361: chown d3 134117460 1 2024-03-22T07:08:39.961 INFO:tasks.workunit.client.0.smithi081.stdout:0/362: dread - d3/dc/d2b/d39/f65 zero size 2024-03-22T07:08:39.961 INFO:tasks.workunit.client.0.smithi081.stdout:0/363: chown d3/dc/d2b/d39/l55 10 1 2024-03-22T07:08:39.963 INFO:tasks.workunit.client.0.smithi081.stdout:7/429: dread d9/d10/d16/d1e/f58 [0,4194304] 0 2024-03-22T07:08:39.964 INFO:tasks.workunit.client.0.smithi081.stdout:6/311: rmdir d3/da/de/d31 39 2024-03-22T07:08:39.965 INFO:tasks.workunit.client.0.smithi081.stdout:0/364: mkdir d3/d32/d35/d74 0 2024-03-22T07:08:39.965 INFO:tasks.workunit.client.0.smithi081.stdout:0/365: chown d3/dc/f3f 61 1 2024-03-22T07:08:39.966 INFO:tasks.workunit.client.0.smithi081.stdout:7/430: mkdir d9/d3f/d52/d99 0 2024-03-22T07:08:39.967 INFO:tasks.workunit.client.0.smithi081.stdout:0/366: truncate d3/f44 276056 0 2024-03-22T07:08:39.967 INFO:tasks.workunit.client.0.smithi081.stdout:0/367: stat d3/dc/d2b/d4d/c6b 0 2024-03-22T07:08:39.967 INFO:tasks.workunit.client.0.smithi081.stdout:0/368: creat d3/dc/d2b/d39/f75 x:0 0 0 2024-03-22T07:08:39.969 INFO:tasks.workunit.client.0.smithi081.stdout:0/369: creat d3/d32/d35/f76 x:0 0 0 2024-03-22T07:08:39.969 INFO:tasks.workunit.client.0.smithi081.stdout:0/370: write d3/d32/f70 [377316,98182] 0 2024-03-22T07:08:39.970 INFO:tasks.workunit.client.0.smithi081.stdout:0/371: mkdir d3/dc/d77 0 2024-03-22T07:08:39.970 INFO:tasks.workunit.client.0.smithi081.stdout:0/372: chown d3/dc/d2b/f48 389476 1 2024-03-22T07:08:39.970 INFO:tasks.workunit.client.0.smithi081.stdout:0/373: creat d3/dc/d33/d62/f78 x:0 0 0 2024-03-22T07:08:39.970 INFO:tasks.workunit.client.0.smithi081.stdout:7/431: rmdir d9/d10/d16/d1e 39 2024-03-22T07:08:39.972 INFO:tasks.workunit.client.0.smithi081.stdout:8/253: dwrite d2/d3/f1b [0,4194304] 0 2024-03-22T07:08:39.974 INFO:tasks.workunit.client.0.smithi081.stdout:0/374: unlink d3/dc/f58 0 2024-03-22T07:08:39.979 INFO:tasks.workunit.client.0.smithi081.stdout:0/375: mknod d3/dc/d40/d69/c79 0 2024-03-22T07:08:39.981 INFO:tasks.workunit.client.0.smithi081.stdout:0/376: symlink d3/dc/d33/d62/l7a 0 2024-03-22T07:08:39.981 INFO:tasks.workunit.client.0.smithi081.stdout:0/377: read - d3/d32/d35/f76 zero size 2024-03-22T07:08:39.982 INFO:tasks.workunit.client.0.smithi081.stdout:0/378: mknod d3/d32/c7b 0 2024-03-22T07:08:39.983 INFO:tasks.workunit.client.0.smithi081.stdout:0/379: symlink d3/dc/d40/d69/l7c 0 2024-03-22T07:08:39.983 INFO:tasks.workunit.client.0.smithi081.stdout:0/380: write d3/dc/d33/f5a [4673810,115589] 0 2024-03-22T07:08:39.994 INFO:tasks.workunit.client.0.smithi081.stdout:8/254: read d2/f11 [2091618,115157] 0 2024-03-22T07:08:39.994 INFO:tasks.workunit.client.0.smithi081.stdout:7/432: dwrite d9/d10/d2a/f78 [0,4194304] 0 2024-03-22T07:08:39.994 INFO:tasks.workunit.client.0.smithi081.stdout:7/433: write d9/d10/d53/f82 [658300,52066] 0 2024-03-22T07:08:39.994 INFO:tasks.workunit.client.0.smithi081.stdout:7/434: chown f7 21364 1 2024-03-22T07:08:39.998 INFO:tasks.workunit.client.0.smithi081.stdout:0/381: write d3/f8 [1104981,93113] 0 2024-03-22T07:08:39.999 INFO:tasks.workunit.client.0.smithi081.stdout:7/435: write d9/d10/d16/d1e/f2c [279515,16755] 0 2024-03-22T07:08:40.000 INFO:tasks.workunit.client.0.smithi081.stdout:0/382: link d3/f4 d3/dc/d33/f7d 0 2024-03-22T07:08:40.002 INFO:tasks.workunit.client.0.smithi081.stdout:7/436: creat d9/d3f/d52/d99/f9a x:0 0 0 2024-03-22T07:08:40.006 INFO:tasks.workunit.client.0.smithi081.stdout:0/383: creat d3/dc/d2b/f7e x:0 0 0 2024-03-22T07:08:40.009 INFO:tasks.workunit.client.0.smithi081.stdout:0/384: link d3/dc/l16 d3/dc/l7f 0 2024-03-22T07:08:40.012 INFO:tasks.workunit.client.0.smithi081.stdout:0/385: dread d3/d32/d35/f38 [0,4194304] 0 2024-03-22T07:08:40.013 INFO:tasks.workunit.client.0.smithi081.stdout:0/386: chown d3/f13 1389996 1 2024-03-22T07:08:40.021 INFO:tasks.workunit.client.0.smithi081.stdout:0/387: write d3/dc/d2b/d39/f3c [2226592,113964] 0 2024-03-22T07:08:40.021 INFO:tasks.workunit.client.0.smithi081.stdout:0/388: stat d3/d32/d35/c51 0 2024-03-22T07:08:40.021 INFO:tasks.workunit.client.0.smithi081.stdout:0/389: chown d3/dc/d40/c4c 325152 1 2024-03-22T07:08:40.022 INFO:tasks.workunit.client.0.smithi081.stdout:8/255: link d2/f15 d2/d3/d19/d1e/f5a 0 2024-03-22T07:08:40.024 INFO:tasks.workunit.client.0.smithi081.stdout:8/256: mkdir d2/d5b 0 2024-03-22T07:08:40.025 INFO:tasks.workunit.client.0.smithi081.stdout:0/390: getdents d3/d32/d35 0 2024-03-22T07:08:40.025 INFO:tasks.workunit.client.0.smithi081.stdout:8/257: write d2/d3/d23/f3b [818528,5529] 0 2024-03-22T07:08:40.025 INFO:tasks.workunit.client.0.smithi081.stdout:8/258: dread - d2/d3/d28/d2b/f46 zero size 2024-03-22T07:08:40.030 INFO:tasks.workunit.client.0.smithi081.stdout:8/259: truncate d2/f24 4087866 0 2024-03-22T07:08:40.030 INFO:tasks.workunit.client.0.smithi081.stdout:8/260: stat d2/d3/d19/d1e/d13/d22 0 2024-03-22T07:08:40.031 INFO:tasks.workunit.client.0.smithi081.stdout:8/261: creat d2/d3/d19/d1e/f5c x:0 0 0 2024-03-22T07:08:40.032 INFO:tasks.workunit.client.0.smithi081.stdout:8/262: readlink d2/d3/d19/d1e/d13/d22/d2f/l31 0 2024-03-22T07:08:40.033 INFO:tasks.workunit.client.1.smithi149.stderr:++ readlink -f fsstress 2024-03-22T07:08:40.034 INFO:tasks.workunit.client.0.smithi081.stdout:8/263: rmdir d2/d5b 0 2024-03-22T07:08:40.035 INFO:tasks.workunit.client.1.smithi149.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-22T07:08:40.035 INFO:tasks.workunit.client.1.smithi149.stderr:+ popd 2024-03-22T07:08:40.035 INFO:tasks.workunit.client.1.smithi149.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T07:08:40.035 INFO:tasks.workunit.client.1.smithi149.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-22T07:08:40.036 INFO:tasks.workunit.client.1.smithi149.stderr:+ popd 2024-03-22T07:08:40.036 INFO:tasks.workunit.client.1.smithi149.stderr:++ mktemp -d -p . 2024-03-22T07:08:40.036 INFO:tasks.workunit.client.0.smithi081.stdout:8/264: mknod d2/d3/d19/d1e/d13/d22/d2f/c5d 0 2024-03-22T07:08:40.037 INFO:tasks.workunit.client.0.smithi081.stdout:8/265: creat d2/f5e x:0 0 0 2024-03-22T07:08:40.037 INFO:tasks.workunit.client.1.smithi149.stderr:+ T=./tmp.n6qV8VL6Wb 2024-03-22T07:08:40.037 INFO:tasks.workunit.client.1.smithi149.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.n6qV8VL6Wb -l 1 -n 1000 -p 10 -v 2024-03-22T07:08:40.037 INFO:tasks.workunit.client.0.smithi081.stdout:8/266: creat d2/d3/d19/d1e/f5f x:0 0 0 2024-03-22T07:08:40.039 INFO:tasks.workunit.client.1.smithi149.stdout:seed = 1711125114 2024-03-22T07:08:40.042 INFO:tasks.workunit.client.1.smithi149.stdout:0/0: stat - no entries 2024-03-22T07:08:40.042 INFO:tasks.workunit.client.1.smithi149.stdout:0/1: read - no filename 2024-03-22T07:08:40.042 INFO:tasks.workunit.client.1.smithi149.stdout:0/2: write - no filename 2024-03-22T07:08:40.043 INFO:tasks.workunit.client.1.smithi149.stdout:0/3: fsync - no filename 2024-03-22T07:08:40.044 INFO:tasks.workunit.client.1.smithi149.stdout:1/0: getdents . 0 2024-03-22T07:08:40.044 INFO:tasks.workunit.client.1.smithi149.stdout:1/1: fsync - no filename 2024-03-22T07:08:40.044 INFO:tasks.workunit.client.1.smithi149.stdout:0/4: symlink l0 0 2024-03-22T07:08:40.045 INFO:tasks.workunit.client.1.smithi149.stdout:0/5: dwrite - no filename 2024-03-22T07:08:40.045 INFO:tasks.workunit.client.1.smithi149.stdout:2/0: dwrite - no filename 2024-03-22T07:08:40.046 INFO:tasks.workunit.client.1.smithi149.stdout:1/2: mknod c0 0 2024-03-22T07:08:40.046 INFO:tasks.workunit.client.1.smithi149.stdout:1/3: dread - no filename 2024-03-22T07:08:40.046 INFO:tasks.workunit.client.1.smithi149.stdout:1/4: write - no filename 2024-03-22T07:08:40.047 INFO:tasks.workunit.client.1.smithi149.stdout:0/6: unlink l0 0 2024-03-22T07:08:40.048 INFO:tasks.workunit.client.1.smithi149.stdout:0/7: dwrite - no filename 2024-03-22T07:08:40.048 INFO:tasks.workunit.client.1.smithi149.stdout:3/0: chown . 172 1 2024-03-22T07:08:40.049 INFO:tasks.workunit.client.1.smithi149.stdout:3/1: chown . 21588843 1 2024-03-22T07:08:40.049 INFO:tasks.workunit.client.1.smithi149.stdout:3/2: dwrite - no filename 2024-03-22T07:08:40.049 INFO:tasks.workunit.client.1.smithi149.stdout:2/1: symlink l0 0 2024-03-22T07:08:40.049 INFO:tasks.workunit.client.1.smithi149.stdout:2/2: chown l0 238960453 1 2024-03-22T07:08:40.050 INFO:tasks.workunit.client.1.smithi149.stdout:1/5: creat f1 x:0 0 0 2024-03-22T07:08:40.052 INFO:tasks.workunit.client.1.smithi149.stdout:4/0: write - no filename 2024-03-22T07:08:40.052 INFO:tasks.workunit.client.1.smithi149.stdout:4/1: fdatasync - no filename 2024-03-22T07:08:40.052 INFO:tasks.workunit.client.1.smithi149.stdout:4/2: read - no filename 2024-03-22T07:08:40.052 INFO:tasks.workunit.client.1.smithi149.stdout:4/3: readlink - no filename 2024-03-22T07:08:40.052 INFO:tasks.workunit.client.1.smithi149.stdout:4/4: stat - no entries 2024-03-22T07:08:40.053 INFO:tasks.workunit.client.0.smithi081.stdout:8/267: dwrite d2/d3/d28/f30 [0,4194304] 0 2024-03-22T07:08:40.053 INFO:tasks.workunit.client.0.smithi081.stdout:8/268: chown f1 10433342 1 2024-03-22T07:08:40.053 INFO:tasks.workunit.client.1.smithi149.stdout:3/3: mkdir d0 0 2024-03-22T07:08:40.054 INFO:tasks.workunit.client.1.smithi149.stdout:3/4: dwrite - no filename 2024-03-22T07:08:40.054 INFO:tasks.workunit.client.1.smithi149.stdout:3/5: chown d0 165 1 2024-03-22T07:08:40.054 INFO:tasks.workunit.client.1.smithi149.stdout:2/3: mkdir d1 0 2024-03-22T07:08:40.056 INFO:tasks.workunit.client.1.smithi149.stdout:4/5: mkdir d0 0 2024-03-22T07:08:40.057 INFO:tasks.workunit.client.1.smithi149.stdout:4/6: write - no filename 2024-03-22T07:08:40.057 INFO:tasks.workunit.client.1.smithi149.stdout:4/7: chown d0 58186 1 2024-03-22T07:08:40.057 INFO:tasks.workunit.client.1.smithi149.stdout:4/8: chown d0 967177 1 2024-03-22T07:08:40.057 INFO:tasks.workunit.client.1.smithi149.stdout:4/9: write - no filename 2024-03-22T07:08:40.057 INFO:tasks.workunit.client.1.smithi149.stdout:4/10: truncate - no filename 2024-03-22T07:08:40.057 INFO:tasks.workunit.client.1.smithi149.stdout:3/6: mknod d0/c1 0 2024-03-22T07:08:40.059 INFO:tasks.workunit.client.1.smithi149.stdout:2/4: rename l0 to d1/l2 0 2024-03-22T07:08:40.062 INFO:tasks.workunit.client.1.smithi149.stdout:5/0: creat f0 x:0 0 0 2024-03-22T07:08:40.062 INFO:tasks.workunit.client.1.smithi149.stdout:5/1: stat f0 0 2024-03-22T07:08:40.062 INFO:tasks.workunit.client.0.smithi081.stdout:8/269: dread d2/d3/d28/f30 [0,4194304] 0 2024-03-22T07:08:40.062 INFO:tasks.workunit.client.0.smithi081.stdout:8/270: creat d2/d3/d19/d1e/d13/f60 x:0 0 0 2024-03-22T07:08:40.063 INFO:tasks.workunit.client.1.smithi149.stdout:4/11: mknod d0/c1 0 2024-03-22T07:08:40.063 INFO:tasks.workunit.client.1.smithi149.stdout:4/12: fsync - no filename 2024-03-22T07:08:40.064 INFO:tasks.workunit.client.0.smithi081.stdout:8/271: mkdir d2/d3/d28/d2b/d61 0 2024-03-22T07:08:40.065 INFO:tasks.workunit.client.1.smithi149.stdout:3/7: rmdir d0 39 2024-03-22T07:08:40.065 INFO:tasks.workunit.client.1.smithi149.stdout:3/8: fdatasync - no filename 2024-03-22T07:08:40.065 INFO:tasks.workunit.client.1.smithi149.stdout:3/9: dwrite - no filename 2024-03-22T07:08:40.065 INFO:tasks.workunit.client.1.smithi149.stdout:3/10: write - no filename 2024-03-22T07:08:40.065 INFO:tasks.workunit.client.1.smithi149.stdout:3/11: dwrite - no filename 2024-03-22T07:08:40.067 INFO:tasks.workunit.client.1.smithi149.stdout:2/5: mkdir d1/d3 0 2024-03-22T07:08:40.068 INFO:tasks.workunit.client.1.smithi149.stdout:0/8: getdents . 0 2024-03-22T07:08:40.068 INFO:tasks.workunit.client.1.smithi149.stdout:0/9: getdents . 0 2024-03-22T07:08:40.068 INFO:tasks.workunit.client.1.smithi149.stdout:0/10: dwrite - no filename 2024-03-22T07:08:40.069 INFO:tasks.workunit.client.1.smithi149.stdout:6/0: creat f0 x:0 0 0 2024-03-22T07:08:40.071 INFO:tasks.workunit.client.1.smithi149.stdout:2/6: creat d1/f4 x:0 0 0 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/0: chown . 330100229 1 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/1: write - no filename 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/2: dwrite - no filename 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/3: getdents . 0 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/4: dread - no filename 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/5: unlink - no file 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/6: unlink - no file 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/7: read - no filename 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/8: fdatasync - no filename 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/9: dwrite - no filename 2024-03-22T07:08:40.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/10: write - no filename 2024-03-22T07:08:40.073 INFO:tasks.workunit.client.1.smithi149.stdout:8/11: read - no filename 2024-03-22T07:08:40.073 INFO:tasks.workunit.client.1.smithi149.stdout:7/0: creat f0 x:0 0 0 2024-03-22T07:08:40.073 INFO:tasks.workunit.client.1.smithi149.stdout:6/1: rename f0 to f1 0 2024-03-22T07:08:40.073 INFO:tasks.workunit.client.1.smithi149.stdout:6/2: rmdir - no directory 2024-03-22T07:08:40.073 INFO:tasks.workunit.client.1.smithi149.stdout:6/3: chown f1 25 1 2024-03-22T07:08:40.074 INFO:tasks.workunit.client.1.smithi149.stdout:2/7: mkdir d1/d3/d5 0 2024-03-22T07:08:40.075 INFO:tasks.workunit.client.1.smithi149.stdout:1/6: dwrite f1 [0,4194304] 0 2024-03-22T07:08:40.077 INFO:tasks.workunit.client.0.smithi081.stdout:8/272: dread d2/d3/f4 [0,4194304] 0 2024-03-22T07:08:40.084 INFO:tasks.workunit.client.0.smithi081.stdout:8/273: symlink d2/d3/d28/d2b/l62 0 2024-03-22T07:08:40.084 INFO:tasks.workunit.client.1.smithi149.stdout:9/0: write - no filename 2024-03-22T07:08:40.084 INFO:tasks.workunit.client.1.smithi149.stdout:9/1: chown . 102 1 2024-03-22T07:08:40.085 INFO:tasks.workunit.client.1.smithi149.stdout:9/2: dread - no filename 2024-03-22T07:08:40.085 INFO:tasks.workunit.client.1.smithi149.stdout:7/1: mkdir d1 0 2024-03-22T07:08:40.085 INFO:tasks.workunit.client.1.smithi149.stdout:7/2: dread - f0 zero size 2024-03-22T07:08:40.085 INFO:tasks.workunit.client.1.smithi149.stdout:8/12: mknod c0 0 2024-03-22T07:08:40.085 INFO:tasks.workunit.client.1.smithi149.stdout:8/13: dwrite - no filename 2024-03-22T07:08:40.085 INFO:tasks.workunit.client.1.smithi149.stdout:6/4: mknod c2 0 2024-03-22T07:08:40.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/8: mkdir d1/d3/d6 0 2024-03-22T07:08:40.096 INFO:tasks.workunit.client.1.smithi149.stdout:1/7: mknod c2 0 2024-03-22T07:08:40.098 INFO:tasks.workunit.client.1.smithi149.stdout:9/3: mknod c0 0 2024-03-22T07:08:40.099 INFO:tasks.workunit.client.1.smithi149.stdout:7/3: creat d1/f2 x:0 0 0 2024-03-22T07:08:40.099 INFO:tasks.workunit.client.1.smithi149.stdout:7/4: chown d1/f2 119270 1 2024-03-22T07:08:40.099 INFO:tasks.workunit.client.1.smithi149.stdout:7/5: dread - d1/f2 zero size 2024-03-22T07:08:40.101 INFO:tasks.workunit.client.1.smithi149.stdout:5/2: fsync f0 0 2024-03-22T07:08:40.106 INFO:tasks.workunit.client.1.smithi149.stdout:6/5: rename c2 to c3 0 2024-03-22T07:08:40.106 INFO:tasks.workunit.client.1.smithi149.stdout:6/6: write f1 [758435,24264] 0 2024-03-22T07:08:40.106 INFO:tasks.workunit.client.1.smithi149.stdout:2/9: creat d1/d3/d6/f7 x:0 0 0 2024-03-22T07:08:40.108 INFO:tasks.workunit.client.1.smithi149.stdout:9/4: symlink l1 0 2024-03-22T07:08:40.109 INFO:tasks.workunit.client.1.smithi149.stdout:1/8: symlink l3 0 2024-03-22T07:08:40.109 INFO:tasks.workunit.client.1.smithi149.stdout:1/9: fdatasync f1 0 2024-03-22T07:08:40.109 INFO:tasks.workunit.client.1.smithi149.stdout:5/3: mknod c1 0 2024-03-22T07:08:40.111 INFO:tasks.workunit.client.1.smithi149.stdout:2/10: mkdir d1/d8 0 2024-03-22T07:08:40.111 INFO:tasks.workunit.client.1.smithi149.stdout:2/11: dread - d1/d3/d6/f7 zero size 2024-03-22T07:08:40.111 INFO:tasks.workunit.client.1.smithi149.stdout:2/12: dread - d1/f4 zero size 2024-03-22T07:08:40.111 INFO:tasks.workunit.client.1.smithi149.stdout:2/13: dread - d1/f4 zero size 2024-03-22T07:08:40.111 INFO:tasks.workunit.client.1.smithi149.stdout:2/14: dread - d1/d3/d6/f7 zero size 2024-03-22T07:08:40.112 INFO:tasks.workunit.client.1.smithi149.stdout:9/5: creat f2 x:0 0 0 2024-03-22T07:08:40.112 INFO:tasks.workunit.client.1.smithi149.stdout:9/6: read - f2 zero size 2024-03-22T07:08:40.115 INFO:tasks.workunit.client.1.smithi149.stdout:2/15: link d1/d3/d6/f7 d1/d3/f9 0 2024-03-22T07:08:40.115 INFO:tasks.workunit.client.1.smithi149.stdout:1/10: mkdir d4 0 2024-03-22T07:08:40.115 INFO:tasks.workunit.client.1.smithi149.stdout:1/11: rename d4 to d4/d5 22 2024-03-22T07:08:40.119 INFO:tasks.workunit.client.1.smithi149.stdout:2/16: mkdir d1/d8/da 0 2024-03-22T07:08:40.119 INFO:tasks.workunit.client.1.smithi149.stdout:2/17: dread - d1/d3/d6/f7 zero size 2024-03-22T07:08:40.119 INFO:tasks.workunit.client.1.smithi149.stdout:2/18: fdatasync d1/d3/f9 0 2024-03-22T07:08:40.566 INFO:tasks.workunit.client.1.smithi149.stdout:8/14: getdents . 0 2024-03-22T07:08:40.629 INFO:tasks.workunit.client.1.smithi149.stdout:7/6: getdents d1 0 2024-03-22T07:08:40.630 INFO:tasks.workunit.client.1.smithi149.stdout:7/7: dread - d1/f2 zero size 2024-03-22T07:08:40.723 INFO:tasks.workunit.client.0.smithi081.stdout:8/274: dwrite d2/d3/d28/f30 [0,4194304] 0 2024-03-22T07:08:40.761 INFO:tasks.workunit.client.0.smithi081.stdout:8/275: unlink d2/d3/d19/d1e/c14 0 2024-03-22T07:08:40.775 INFO:tasks.workunit.client.0.smithi081.stdout:8/276: mkdir d2/d3/d19/d1e/d63 0 2024-03-22T07:08:40.785 INFO:tasks.workunit.client.0.smithi081.stdout:8/277: rmdir d2/d3/d19/d1e/d13/d22/d38 39 2024-03-22T07:08:40.795 INFO:tasks.workunit.client.1.smithi149.stdout:5/4: dwrite f0 [0,4194304] 0 2024-03-22T07:08:40.848 INFO:tasks.workunit.client.1.smithi149.stdout:1/12: read f1 [3788985,54985] 0 2024-03-22T07:08:40.848 INFO:tasks.workunit.client.1.smithi149.stdout:1/13: chown c2 832 1 2024-03-22T07:08:40.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:40 smithi149 ceph-mon[36068]: Deploying cephadm binary to smithi081 2024-03-22T07:08:40.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:40 smithi149 ceph-mon[36068]: Deploying cephadm binary to smithi149 2024-03-22T07:08:40.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:40 smithi149 ceph-mon[36068]: mgrmap e22: smithi149.sgmtoa(active, since 1.01811s) 2024-03-22T07:08:40.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:40 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:40.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:40 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:40.861 INFO:tasks.workunit.client.0.smithi081.stdout:8/278: dread d2/d3/d19/d1e/f41 [0,4194304] 0 2024-03-22T07:08:40.863 INFO:tasks.workunit.client.0.smithi081.stdout:8/279: mkdir d2/d3/d19/d64 0 2024-03-22T07:08:40.870 INFO:tasks.workunit.client.0.smithi081.stdout:8/280: creat d2/d3/d19/d1e/d63/f65 x:0 0 0 2024-03-22T07:08:40.873 INFO:tasks.workunit.client.0.smithi081.stdout:8/281: mkdir d2/d3/d19/d1e/d13/d66 0 2024-03-22T07:08:40.873 INFO:tasks.workunit.client.0.smithi081.stdout:8/282: chown d2/d3/d23/f53 62393305 1 2024-03-22T07:08:40.874 INFO:tasks.workunit.client.0.smithi081.stdout:8/283: fdatasync d2/d3/d23/f53 0 2024-03-22T07:08:40.874 INFO:tasks.workunit.client.0.smithi081.stdout:8/284: creat d2/d3/d28/d2b/f67 x:0 0 0 2024-03-22T07:08:41.003 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:40 smithi081 ceph-mon[29245]: Deploying cephadm binary to smithi081 2024-03-22T07:08:41.003 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:40 smithi081 ceph-mon[29245]: Deploying cephadm binary to smithi149 2024-03-22T07:08:41.003 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:40 smithi081 ceph-mon[29245]: mgrmap e22: smithi149.sgmtoa(active, since 1.01811s) 2024-03-22T07:08:41.003 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:40 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:41.003 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:40 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:41.066 INFO:tasks.workunit.client.1.smithi149.stdout:6/7: dwrite f1 [0,4194304] 0 2024-03-22T07:08:41.074 INFO:tasks.workunit.client.1.smithi149.stdout:9/7: dwrite f2 [0,4194304] 0 2024-03-22T07:08:41.084 INFO:tasks.workunit.client.1.smithi149.stdout:6/8: dread f1 [0,4194304] 0 2024-03-22T07:08:41.217 INFO:tasks.workunit.client.1.smithi149.stdout:8/15: mkdir d1 0 2024-03-22T07:08:41.217 INFO:tasks.workunit.client.1.smithi149.stdout:8/16: chown c0 0 1 2024-03-22T07:08:41.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:41 smithi081 ceph-mon[29245]: pgmap v3: 65 pgs: 65 active+clean; 814 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:08:41.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:41 smithi081 ceph-mon[29245]: [22/Mar/2024:07:08:41] ENGINE Bus STARTING 2024-03-22T07:08:41.821 INFO:tasks.workunit.client.1.smithi149.stdout:9/8: read f2 [2211640,62594] 0 2024-03-22T07:08:41.821 INFO:tasks.workunit.client.1.smithi149.stdout:9/9: chown c0 5226 1 2024-03-22T07:08:41.821 INFO:tasks.workunit.client.1.smithi149.stdout:9/10: chown l1 0 1 2024-03-22T07:08:41.830 INFO:tasks.workunit.client.1.smithi149.stdout:9/11: dread f2 [0,4194304] 0 2024-03-22T07:08:41.839 INFO:tasks.workunit.client.1.smithi149.stdout:9/12: dread f2 [0,4194304] 0 2024-03-22T07:08:41.839 INFO:tasks.workunit.client.1.smithi149.stdout:9/13: write f2 [1584421,96501] 0 2024-03-22T07:08:41.839 INFO:tasks.workunit.client.1.smithi149.stdout:9/14: creat f3 x:0 0 0 2024-03-22T07:08:41.845 INFO:tasks.workunit.client.1.smithi149.stdout:9/15: fsync f2 0 2024-03-22T07:08:41.845 INFO:tasks.workunit.client.1.smithi149.stdout:9/16: dread - f3 zero size 2024-03-22T07:08:41.845 INFO:tasks.workunit.client.1.smithi149.stdout:9/17: fdatasync f2 0 2024-03-22T07:08:41.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:41 smithi149 ceph-mon[36068]: pgmap v3: 65 pgs: 65 active+clean; 814 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:08:41.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:41 smithi149 ceph-mon[36068]: [22/Mar/2024:07:08:41] ENGINE Bus STARTING 2024-03-22T07:08:42.129 INFO:tasks.workunit.client.1.smithi149.stdout:5/5: mknod c2 0 2024-03-22T07:08:42.129 INFO:tasks.workunit.client.1.smithi149.stdout:5/6: chown c1 28672862 1 2024-03-22T07:08:42.207 INFO:tasks.workunit.client.1.smithi149.stdout:1/14: creat d4/f6 x:0 0 0 2024-03-22T07:08:42.211 INFO:tasks.workunit.client.1.smithi149.stdout:1/15: dread f1 [0,4194304] 0 2024-03-22T07:08:42.211 INFO:tasks.workunit.client.1.smithi149.stdout:1/16: write f1 [4368884,95464] 0 2024-03-22T07:08:42.211 INFO:tasks.workunit.client.1.smithi149.stdout:1/17: write d4/f6 [795953,14045] 0 2024-03-22T07:08:42.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:42 smithi149 ceph-mon[36068]: pgmap v4: 65 pgs: 65 active+clean; 814 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:08:42.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:42 smithi149 ceph-mon[36068]: [22/Mar/2024:07:08:41] ENGINE Serving on https://172.21.15.149:7150 2024-03-22T07:08:42.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:42 smithi149 ceph-mon[36068]: [22/Mar/2024:07:08:41] ENGINE Serving on http://172.21.15.149:8765 2024-03-22T07:08:42.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:42 smithi149 ceph-mon[36068]: [22/Mar/2024:07:08:41] ENGINE Bus STARTED 2024-03-22T07:08:42.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:42 smithi149 ceph-mon[36068]: mgrmap e23: smithi149.sgmtoa(active, since 2s) 2024-03-22T07:08:43.027 INFO:tasks.workunit.client.1.smithi149.stdout:6/9: symlink l4 0 2024-03-22T07:08:43.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:42 smithi081 ceph-mon[29245]: pgmap v4: 65 pgs: 65 active+clean; 814 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:08:43.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:42 smithi081 ceph-mon[29245]: [22/Mar/2024:07:08:41] ENGINE Serving on https://172.21.15.149:7150 2024-03-22T07:08:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:42 smithi081 ceph-mon[29245]: [22/Mar/2024:07:08:41] ENGINE Serving on http://172.21.15.149:8765 2024-03-22T07:08:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:42 smithi081 ceph-mon[29245]: [22/Mar/2024:07:08:41] ENGINE Bus STARTED 2024-03-22T07:08:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:42 smithi081 ceph-mon[29245]: mgrmap e23: smithi149.sgmtoa(active, since 2s) 2024-03-22T07:08:43.426 INFO:tasks.workunit.client.1.smithi149.stdout:8/17: mkdir d1/d2 0 2024-03-22T07:08:43.964 INFO:tasks.workunit.client.1.smithi149.stdout:7/8: symlink d1/l3 0 2024-03-22T07:08:43.964 INFO:tasks.workunit.client.1.smithi149.stdout:7/9: chown d1/f2 15 1 2024-03-22T07:08:44.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.479+0000 7f1cbcd91700 1 -- 172.21.15.81:0/942897621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fbd00 msgr2=0x7f1cb80fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:44.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.479+0000 7f1cbcd91700 1 --2- 172.21.15.81:0/942897621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fbd00 0x7f1cb80fc110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8004940 tx=0x7f1ca8009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.479+0000 7f1cbcd91700 1 -- 172.21.15.81:0/942897621 shutdown_connections 2024-03-22T07:08:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.479+0000 7f1cbcd91700 1 --2- 172.21.15.81:0/942897621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb80fcfb0 0x7f1cb80fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.479+0000 7f1cbcd91700 1 --2- 172.21.15.81:0/942897621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fbd00 0x7f1cb80fc110 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.479+0000 7f1cbcd91700 1 -- 172.21.15.81:0/942897621 >> 172.21.15.81:0/942897621 conn(0x7f1cb80f7440 msgr2=0x7f1cb80f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.479+0000 7f1cbcd91700 1 -- 172.21.15.81:0/942897621 shutdown_connections 2024-03-22T07:08:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.479+0000 7f1cbcd91700 1 -- 172.21.15.81:0/942897621 wait complete. 2024-03-22T07:08:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.480+0000 7f1cbcd91700 1 Processor -- start 2024-03-22T07:08:44.482 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.480+0000 7f1cbcd91700 1 -- start start 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.480+0000 7f1cbcd91700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb80fbd00 0x7f1cb810c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.480+0000 7f1cbcd91700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fcfb0 0x7f1cb810ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cbcd91700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb810d2f0 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cbcd91700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb810d460 con 0x7f1cb80fbd00 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cb5d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fcfb0 0x7f1cb810ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cb5d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fcfb0 0x7f1cb810ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39442/0 (socket says 172.21.15.81:39442) 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cb5d9b700 1 -- 172.21.15.81:0/336257526 learned_addr learned my addr 172.21.15.81:0/336257526 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cb659c700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb80fbd00 0x7f1cb810c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cb5d9b700 1 -- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb80fbd00 msgr2=0x7f1cb810c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cb5d9b700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb80fbd00 0x7f1cb810c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cb5d9b700 1 -- 172.21.15.81:0/336257526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8005040 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.481+0000 7f1cb5d9b700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fcfb0 0x7f1cb810ccf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00eb40 tx=0x7f1cac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.482+0000 7f1ca6ffd700 1 -- 172.21.15.81:0/336257526 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cac007cc0 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.482+0000 7f1ca6ffd700 1 -- 172.21.15.81:0/336257526 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cac00f040 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.482+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb8066350 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.482+0000 7f1ca6ffd700 1 -- 172.21.15.81:0/336257526 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cac00ca10 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.482+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb8066890 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.483+0000 7f1ca6ffd700 1 -- 172.21.15.81:0/336257526 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 24) v1 ==== 45771+0+0 (secure 0 0 0) 0x7f1cac00cb70 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.483+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb804c710 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.484+0000 7f1ca6ffd700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f1ca0038840 0x7f1ca003acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:44.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.484+0000 7f1ca6ffd700 1 -- 172.21.15.81:0/336257526 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f1cac04be20 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.484+0000 7f1cb659c700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f1ca0038840 0x7f1ca003acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:44.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.485+0000 7f1cb659c700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f1ca0038840 0x7f1ca003acf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009b30 tx=0x7f1ca8009e30 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:44.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.489+0000 7f1ca6ffd700 1 -- 172.21.15.81:0/336257526 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f1cac012d70 con 0x7f1cb80fcfb0 2024-03-22T07:08:44.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.724+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cb8066e10 con 0x7f1ca0038840 2024-03-22T07:08:44.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.733+0000 7f1ca6ffd700 1 -- 172.21.15.81:0/336257526 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f1cb8066e10 con 0x7f1ca0038840 2024-03-22T07:08:44.734 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:44 smithi081 ceph-mon[29245]: pgmap v5: 65 pgs: 65 active+clean; 814 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:08:44.734 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:44 smithi081 ceph-mon[29245]: mgrmap e24: smithi149.sgmtoa(active, since 4s) 2024-03-22T07:08:44.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f1ca0038840 msgr2=0x7f1ca003acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:44.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f1ca0038840 0x7f1ca003acf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009b30 tx=0x7f1ca8009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:44.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fcfb0 msgr2=0x7f1cb810ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:44.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fcfb0 0x7f1cb810ccf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00eb40 tx=0x7f1cac00b040 comp rx=0 tx=0).stop 2024-03-22T07:08:44.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 shutdown_connections 2024-03-22T07:08:44.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f1ca0038840 0x7f1ca003acf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:44.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cb80fbd00 0x7f1cb810c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:44.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 --2- 172.21.15.81:0/336257526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1cb80fcfb0 0x7f1cb810ccf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:44.736 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 >> 172.21.15.81:0/336257526 conn(0x7f1cb80f7440 msgr2=0x7f1cb80f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:44.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 shutdown_connections 2024-03-22T07:08:44.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.736+0000 7f1cbcd91700 1 -- 172.21.15.81:0/336257526 wait complete. 2024-03-22T07:08:44.747 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:08:44.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:44 smithi149 ceph-mon[36068]: pgmap v5: 65 pgs: 65 active+clean; 814 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:08:44.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:44 smithi149 ceph-mon[36068]: mgrmap e24: smithi149.sgmtoa(active, since 4s) 2024-03-22T07:08:44.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.885+0000 7fd64ee9c700 1 -- 172.21.15.81:0/241518070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6480fbcd0 msgr2=0x7fd6480fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:44.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.885+0000 7fd64ee9c700 1 --2- 172.21.15.81:0/241518070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6480fbcd0 0x7fd6480fc0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd63c004940 tx=0x7fd63c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.885+0000 7fd64ee9c700 1 -- 172.21.15.81:0/241518070 shutdown_connections 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.885+0000 7fd64ee9c700 1 --2- 172.21.15.81:0/241518070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6480fcf80 0x7fd6480fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.885+0000 7fd64ee9c700 1 --2- 172.21.15.81:0/241518070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6480fbcd0 0x7fd6480fc0e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.885+0000 7fd64ee9c700 1 -- 172.21.15.81:0/241518070 >> 172.21.15.81:0/241518070 conn(0x7fd6480f7430 msgr2=0x7fd6480f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.885+0000 7fd64ee9c700 1 -- 172.21.15.81:0/241518070 shutdown_connections 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.885+0000 7fd64ee9c700 1 -- 172.21.15.81:0/241518070 wait complete. 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd64ee9c700 1 Processor -- start 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd64ee9c700 1 -- start start 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd64ee9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6480fbcd0 0x7fd6480ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd64ee9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6480fcf80 0x7fd6480fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd64ee9c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480fe850 con 0x7fd6480fcf80 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd64ee9c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480fe990 con 0x7fd6480fbcd0 2024-03-22T07:08:44.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd647fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6480fcf80 0x7fd6480fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:44.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd647fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6480fcf80 0x7fd6480fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39280/0 (socket says 172.21.15.81:39280) 2024-03-22T07:08:44.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd647fff700 1 -- 172.21.15.81:0/591424307 learned_addr learned my addr 172.21.15.81:0/591424307 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:44.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.886+0000 7fd64cc38700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6480fbcd0 0x7fd6480ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:44.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.887+0000 7fd64cc38700 1 -- 172.21.15.81:0/591424307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6480fcf80 msgr2=0x7fd6480fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:44.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.887+0000 7fd64cc38700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6480fcf80 0x7fd6480fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:44.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.887+0000 7fd64cc38700 1 -- 172.21.15.81:0/591424307 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c005040 con 0x7fd6480fbcd0 2024-03-22T07:08:44.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.887+0000 7fd64cc38700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6480fbcd0 0x7fd6480ffda0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd63c000c00 tx=0x7fd63c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:44.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.887+0000 7fd6457fa700 1 -- 172.21.15.81:0/591424307 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd63c00e030 con 0x7fd6480fbcd0 2024-03-22T07:08:44.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.887+0000 7fd6457fa700 1 -- 172.21.15.81:0/591424307 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd63c013de0 con 0x7fd6480fbcd0 2024-03-22T07:08:44.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.887+0000 7fd6457fa700 1 -- 172.21.15.81:0/591424307 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd63c01ce50 con 0x7fd6480fbcd0 2024-03-22T07:08:44.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.887+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6480fec10 con 0x7fd6480fbcd0 2024-03-22T07:08:44.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.887+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6480ff100 con 0x7fd6480fbcd0 2024-03-22T07:08:44.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.888+0000 7fd6457fa700 1 -- 172.21.15.81:0/591424307 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 45771+0+0 (secure 0 0 0) 0x7fd63c00b4a0 con 0x7fd6480fbcd0 2024-03-22T07:08:44.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.888+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd648101b10 con 0x7fd6480fbcd0 2024-03-22T07:08:44.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.889+0000 7fd6457fa700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fd6380387d0 0x7fd63803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:44.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.889+0000 7fd6457fa700 1 -- 172.21.15.81:0/591424307 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fd63c04db00 con 0x7fd6480fbcd0 2024-03-22T07:08:44.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.889+0000 7fd647fff700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fd6380387d0 0x7fd63803ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:44.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.889+0000 7fd647fff700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fd6380387d0 0x7fd63803ac80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd634008fd0 tx=0x7fd63400b040 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:44.894 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:44.894+0000 7fd6457fa700 1 -- 172.21.15.81:0/591424307 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fd63c025400 con 0x7fd6480fbcd0 2024-03-22T07:08:45.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.143+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd64805e400 con 0x7fd6380387d0 2024-03-22T07:08:45.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.157+0000 7fd6457fa700 1 -- 172.21.15.81:0/591424307 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fd64805e400 con 0x7fd6380387d0 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.159+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fd6380387d0 msgr2=0x7fd63803ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.159+0000 7fd64ee9c700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fd6380387d0 0x7fd63803ac80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd634008fd0 tx=0x7fd63400b040 comp rx=0 tx=0).stop 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.159+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6480fbcd0 msgr2=0x7fd6480ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.159+0000 7fd64ee9c700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6480fbcd0 0x7fd6480ffda0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd63c000c00 tx=0x7fd63c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.159+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 shutdown_connections 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.159+0000 7fd64ee9c700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fd6380387d0 0x7fd63803ac80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.159+0000 7fd64ee9c700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6480fbcd0 0x7fd6480ffda0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.159+0000 7fd64ee9c700 1 --2- 172.21.15.81:0/591424307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6480fcf80 0x7fd6480fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.159+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 >> 172.21.15.81:0/591424307 conn(0x7fd6480f7430 msgr2=0x7fd648065b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.160+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 shutdown_connections 2024-03-22T07:08:45.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.160+0000 7fd64ee9c700 1 -- 172.21.15.81:0/591424307 wait complete. 2024-03-22T07:08:45.202 INFO:tasks.workunit.client.1.smithi149.stdout:9/18: mknod c4 0 2024-03-22T07:08:45.202 INFO:tasks.workunit.client.1.smithi149.stdout:9/19: chown c4 203799472 1 2024-03-22T07:08:45.208 INFO:tasks.workunit.client.1.smithi149.stdout:9/20: dread f2 [0,4194304] 0 2024-03-22T07:08:45.213 INFO:tasks.workunit.client.1.smithi149.stdout:9/21: dread f2 [0,4194304] 0 2024-03-22T07:08:45.213 INFO:tasks.workunit.client.1.smithi149.stdout:9/22: dread - f3 zero size 2024-03-22T07:08:45.213 INFO:tasks.workunit.client.1.smithi149.stdout:9/23: chown f2 12 1 2024-03-22T07:08:45.266 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.266+0000 7f8f2556b700 1 -- 172.21.15.81:0/1039485355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 msgr2=0x7f8f200fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.266+0000 7f8f2556b700 1 --2- 172.21.15.81:0/1039485355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 0x7f8f200fd3e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8f14004940 tx=0x7f8f14009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.266+0000 7f8f2556b700 1 -- 172.21.15.81:0/1039485355 shutdown_connections 2024-03-22T07:08:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.266+0000 7f8f2556b700 1 --2- 172.21.15.81:0/1039485355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 0x7f8f200fd3e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.266+0000 7f8f2556b700 1 --2- 172.21.15.81:0/1039485355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f200fbe00 0x7f8f200fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.266+0000 7f8f2556b700 1 -- 172.21.15.81:0/1039485355 >> 172.21.15.81:0/1039485355 conn(0x7f8f200f74e0 msgr2=0x7f8f200f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.266+0000 7f8f2556b700 1 -- 172.21.15.81:0/1039485355 shutdown_connections 2024-03-22T07:08:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.266+0000 7f8f2556b700 1 -- 172.21.15.81:0/1039485355 wait complete. 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.267+0000 7f8f2556b700 1 Processor -- start 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.267+0000 7f8f2556b700 1 -- start start 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.267+0000 7f8f2556b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f200fbe00 0x7f8f200ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.267+0000 7f8f2556b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 0x7f8f200fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.267+0000 7f8f2556b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200fe7f0 con 0x7f8f200fcf70 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.267+0000 7f8f2556b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200fe930 con 0x7f8f200fbe00 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.267+0000 7f8f1e7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 0x7f8f200fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f1e7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 0x7f8f200fe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39298/0 (socket says 172.21.15.81:39298) 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f1e7fc700 1 -- 172.21.15.81:0/2299059175 learned_addr learned my addr 172.21.15.81:0/2299059175 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f1effd700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f200fbe00 0x7f8f200ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f1e7fc700 1 -- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f200fbe00 msgr2=0x7f8f200ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f1e7fc700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f200fbe00 0x7f8f200ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f1e7fc700 1 -- 172.21.15.81:0/2299059175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f14005040 con 0x7f8f200fcf70 2024-03-22T07:08:45.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f1e7fc700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 0x7f8f200fe2b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f14013870 tx=0x7f8f14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:45.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f077fe700 1 -- 172.21.15.81:0/2299059175 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f14021070 con 0x7f8f200fcf70 2024-03-22T07:08:45.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f200febb0 con 0x7f8f200fcf70 2024-03-22T07:08:45.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f077fe700 1 -- 172.21.15.81:0/2299059175 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f14009430 con 0x7f8f200fcf70 2024-03-22T07:08:45.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.268+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f200ff0f0 con 0x7f8f200fcf70 2024-03-22T07:08:45.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.269+0000 7f8f077fe700 1 -- 172.21.15.81:0/2299059175 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f1400e070 con 0x7f8f200fcf70 2024-03-22T07:08:45.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.270+0000 7f8f077fe700 1 -- 172.21.15.81:0/2299059175 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 24) v1 ==== 45771+0+0 (secure 0 0 0) 0x7f8f14026410 con 0x7f8f200fcf70 2024-03-22T07:08:45.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.270+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c0052b0 con 0x7f8f200fcf70 2024-03-22T07:08:45.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.270+0000 7f8f077fe700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8f080387d0 0x7f8f0803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:45.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.270+0000 7f8f077fe700 1 -- 172.21.15.81:0/2299059175 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8f1401c400 con 0x7f8f200fcf70 2024-03-22T07:08:45.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.270+0000 7f8f1effd700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8f080387d0 0x7f8f0803ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:45.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.271+0000 7f8f1effd700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8f080387d0 0x7f8f0803ac80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f200ffad0 tx=0x7f8f1000b040 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:45.279 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.279+0000 7f8f077fe700 1 -- 172.21.15.81:0/2299059175 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f8f14024e10 con 0x7f8f200fcf70 2024-03-22T07:08:45.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.520+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f0c000bc0 con 0x7f8f080387d0 2024-03-22T07:08:45.528 INFO:tasks.workunit.client.1.smithi149.stdout:5/7: symlink l3 0 2024-03-22T07:08:45.528 INFO:tasks.workunit.client.1.smithi149.stdout:5/8: truncate f0 4431136 0 2024-03-22T07:08:45.528 INFO:tasks.workunit.client.1.smithi149.stdout:5/9: stat c1 0 2024-03-22T07:08:45.530 INFO:tasks.workunit.client.1.smithi149.stdout:1/18: mkdir d4/d7 0 2024-03-22T07:08:45.530 INFO:tasks.workunit.client.1.smithi149.stdout:1/19: chown l3 145628378 1 2024-03-22T07:08:45.530 INFO:tasks.workunit.client.1.smithi149.stdout:1/20: chown c0 313 1 2024-03-22T07:08:45.530 INFO:tasks.workunit.client.1.smithi149.stdout:1/21: creat d4/f8 x:0 0 0 2024-03-22T07:08:45.548 INFO:tasks.workunit.client.1.smithi149.stdout:5/10: dwrite f0 [0,4194304] 0 2024-03-22T07:08:45.548 INFO:tasks.workunit.client.1.smithi149.stdout:5/11: readlink l3 0 2024-03-22T07:08:45.552 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.552+0000 7f8f077fe700 1 -- 172.21.15.81:0/2299059175 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f8f0c000bc0 con 0x7f8f080387d0 2024-03-22T07:08:45.552 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (6m) 115s ago 7m 25.9M - 0.25.0 c8568f914cd2 f83c4e446eae 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (7m) 115s ago 7m 6660k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (6m) 12s ago 6m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (6m) 115s ago 7m 82.4M - 9.4.7 954c08fa6188 5b1c9b6c3c44 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (2m) 115s ago 2m 10.4M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (2m) 115s ago 2m 16.5M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (2m) 12s ago 2m 14.0M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (2m) 12s ago 2m 67.6M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:9283 running (9m) 115s ago 9m 480M - 17.2.7-726-g9c4e0c34 837124da804f 3ef26f13517a 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283 running (16s) 12s ago 6m 51.5M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ab0200c8a736 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (9m) 115s ago 9m 47.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (6m) 12s ago 6m 39.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (7m) 115s ago 7m 13.7M - 1.5.0 0da6a335fe13 c04b2c4a7e70 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (6m) 12s ago 6m 15.4M - 1.5.0 0da6a335fe13 fbdea05540dc 2024-03-22T07:08:45.553 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (5m) 115s ago 5m 45.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:08:45.554 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (5m) 115s ago 5m 46.5M 4288M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:08:45.554 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (5m) 115s ago 5m 43.0M 4288M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:08:45.554 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (4m) 12s ago 4m 132M 1554M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:08:45.554 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (4m) 12s ago 4m 124M 1554M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:08:45.554 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (3m) 12s ago 3m 104M 1554M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:08:45.554 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (6m) 115s ago 7m 56.0M - 2.43.0 a07b618ecd1d 92a93d9b8284 2024-03-22T07:08:45.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8f080387d0 msgr2=0x7f8f0803ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8f080387d0 0x7f8f0803ac80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f200ffad0 tx=0x7f8f1000b040 comp rx=0 tx=0).stop 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 msgr2=0x7f8f200fe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 0x7f8f200fe2b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8f14013870 tx=0x7f8f14009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 shutdown_connections 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8f080387d0 0x7f8f0803ac80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f200fbe00 0x7f8f200ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 --2- 172.21.15.81:0/2299059175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f200fcf70 0x7f8f200fe2b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 >> 172.21.15.81:0/2299059175 conn(0x7f8f200f74e0 msgr2=0x7f8f20066420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 shutdown_connections 2024-03-22T07:08:45.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.555+0000 7f8f2556b700 1 -- 172.21.15.81:0/2299059175 wait complete. 2024-03-22T07:08:45.652 INFO:tasks.workunit.client.1.smithi149.stdout:8/18: mkdir d1/d2/d3 0 2024-03-22T07:08:45.652 INFO:tasks.workunit.client.1.smithi149.stdout:8/19: read - no filename 2024-03-22T07:08:45.652 INFO:tasks.workunit.client.1.smithi149.stdout:8/20: dread - no filename 2024-03-22T07:08:45.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.691+0000 7f33f5837700 1 -- 172.21.15.81:0/1985734440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 msgr2=0x7f33f00fd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:45.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.691+0000 7f33f5837700 1 --2- 172.21.15.81:0/1985734440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 0x7f33f00fd6a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f33e0004940 tx=0x7f33e0009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:45.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.691+0000 7f33f5837700 1 -- 172.21.15.81:0/1985734440 shutdown_connections 2024-03-22T07:08:45.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.691+0000 7f33f5837700 1 --2- 172.21.15.81:0/1985734440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33f00fdbe0 0x7f33f00fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.691+0000 7f33f5837700 1 --2- 172.21.15.81:0/1985734440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 0x7f33f00fd6a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.691+0000 7f33f5837700 1 -- 172.21.15.81:0/1985734440 >> 172.21.15.81:0/1985734440 conn(0x7f33f00f6ba0 msgr2=0x7f33f00f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:45.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.691+0000 7f33f5837700 1 -- 172.21.15.81:0/1985734440 shutdown_connections 2024-03-22T07:08:45.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.691+0000 7f33f5837700 1 -- 172.21.15.81:0/1985734440 wait complete. 2024-03-22T07:08:45.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.692+0000 7f33f5837700 1 Processor -- start 2024-03-22T07:08:45.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.692+0000 7f33f5837700 1 -- start start 2024-03-22T07:08:45.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33f5837700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 0x7f33f010fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:45.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33f5837700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33f0110140 0x7f33f0110f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:45.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33f5837700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f0110600 con 0x7f33f00fd290 2024-03-22T07:08:45.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33f5837700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f0110740 con 0x7f33f0110140 2024-03-22T07:08:45.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33eeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 0x7f33f010fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33eeffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 0x7f33f010fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39326/0 (socket says 172.21.15.81:39326) 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33eeffd700 1 -- 172.21.15.81:0/2138666183 learned_addr learned my addr 172.21.15.81:0/2138666183 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33ee7fc700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33f0110140 0x7f33f0110f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33eeffd700 1 -- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33f0110140 msgr2=0x7f33f0110f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33eeffd700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33f0110140 0x7f33f0110f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33eeffd700 1 -- 172.21.15.81:0/2138666183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e0005040 con 0x7f33f00fd290 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33ee7fc700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33f0110140 0x7f33f0110f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.693+0000 7f33eeffd700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 0x7f33f010fc00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.694+0000 7f33d77fe700 1 -- 172.21.15.81:0/2138666183 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33e000e030 con 0x7f33f00fd290 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.694+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f0111590 con 0x7f33f00fd290 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.694+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f0111aa0 con 0x7f33f00fd290 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.694+0000 7f33d77fe700 1 -- 172.21.15.81:0/2138666183 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33e0013de0 con 0x7f33f00fd290 2024-03-22T07:08:45.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.694+0000 7f33d77fe700 1 -- 172.21.15.81:0/2138666183 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33e001cb70 con 0x7f33f00fd290 2024-03-22T07:08:45.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.695+0000 7f33d77fe700 1 -- 172.21.15.81:0/2138666183 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 24) v1 ==== 45771+0+0 (secure 0 0 0) 0x7f33e001ccd0 con 0x7f33f00fd290 2024-03-22T07:08:45.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.695+0000 7f33d77fe700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f33d8038880 0x7f33d803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:45.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.695+0000 7f33d77fe700 1 -- 172.21.15.81:0/2138666183 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f33e004e7e0 con 0x7f33f00fd290 2024-03-22T07:08:45.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.695+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc0052e0 con 0x7f33f00fd290 2024-03-22T07:08:45.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.696+0000 7f33ee7fc700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f33d8038880 0x7f33d803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:45.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.698+0000 7f33ee7fc700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f33d8038880 0x7f33d803ad30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009b80 tx=0x7f33e400b040 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:45.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:45.700+0000 7f33d77fe700 1 -- 172.21.15.81:0/2138666183 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f33e00125c0 con 0x7f33f00fd290 2024-03-22T07:08:45.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:45 smithi081 ceph-mon[29245]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:45.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:45 smithi081 ceph-mon[29245]: from='client.24369 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:46.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:45 smithi149 ceph-mon[36068]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:46.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:45 smithi149 ceph-mon[36068]: from='client.24369 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:46.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.127+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f33dc005860 con 0x7f33f00fd290 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.127+0000 7f33d77fe700 1 -- 172.21.15.81:0/2138666183 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f33e001a640 con 0x7f33f00fd290 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:08:46.128 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:08:46.129 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:08:46.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:08:46.129 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:08:46.129 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T07:08:46.129 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:08:46.129 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:08:46.129 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:08:46.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f33d8038880 msgr2=0x7f33d803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:46.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f33d8038880 0x7f33d803ad30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009b80 tx=0x7f33e400b040 comp rx=0 tx=0).stop 2024-03-22T07:08:46.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 msgr2=0x7f33f010fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:46.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 0x7f33f010fc00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e0009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:46.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 shutdown_connections 2024-03-22T07:08:46.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f33d8038880 0x7f33d803ad30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33f00fd290 0x7f33f010fc00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 --2- 172.21.15.81:0/2138666183 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33f0110140 0x7f33f0110f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 >> 172.21.15.81:0/2138666183 conn(0x7f33f00f6ba0 msgr2=0x7f33f00fb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:46.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 shutdown_connections 2024-03-22T07:08:46.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.130+0000 7f33f5837700 1 -- 172.21.15.81:0/2138666183 wait complete. 2024-03-22T07:08:46.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.242+0000 7f5481299700 1 -- 172.21.15.81:0/4114739439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c0fff90 msgr2=0x7f547c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:46.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.242+0000 7f5481299700 1 --2- 172.21.15.81:0/4114739439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c0fff90 0x7f547c1023f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5470004940 tx=0x7f5470009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:46.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.242+0000 7f5481299700 1 -- 172.21.15.81:0/4114739439 shutdown_connections 2024-03-22T07:08:46.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.242+0000 7f5481299700 1 --2- 172.21.15.81:0/4114739439 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c0fff90 0x7f547c1023f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.242+0000 7f5481299700 1 --2- 172.21.15.81:0/4114739439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f547c066ba0 0x7f547c0ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.242+0000 7f5481299700 1 -- 172.21.15.81:0/4114739439 >> 172.21.15.81:0/4114739439 conn(0x7f547c0f7480 msgr2=0x7f547c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:46.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.242+0000 7f5481299700 1 -- 172.21.15.81:0/4114739439 shutdown_connections 2024-03-22T07:08:46.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.242+0000 7f5481299700 1 -- 172.21.15.81:0/4114739439 wait complete. 2024-03-22T07:08:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f5481299700 1 Processor -- start 2024-03-22T07:08:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f5481299700 1 -- start start 2024-03-22T07:08:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f5481299700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c066ba0 0x7f547c10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f5481299700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f547c0fff90 0x7f547c10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f5481299700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c10e570 con 0x7f547c066ba0 2024-03-22T07:08:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f5481299700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c10edc0 con 0x7f547c0fff90 2024-03-22T07:08:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f547affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c066ba0 0x7f547c10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f547a7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f547c0fff90 0x7f547c10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f547affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c066ba0 0x7f547c10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39338/0 (socket says 172.21.15.81:39338) 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.243+0000 7f547affd700 1 -- 172.21.15.81:0/997549360 learned_addr learned my addr 172.21.15.81:0/997549360 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.244+0000 7f547affd700 1 -- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f547c0fff90 msgr2=0x7f547c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.244+0000 7f547affd700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f547c0fff90 0x7f547c10df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.244+0000 7f547affd700 1 -- 172.21.15.81:0/997549360 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470005040 con 0x7f547c066ba0 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.244+0000 7f547a7fc700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f547c0fff90 0x7f547c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.244+0000 7f547affd700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c066ba0 0x7f547c10da30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f546c009b80 tx=0x7f546c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.244+0000 7f54637fe700 1 -- 172.21.15.81:0/997549360 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546c004e70 con 0x7f547c066ba0 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.244+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c10f0a0 con 0x7f547c066ba0 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.244+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c10f5e0 con 0x7f547c066ba0 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.244+0000 7f54637fe700 1 -- 172.21.15.81:0/997549360 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f546c00f040 con 0x7f547c066ba0 2024-03-22T07:08:46.246 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.245+0000 7f54637fe700 1 -- 172.21.15.81:0/997549360 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546c00da60 con 0x7f547c066ba0 2024-03-22T07:08:46.247 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.245+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c05f5f0 con 0x7f547c066ba0 2024-03-22T07:08:46.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.250+0000 7f54637fe700 1 -- 172.21.15.81:0/997549360 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 24) v1 ==== 45771+0+0 (secure 0 0 0) 0x7f546c007cc0 con 0x7f547c066ba0 2024-03-22T07:08:46.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.250+0000 7f54637fe700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f546403cc90 0x7f546403f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:46.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.250+0000 7f54637fe700 1 -- 172.21.15.81:0/997549360 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f546c04c690 con 0x7f547c066ba0 2024-03-22T07:08:46.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.250+0000 7f547a7fc700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f546403cc90 0x7f546403f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:46.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.252+0000 7f547a7fc700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f546403cc90 0x7f546403f140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f547000b970 tx=0x7f5470009e30 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:46.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.255+0000 7f54637fe700 1 -- 172.21.15.81:0/997549360 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f546c024920 con 0x7f547c066ba0 2024-03-22T07:08:46.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.534+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f547c10fa20 con 0x7f546403cc90 2024-03-22T07:08:46.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.536+0000 7f54637fe700 1 -- 172.21.15.81:0/997549360 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f547c10fa20 con 0x7f546403cc90 2024-03-22T07:08:46.536 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:08:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:08:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:08:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:08:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [], 2024-03-22T07:08:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "1/2 daemons upgraded", 2024-03-22T07:08:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "", 2024-03-22T07:08:46.537 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:08:46.537 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:08:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.539+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f546403cc90 msgr2=0x7f546403f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.539+0000 7f5481299700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f546403cc90 0x7f546403f140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f547000b970 tx=0x7f5470009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.539+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c066ba0 msgr2=0x7f547c10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:46.539 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.539+0000 7f5481299700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c066ba0 0x7f547c10da30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f546c009b80 tx=0x7f546c004600 comp rx=0 tx=0).stop 2024-03-22T07:08:46.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.539+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 shutdown_connections 2024-03-22T07:08:46.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.539+0000 7f5481299700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f546403cc90 0x7f546403f140 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.539+0000 7f5481299700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f547c066ba0 0x7f547c10da30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.539+0000 7f5481299700 1 --2- 172.21.15.81:0/997549360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f547c0fff90 0x7f547c10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.539+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 >> 172.21.15.81:0/997549360 conn(0x7f547c0f7480 msgr2=0x7f547c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:46.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.540+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 shutdown_connections 2024-03-22T07:08:46.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:08:46.540+0000 7f5481299700 1 -- 172.21.15.81:0/997549360 wait complete. 2024-03-22T07:08:46.564 INFO:tasks.workunit.client.1.smithi149.stdout:9/24: mknod c5 0 2024-03-22T07:08:46.569 INFO:tasks.workunit.client.1.smithi149.stdout:1/22: mknod d4/d7/c9 0 2024-03-22T07:08:46.569 INFO:tasks.workunit.client.1.smithi149.stdout:1/23: fsync d4/f6 0 2024-03-22T07:08:46.569 INFO:tasks.workunit.client.1.smithi149.stdout:1/24: chown d4/f8 75273425 1 2024-03-22T07:08:46.569 INFO:tasks.workunit.client.1.smithi149.stdout:1/25: chown c2 145 1 2024-03-22T07:08:46.572 INFO:tasks.workunit.client.1.smithi149.stdout:5/12: unlink c1 0 2024-03-22T07:08:46.572 INFO:tasks.workunit.client.1.smithi149.stdout:5/13: fdatasync f0 0 2024-03-22T07:08:46.572 INFO:tasks.workunit.client.1.smithi149.stdout:8/21: rmdir d1/d2 39 2024-03-22T07:08:46.573 INFO:tasks.workunit.client.1.smithi149.stdout:9/25: symlink l6 0 2024-03-22T07:08:46.574 INFO:tasks.workunit.client.1.smithi149.stdout:1/26: mknod d4/ca 0 2024-03-22T07:08:46.575 INFO:tasks.workunit.client.1.smithi149.stdout:1/27: creat d4/fb x:0 0 0 2024-03-22T07:08:46.576 INFO:tasks.workunit.client.1.smithi149.stdout:8/22: creat d1/d2/f4 x:0 0 0 2024-03-22T07:08:46.577 INFO:tasks.workunit.client.1.smithi149.stdout:9/26: rename f2 to f7 0 2024-03-22T07:08:46.579 INFO:tasks.workunit.client.1.smithi149.stdout:1/28: creat d4/d7/fc x:0 0 0 2024-03-22T07:08:46.579 INFO:tasks.workunit.client.1.smithi149.stdout:1/29: readlink l3 0 2024-03-22T07:08:46.579 INFO:tasks.workunit.client.1.smithi149.stdout:1/30: truncate d4/f8 763540 0 2024-03-22T07:08:46.582 INFO:tasks.workunit.client.1.smithi149.stdout:1/31: rename c0 to d4/cd 0 2024-03-22T07:08:46.582 INFO:tasks.workunit.client.1.smithi149.stdout:1/32: stat f1 0 2024-03-22T07:08:46.582 INFO:tasks.workunit.client.1.smithi149.stdout:1/33: fdatasync d4/d7/fc 0 2024-03-22T07:08:46.582 INFO:tasks.workunit.client.1.smithi149.stdout:1/34: creat d4/d7/fe x:0 0 0 2024-03-22T07:08:46.583 INFO:tasks.workunit.client.1.smithi149.stdout:1/35: dread - d4/fb zero size 2024-03-22T07:08:46.583 INFO:tasks.workunit.client.1.smithi149.stdout:1/36: dread - d4/fb zero size 2024-03-22T07:08:46.583 INFO:tasks.workunit.client.1.smithi149.stdout:1/37: write d4/f6 [846553,42485] 0 2024-03-22T07:08:46.583 INFO:tasks.workunit.client.1.smithi149.stdout:1/38: dread d4/f8 [0,4194304] 0 2024-03-22T07:08:46.586 INFO:tasks.workunit.client.1.smithi149.stdout:1/39: dread f1 [0,4194304] 0 2024-03-22T07:08:46.587 INFO:tasks.workunit.client.1.smithi149.stdout:1/40: symlink d4/lf 0 2024-03-22T07:08:46.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:46 smithi081 ceph-mon[29245]: pgmap v6: 65 pgs: 65 active+clean; 814 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:08:46.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:46 smithi081 ceph-mon[29245]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:46.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:46 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2138666183' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:08:46.920 INFO:tasks.workunit.client.1.smithi149.stdout:8/23: fsync d1/d2/f4 0 2024-03-22T07:08:46.920 INFO:tasks.workunit.client.1.smithi149.stdout:8/24: write d1/d2/f4 [652451,16907] 0 2024-03-22T07:08:46.920 INFO:tasks.workunit.client.1.smithi149.stdout:9/27: getdents . 0 2024-03-22T07:08:46.920 INFO:tasks.workunit.client.1.smithi149.stdout:9/28: dread - f3 zero size 2024-03-22T07:08:46.922 INFO:tasks.workunit.client.1.smithi149.stdout:8/25: rename d1/d2/f4 to d1/f5 0 2024-03-22T07:08:46.923 INFO:tasks.workunit.client.1.smithi149.stdout:9/29: creat f8 x:0 0 0 2024-03-22T07:08:46.924 INFO:tasks.workunit.client.1.smithi149.stdout:9/30: creat f9 x:0 0 0 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:4/13: sync 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:7/10: sync 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:4/14: dread - no filename 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:6/10: sync 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:3/12: sync 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:3/13: dwrite - no filename 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:3/14: dwrite - no filename 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:2/19: sync 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:0/11: sync 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:2/20: fdatasync d1/f4 0 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:2/21: dread - d1/d3/f9 zero size 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:6/11: write f1 [4934404,98013] 0 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:2/22: getdents d1/d8/da 0 2024-03-22T07:08:46.925 INFO:tasks.workunit.client.1.smithi149.stdout:6/12: creat f5 x:0 0 0 2024-03-22T07:08:46.926 INFO:tasks.workunit.client.1.smithi149.stdout:6/13: creat f6 x:0 0 0 2024-03-22T07:08:46.926 INFO:tasks.workunit.client.1.smithi149.stdout:8/26: creat d1/d2/d3/f6 x:0 0 0 2024-03-22T07:08:46.929 INFO:tasks.workunit.client.1.smithi149.stdout:9/31: mknod ca 0 2024-03-22T07:08:46.929 INFO:tasks.workunit.client.1.smithi149.stdout:9/32: write f9 [139273,113979] 0 2024-03-22T07:08:46.929 INFO:tasks.workunit.client.1.smithi149.stdout:9/33: chown f7 3320 1 2024-03-22T07:08:46.929 INFO:tasks.workunit.client.1.smithi149.stdout:9/34: stat l1 0 2024-03-22T07:08:46.929 INFO:tasks.workunit.client.1.smithi149.stdout:1/41: getdents d4 0 2024-03-22T07:08:46.931 INFO:tasks.workunit.client.1.smithi149.stdout:3/15: mkdir d0/d2 0 2024-03-22T07:08:46.931 INFO:tasks.workunit.client.1.smithi149.stdout:3/16: rename d0 to d0/d3 22 2024-03-22T07:08:46.931 INFO:tasks.workunit.client.1.smithi149.stdout:3/17: truncate - no filename 2024-03-22T07:08:46.933 INFO:tasks.workunit.client.1.smithi149.stdout:0/12: mkdir d1 0 2024-03-22T07:08:46.933 INFO:tasks.workunit.client.1.smithi149.stdout:0/13: dwrite - no filename 2024-03-22T07:08:46.933 INFO:tasks.workunit.client.1.smithi149.stdout:0/14: write - no filename 2024-03-22T07:08:46.933 INFO:tasks.workunit.client.1.smithi149.stdout:0/15: dwrite - no filename 2024-03-22T07:08:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/16: fsync - no filename 2024-03-22T07:08:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/17: dwrite - no filename 2024-03-22T07:08:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/18: write - no filename 2024-03-22T07:08:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/19: dwrite - no filename 2024-03-22T07:08:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/20: fdatasync - no filename 2024-03-22T07:08:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/21: dread - no filename 2024-03-22T07:08:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/22: getdents d1 0 2024-03-22T07:08:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/23: chown d1 27 1 2024-03-22T07:08:46.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/24: dwrite - no filename 2024-03-22T07:08:46.936 INFO:tasks.workunit.client.1.smithi149.stdout:2/23: creat d1/d3/fb x:0 0 0 2024-03-22T07:08:46.939 INFO:tasks.workunit.client.1.smithi149.stdout:6/14: mknod c7 0 2024-03-22T07:08:46.940 INFO:tasks.workunit.client.1.smithi149.stdout:8/27: unlink c0 0 2024-03-22T07:08:46.942 INFO:tasks.workunit.client.1.smithi149.stdout:1/42: mkdir d4/d10 0 2024-03-22T07:08:46.943 INFO:tasks.workunit.client.1.smithi149.stdout:7/11: dwrite f0 [0,4194304] 0 2024-03-22T07:08:46.944 INFO:tasks.workunit.client.1.smithi149.stdout:7/12: chown f0 114984535 1 2024-03-22T07:08:46.944 INFO:tasks.workunit.client.1.smithi149.stdout:3/18: mkdir d0/d4 0 2024-03-22T07:08:46.945 INFO:tasks.workunit.client.1.smithi149.stdout:0/25: symlink d1/l2 0 2024-03-22T07:08:46.945 INFO:tasks.workunit.client.1.smithi149.stdout:0/26: write - no filename 2024-03-22T07:08:46.945 INFO:tasks.workunit.client.1.smithi149.stdout:0/27: truncate - no filename 2024-03-22T07:08:46.947 INFO:tasks.workunit.client.1.smithi149.stdout:2/24: mkdir d1/dc 0 2024-03-22T07:08:46.947 INFO:tasks.workunit.client.1.smithi149.stdout:2/25: dread - d1/f4 zero size 2024-03-22T07:08:46.949 INFO:tasks.workunit.client.1.smithi149.stdout:6/15: link f6 f8 0 2024-03-22T07:08:46.949 INFO:tasks.workunit.client.1.smithi149.stdout:6/16: dread - f8 zero size 2024-03-22T07:08:46.949 INFO:tasks.workunit.client.1.smithi149.stdout:6/17: dread - f5 zero size 2024-03-22T07:08:46.950 INFO:tasks.workunit.client.1.smithi149.stdout:8/28: mknod d1/c7 0 2024-03-22T07:08:46.951 INFO:tasks.workunit.client.1.smithi149.stdout:1/43: creat d4/f11 x:0 0 0 2024-03-22T07:08:46.951 INFO:tasks.workunit.client.1.smithi149.stdout:1/44: write d4/f11 [510221,17580] 0 2024-03-22T07:08:46.951 INFO:tasks.workunit.client.1.smithi149.stdout:1/45: chown d4 21761252 1 2024-03-22T07:08:46.954 INFO:tasks.workunit.client.1.smithi149.stdout:7/13: rename d1/f2 to d1/f4 0 2024-03-22T07:08:46.954 INFO:tasks.workunit.client.1.smithi149.stdout:7/14: stat d1 0 2024-03-22T07:08:46.954 INFO:tasks.workunit.client.1.smithi149.stdout:3/19: symlink d0/d2/l5 0 2024-03-22T07:08:46.955 INFO:tasks.workunit.client.1.smithi149.stdout:3/20: dwrite - no filename 2024-03-22T07:08:46.955 INFO:tasks.workunit.client.1.smithi149.stdout:0/28: mknod d1/c3 0 2024-03-22T07:08:46.955 INFO:tasks.workunit.client.1.smithi149.stdout:0/29: dread - no filename 2024-03-22T07:08:46.955 INFO:tasks.workunit.client.1.smithi149.stdout:0/30: truncate - no filename 2024-03-22T07:08:46.956 INFO:tasks.workunit.client.1.smithi149.stdout:2/26: rename d1/d8/da to d1/d3/dd 0 2024-03-22T07:08:46.958 INFO:tasks.workunit.client.1.smithi149.stdout:1/46: creat d4/d10/f12 x:0 0 0 2024-03-22T07:08:46.961 INFO:tasks.workunit.client.1.smithi149.stdout:3/21: link d0/d2/l5 d0/d4/l6 0 2024-03-22T07:08:46.961 INFO:tasks.workunit.client.1.smithi149.stdout:3/22: truncate - no filename 2024-03-22T07:08:46.961 INFO:tasks.workunit.client.1.smithi149.stdout:3/23: rename d0/d2 to d0/d2/d7 22 2024-03-22T07:08:46.962 INFO:tasks.workunit.client.1.smithi149.stdout:0/31: mknod d1/c4 0 2024-03-22T07:08:46.963 INFO:tasks.workunit.client.1.smithi149.stdout:0/32: stat d1 0 2024-03-22T07:08:46.968 INFO:tasks.workunit.client.1.smithi149.stdout:3/24: mkdir d0/d4/d8 0 2024-03-22T07:08:46.970 INFO:tasks.workunit.client.1.smithi149.stdout:0/33: link d1/c3 d1/c5 0 2024-03-22T07:08:46.972 INFO:tasks.workunit.client.1.smithi149.stdout:6/18: dwrite f8 [0,4194304] 0 2024-03-22T07:08:46.973 INFO:tasks.workunit.client.1.smithi149.stdout:3/25: link d0/d2/l5 d0/l9 0 2024-03-22T07:08:46.977 INFO:tasks.workunit.client.1.smithi149.stdout:3/26: symlink d0/d4/d8/la 0 2024-03-22T07:08:46.989 INFO:tasks.workunit.client.1.smithi149.stdout:6/19: dwrite f8 [0,4194304] 0 2024-03-22T07:08:46.991 INFO:tasks.workunit.client.1.smithi149.stdout:6/20: mkdir d9 0 2024-03-22T07:08:46.991 INFO:tasks.workunit.client.1.smithi149.stdout:6/21: write f5 [1032609,10102] 0 2024-03-22T07:08:46.993 INFO:tasks.workunit.client.1.smithi149.stdout:9/35: fsync f8 0 2024-03-22T07:08:46.994 INFO:tasks.workunit.client.1.smithi149.stdout:9/36: mkdir db 0 2024-03-22T07:08:46.995 INFO:tasks.workunit.client.1.smithi149.stdout:9/37: creat db/fc x:0 0 0 2024-03-22T07:08:46.995 INFO:tasks.workunit.client.1.smithi149.stdout:9/38: write f8 [930097,15423] 0 2024-03-22T07:08:46.995 INFO:tasks.workunit.client.1.smithi149.stdout:9/39: chown c5 3036 1 2024-03-22T07:08:46.998 INFO:tasks.workunit.client.1.smithi149.stdout:8/29: truncate d1/f5 611108 0 2024-03-22T07:08:47.000 INFO:tasks.workunit.client.1.smithi149.stdout:2/27: getdents d1 0 2024-03-22T07:08:47.002 INFO:tasks.workunit.client.1.smithi149.stdout:2/28: rename d1/l2 to d1/d3/le 0 2024-03-22T07:08:47.003 INFO:tasks.workunit.client.1.smithi149.stdout:7/15: truncate f0 2829794 0 2024-03-22T07:08:47.003 INFO:tasks.workunit.client.1.smithi149.stdout:7/16: chown f0 5548 1 2024-03-22T07:08:47.004 INFO:tasks.workunit.client.1.smithi149.stdout:7/17: chown d1/l3 23 1 2024-03-22T07:08:47.004 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:46 smithi149 ceph-mon[36068]: pgmap v6: 65 pgs: 65 active+clean; 814 MiB data, 4.1 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:08:47.004 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:46 smithi149 ceph-mon[36068]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:47.004 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:46 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2138666183' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:08:47.005 INFO:tasks.workunit.client.1.smithi149.stdout:2/29: creat d1/d3/d5/ff x:0 0 0 2024-03-22T07:08:47.006 INFO:tasks.workunit.client.1.smithi149.stdout:2/30: creat d1/f10 x:0 0 0 2024-03-22T07:08:47.006 INFO:tasks.workunit.client.1.smithi149.stdout:2/31: dread - d1/d3/f9 zero size 2024-03-22T07:08:47.012 INFO:tasks.workunit.client.1.smithi149.stdout:6/22: truncate f8 102393 0 2024-03-22T07:08:47.014 INFO:tasks.workunit.client.1.smithi149.stdout:6/23: mkdir d9/da 0 2024-03-22T07:08:47.017 INFO:tasks.workunit.client.1.smithi149.stdout:6/24: mknod d9/da/cb 0 2024-03-22T07:08:47.024 INFO:tasks.workunit.client.1.smithi149.stdout:6/25: write f1 [1956325,74615] 0 2024-03-22T07:08:47.026 INFO:tasks.workunit.client.1.smithi149.stdout:6/26: rmdir d9 39 2024-03-22T07:08:47.028 INFO:tasks.workunit.client.1.smithi149.stdout:9/40: dwrite f7 [0,4194304] 0 2024-03-22T07:08:47.028 INFO:tasks.workunit.client.1.smithi149.stdout:9/41: creat db/fd x:0 0 0 2024-03-22T07:08:47.028 INFO:tasks.workunit.client.1.smithi149.stdout:9/42: chown db 24529126 1 2024-03-22T07:08:47.028 INFO:tasks.workunit.client.1.smithi149.stdout:9/43: chown db/fc 24279431 1 2024-03-22T07:08:47.032 INFO:tasks.workunit.client.1.smithi149.stdout:8/30: dwrite d1/d2/d3/f6 [0,4194304] 0 2024-03-22T07:08:47.033 INFO:tasks.workunit.client.1.smithi149.stdout:6/27: truncate f8 751837 0 2024-03-22T07:08:47.033 INFO:tasks.workunit.client.1.smithi149.stdout:6/28: stat d9/da/cb 0 2024-03-22T07:08:47.033 INFO:tasks.workunit.client.1.smithi149.stdout:6/29: chown d9/da/cb 22 1 2024-03-22T07:08:47.035 INFO:tasks.workunit.client.1.smithi149.stdout:8/31: symlink d1/l8 0 2024-03-22T07:08:47.036 INFO:tasks.workunit.client.1.smithi149.stdout:6/30: creat d9/fc x:0 0 0 2024-03-22T07:08:47.037 INFO:tasks.workunit.client.1.smithi149.stdout:9/44: getdents db 0 2024-03-22T07:08:47.037 INFO:tasks.workunit.client.1.smithi149.stdout:9/45: truncate f3 1045750 0 2024-03-22T07:08:47.038 INFO:tasks.workunit.client.1.smithi149.stdout:9/46: creat db/fe x:0 0 0 2024-03-22T07:08:47.040 INFO:tasks.workunit.client.1.smithi149.stdout:2/32: dwrite d1/f10 [0,4194304] 0 2024-03-22T07:08:47.040 INFO:tasks.workunit.client.1.smithi149.stdout:2/33: chown d1/d3 686433865 1 2024-03-22T07:08:47.041 INFO:tasks.workunit.client.1.smithi149.stdout:2/34: write d1/d3/f9 [305799,73088] 0 2024-03-22T07:08:47.041 INFO:tasks.workunit.client.1.smithi149.stdout:7/18: dwrite f0 [0,4194304] 0 2024-03-22T07:08:47.042 INFO:tasks.workunit.client.1.smithi149.stdout:6/31: getdents d9 0 2024-03-22T07:08:47.053 INFO:tasks.workunit.client.1.smithi149.stdout:8/32: write d1/d2/d3/f6 [2462472,120961] 0 2024-03-22T07:08:47.054 INFO:tasks.workunit.client.1.smithi149.stdout:2/35: dread d1/d3/f9 [0,4194304] 0 2024-03-22T07:08:47.054 INFO:tasks.workunit.client.1.smithi149.stdout:8/33: mkdir d1/d2/d9 0 2024-03-22T07:08:47.055 INFO:tasks.workunit.client.1.smithi149.stdout:2/36: write d1/f10 [4892056,20627] 0 2024-03-22T07:08:47.057 INFO:tasks.workunit.client.1.smithi149.stdout:2/37: symlink d1/d8/l11 0 2024-03-22T07:08:47.059 INFO:tasks.workunit.client.1.smithi149.stdout:6/32: dread f1 [4194304,4194304] 0 2024-03-22T07:08:47.059 INFO:tasks.workunit.client.1.smithi149.stdout:2/38: mknod d1/d3/d6/c12 0 2024-03-22T07:08:47.059 INFO:tasks.workunit.client.1.smithi149.stdout:2/39: write d1/f4 [14305,104614] 0 2024-03-22T07:08:47.059 INFO:tasks.workunit.client.1.smithi149.stdout:2/40: chown d1/d3/d6/c12 61 1 2024-03-22T07:08:47.059 INFO:tasks.workunit.client.1.smithi149.stdout:2/41: chown d1/f10 1095678774 1 2024-03-22T07:08:47.059 INFO:tasks.workunit.client.1.smithi149.stdout:2/42: creat d1/d3/f13 x:0 0 0 2024-03-22T07:08:47.060 INFO:tasks.workunit.client.1.smithi149.stdout:6/33: rmdir d9 39 2024-03-22T07:08:47.060 INFO:tasks.workunit.client.1.smithi149.stdout:6/34: chown f5 43665 1 2024-03-22T07:08:47.062 INFO:tasks.workunit.client.1.smithi149.stdout:2/43: symlink d1/d8/l14 0 2024-03-22T07:08:47.063 INFO:tasks.workunit.client.1.smithi149.stdout:2/44: mknod d1/d3/c15 0 2024-03-22T07:08:47.072 INFO:tasks.workunit.client.1.smithi149.stdout:5/14: sync 2024-03-22T07:08:47.072 INFO:tasks.workunit.client.1.smithi149.stdout:5/15: fdatasync f0 0 2024-03-22T07:08:47.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/34: fdatasync d1/d2/d3/f6 0 2024-03-22T07:08:47.073 INFO:tasks.workunit.client.1.smithi149.stdout:9/47: dwrite f9 [0,4194304] 0 2024-03-22T07:08:47.073 INFO:tasks.workunit.client.1.smithi149.stdout:7/19: dwrite d1/f4 [0,4194304] 0 2024-03-22T07:08:47.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/45: dread d1/f10 [0,4194304] 0 2024-03-22T07:08:47.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/46: read - d1/d3/f13 zero size 2024-03-22T07:08:47.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/47: creat d1/d3/d6/f16 x:0 0 0 2024-03-22T07:08:47.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/48: fsync d1/f4 0 2024-03-22T07:08:47.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/49: write d1/d3/d5/ff [418380,84234] 0 2024-03-22T07:08:47.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/50: dread - d1/d3/fb zero size 2024-03-22T07:08:47.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/51: dread - d1/d3/f13 zero size 2024-03-22T07:08:47.085 INFO:tasks.workunit.client.1.smithi149.stdout:2/52: dread - d1/d3/fb zero size 2024-03-22T07:08:47.085 INFO:tasks.workunit.client.1.smithi149.stdout:9/48: mknod db/cf 0 2024-03-22T07:08:47.086 INFO:tasks.workunit.client.1.smithi149.stdout:2/53: write d1/d3/d5/ff [214311,11923] 0 2024-03-22T07:08:47.086 INFO:tasks.workunit.client.1.smithi149.stdout:9/49: write f3 [482301,6619] 0 2024-03-22T07:08:47.089 INFO:tasks.workunit.client.1.smithi149.stdout:9/50: dread - db/fd zero size 2024-03-22T07:08:47.089 INFO:tasks.workunit.client.1.smithi149.stdout:9/51: stat f3 0 2024-03-22T07:08:47.093 INFO:tasks.workunit.client.1.smithi149.stdout:8/35: dwrite d1/f5 [0,4194304] 0 2024-03-22T07:08:47.093 INFO:tasks.workunit.client.1.smithi149.stdout:8/36: stat d1 0 2024-03-22T07:08:47.093 INFO:tasks.workunit.client.1.smithi149.stdout:8/37: read d1/d2/d3/f6 [1000664,9622] 0 2024-03-22T07:08:47.093 INFO:tasks.workunit.client.1.smithi149.stdout:8/38: chown d1 77 1 2024-03-22T07:08:47.094 INFO:tasks.workunit.client.1.smithi149.stdout:9/52: mknod db/c10 0 2024-03-22T07:08:47.097 INFO:tasks.workunit.client.1.smithi149.stdout:8/39: mkdir d1/d2/d9/da 0 2024-03-22T07:08:47.097 INFO:tasks.workunit.client.1.smithi149.stdout:7/20: dread f0 [0,4194304] 0 2024-03-22T07:08:47.097 INFO:tasks.workunit.client.1.smithi149.stdout:8/40: readlink d1/l8 0 2024-03-22T07:08:47.099 INFO:tasks.workunit.client.1.smithi149.stdout:7/21: rename d1/f4 to d1/f5 0 2024-03-22T07:08:47.107 INFO:tasks.workunit.client.1.smithi149.stdout:7/22: write f0 [976749,63519] 0 2024-03-22T07:08:47.107 INFO:tasks.workunit.client.1.smithi149.stdout:7/23: read f0 [3699426,123326] 0 2024-03-22T07:08:47.107 INFO:tasks.workunit.client.1.smithi149.stdout:9/53: write f9 [2108847,51588] 0 2024-03-22T07:08:47.107 INFO:tasks.workunit.client.1.smithi149.stdout:8/41: dread d1/d2/d3/f6 [0,4194304] 0 2024-03-22T07:08:47.108 INFO:tasks.workunit.client.1.smithi149.stdout:7/24: truncate d1/f5 2281971 0 2024-03-22T07:08:47.108 INFO:tasks.workunit.client.1.smithi149.stdout:7/25: chown f0 2878 1 2024-03-22T07:08:47.110 INFO:tasks.workunit.client.1.smithi149.stdout:9/54: dread - db/fc zero size 2024-03-22T07:08:47.110 INFO:tasks.workunit.client.1.smithi149.stdout:9/55: readlink l6 0 2024-03-22T07:08:47.110 INFO:tasks.workunit.client.1.smithi149.stdout:9/56: write db/fe [286928,89555] 0 2024-03-22T07:08:47.110 INFO:tasks.workunit.client.1.smithi149.stdout:9/57: chown db/fe 756 1 2024-03-22T07:08:47.111 INFO:tasks.workunit.client.1.smithi149.stdout:7/26: unlink f0 0 2024-03-22T07:08:47.113 INFO:tasks.workunit.client.1.smithi149.stdout:9/58: unlink c0 0 2024-03-22T07:08:47.114 INFO:tasks.workunit.client.1.smithi149.stdout:7/27: mkdir d1/d6 0 2024-03-22T07:08:47.116 INFO:tasks.workunit.client.1.smithi149.stdout:2/54: dwrite d1/f10 [0,4194304] 0 2024-03-22T07:08:47.116 INFO:tasks.workunit.client.1.smithi149.stdout:2/55: read d1/f4 [38162,25223] 0 2024-03-22T07:08:47.116 INFO:tasks.workunit.client.1.smithi149.stdout:2/56: write d1/d3/fb [374500,18564] 0 2024-03-22T07:08:47.117 INFO:tasks.workunit.client.1.smithi149.stdout:7/28: rmdir d1 39 2024-03-22T07:08:47.118 INFO:tasks.workunit.client.1.smithi149.stdout:9/59: symlink db/l11 0 2024-03-22T07:08:47.118 INFO:tasks.workunit.client.1.smithi149.stdout:7/29: link d1/l3 d1/d6/l7 0 2024-03-22T07:08:47.118 INFO:tasks.workunit.client.1.smithi149.stdout:7/30: stat d1 0 2024-03-22T07:08:47.119 INFO:tasks.workunit.client.1.smithi149.stdout:5/16: dwrite f0 [4194304,4194304] 0 2024-03-22T07:08:47.119 INFO:tasks.workunit.client.1.smithi149.stdout:5/17: chown f0 353718 1 2024-03-22T07:08:47.119 INFO:tasks.workunit.client.1.smithi149.stdout:2/57: getdents d1/d3/d6 0 2024-03-22T07:08:47.119 INFO:tasks.workunit.client.1.smithi149.stdout:2/58: chown d1/d3/f9 25473106 1 2024-03-22T07:08:47.120 INFO:tasks.workunit.client.1.smithi149.stdout:9/60: mkdir db/d12 0 2024-03-22T07:08:47.120 INFO:tasks.workunit.client.1.smithi149.stdout:9/61: write db/fd [179552,57711] 0 2024-03-22T07:08:47.121 INFO:tasks.workunit.client.1.smithi149.stdout:9/62: read f3 [449236,40760] 0 2024-03-22T07:08:47.124 INFO:tasks.workunit.client.1.smithi149.stdout:2/59: creat d1/d8/f17 x:0 0 0 2024-03-22T07:08:47.125 INFO:tasks.workunit.client.1.smithi149.stdout:2/60: write d1/d3/fb [1111893,8974] 0 2024-03-22T07:08:47.127 INFO:tasks.workunit.client.1.smithi149.stdout:5/18: getdents . 0 2024-03-22T07:08:47.127 INFO:tasks.workunit.client.1.smithi149.stdout:5/19: chown c2 1567 1 2024-03-22T07:08:47.131 INFO:tasks.workunit.client.1.smithi149.stdout:8/42: dwrite d1/d2/d3/f6 [0,4194304] 0 2024-03-22T07:08:47.131 INFO:tasks.workunit.client.1.smithi149.stdout:9/63: link db/l11 db/l13 0 2024-03-22T07:08:47.133 INFO:tasks.workunit.client.1.smithi149.stdout:2/61: getdents d1/d3/d5 0 2024-03-22T07:08:47.141 INFO:tasks.workunit.client.1.smithi149.stdout:8/43: dread d1/f5 [0,4194304] 0 2024-03-22T07:08:47.142 INFO:tasks.workunit.client.1.smithi149.stdout:5/20: read f0 [5094441,26926] 0 2024-03-22T07:08:47.144 INFO:tasks.workunit.client.1.smithi149.stdout:8/44: rename d1/f5 to d1/d2/d9/da/fb 0 2024-03-22T07:08:47.145 INFO:tasks.workunit.client.1.smithi149.stdout:7/31: dwrite d1/f5 [0,4194304] 0 2024-03-22T07:08:47.147 INFO:tasks.workunit.client.1.smithi149.stdout:5/21: creat f4 x:0 0 0 2024-03-22T07:08:47.147 INFO:tasks.workunit.client.1.smithi149.stdout:5/22: dread - f4 zero size 2024-03-22T07:08:47.147 INFO:tasks.workunit.client.1.smithi149.stdout:5/23: chown c2 0 1 2024-03-22T07:08:47.149 INFO:tasks.workunit.client.1.smithi149.stdout:5/24: symlink l5 0 2024-03-22T07:08:47.149 INFO:tasks.workunit.client.1.smithi149.stdout:5/25: creat f6 x:0 0 0 2024-03-22T07:08:47.150 INFO:tasks.workunit.client.1.smithi149.stdout:5/26: mkdir d7 0 2024-03-22T07:08:47.152 INFO:tasks.workunit.client.1.smithi149.stdout:5/27: unlink f4 0 2024-03-22T07:08:47.152 INFO:tasks.workunit.client.1.smithi149.stdout:5/28: dread - f6 zero size 2024-03-22T07:08:47.153 INFO:tasks.workunit.client.1.smithi149.stdout:5/29: rename f6 to d7/f8 0 2024-03-22T07:08:47.153 INFO:tasks.workunit.client.1.smithi149.stdout:8/45: write d1/d2/d3/f6 [3527579,121145] 0 2024-03-22T07:08:47.154 INFO:tasks.workunit.client.1.smithi149.stdout:2/62: dwrite d1/d3/d6/f16 [0,4194304] 0 2024-03-22T07:08:47.154 INFO:tasks.workunit.client.1.smithi149.stdout:5/30: rmdir d7 39 2024-03-22T07:08:47.155 INFO:tasks.workunit.client.1.smithi149.stdout:8/46: fsync d1/d2/d9/da/fb 0 2024-03-22T07:08:47.156 INFO:tasks.workunit.client.1.smithi149.stdout:7/32: dread d1/f5 [0,4194304] 0 2024-03-22T07:08:47.156 INFO:tasks.workunit.client.1.smithi149.stdout:2/63: symlink d1/d8/l18 0 2024-03-22T07:08:47.157 INFO:tasks.workunit.client.1.smithi149.stdout:2/64: creat d1/d8/f19 x:0 0 0 2024-03-22T07:08:47.157 INFO:tasks.workunit.client.1.smithi149.stdout:2/65: getdents d1/d3/dd 0 2024-03-22T07:08:47.157 INFO:tasks.workunit.client.1.smithi149.stdout:5/31: creat d7/f9 x:0 0 0 2024-03-22T07:08:47.158 INFO:tasks.workunit.client.1.smithi149.stdout:7/33: creat d1/d6/f8 x:0 0 0 2024-03-22T07:08:47.161 INFO:tasks.workunit.client.1.smithi149.stdout:7/34: rename d1/l3 to d1/l9 0 2024-03-22T07:08:47.164 INFO:tasks.workunit.client.1.smithi149.stdout:8/47: read d1/d2/d9/da/fb [567013,116285] 0 2024-03-22T07:08:47.167 INFO:tasks.workunit.client.1.smithi149.stdout:2/66: dread d1/d3/d6/f16 [0,4194304] 0 2024-03-22T07:08:47.171 INFO:tasks.workunit.client.1.smithi149.stdout:7/35: dread d1/f5 [0,4194304] 0 2024-03-22T07:08:47.172 INFO:tasks.workunit.client.1.smithi149.stdout:7/36: symlink d1/d6/la 0 2024-03-22T07:08:47.175 INFO:tasks.workunit.client.1.smithi149.stdout:2/67: dread d1/d3/d6/f16 [0,4194304] 0 2024-03-22T07:08:47.180 INFO:tasks.workunit.client.1.smithi149.stdout:5/32: dwrite f0 [0,4194304] 0 2024-03-22T07:08:47.183 INFO:tasks.workunit.client.1.smithi149.stdout:5/33: mkdir d7/da 0 2024-03-22T07:08:47.183 INFO:tasks.workunit.client.1.smithi149.stdout:5/34: chown d7 3027 1 2024-03-22T07:08:47.185 INFO:tasks.workunit.client.1.smithi149.stdout:5/35: mkdir d7/db 0 2024-03-22T07:08:47.185 INFO:tasks.workunit.client.1.smithi149.stdout:8/48: dread d1/d2/d3/f6 [0,4194304] 0 2024-03-22T07:08:47.187 INFO:tasks.workunit.client.1.smithi149.stdout:5/36: creat d7/db/fc x:0 0 0 2024-03-22T07:08:47.187 INFO:tasks.workunit.client.1.smithi149.stdout:5/37: dread - d7/f9 zero size 2024-03-22T07:08:47.188 INFO:tasks.workunit.client.1.smithi149.stdout:8/49: rename d1/l8 to d1/d2/d9/lc 0 2024-03-22T07:08:47.188 INFO:tasks.workunit.client.1.smithi149.stdout:8/50: creat d1/d2/d3/fd x:0 0 0 2024-03-22T07:08:47.188 INFO:tasks.workunit.client.1.smithi149.stdout:8/51: readlink d1/d2/d9/lc 0 2024-03-22T07:08:47.190 INFO:tasks.workunit.client.1.smithi149.stdout:8/52: mkdir d1/d2/d9/de 0 2024-03-22T07:08:47.190 INFO:tasks.workunit.client.1.smithi149.stdout:7/37: dwrite d1/d6/f8 [0,4194304] 0 2024-03-22T07:08:47.190 INFO:tasks.workunit.client.1.smithi149.stdout:8/53: write d1/d2/d3/fd [292520,129879] 0 2024-03-22T07:08:47.190 INFO:tasks.workunit.client.1.smithi149.stdout:8/54: stat d1 0 2024-03-22T07:08:47.192 INFO:tasks.workunit.client.1.smithi149.stdout:8/55: truncate d1/d2/d9/da/fb 281078 0 2024-03-22T07:08:47.193 INFO:tasks.workunit.client.1.smithi149.stdout:8/56: rename d1/d2/d9/de to d1/d2/d9/df 0 2024-03-22T07:08:47.193 INFO:tasks.workunit.client.1.smithi149.stdout:8/57: write d1/d2/d3/f6 [3338045,83739] 0 2024-03-22T07:08:47.194 INFO:tasks.workunit.client.1.smithi149.stdout:8/58: mknod d1/d2/d9/da/c10 0 2024-03-22T07:08:47.195 INFO:tasks.workunit.client.1.smithi149.stdout:8/59: mkdir d1/d2/d9/d11 0 2024-03-22T07:08:47.195 INFO:tasks.workunit.client.1.smithi149.stdout:8/60: chown d1/d2/d9/d11 2613 1 2024-03-22T07:08:47.196 INFO:tasks.workunit.client.1.smithi149.stdout:8/61: symlink d1/d2/l12 0 2024-03-22T07:08:47.196 INFO:tasks.workunit.client.1.smithi149.stdout:8/62: mknod d1/d2/d9/d11/c13 0 2024-03-22T07:08:47.196 INFO:tasks.workunit.client.1.smithi149.stdout:8/63: chown d1/d2/d9 2054 1 2024-03-22T07:08:47.197 INFO:tasks.workunit.client.1.smithi149.stdout:8/64: symlink d1/d2/d9/da/l14 0 2024-03-22T07:08:47.198 INFO:tasks.workunit.client.1.smithi149.stdout:2/68: dwrite d1/d3/d6/f7 [0,4194304] 0 2024-03-22T07:08:47.198 INFO:tasks.workunit.client.1.smithi149.stdout:2/69: write d1/f4 [442023,116123] 0 2024-03-22T07:08:47.199 INFO:tasks.workunit.client.1.smithi149.stdout:2/70: truncate d1/d3/f13 75595 0 2024-03-22T07:08:47.199 INFO:tasks.workunit.client.1.smithi149.stdout:5/38: getdents d7 0 2024-03-22T07:08:47.202 INFO:tasks.workunit.client.1.smithi149.stdout:7/38: dread d1/f5 [0,4194304] 0 2024-03-22T07:08:47.202 INFO:tasks.workunit.client.1.smithi149.stdout:7/39: rename d1 to d1/d6/db 22 2024-03-22T07:08:47.202 INFO:tasks.workunit.client.1.smithi149.stdout:7/40: write d1/f5 [4770817,75538] 0 2024-03-22T07:08:47.203 INFO:tasks.workunit.client.1.smithi149.stdout:5/39: creat d7/da/fd x:0 0 0 2024-03-22T07:08:47.203 INFO:tasks.workunit.client.1.smithi149.stdout:7/41: symlink d1/lc 0 2024-03-22T07:08:47.205 INFO:tasks.workunit.client.1.smithi149.stdout:5/40: rename f0 to d7/da/fe 0 2024-03-22T07:08:47.213 INFO:tasks.workunit.client.1.smithi149.stdout:7/42: dread d1/f5 [4194304,4194304] 0 2024-03-22T07:08:47.221 INFO:tasks.workunit.client.1.smithi149.stdout:7/43: getdents d1 0 2024-03-22T07:08:47.231 INFO:tasks.workunit.client.1.smithi149.stdout:5/41: dwrite d7/da/fe [0,4194304] 0 2024-03-22T07:08:47.232 INFO:tasks.workunit.client.1.smithi149.stdout:5/42: rename l3 to d7/da/lf 0 2024-03-22T07:08:47.234 INFO:tasks.workunit.client.1.smithi149.stdout:5/43: link c2 d7/c10 0 2024-03-22T07:08:47.234 INFO:tasks.workunit.client.1.smithi149.stdout:5/44: readlink d7/da/lf 0 2024-03-22T07:08:47.234 INFO:tasks.workunit.client.1.smithi149.stdout:5/45: write d7/f9 [290042,127225] 0 2024-03-22T07:08:47.234 INFO:tasks.workunit.client.1.smithi149.stdout:5/46: write d7/db/fc [571877,117639] 0 2024-03-22T07:08:47.254 INFO:tasks.workunit.client.1.smithi149.stdout:2/71: dwrite d1/d3/fb [0,4194304] 0 2024-03-22T07:08:47.254 INFO:tasks.workunit.client.1.smithi149.stdout:8/65: dwrite d1/d2/d3/fd [0,4194304] 0 2024-03-22T07:08:47.254 INFO:tasks.workunit.client.1.smithi149.stdout:7/44: dwrite d1/d6/f8 [0,4194304] 0 2024-03-22T07:08:47.254 INFO:tasks.workunit.client.1.smithi149.stdout:7/45: creat d1/d6/fd x:0 0 0 2024-03-22T07:08:47.254 INFO:tasks.workunit.client.1.smithi149.stdout:7/46: readlink d1/lc 0 2024-03-22T07:08:47.256 INFO:tasks.workunit.client.1.smithi149.stdout:2/72: unlink d1/d8/l11 0 2024-03-22T07:08:47.256 INFO:tasks.workunit.client.1.smithi149.stdout:2/73: write d1/f4 [1481137,120617] 0 2024-03-22T07:08:47.256 INFO:tasks.workunit.client.1.smithi149.stdout:2/74: creat d1/f1a x:0 0 0 2024-03-22T07:08:47.262 INFO:tasks.workunit.client.1.smithi149.stdout:8/66: unlink d1/c7 0 2024-03-22T07:08:47.270 INFO:tasks.workunit.client.1.smithi149.stdout:2/75: creat d1/dc/f1b x:0 0 0 2024-03-22T07:08:47.271 INFO:tasks.workunit.client.1.smithi149.stdout:7/47: rmdir d1/d6 39 2024-03-22T07:08:47.273 INFO:tasks.workunit.client.1.smithi149.stdout:2/76: link d1/f1a d1/d3/d6/f1c 0 2024-03-22T07:08:47.273 INFO:tasks.workunit.client.1.smithi149.stdout:2/77: truncate d1/d3/f13 189847 0 2024-03-22T07:08:47.273 INFO:tasks.workunit.client.1.smithi149.stdout:7/48: mknod d1/ce 0 2024-03-22T07:08:47.274 INFO:tasks.workunit.client.1.smithi149.stdout:2/78: symlink d1/d3/l1d 0 2024-03-22T07:08:47.274 INFO:tasks.workunit.client.1.smithi149.stdout:2/79: chown d1/f4 7 1 2024-03-22T07:08:47.275 INFO:tasks.workunit.client.1.smithi149.stdout:7/49: mknod d1/d6/cf 0 2024-03-22T07:08:47.276 INFO:tasks.workunit.client.1.smithi149.stdout:7/50: creat d1/f10 x:0 0 0 2024-03-22T07:08:47.276 INFO:tasks.workunit.client.1.smithi149.stdout:7/51: chown d1/d6/cf 13065824 1 2024-03-22T07:08:47.285 INFO:tasks.workunit.client.1.smithi149.stdout:7/52: dread d1/f5 [0,4194304] 0 2024-03-22T07:08:47.287 INFO:tasks.workunit.client.1.smithi149.stdout:7/53: mknod d1/d6/c11 0 2024-03-22T07:08:47.287 INFO:tasks.workunit.client.1.smithi149.stdout:7/54: read d1/f5 [4846050,123714] 0 2024-03-22T07:08:47.287 INFO:tasks.workunit.client.1.smithi149.stdout:7/55: fsync d1/d6/f8 0 2024-03-22T07:08:47.287 INFO:tasks.workunit.client.1.smithi149.stdout:7/56: chown d1/lc 45406 1 2024-03-22T07:08:47.289 INFO:tasks.workunit.client.1.smithi149.stdout:1/47: sync 2024-03-22T07:08:47.289 INFO:tasks.workunit.client.1.smithi149.stdout:4/15: sync 2024-03-22T07:08:47.289 INFO:tasks.workunit.client.1.smithi149.stdout:1/48: chown d4/f8 2816 1 2024-03-22T07:08:47.289 INFO:tasks.workunit.client.1.smithi149.stdout:1/49: creat d4/d10/f13 x:0 0 0 2024-03-22T07:08:47.290 INFO:tasks.workunit.client.1.smithi149.stdout:4/16: mknod d0/c2 0 2024-03-22T07:08:47.291 INFO:tasks.workunit.client.1.smithi149.stdout:1/50: write d4/d7/fc [682769,128913] 0 2024-03-22T07:08:47.292 INFO:tasks.workunit.client.1.smithi149.stdout:4/17: creat d0/f3 x:0 0 0 2024-03-22T07:08:47.292 INFO:tasks.workunit.client.1.smithi149.stdout:4/18: rename d0 to d0/d4 22 2024-03-22T07:08:47.292 INFO:tasks.workunit.client.1.smithi149.stdout:4/19: read - d0/f3 zero size 2024-03-22T07:08:47.292 INFO:tasks.workunit.client.1.smithi149.stdout:1/51: chown d4/ca 89148318 1 2024-03-22T07:08:47.292 INFO:tasks.workunit.client.1.smithi149.stdout:4/20: symlink d0/l5 0 2024-03-22T07:08:47.293 INFO:tasks.workunit.client.1.smithi149.stdout:4/21: symlink d0/l6 0 2024-03-22T07:08:47.295 INFO:tasks.workunit.client.1.smithi149.stdout:0/34: sync 2024-03-22T07:08:47.295 INFO:tasks.workunit.client.1.smithi149.stdout:3/27: sync 2024-03-22T07:08:47.295 INFO:tasks.workunit.client.1.smithi149.stdout:6/35: sync 2024-03-22T07:08:47.295 INFO:tasks.workunit.client.1.smithi149.stdout:0/35: chown d1/c3 6869 1 2024-03-22T07:08:47.295 INFO:tasks.workunit.client.1.smithi149.stdout:0/36: dwrite - no filename 2024-03-22T07:08:47.295 INFO:tasks.workunit.client.1.smithi149.stdout:0/37: dwrite - no filename 2024-03-22T07:08:47.295 INFO:tasks.workunit.client.1.smithi149.stdout:9/64: sync 2024-03-22T07:08:47.296 INFO:tasks.workunit.client.1.smithi149.stdout:3/28: symlink d0/d4/d8/lb 0 2024-03-22T07:08:47.297 INFO:tasks.workunit.client.1.smithi149.stdout:6/36: rmdir d9 39 2024-03-22T07:08:47.297 INFO:tasks.workunit.client.1.smithi149.stdout:6/37: fdatasync d9/fc 0 2024-03-22T07:08:47.297 INFO:tasks.workunit.client.1.smithi149.stdout:6/38: truncate d9/fc 836169 0 2024-03-22T07:08:47.297 INFO:tasks.workunit.client.1.smithi149.stdout:6/39: write f8 [1123612,25162] 0 2024-03-22T07:08:47.297 INFO:tasks.workunit.client.1.smithi149.stdout:6/40: write d9/fc [1605587,47492] 0 2024-03-22T07:08:47.297 INFO:tasks.workunit.client.1.smithi149.stdout:0/38: mknod d1/c6 0 2024-03-22T07:08:47.298 INFO:tasks.workunit.client.1.smithi149.stdout:9/65: dread f3 [0,4194304] 0 2024-03-22T07:08:47.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/66: write f8 [1866384,120846] 0 2024-03-22T07:08:47.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/67: fdatasync db/fe 0 2024-03-22T07:08:47.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/68: truncate db/fd 623640 0 2024-03-22T07:08:47.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/69: truncate db/fc 428633 0 2024-03-22T07:08:47.299 INFO:tasks.workunit.client.1.smithi149.stdout:8/67: dwrite d1/d2/d3/fd [0,4194304] 0 2024-03-22T07:08:47.299 INFO:tasks.workunit.client.1.smithi149.stdout:8/68: chown d1/d2/d9 35 1 2024-03-22T07:08:47.299 INFO:tasks.workunit.client.1.smithi149.stdout:3/29: unlink d0/d4/d8/la 0 2024-03-22T07:08:47.300 INFO:tasks.workunit.client.1.smithi149.stdout:6/41: creat d9/fd x:0 0 0 2024-03-22T07:08:47.302 INFO:tasks.workunit.client.1.smithi149.stdout:9/70: creat db/d12/f14 x:0 0 0 2024-03-22T07:08:47.303 INFO:tasks.workunit.client.1.smithi149.stdout:8/69: creat d1/d2/d9/df/f15 x:0 0 0 2024-03-22T07:08:47.303 INFO:tasks.workunit.client.1.smithi149.stdout:8/70: fsync d1/d2/d3/fd 0 2024-03-22T07:08:47.303 INFO:tasks.workunit.client.1.smithi149.stdout:8/71: fsync d1/d2/d3/f6 0 2024-03-22T07:08:47.304 INFO:tasks.workunit.client.1.smithi149.stdout:2/80: dwrite d1/d8/f17 [0,4194304] 0 2024-03-22T07:08:47.304 INFO:tasks.workunit.client.1.smithi149.stdout:3/30: symlink d0/lc 0 2024-03-22T07:08:47.305 INFO:tasks.workunit.client.1.smithi149.stdout:9/71: dread f3 [0,4194304] 0 2024-03-22T07:08:47.305 INFO:tasks.workunit.client.1.smithi149.stdout:9/72: fdatasync f3 0 2024-03-22T07:08:47.305 INFO:tasks.workunit.client.1.smithi149.stdout:9/73: chown db/l13 116822752 1 2024-03-22T07:08:47.305 INFO:tasks.workunit.client.1.smithi149.stdout:8/72: dread d1/d2/d9/da/fb [0,4194304] 0 2024-03-22T07:08:47.308 INFO:tasks.workunit.client.1.smithi149.stdout:7/57: dwrite d1/f5 [0,4194304] 0 2024-03-22T07:08:47.309 INFO:tasks.workunit.client.1.smithi149.stdout:7/58: truncate d1/f10 631989 0 2024-03-22T07:08:47.309 INFO:tasks.workunit.client.1.smithi149.stdout:3/31: mknod d0/cd 0 2024-03-22T07:08:47.311 INFO:tasks.workunit.client.1.smithi149.stdout:7/59: symlink d1/l12 0 2024-03-22T07:08:47.311 INFO:tasks.workunit.client.1.smithi149.stdout:8/73: dread d1/d2/d9/da/fb [0,4194304] 0 2024-03-22T07:08:47.311 INFO:tasks.workunit.client.1.smithi149.stdout:3/32: creat d0/d4/fe x:0 0 0 2024-03-22T07:08:47.312 INFO:tasks.workunit.client.1.smithi149.stdout:2/81: rmdir d1/d3 39 2024-03-22T07:08:47.312 INFO:tasks.workunit.client.1.smithi149.stdout:2/82: stat d1/d8/f17 0 2024-03-22T07:08:47.314 INFO:tasks.workunit.client.1.smithi149.stdout:3/33: link d0/cd d0/cf 0 2024-03-22T07:08:47.315 INFO:tasks.workunit.client.1.smithi149.stdout:2/83: link d1/d3/f9 d1/dc/f1e 0 2024-03-22T07:08:47.315 INFO:tasks.workunit.client.1.smithi149.stdout:2/84: dread - d1/d3/d6/f1c zero size 2024-03-22T07:08:47.316 INFO:tasks.workunit.client.1.smithi149.stdout:9/74: dread f7 [0,4194304] 0 2024-03-22T07:08:47.317 INFO:tasks.workunit.client.1.smithi149.stdout:3/34: link d0/cf d0/d4/d8/c10 0 2024-03-22T07:08:47.318 INFO:tasks.workunit.client.1.smithi149.stdout:3/35: truncate d0/d4/fe 253074 0 2024-03-22T07:08:47.319 INFO:tasks.workunit.client.1.smithi149.stdout:4/22: rmdir d0 39 2024-03-22T07:08:47.320 INFO:tasks.workunit.client.1.smithi149.stdout:9/75: unlink db/l13 0 2024-03-22T07:08:47.321 INFO:tasks.workunit.client.1.smithi149.stdout:0/39: getdents d1 0 2024-03-22T07:08:47.322 INFO:tasks.workunit.client.1.smithi149.stdout:6/42: dwrite d9/fd [0,4194304] 0 2024-03-22T07:08:47.323 INFO:tasks.workunit.client.1.smithi149.stdout:4/23: getdents d0 0 2024-03-22T07:08:47.325 INFO:tasks.workunit.client.1.smithi149.stdout:0/40: symlink d1/l7 0 2024-03-22T07:08:47.325 INFO:tasks.workunit.client.1.smithi149.stdout:0/41: readlink d1/l7 0 2024-03-22T07:08:47.327 INFO:tasks.workunit.client.1.smithi149.stdout:8/74: truncate d1/d2/d3/fd 929382 0 2024-03-22T07:08:47.327 INFO:tasks.workunit.client.1.smithi149.stdout:8/75: chown d1/d2/d3 270432305 1 2024-03-22T07:08:47.327 INFO:tasks.workunit.client.1.smithi149.stdout:8/76: write d1/d2/d9/da/fb [1153854,3218] 0 2024-03-22T07:08:47.327 INFO:tasks.workunit.client.1.smithi149.stdout:0/42: chown d1/c3 184396738 1 2024-03-22T07:08:47.327 INFO:tasks.workunit.client.1.smithi149.stdout:0/43: dread - no filename 2024-03-22T07:08:47.330 INFO:tasks.workunit.client.1.smithi149.stdout:8/77: creat d1/f16 x:0 0 0 2024-03-22T07:08:47.330 INFO:tasks.workunit.client.1.smithi149.stdout:0/44: rmdir d1 39 2024-03-22T07:08:47.331 INFO:tasks.workunit.client.1.smithi149.stdout:7/60: getdents d1 0 2024-03-22T07:08:47.331 INFO:tasks.workunit.client.1.smithi149.stdout:7/61: write d1/d6/fd [1040717,91695] 0 2024-03-22T07:08:47.333 INFO:tasks.workunit.client.1.smithi149.stdout:3/36: fsync d0/d4/fe 0 2024-03-22T07:08:47.334 INFO:tasks.workunit.client.1.smithi149.stdout:3/37: symlink d0/d2/l11 0 2024-03-22T07:08:47.335 INFO:tasks.workunit.client.1.smithi149.stdout:3/38: chown d0/d4/l6 80376325 1 2024-03-22T07:08:47.335 INFO:tasks.workunit.client.1.smithi149.stdout:6/43: dread d9/fd [0,4194304] 0 2024-03-22T07:08:47.337 INFO:tasks.workunit.client.1.smithi149.stdout:3/39: dread d0/d4/fe [0,4194304] 0 2024-03-22T07:08:47.337 INFO:tasks.workunit.client.1.smithi149.stdout:0/45: link d1/c5 d1/c8 0 2024-03-22T07:08:47.338 INFO:tasks.workunit.client.1.smithi149.stdout:6/44: rmdir d9/da 39 2024-03-22T07:08:47.340 INFO:tasks.workunit.client.1.smithi149.stdout:2/85: rename d1/dc to d1/d3/d5/d1f 0 2024-03-22T07:08:47.341 INFO:tasks.workunit.client.1.smithi149.stdout:4/24: dwrite d0/f3 [0,4194304] 0 2024-03-22T07:08:47.342 INFO:tasks.workunit.client.1.smithi149.stdout:6/45: symlink d9/le 0 2024-03-22T07:08:47.346 INFO:tasks.workunit.client.1.smithi149.stdout:7/62: dread d1/d6/fd [0,4194304] 0 2024-03-22T07:08:47.346 INFO:tasks.workunit.client.1.smithi149.stdout:7/63: chown d1/d6/cf 14569774 1 2024-03-22T07:08:47.347 INFO:tasks.workunit.client.1.smithi149.stdout:2/86: truncate d1/d3/d5/d1f/f1e 520039 0 2024-03-22T07:08:47.347 INFO:tasks.workunit.client.1.smithi149.stdout:2/87: write d1/d8/f19 [521625,18224] 0 2024-03-22T07:08:47.348 INFO:tasks.workunit.client.1.smithi149.stdout:0/46: mkdir d1/d9 0 2024-03-22T07:08:47.350 INFO:tasks.workunit.client.1.smithi149.stdout:9/76: dwrite db/fd [0,4194304] 0 2024-03-22T07:08:47.359 INFO:tasks.workunit.client.1.smithi149.stdout:2/88: symlink d1/l20 0 2024-03-22T07:08:47.360 INFO:tasks.workunit.client.1.smithi149.stdout:4/25: write d0/f3 [1494850,4946] 0 2024-03-22T07:08:47.360 INFO:tasks.workunit.client.1.smithi149.stdout:2/89: rmdir d1/d3/d6 39 2024-03-22T07:08:47.360 INFO:tasks.workunit.client.1.smithi149.stdout:2/90: write d1/d8/f19 [588272,13479] 0 2024-03-22T07:08:47.360 INFO:tasks.workunit.client.1.smithi149.stdout:4/26: write d0/f3 [3998475,91482] 0 2024-03-22T07:08:47.361 INFO:tasks.workunit.client.1.smithi149.stdout:4/27: write d0/f3 [884373,104704] 0 2024-03-22T07:08:47.363 INFO:tasks.workunit.client.1.smithi149.stdout:2/91: creat d1/d3/dd/f21 x:0 0 0 2024-03-22T07:08:47.363 INFO:tasks.workunit.client.1.smithi149.stdout:2/92: creat d1/d3/dd/f22 x:0 0 0 2024-03-22T07:08:47.363 INFO:tasks.workunit.client.1.smithi149.stdout:4/28: symlink d0/l7 0 2024-03-22T07:08:47.363 INFO:tasks.workunit.client.1.smithi149.stdout:4/29: write d0/f3 [1444595,118804] 0 2024-03-22T07:08:47.365 INFO:tasks.workunit.client.1.smithi149.stdout:9/77: read f7 [2909061,4128] 0 2024-03-22T07:08:47.365 INFO:tasks.workunit.client.1.smithi149.stdout:9/78: creat db/d12/f15 x:0 0 0 2024-03-22T07:08:47.366 INFO:tasks.workunit.client.1.smithi149.stdout:2/93: mkdir d1/d8/d23 0 2024-03-22T07:08:47.366 INFO:tasks.workunit.client.1.smithi149.stdout:9/79: read f7 [2899197,19266] 0 2024-03-22T07:08:47.366 INFO:tasks.workunit.client.1.smithi149.stdout:4/30: mkdir d0/d8 0 2024-03-22T07:08:47.369 INFO:tasks.workunit.client.1.smithi149.stdout:2/94: unlink d1/d3/dd/f22 0 2024-03-22T07:08:47.370 INFO:tasks.workunit.client.1.smithi149.stdout:4/31: mknod d0/c9 0 2024-03-22T07:08:47.370 INFO:tasks.workunit.client.1.smithi149.stdout:4/32: write d0/f3 [3400554,59264] 0 2024-03-22T07:08:47.372 INFO:tasks.workunit.client.1.smithi149.stdout:2/95: rename d1/d3/d6/f1c to d1/d8/f24 0 2024-03-22T07:08:47.372 INFO:tasks.workunit.client.1.smithi149.stdout:2/96: fsync d1/d3/d5/d1f/f1b 0 2024-03-22T07:08:47.373 INFO:tasks.workunit.client.1.smithi149.stdout:4/33: rename d0/l5 to d0/la 0 2024-03-22T07:08:47.373 INFO:tasks.workunit.client.1.smithi149.stdout:4/34: chown d0/la 240 1 2024-03-22T07:08:47.375 INFO:tasks.workunit.client.1.smithi149.stdout:2/97: creat d1/d3/f25 x:0 0 0 2024-03-22T07:08:47.375 INFO:tasks.workunit.client.1.smithi149.stdout:2/98: stat d1/d3/f25 0 2024-03-22T07:08:47.375 INFO:tasks.workunit.client.1.smithi149.stdout:9/80: dread f7 [0,4194304] 0 2024-03-22T07:08:47.376 INFO:tasks.workunit.client.1.smithi149.stdout:9/81: write f3 [718289,12577] 0 2024-03-22T07:08:47.376 INFO:tasks.workunit.client.1.smithi149.stdout:4/35: symlink d0/lb 0 2024-03-22T07:08:47.377 INFO:tasks.workunit.client.1.smithi149.stdout:2/99: mkdir d1/d3/d26 0 2024-03-22T07:08:47.377 INFO:tasks.workunit.client.1.smithi149.stdout:2/100: fdatasync d1/d3/f9 0 2024-03-22T07:08:47.378 INFO:tasks.workunit.client.1.smithi149.stdout:9/82: creat db/f16 x:0 0 0 2024-03-22T07:08:47.378 INFO:tasks.workunit.client.1.smithi149.stdout:9/83: readlink db/l11 0 2024-03-22T07:08:47.378 INFO:tasks.workunit.client.1.smithi149.stdout:9/84: write db/f16 [682437,150] 0 2024-03-22T07:08:47.378 INFO:tasks.workunit.client.1.smithi149.stdout:4/36: mkdir d0/d8/dc 0 2024-03-22T07:08:47.378 INFO:tasks.workunit.client.1.smithi149.stdout:9/85: read db/fc [166434,104926] 0 2024-03-22T07:08:47.380 INFO:tasks.workunit.client.1.smithi149.stdout:9/86: rename db/fe to db/d12/f17 0 2024-03-22T07:08:47.380 INFO:tasks.workunit.client.1.smithi149.stdout:9/87: readlink db/l11 0 2024-03-22T07:08:47.380 INFO:tasks.workunit.client.1.smithi149.stdout:9/88: stat c5 0 2024-03-22T07:08:47.380 INFO:tasks.workunit.client.1.smithi149.stdout:9/89: dread - db/d12/f14 zero size 2024-03-22T07:08:47.381 INFO:tasks.workunit.client.1.smithi149.stdout:9/90: unlink db/f16 0 2024-03-22T07:08:47.381 INFO:tasks.workunit.client.1.smithi149.stdout:9/91: chown db/fc 10 1 2024-03-22T07:08:47.381 INFO:tasks.workunit.client.1.smithi149.stdout:9/92: fdatasync db/d12/f17 0 2024-03-22T07:08:47.382 INFO:tasks.workunit.client.1.smithi149.stdout:9/93: truncate db/d12/f17 1058890 0 2024-03-22T07:08:47.383 INFO:tasks.workunit.client.1.smithi149.stdout:9/94: rename db/l11 to db/l18 0 2024-03-22T07:08:47.386 INFO:tasks.workunit.client.1.smithi149.stdout:9/95: mkdir db/d19 0 2024-03-22T07:08:47.386 INFO:tasks.workunit.client.1.smithi149.stdout:9/96: write f3 [1691999,14088] 0 2024-03-22T07:08:47.386 INFO:tasks.workunit.client.1.smithi149.stdout:9/97: chown db/c10 7831 1 2024-03-22T07:08:47.386 INFO:tasks.workunit.client.1.smithi149.stdout:9/98: fsync f8 0 2024-03-22T07:08:47.386 INFO:tasks.workunit.client.1.smithi149.stdout:9/99: write db/d12/f14 [113280,89565] 0 2024-03-22T07:08:47.393 INFO:tasks.workunit.client.1.smithi149.stdout:9/100: dread f7 [0,4194304] 0 2024-03-22T07:08:47.395 INFO:tasks.workunit.client.1.smithi149.stdout:6/46: dwrite d9/fc [0,4194304] 0 2024-03-22T07:08:47.395 INFO:tasks.workunit.client.1.smithi149.stdout:6/47: chown d9/le 84305521 1 2024-03-22T07:08:47.395 INFO:tasks.workunit.client.1.smithi149.stdout:3/40: dwrite d0/d4/fe [0,4194304] 0 2024-03-22T07:08:47.398 INFO:tasks.workunit.client.1.smithi149.stdout:8/78: dwrite d1/d2/d3/fd [0,4194304] 0 2024-03-22T07:08:47.398 INFO:tasks.workunit.client.1.smithi149.stdout:6/48: mkdir d9/df 0 2024-03-22T07:08:47.403 INFO:tasks.workunit.client.1.smithi149.stdout:2/101: dwrite d1/d3/d5/ff [0,4194304] 0 2024-03-22T07:08:47.403 INFO:tasks.workunit.client.1.smithi149.stdout:2/102: dread - d1/d3/dd/f21 zero size 2024-03-22T07:08:47.403 INFO:tasks.workunit.client.1.smithi149.stdout:8/79: creat d1/d2/d9/d11/f17 x:0 0 0 2024-03-22T07:08:47.403 INFO:tasks.workunit.client.1.smithi149.stdout:3/41: mknod d0/d4/c12 0 2024-03-22T07:08:47.404 INFO:tasks.workunit.client.1.smithi149.stdout:7/64: dwrite d1/d6/f8 [0,4194304] 0 2024-03-22T07:08:47.404 INFO:tasks.workunit.client.1.smithi149.stdout:7/65: fdatasync d1/d6/fd 0 2024-03-22T07:08:47.404 INFO:tasks.workunit.client.1.smithi149.stdout:7/66: readlink d1/d6/la 0 2024-03-22T07:08:47.404 INFO:tasks.workunit.client.1.smithi149.stdout:7/67: stat d1/f10 0 2024-03-22T07:08:47.404 INFO:tasks.workunit.client.1.smithi149.stdout:7/68: write d1/d6/f8 [4233995,100517] 0 2024-03-22T07:08:47.404 INFO:tasks.workunit.client.1.smithi149.stdout:7/69: chown d1/f10 2 1 2024-03-22T07:08:47.411 INFO:tasks.workunit.client.1.smithi149.stdout:8/80: mkdir d1/d2/d18 0 2024-03-22T07:08:47.411 INFO:tasks.workunit.client.1.smithi149.stdout:8/81: write d1/d2/d9/d11/f17 [837878,73405] 0 2024-03-22T07:08:47.411 INFO:tasks.workunit.client.1.smithi149.stdout:8/82: stat d1/d2/d9/da/c10 0 2024-03-22T07:08:47.418 INFO:tasks.workunit.client.1.smithi149.stdout:9/101: dwrite f7 [0,4194304] 0 2024-03-22T07:08:47.424 INFO:tasks.workunit.client.1.smithi149.stdout:3/42: write d0/d4/fe [3634651,119832] 0 2024-03-22T07:08:47.425 INFO:tasks.workunit.client.1.smithi149.stdout:9/102: read f8 [29529,4227] 0 2024-03-22T07:08:47.427 INFO:tasks.workunit.client.1.smithi149.stdout:3/43: creat d0/f13 x:0 0 0 2024-03-22T07:08:47.427 INFO:tasks.workunit.client.1.smithi149.stdout:3/44: chown d0/lc 0 1 2024-03-22T07:08:47.427 INFO:tasks.workunit.client.1.smithi149.stdout:9/103: mkdir db/d12/d1a 0 2024-03-22T07:08:47.429 INFO:tasks.workunit.client.1.smithi149.stdout:3/45: mknod d0/c14 0 2024-03-22T07:08:47.430 INFO:tasks.workunit.client.1.smithi149.stdout:3/46: mknod d0/d2/c15 0 2024-03-22T07:08:47.434 INFO:tasks.workunit.client.1.smithi149.stdout:3/47: symlink d0/d4/l16 0 2024-03-22T07:08:47.434 INFO:tasks.workunit.client.1.smithi149.stdout:3/48: write d0/f13 [30356,103533] 0 2024-03-22T07:08:47.434 INFO:tasks.workunit.client.1.smithi149.stdout:3/49: stat d0/f13 0 2024-03-22T07:08:47.434 INFO:tasks.workunit.client.1.smithi149.stdout:3/50: chown d0/d4/d8 31068 1 2024-03-22T07:08:47.446 INFO:tasks.workunit.client.1.smithi149.stdout:4/37: dwrite d0/f3 [0,4194304] 0 2024-03-22T07:08:47.446 INFO:tasks.workunit.client.1.smithi149.stdout:4/38: write d0/f3 [4635478,11082] 0 2024-03-22T07:08:47.447 INFO:tasks.workunit.client.1.smithi149.stdout:3/51: fdatasync d0/d4/fe 0 2024-03-22T07:08:47.448 INFO:tasks.workunit.client.1.smithi149.stdout:6/49: dwrite f8 [0,4194304] 0 2024-03-22T07:08:47.448 INFO:tasks.workunit.client.1.smithi149.stdout:9/104: dwrite db/d12/f14 [0,4194304] 0 2024-03-22T07:08:47.449 INFO:tasks.workunit.client.1.smithi149.stdout:2/103: dwrite d1/f1a [0,4194304] 0 2024-03-22T07:08:47.449 INFO:tasks.workunit.client.1.smithi149.stdout:2/104: chown d1/d3/fb 14 1 2024-03-22T07:08:47.449 INFO:tasks.workunit.client.1.smithi149.stdout:9/105: write db/fc [665712,70721] 0 2024-03-22T07:08:47.449 INFO:tasks.workunit.client.1.smithi149.stdout:4/39: fsync d0/f3 0 2024-03-22T07:08:47.449 INFO:tasks.workunit.client.1.smithi149.stdout:4/40: chown d0/l6 124723785 1 2024-03-22T07:08:47.450 INFO:tasks.workunit.client.1.smithi149.stdout:6/50: truncate f5 692502 0 2024-03-22T07:08:47.450 INFO:tasks.workunit.client.1.smithi149.stdout:6/51: fdatasync f1 0 2024-03-22T07:08:47.458 INFO:tasks.workunit.client.1.smithi149.stdout:2/105: rename d1/d8/f17 to d1/f27 0 2024-03-22T07:08:47.458 INFO:tasks.workunit.client.1.smithi149.stdout:2/106: write d1/d3/d5/d1f/f1e [1436751,18365] 0 2024-03-22T07:08:47.458 INFO:tasks.workunit.client.1.smithi149.stdout:2/107: chown d1/d3/d5/d1f 452 1 2024-03-22T07:08:47.459 INFO:tasks.workunit.client.1.smithi149.stdout:9/106: symlink db/d12/l1b 0 2024-03-22T07:08:47.459 INFO:tasks.workunit.client.1.smithi149.stdout:9/107: chown db/cf 69103478 1 2024-03-22T07:08:47.460 INFO:tasks.workunit.client.1.smithi149.stdout:4/41: dread d0/f3 [0,4194304] 0 2024-03-22T07:08:47.461 INFO:tasks.workunit.client.1.smithi149.stdout:6/52: mknod d9/da/c10 0 2024-03-22T07:08:47.462 INFO:tasks.workunit.client.1.smithi149.stdout:7/70: truncate d1/d6/f8 1499928 0 2024-03-22T07:08:47.464 INFO:tasks.workunit.client.1.smithi149.stdout:3/52: fsync d0/f13 0 2024-03-22T07:08:47.465 INFO:tasks.workunit.client.1.smithi149.stdout:9/108: mkdir db/d19/d1c 0 2024-03-22T07:08:47.469 INFO:tasks.workunit.client.1.smithi149.stdout:7/71: creat d1/d6/f13 x:0 0 0 2024-03-22T07:08:47.469 INFO:tasks.workunit.client.1.smithi149.stdout:7/72: chown d1/lc 0 1 2024-03-22T07:08:47.469 INFO:tasks.workunit.client.1.smithi149.stdout:7/73: write d1/f10 [877939,48479] 0 2024-03-22T07:08:47.469 INFO:tasks.workunit.client.1.smithi149.stdout:7/74: getdents d1 0 2024-03-22T07:08:47.469 INFO:tasks.workunit.client.1.smithi149.stdout:7/75: write d1/f5 [5231510,22591] 0 2024-03-22T07:08:47.469 INFO:tasks.workunit.client.1.smithi149.stdout:7/76: chown d1/l9 93348 1 2024-03-22T07:08:47.470 INFO:tasks.workunit.client.1.smithi149.stdout:3/53: link d0/cd d0/c17 0 2024-03-22T07:08:47.472 INFO:tasks.workunit.client.1.smithi149.stdout:9/109: dread f8 [0,4194304] 0 2024-03-22T07:08:47.475 INFO:tasks.workunit.client.1.smithi149.stdout:9/110: rename c5 to db/d12/c1d 0 2024-03-22T07:08:47.477 INFO:tasks.workunit.client.1.smithi149.stdout:9/111: link ca db/d12/d1a/c1e 0 2024-03-22T07:08:47.477 INFO:tasks.workunit.client.1.smithi149.stdout:9/112: chown db/fd 10155259 1 2024-03-22T07:08:47.477 INFO:tasks.workunit.client.1.smithi149.stdout:2/108: dwrite d1/f10 [4194304,4194304] 0 2024-03-22T07:08:47.479 INFO:tasks.workunit.client.1.smithi149.stdout:4/42: truncate d0/f3 2428750 0 2024-03-22T07:08:47.481 INFO:tasks.workunit.client.1.smithi149.stdout:2/109: creat d1/d3/d5/f28 x:0 0 0 2024-03-22T07:08:47.481 INFO:tasks.workunit.client.1.smithi149.stdout:2/110: creat d1/d3/f29 x:0 0 0 2024-03-22T07:08:47.485 INFO:tasks.workunit.client.1.smithi149.stdout:5/47: sync 2024-03-22T07:08:47.485 INFO:tasks.workunit.client.1.smithi149.stdout:5/48: write d7/da/fe [7915676,68896] 0 2024-03-22T07:08:47.485 INFO:tasks.workunit.client.1.smithi149.stdout:5/49: rename d7 to d7/d11 22 2024-03-22T07:08:47.485 INFO:tasks.workunit.client.1.smithi149.stdout:0/47: sync 2024-03-22T07:08:47.485 INFO:tasks.workunit.client.1.smithi149.stdout:1/52: sync 2024-03-22T07:08:47.486 INFO:tasks.workunit.client.1.smithi149.stdout:0/48: dwrite - no filename 2024-03-22T07:08:47.486 INFO:tasks.workunit.client.1.smithi149.stdout:0/49: dwrite - no filename 2024-03-22T07:08:47.486 INFO:tasks.workunit.client.1.smithi149.stdout:0/50: write - no filename 2024-03-22T07:08:47.486 INFO:tasks.workunit.client.1.smithi149.stdout:1/53: truncate d4/d10/f13 679875 0 2024-03-22T07:08:47.489 INFO:tasks.workunit.client.1.smithi149.stdout:9/113: write db/d12/f14 [4125978,37768] 0 2024-03-22T07:08:47.489 INFO:tasks.workunit.client.1.smithi149.stdout:1/54: readlink d4/lf 0 2024-03-22T07:08:47.489 INFO:tasks.workunit.client.1.smithi149.stdout:1/55: fsync d4/f8 0 2024-03-22T07:08:47.489 INFO:tasks.workunit.client.1.smithi149.stdout:1/56: chown l3 0 1 2024-03-22T07:08:47.489 INFO:tasks.workunit.client.1.smithi149.stdout:0/51: rename d1 to d1/da 22 2024-03-22T07:08:47.493 INFO:tasks.workunit.client.1.smithi149.stdout:9/114: creat db/d19/f1f x:0 0 0 2024-03-22T07:08:47.494 INFO:tasks.workunit.client.1.smithi149.stdout:3/54: dwrite d0/d4/fe [4194304,4194304] 0 2024-03-22T07:08:47.494 INFO:tasks.workunit.client.1.smithi149.stdout:2/111: dread d1/d3/d6/f16 [0,4194304] 0 2024-03-22T07:08:47.494 INFO:tasks.workunit.client.1.smithi149.stdout:2/112: chown d1/d8/f24 15 1 2024-03-22T07:08:47.494 INFO:tasks.workunit.client.1.smithi149.stdout:2/113: readlink d1/d8/l18 0 2024-03-22T07:08:47.494 INFO:tasks.workunit.client.1.smithi149.stdout:2/114: chown d1/d3/f29 77074 1 2024-03-22T07:08:47.495 INFO:tasks.workunit.client.1.smithi149.stdout:0/52: creat d1/fb x:0 0 0 2024-03-22T07:08:47.498 INFO:tasks.workunit.client.1.smithi149.stdout:9/115: unlink f7 0 2024-03-22T07:08:47.501 INFO:tasks.workunit.client.1.smithi149.stdout:3/55: rename d0/c17 to d0/c18 0 2024-03-22T07:08:47.501 INFO:tasks.workunit.client.1.smithi149.stdout:3/56: write d0/f13 [622954,120762] 0 2024-03-22T07:08:47.501 INFO:tasks.workunit.client.1.smithi149.stdout:1/57: getdents d4/d7 0 2024-03-22T07:08:47.502 INFO:tasks.workunit.client.1.smithi149.stdout:7/77: dwrite d1/f10 [0,4194304] 0 2024-03-22T07:08:47.504 INFO:tasks.workunit.client.1.smithi149.stdout:2/115: link d1/d3/f9 d1/f2a 0 2024-03-22T07:08:47.504 INFO:tasks.workunit.client.1.smithi149.stdout:2/116: dread - d1/d3/dd/f21 zero size 2024-03-22T07:08:47.504 INFO:tasks.workunit.client.1.smithi149.stdout:2/117: chown d1/f4 24840781 1 2024-03-22T07:08:47.504 INFO:tasks.workunit.client.1.smithi149.stdout:2/118: fdatasync d1/f10 0 2024-03-22T07:08:47.508 INFO:tasks.workunit.client.1.smithi149.stdout:2/119: write d1/d3/d6/f7 [1038194,68488] 0 2024-03-22T07:08:47.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/50: dread d7/da/fe [0,4194304] 0 2024-03-22T07:08:47.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/51: read - d7/f8 zero size 2024-03-22T07:08:47.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/52: creat d7/da/f12 x:0 0 0 2024-03-22T07:08:47.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/53: rename d7 to d7/d13 22 2024-03-22T07:08:47.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/54: chown d7/db/fc 253 1 2024-03-22T07:08:47.514 INFO:tasks.workunit.client.1.smithi149.stdout:9/116: dread db/d12/f14 [0,4194304] 0 2024-03-22T07:08:47.514 INFO:tasks.workunit.client.1.smithi149.stdout:9/117: chown db/cf 7131235 1 2024-03-22T07:08:47.514 INFO:tasks.workunit.client.1.smithi149.stdout:9/118: write f9 [4886879,78536] 0 2024-03-22T07:08:47.518 INFO:tasks.workunit.client.1.smithi149.stdout:7/78: read d1/f5 [59585,70280] 0 2024-03-22T07:08:47.529 INFO:tasks.workunit.client.1.smithi149.stdout:2/120: dread d1/f27 [0,4194304] 0 2024-03-22T07:08:47.529 INFO:tasks.workunit.client.1.smithi149.stdout:2/121: fdatasync d1/d3/d6/f7 0 2024-03-22T07:08:47.530 INFO:tasks.workunit.client.1.smithi149.stdout:2/122: read - d1/d3/d5/f28 zero size 2024-03-22T07:08:47.530 INFO:tasks.workunit.client.1.smithi149.stdout:2/123: creat d1/d3/d5/f2b x:0 0 0 2024-03-22T07:08:47.530 INFO:tasks.workunit.client.1.smithi149.stdout:4/43: dwrite d0/f3 [0,4194304] 0 2024-03-22T07:08:47.539 INFO:tasks.workunit.client.1.smithi149.stdout:7/79: dwrite d1/d6/f8 [0,4194304] 0 2024-03-22T07:08:47.580 INFO:tasks.workunit.client.1.smithi149.stdout:9/119: creat db/d19/d1c/f20 x:0 0 0 2024-03-22T07:08:47.581 INFO:tasks.workunit.client.1.smithi149.stdout:3/57: getdents d0/d4 0 2024-03-22T07:08:47.582 INFO:tasks.workunit.client.1.smithi149.stdout:1/58: link c2 d4/c14 0 2024-03-22T07:08:47.582 INFO:tasks.workunit.client.1.smithi149.stdout:1/59: chown d4/d7 45 1 2024-03-22T07:08:47.584 INFO:tasks.workunit.client.1.smithi149.stdout:2/124: symlink d1/d3/d26/l2c 0 2024-03-22T07:08:47.584 INFO:tasks.workunit.client.1.smithi149.stdout:2/125: dread - d1/d3/d5/f28 zero size 2024-03-22T07:08:47.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/44: creat d0/d8/fd x:0 0 0 2024-03-22T07:08:47.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/45: write d0/f3 [4398242,44768] 0 2024-03-22T07:08:47.588 INFO:tasks.workunit.client.1.smithi149.stdout:2/126: dread d1/f4 [0,4194304] 0 2024-03-22T07:08:47.590 INFO:tasks.workunit.client.1.smithi149.stdout:4/46: dread d0/f3 [4194304,4194304] 0 2024-03-22T07:08:47.592 INFO:tasks.workunit.client.1.smithi149.stdout:9/120: unlink db/d12/c1d 0 2024-03-22T07:08:47.593 INFO:tasks.workunit.client.1.smithi149.stdout:3/58: creat d0/d4/d8/f19 x:0 0 0 2024-03-22T07:08:47.593 INFO:tasks.workunit.client.1.smithi149.stdout:1/60: creat d4/f15 x:0 0 0 2024-03-22T07:08:47.595 INFO:tasks.workunit.client.1.smithi149.stdout:4/47: creat d0/d8/dc/fe x:0 0 0 2024-03-22T07:08:47.596 INFO:tasks.workunit.client.1.smithi149.stdout:9/121: unlink f8 0 2024-03-22T07:08:47.598 INFO:tasks.workunit.client.1.smithi149.stdout:2/127: dread d1/d8/f24 [0,4194304] 0 2024-03-22T07:08:47.599 INFO:tasks.workunit.client.1.smithi149.stdout:2/128: mknod d1/d3/d26/c2d 0 2024-03-22T07:08:47.599 INFO:tasks.workunit.client.1.smithi149.stdout:9/122: dread f3 [0,4194304] 0 2024-03-22T07:08:47.600 INFO:tasks.workunit.client.1.smithi149.stdout:2/129: rename d1/d8/l18 to d1/d3/d26/l2e 0 2024-03-22T07:08:47.604 INFO:tasks.workunit.client.1.smithi149.stdout:2/130: rename d1/d8/d23 to d1/d3/d26/d2f 0 2024-03-22T07:08:47.607 INFO:tasks.workunit.client.1.smithi149.stdout:2/131: symlink d1/d3/d5/d1f/l30 0 2024-03-22T07:08:47.614 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:47 smithi149 ceph-mon[36068]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:47.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:47 smithi149 ceph-mon[36068]: Standby manager daemon smithi081.bcxarw started 2024-03-22T07:08:47.615 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:47 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.81:0/3007820561' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi081.bcxarw/crt"}]: dispatch 2024-03-22T07:08:47.615 INFO:tasks.workunit.client.1.smithi149.stdout:2/132: dread d1/d8/f24 [0,4194304] 0 2024-03-22T07:08:47.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/133: write d1/f4 [914345,109637] 0 2024-03-22T07:08:47.616 INFO:tasks.workunit.client.1.smithi149.stdout:9/123: truncate f9 4589073 0 2024-03-22T07:08:47.616 INFO:tasks.workunit.client.1.smithi149.stdout:9/124: readlink l1 0 2024-03-22T07:08:47.617 INFO:tasks.workunit.client.1.smithi149.stdout:2/134: symlink d1/d3/d26/l31 0 2024-03-22T07:08:47.617 INFO:tasks.workunit.client.1.smithi149.stdout:2/135: write d1/d3/f9 [1571011,19488] 0 2024-03-22T07:08:47.617 INFO:tasks.workunit.client.1.smithi149.stdout:2/136: write d1/d3/f13 [770921,88405] 0 2024-03-22T07:08:47.619 INFO:tasks.workunit.client.1.smithi149.stdout:8/83: sync 2024-03-22T07:08:47.619 INFO:tasks.workunit.client.1.smithi149.stdout:8/84: chown d1/d2/d9/d11/c13 6 1 2024-03-22T07:08:47.620 INFO:tasks.workunit.client.1.smithi149.stdout:8/85: creat d1/d2/d9/df/f19 x:0 0 0 2024-03-22T07:08:47.620 INFO:tasks.workunit.client.1.smithi149.stdout:8/86: fdatasync d1/d2/d9/df/f15 0 2024-03-22T07:08:47.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/87: fdatasync d1/d2/d3/f6 0 2024-03-22T07:08:47.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/88: creat d1/d2/d3/f1a x:0 0 0 2024-03-22T07:08:47.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/89: truncate d1/d2/d9/d11/f17 1323186 0 2024-03-22T07:08:47.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/90: write d1/d2/d9/da/fb [1692780,99414] 0 2024-03-22T07:08:47.621 INFO:tasks.workunit.client.1.smithi149.stdout:8/91: fsync d1/d2/d9/d11/f17 0 2024-03-22T07:08:47.625 INFO:tasks.workunit.client.1.smithi149.stdout:1/61: fsync d4/f15 0 2024-03-22T07:08:47.625 INFO:tasks.workunit.client.1.smithi149.stdout:8/92: creat d1/d2/d9/f1b x:0 0 0 2024-03-22T07:08:47.625 INFO:tasks.workunit.client.1.smithi149.stdout:4/48: getdents d0/d8/dc 0 2024-03-22T07:08:47.625 INFO:tasks.workunit.client.1.smithi149.stdout:8/93: creat d1/d2/f1c x:0 0 0 2024-03-22T07:08:47.625 INFO:tasks.workunit.client.1.smithi149.stdout:8/94: readlink d1/d2/l12 0 2024-03-22T07:08:47.625 INFO:tasks.workunit.client.1.smithi149.stdout:8/95: read - d1/d2/d9/df/f19 zero size 2024-03-22T07:08:47.625 INFO:tasks.workunit.client.1.smithi149.stdout:8/96: creat d1/d2/d9/f1d x:0 0 0 2024-03-22T07:08:47.630 INFO:tasks.workunit.client.1.smithi149.stdout:8/97: mkdir d1/d2/d9/df/d1e 0 2024-03-22T07:08:47.632 INFO:tasks.workunit.client.1.smithi149.stdout:8/98: truncate d1/d2/d3/f6 3550556 0 2024-03-22T07:08:47.632 INFO:tasks.workunit.client.1.smithi149.stdout:8/99: creat d1/d2/d3/f1f x:0 0 0 2024-03-22T07:08:47.632 INFO:tasks.workunit.client.1.smithi149.stdout:8/100: write d1/d2/d9/df/f15 [32725,58862] 0 2024-03-22T07:08:47.632 INFO:tasks.workunit.client.1.smithi149.stdout:8/101: truncate d1/d2/d3/f1a 337486 0 2024-03-22T07:08:47.634 INFO:tasks.workunit.client.1.smithi149.stdout:8/102: rmdir d1/d2/d9/df/d1e 0 2024-03-22T07:08:47.634 INFO:tasks.workunit.client.1.smithi149.stdout:8/103: dread - d1/f16 zero size 2024-03-22T07:08:47.635 INFO:tasks.workunit.client.1.smithi149.stdout:8/104: mkdir d1/d20 0 2024-03-22T07:08:47.642 INFO:tasks.workunit.client.1.smithi149.stdout:1/62: dread f1 [0,4194304] 0 2024-03-22T07:08:47.643 INFO:tasks.workunit.client.1.smithi149.stdout:1/63: mknod d4/c16 0 2024-03-22T07:08:47.661 INFO:tasks.workunit.client.1.smithi149.stdout:2/137: dwrite d1/d3/d5/ff [0,4194304] 0 2024-03-22T07:08:47.662 INFO:tasks.workunit.client.1.smithi149.stdout:4/49: dwrite d0/d8/fd [0,4194304] 0 2024-03-22T07:08:47.662 INFO:tasks.workunit.client.1.smithi149.stdout:4/50: dread - d0/d8/dc/fe zero size 2024-03-22T07:08:47.662 INFO:tasks.workunit.client.1.smithi149.stdout:1/64: dwrite f1 [0,4194304] 0 2024-03-22T07:08:47.662 INFO:tasks.workunit.client.1.smithi149.stdout:2/138: mkdir d1/d3/d6/d32 0 2024-03-22T07:08:47.664 INFO:tasks.workunit.client.1.smithi149.stdout:1/65: unlink d4/d7/fc 0 2024-03-22T07:08:47.668 INFO:tasks.workunit.client.1.smithi149.stdout:2/139: getdents d1/d3/dd 0 2024-03-22T07:08:47.668 INFO:tasks.workunit.client.1.smithi149.stdout:2/140: chown d1/d3/d26/l2c 32204 1 2024-03-22T07:08:47.668 INFO:tasks.workunit.client.1.smithi149.stdout:1/66: getdents d4/d10 0 2024-03-22T07:08:47.668 INFO:tasks.workunit.client.1.smithi149.stdout:1/67: chown d4/f15 149 1 2024-03-22T07:08:47.669 INFO:tasks.workunit.client.1.smithi149.stdout:9/125: dwrite db/d19/f1f [0,4194304] 0 2024-03-22T07:08:47.669 INFO:tasks.workunit.client.1.smithi149.stdout:1/68: creat d4/f17 x:0 0 0 2024-03-22T07:08:47.670 INFO:tasks.workunit.client.1.smithi149.stdout:4/51: dread d0/d8/fd [0,4194304] 0 2024-03-22T07:08:47.672 INFO:tasks.workunit.client.1.smithi149.stdout:8/105: write d1/d2/d3/f6 [4295700,6344] 0 2024-03-22T07:08:47.680 INFO:tasks.workunit.client.1.smithi149.stdout:2/141: write d1/d3/d5/ff [161499,80193] 0 2024-03-22T07:08:47.683 INFO:tasks.workunit.client.1.smithi149.stdout:1/69: unlink d4/f8 0 2024-03-22T07:08:47.683 INFO:tasks.workunit.client.1.smithi149.stdout:8/106: read d1/d2/d3/fd [3447388,87660] 0 2024-03-22T07:08:47.711 INFO:tasks.workunit.client.1.smithi149.stdout:2/142: truncate d1/d8/f19 317860 0 2024-03-22T07:08:47.711 INFO:tasks.workunit.client.1.smithi149.stdout:2/143: fsync d1/d8/f24 0 2024-03-22T07:08:47.712 INFO:tasks.workunit.client.1.smithi149.stdout:9/126: getdents db/d19 0 2024-03-22T07:08:47.712 INFO:tasks.workunit.client.1.smithi149.stdout:9/127: read f9 [3794821,30526] 0 2024-03-22T07:08:47.713 INFO:tasks.workunit.client.1.smithi149.stdout:1/70: rename d4/c14 to d4/d7/c18 0 2024-03-22T07:08:47.715 INFO:tasks.workunit.client.1.smithi149.stdout:8/107: link d1/f16 d1/f21 0 2024-03-22T07:08:47.715 INFO:tasks.workunit.client.1.smithi149.stdout:8/108: write d1/d2/d9/df/f15 [733708,73072] 0 2024-03-22T07:08:47.716 INFO:tasks.workunit.client.1.smithi149.stdout:2/144: dread d1/f1a [0,4194304] 0 2024-03-22T07:08:47.735 INFO:tasks.workunit.client.1.smithi149.stdout:6/53: sync 2024-03-22T07:08:47.876 INFO:tasks.workunit.client.1.smithi149.stdout:1/71: mknod d4/d10/c19 0 2024-03-22T07:08:47.878 INFO:tasks.workunit.client.1.smithi149.stdout:8/109: mkdir d1/d2/d9/da/d22 0 2024-03-22T07:08:47.878 INFO:tasks.workunit.client.1.smithi149.stdout:8/110: creat d1/d2/f23 x:0 0 0 2024-03-22T07:08:47.878 INFO:tasks.workunit.client.1.smithi149.stdout:8/111: write d1/f16 [810935,60142] 0 2024-03-22T07:08:47.878 INFO:tasks.workunit.client.1.smithi149.stdout:8/112: stat d1/d2/d9/f1b 0 2024-03-22T07:08:47.878 INFO:tasks.workunit.client.1.smithi149.stdout:2/145: rmdir d1/d3 39 2024-03-22T07:08:47.879 INFO:tasks.workunit.client.1.smithi149.stdout:6/54: creat d9/df/f11 x:0 0 0 2024-03-22T07:08:47.880 INFO:tasks.workunit.client.1.smithi149.stdout:9/128: mkdir db/d21 0 2024-03-22T07:08:47.881 INFO:tasks.workunit.client.1.smithi149.stdout:1/72: write d4/fb [902296,70286] 0 2024-03-22T07:08:47.881 INFO:tasks.workunit.client.1.smithi149.stdout:1/73: truncate d4/fb 1426663 0 2024-03-22T07:08:47.883 INFO:tasks.workunit.client.1.smithi149.stdout:2/146: creat d1/d3/d26/d2f/f33 x:0 0 0 2024-03-22T07:08:47.885 INFO:tasks.workunit.client.1.smithi149.stdout:6/55: unlink d9/le 0 2024-03-22T07:08:47.887 INFO:tasks.workunit.client.1.smithi149.stdout:1/74: mknod d4/d10/c1a 0 2024-03-22T07:08:47.896 INFO:tasks.workunit.client.1.smithi149.stdout:8/113: dwrite d1/d2/d9/d11/f17 [0,4194304] 0 2024-03-22T07:08:47.898 INFO:tasks.workunit.client.1.smithi149.stdout:6/56: dread f6 [0,4194304] 0 2024-03-22T07:08:47.898 INFO:tasks.workunit.client.1.smithi149.stdout:6/57: stat f1 0 2024-03-22T07:08:47.901 INFO:tasks.workunit.client.1.smithi149.stdout:0/53: sync 2024-03-22T07:08:47.901 INFO:tasks.workunit.client.1.smithi149.stdout:5/55: sync 2024-03-22T07:08:47.901 INFO:tasks.workunit.client.1.smithi149.stdout:5/56: readlink d7/da/lf 0 2024-03-22T07:08:47.914 INFO:tasks.workunit.client.1.smithi149.stdout:8/114: dwrite d1/d2/d3/f1a [0,4194304] 0 2024-03-22T07:08:47.938 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:47 smithi081 ceph-mon[29245]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:47.938 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:47 smithi081 ceph-mon[29245]: Standby manager daemon smithi081.bcxarw started 2024-03-22T07:08:47.938 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:47 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.81:0/3007820561' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi081.bcxarw/crt"}]: dispatch 2024-03-22T07:08:47.956 INFO:tasks.workunit.client.1.smithi149.stdout:9/129: mkdir db/d22 0 2024-03-22T07:08:47.956 INFO:tasks.workunit.client.1.smithi149.stdout:9/130: stat db/d19/d1c 0 2024-03-22T07:08:47.956 INFO:tasks.workunit.client.1.smithi149.stdout:9/131: write db/d12/f15 [512554,59203] 0 2024-03-22T07:08:47.956 INFO:tasks.workunit.client.1.smithi149.stdout:9/132: write db/fc [1343512,33008] 0 2024-03-22T07:08:47.957 INFO:tasks.workunit.client.1.smithi149.stdout:1/75: creat d4/f1b x:0 0 0 2024-03-22T07:08:47.957 INFO:tasks.workunit.client.1.smithi149.stdout:1/76: fsync d4/d7/fe 0 2024-03-22T07:08:47.957 INFO:tasks.workunit.client.1.smithi149.stdout:1/77: stat d4/d10/f12 0 2024-03-22T07:08:47.957 INFO:tasks.workunit.client.1.smithi149.stdout:1/78: dread - d4/f1b zero size 2024-03-22T07:08:47.957 INFO:tasks.workunit.client.1.smithi149.stdout:1/79: dread - d4/f17 zero size 2024-03-22T07:08:47.958 INFO:tasks.workunit.client.1.smithi149.stdout:1/80: readlink d4/lf 0 2024-03-22T07:08:47.958 INFO:tasks.workunit.client.1.smithi149.stdout:1/81: fsync d4/d10/f12 0 2024-03-22T07:08:47.958 INFO:tasks.workunit.client.1.smithi149.stdout:1/82: write d4/d10/f12 [442581,117874] 0 2024-03-22T07:08:47.958 INFO:tasks.workunit.client.1.smithi149.stdout:1/83: readlink d4/lf 0 2024-03-22T07:08:47.958 INFO:tasks.workunit.client.1.smithi149.stdout:1/84: write d4/f15 [40958,7199] 0 2024-03-22T07:08:47.958 INFO:tasks.workunit.client.1.smithi149.stdout:6/58: rmdir d9/da 39 2024-03-22T07:08:47.959 INFO:tasks.workunit.client.1.smithi149.stdout:0/54: mknod d1/d9/cc 0 2024-03-22T07:08:47.959 INFO:tasks.workunit.client.1.smithi149.stdout:5/57: mknod d7/da/c14 0 2024-03-22T07:08:47.962 INFO:tasks.workunit.client.1.smithi149.stdout:6/59: unlink c7 0 2024-03-22T07:08:47.966 INFO:tasks.workunit.client.1.smithi149.stdout:0/55: creat d1/d9/fd x:0 0 0 2024-03-22T07:08:47.966 INFO:tasks.workunit.client.1.smithi149.stdout:6/60: creat d9/da/f12 x:0 0 0 2024-03-22T07:08:47.966 INFO:tasks.workunit.client.1.smithi149.stdout:0/56: write d1/d9/fd [121366,45096] 0 2024-03-22T07:08:47.966 INFO:tasks.workunit.client.1.smithi149.stdout:5/58: unlink l5 0 2024-03-22T07:08:47.966 INFO:tasks.workunit.client.1.smithi149.stdout:5/59: fdatasync d7/f8 0 2024-03-22T07:08:47.966 INFO:tasks.workunit.client.1.smithi149.stdout:5/60: truncate d7/f8 226252 0 2024-03-22T07:08:47.966 INFO:tasks.workunit.client.1.smithi149.stdout:5/61: write d7/f9 [506403,57769] 0 2024-03-22T07:08:47.967 INFO:tasks.workunit.client.1.smithi149.stdout:6/61: mknod d9/da/c13 0 2024-03-22T07:08:47.969 INFO:tasks.workunit.client.1.smithi149.stdout:5/62: truncate d7/db/fc 205143 0 2024-03-22T07:08:47.969 INFO:tasks.workunit.client.1.smithi149.stdout:5/63: stat d7/da 0 2024-03-22T07:08:47.980 INFO:tasks.workunit.client.1.smithi149.stdout:7/80: sync 2024-03-22T07:08:47.984 INFO:tasks.workunit.client.1.smithi149.stdout:7/81: link d1/lc d1/d6/l14 0 2024-03-22T07:08:47.984 INFO:tasks.workunit.client.1.smithi149.stdout:7/82: readlink d1/l9 0 2024-03-22T07:08:47.986 INFO:tasks.workunit.client.1.smithi149.stdout:7/83: symlink d1/l15 0 2024-03-22T07:08:47.986 INFO:tasks.workunit.client.1.smithi149.stdout:7/84: creat d1/d6/f16 x:0 0 0 2024-03-22T07:08:47.990 INFO:tasks.workunit.client.1.smithi149.stdout:7/85: mknod d1/c17 0 2024-03-22T07:08:47.991 INFO:tasks.workunit.client.1.smithi149.stdout:7/86: mkdir d1/d18 0 2024-03-22T07:08:47.993 INFO:tasks.workunit.client.1.smithi149.stdout:7/87: creat d1/f19 x:0 0 0 2024-03-22T07:08:47.993 INFO:tasks.workunit.client.1.smithi149.stdout:7/88: fdatasync d1/d6/f8 0 2024-03-22T07:08:47.993 INFO:tasks.workunit.client.1.smithi149.stdout:7/89: readlink d1/l12 0 2024-03-22T07:08:47.996 INFO:tasks.workunit.client.1.smithi149.stdout:7/90: write d1/d6/fd [904659,66349] 0 2024-03-22T07:08:47.997 INFO:tasks.workunit.client.1.smithi149.stdout:7/91: symlink d1/d6/l1a 0 2024-03-22T07:08:47.998 INFO:tasks.workunit.client.1.smithi149.stdout:7/92: mkdir d1/d18/d1b 0 2024-03-22T07:08:47.998 INFO:tasks.workunit.client.1.smithi149.stdout:9/133: dwrite db/d19/f1f [0,4194304] 0 2024-03-22T07:08:47.998 INFO:tasks.workunit.client.1.smithi149.stdout:7/93: mkdir d1/d1c 0 2024-03-22T07:08:48.005 INFO:tasks.workunit.client.1.smithi149.stdout:9/134: unlink db/l18 0 2024-03-22T07:08:48.005 INFO:tasks.workunit.client.1.smithi149.stdout:9/135: getdents db/d22 0 2024-03-22T07:08:48.007 INFO:tasks.workunit.client.1.smithi149.stdout:9/136: creat db/d21/f23 x:0 0 0 2024-03-22T07:08:48.008 INFO:tasks.workunit.client.1.smithi149.stdout:9/137: mknod db/d12/c24 0 2024-03-22T07:08:48.008 INFO:tasks.workunit.client.1.smithi149.stdout:9/138: chown db/d12/f17 770211681 1 2024-03-22T07:08:48.009 INFO:tasks.workunit.client.1.smithi149.stdout:6/62: dwrite d9/fc [4194304,4194304] 0 2024-03-22T07:08:48.011 INFO:tasks.workunit.client.1.smithi149.stdout:6/63: mknod d9/df/c14 0 2024-03-22T07:08:48.012 INFO:tasks.workunit.client.1.smithi149.stdout:0/57: dwrite d1/d9/fd [0,4194304] 0 2024-03-22T07:08:48.012 INFO:tasks.workunit.client.1.smithi149.stdout:0/58: write d1/fb [36321,78159] 0 2024-03-22T07:08:48.014 INFO:tasks.workunit.client.1.smithi149.stdout:5/64: dwrite d7/f9 [0,4194304] 0 2024-03-22T07:08:48.014 INFO:tasks.workunit.client.1.smithi149.stdout:0/59: link d1/c5 d1/ce 0 2024-03-22T07:08:48.014 INFO:tasks.workunit.client.1.smithi149.stdout:0/60: creat d1/d9/ff x:0 0 0 2024-03-22T07:08:48.014 INFO:tasks.workunit.client.1.smithi149.stdout:1/85: dwrite d4/f6 [0,4194304] 0 2024-03-22T07:08:48.014 INFO:tasks.workunit.client.1.smithi149.stdout:9/139: dread f9 [0,4194304] 0 2024-03-22T07:08:48.022 INFO:tasks.workunit.client.1.smithi149.stdout:7/94: rename d1/d6 to d1/d18/d1b/d1d 0 2024-03-22T07:08:48.022 INFO:tasks.workunit.client.1.smithi149.stdout:7/95: chown d1/d18/d1b/d1d/cf 2 1 2024-03-22T07:08:48.023 INFO:tasks.workunit.client.1.smithi149.stdout:1/86: symlink d4/l1c 0 2024-03-22T07:08:48.024 INFO:tasks.workunit.client.1.smithi149.stdout:7/96: creat d1/d18/f1e x:0 0 0 2024-03-22T07:08:48.027 INFO:tasks.workunit.client.1.smithi149.stdout:9/140: dread db/d19/f1f [0,4194304] 0 2024-03-22T07:08:48.027 INFO:tasks.workunit.client.1.smithi149.stdout:9/141: truncate db/d12/f17 2099680 0 2024-03-22T07:08:48.028 INFO:tasks.workunit.client.1.smithi149.stdout:7/97: unlink d1/d18/d1b/d1d/cf 0 2024-03-22T07:08:48.028 INFO:tasks.workunit.client.1.smithi149.stdout:7/98: truncate d1/d18/d1b/d1d/fd 1362919 0 2024-03-22T07:08:48.028 INFO:tasks.workunit.client.1.smithi149.stdout:7/99: readlink d1/l12 0 2024-03-22T07:08:48.029 INFO:tasks.workunit.client.1.smithi149.stdout:0/61: rename d1/c4 to d1/d9/c10 0 2024-03-22T07:08:48.031 INFO:tasks.workunit.client.1.smithi149.stdout:6/64: dwrite d9/da/f12 [0,4194304] 0 2024-03-22T07:08:48.031 INFO:tasks.workunit.client.1.smithi149.stdout:6/65: creat d9/df/f15 x:0 0 0 2024-03-22T07:08:48.031 INFO:tasks.workunit.client.1.smithi149.stdout:6/66: creat d9/df/f16 x:0 0 0 2024-03-22T07:08:48.032 INFO:tasks.workunit.client.1.smithi149.stdout:1/87: link c2 d4/c1d 0 2024-03-22T07:08:48.032 INFO:tasks.workunit.client.1.smithi149.stdout:1/88: fsync d4/f6 0 2024-03-22T07:08:48.032 INFO:tasks.workunit.client.1.smithi149.stdout:9/142: mknod db/d22/c25 0 2024-03-22T07:08:48.032 INFO:tasks.workunit.client.1.smithi149.stdout:6/67: write f5 [27710,72125] 0 2024-03-22T07:08:48.033 INFO:tasks.workunit.client.1.smithi149.stdout:7/100: creat d1/d18/d1b/d1d/f1f x:0 0 0 2024-03-22T07:08:48.035 INFO:tasks.workunit.client.1.smithi149.stdout:3/59: sync 2024-03-22T07:08:48.035 INFO:tasks.workunit.client.1.smithi149.stdout:3/60: creat d0/f1a x:0 0 0 2024-03-22T07:08:48.035 INFO:tasks.workunit.client.1.smithi149.stdout:4/52: sync 2024-03-22T07:08:48.037 INFO:tasks.workunit.client.1.smithi149.stdout:5/65: truncate d7/f9 2763553 0 2024-03-22T07:08:48.038 INFO:tasks.workunit.client.1.smithi149.stdout:5/66: chown c2 7 1 2024-03-22T07:08:48.038 INFO:tasks.workunit.client.1.smithi149.stdout:1/89: truncate f1 2741647 0 2024-03-22T07:08:48.041 INFO:tasks.workunit.client.1.smithi149.stdout:7/101: mknod d1/d18/d1b/c20 0 2024-03-22T07:08:48.042 INFO:tasks.workunit.client.1.smithi149.stdout:7/102: fsync d1/d18/d1b/d1d/f16 0 2024-03-22T07:08:48.042 INFO:tasks.workunit.client.1.smithi149.stdout:7/103: creat d1/f21 x:0 0 0 2024-03-22T07:08:48.042 INFO:tasks.workunit.client.1.smithi149.stdout:9/143: dread db/fd [0,4194304] 0 2024-03-22T07:08:48.042 INFO:tasks.workunit.client.1.smithi149.stdout:9/144: fdatasync db/d19/d1c/f20 0 2024-03-22T07:08:48.044 INFO:tasks.workunit.client.1.smithi149.stdout:3/61: rename d0/cd to d0/d4/d8/c1b 0 2024-03-22T07:08:48.044 INFO:tasks.workunit.client.1.smithi149.stdout:3/62: chown d0/d4/l16 20551814 1 2024-03-22T07:08:48.044 INFO:tasks.workunit.client.1.smithi149.stdout:3/63: fdatasync d0/f13 0 2024-03-22T07:08:48.044 INFO:tasks.workunit.client.1.smithi149.stdout:4/53: symlink d0/lf 0 2024-03-22T07:08:48.047 INFO:tasks.workunit.client.1.smithi149.stdout:1/90: symlink d4/d10/l1e 0 2024-03-22T07:08:48.047 INFO:tasks.workunit.client.1.smithi149.stdout:6/68: rmdir d9/da 39 2024-03-22T07:08:48.049 INFO:tasks.workunit.client.1.smithi149.stdout:7/104: truncate d1/f10 1457403 0 2024-03-22T07:08:48.051 INFO:tasks.workunit.client.1.smithi149.stdout:9/145: rename db/d12/f15 to db/d21/f26 0 2024-03-22T07:08:48.051 INFO:tasks.workunit.client.1.smithi149.stdout:9/146: read db/d12/f14 [506527,22030] 0 2024-03-22T07:08:48.052 INFO:tasks.workunit.client.1.smithi149.stdout:3/64: mkdir d0/d4/d8/d1c 0 2024-03-22T07:08:48.052 INFO:tasks.workunit.client.1.smithi149.stdout:3/65: write d0/d4/d8/f19 [284172,15877] 0 2024-03-22T07:08:48.052 INFO:tasks.workunit.client.1.smithi149.stdout:3/66: creat d0/d4/d8/f1d x:0 0 0 2024-03-22T07:08:48.052 INFO:tasks.workunit.client.1.smithi149.stdout:3/67: chown d0/d2/l11 5986 1 2024-03-22T07:08:48.052 INFO:tasks.workunit.client.1.smithi149.stdout:4/54: symlink d0/d8/l10 0 2024-03-22T07:08:48.052 INFO:tasks.workunit.client.1.smithi149.stdout:9/147: dread db/fc [0,4194304] 0 2024-03-22T07:08:48.053 INFO:tasks.workunit.client.1.smithi149.stdout:5/67: getdents d7/da 0 2024-03-22T07:08:48.056 INFO:tasks.workunit.client.1.smithi149.stdout:6/69: unlink d9/fc 0 2024-03-22T07:08:48.056 INFO:tasks.workunit.client.1.smithi149.stdout:7/105: truncate d1/f5 2264885 0 2024-03-22T07:08:48.058 INFO:tasks.workunit.client.1.smithi149.stdout:3/68: creat d0/d4/f1e x:0 0 0 2024-03-22T07:08:48.059 INFO:tasks.workunit.client.1.smithi149.stdout:4/55: write d0/d8/fd [3058173,13080] 0 2024-03-22T07:08:48.061 INFO:tasks.workunit.client.1.smithi149.stdout:5/68: link d7/da/c14 d7/c15 0 2024-03-22T07:08:48.061 INFO:tasks.workunit.client.1.smithi149.stdout:5/69: fsync d7/da/fd 0 2024-03-22T07:08:48.062 INFO:tasks.workunit.client.1.smithi149.stdout:1/91: mkdir d4/d1f 0 2024-03-22T07:08:48.062 INFO:tasks.workunit.client.1.smithi149.stdout:1/92: dread - d4/f17 zero size 2024-03-22T07:08:48.062 INFO:tasks.workunit.client.1.smithi149.stdout:1/93: chown d4/d10/f12 904983 1 2024-03-22T07:08:48.062 INFO:tasks.workunit.client.1.smithi149.stdout:1/94: chown d4/f6 3826 1 2024-03-22T07:08:48.062 INFO:tasks.workunit.client.1.smithi149.stdout:1/95: read - d4/d7/fe zero size 2024-03-22T07:08:48.062 INFO:tasks.workunit.client.1.smithi149.stdout:1/96: stat d4/f6 0 2024-03-22T07:08:48.063 INFO:tasks.workunit.client.1.smithi149.stdout:7/106: rename d1/d18/d1b/d1d/c11 to d1/d18/d1b/d1d/c22 0 2024-03-22T07:08:48.065 INFO:tasks.workunit.client.1.smithi149.stdout:9/148: symlink db/l27 0 2024-03-22T07:08:48.065 INFO:tasks.workunit.client.1.smithi149.stdout:9/149: fsync db/d12/f17 0 2024-03-22T07:08:48.065 INFO:tasks.workunit.client.1.smithi149.stdout:9/150: chown db/d19/f1f 3754 1 2024-03-22T07:08:48.065 INFO:tasks.workunit.client.1.smithi149.stdout:6/70: dread f6 [0,4194304] 0 2024-03-22T07:08:48.065 INFO:tasks.workunit.client.1.smithi149.stdout:6/71: creat d9/f17 x:0 0 0 2024-03-22T07:08:48.068 INFO:tasks.workunit.client.1.smithi149.stdout:9/151: dread f3 [0,4194304] 0 2024-03-22T07:08:48.071 INFO:tasks.workunit.client.1.smithi149.stdout:5/70: symlink d7/l16 0 2024-03-22T07:08:48.071 INFO:tasks.workunit.client.1.smithi149.stdout:5/71: fdatasync d7/db/fc 0 2024-03-22T07:08:48.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/72: symlink d9/df/l18 0 2024-03-22T07:08:48.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/73: read - d9/df/f11 zero size 2024-03-22T07:08:48.076 INFO:tasks.workunit.client.1.smithi149.stdout:9/152: rename db/d12/d1a to db/d19/d28 0 2024-03-22T07:08:48.078 INFO:tasks.workunit.client.1.smithi149.stdout:1/97: mkdir d4/d20 0 2024-03-22T07:08:48.080 INFO:tasks.workunit.client.1.smithi149.stdout:9/153: symlink db/d19/d28/l29 0 2024-03-22T07:08:48.082 INFO:tasks.workunit.client.1.smithi149.stdout:1/98: dread d4/f11 [0,4194304] 0 2024-03-22T07:08:48.083 INFO:tasks.workunit.client.1.smithi149.stdout:4/56: truncate d0/d8/fd 2238603 0 2024-03-22T07:08:48.083 INFO:tasks.workunit.client.1.smithi149.stdout:7/107: dwrite d1/f19 [0,4194304] 0 2024-03-22T07:08:48.083 INFO:tasks.workunit.client.1.smithi149.stdout:7/108: fdatasync d1/d18/d1b/d1d/f16 0 2024-03-22T07:08:48.083 INFO:tasks.workunit.client.1.smithi149.stdout:1/99: symlink d4/d1f/l21 0 2024-03-22T07:08:48.084 INFO:tasks.workunit.client.1.smithi149.stdout:1/100: write d4/fb [1467807,91559] 0 2024-03-22T07:08:48.084 INFO:tasks.workunit.client.1.smithi149.stdout:5/72: getdents d7 0 2024-03-22T07:08:48.084 INFO:tasks.workunit.client.1.smithi149.stdout:5/73: write d7/da/fe [5892351,103563] 0 2024-03-22T07:08:48.086 INFO:tasks.workunit.client.1.smithi149.stdout:4/57: mknod d0/d8/c11 0 2024-03-22T07:08:48.087 INFO:tasks.workunit.client.1.smithi149.stdout:7/109: link d1/lc d1/d18/d1b/d1d/l23 0 2024-03-22T07:08:48.087 INFO:tasks.workunit.client.1.smithi149.stdout:1/101: creat d4/d20/f22 x:0 0 0 2024-03-22T07:08:48.088 INFO:tasks.workunit.client.1.smithi149.stdout:4/58: mknod d0/c12 0 2024-03-22T07:08:48.088 INFO:tasks.workunit.client.1.smithi149.stdout:4/59: chown d0/c12 1536446 1 2024-03-22T07:08:48.088 INFO:tasks.workunit.client.1.smithi149.stdout:4/60: stat d0/d8/l10 0 2024-03-22T07:08:48.092 INFO:tasks.workunit.client.1.smithi149.stdout:5/74: write d7/f9 [1902446,104014] 0 2024-03-22T07:08:48.092 INFO:tasks.workunit.client.1.smithi149.stdout:5/75: chown d7/db/fc 85442927 1 2024-03-22T07:08:48.095 INFO:tasks.workunit.client.1.smithi149.stdout:6/74: dwrite d9/fd [4194304,4194304] 0 2024-03-22T07:08:48.099 INFO:tasks.workunit.client.1.smithi149.stdout:4/61: getdents d0/d8 0 2024-03-22T07:08:48.099 INFO:tasks.workunit.client.1.smithi149.stdout:4/62: chown d0/d8/dc/fe 1 1 2024-03-22T07:08:48.101 INFO:tasks.workunit.client.1.smithi149.stdout:1/102: read d4/f6 [169357,71887] 0 2024-03-22T07:08:48.101 INFO:tasks.workunit.client.1.smithi149.stdout:1/103: chown d4/f11 32212964 1 2024-03-22T07:08:48.102 INFO:tasks.workunit.client.1.smithi149.stdout:1/104: mkdir d4/d20/d23 0 2024-03-22T07:08:48.103 INFO:tasks.workunit.client.1.smithi149.stdout:5/76: truncate d7/f9 216684 0 2024-03-22T07:08:48.103 INFO:tasks.workunit.client.1.smithi149.stdout:5/77: write d7/da/fe [4817203,12710] 0 2024-03-22T07:08:48.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/105: link d4/cd d4/d20/d23/c24 0 2024-03-22T07:08:48.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/106: write d4/f6 [2743347,117750] 0 2024-03-22T07:08:48.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/107: chown d4/d7/c18 148161558 1 2024-03-22T07:08:48.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/108: write d4/d10/f13 [1085698,3783] 0 2024-03-22T07:08:48.107 INFO:tasks.workunit.client.1.smithi149.stdout:1/109: rename d4/l1c to d4/d20/l25 0 2024-03-22T07:08:48.109 INFO:tasks.workunit.client.1.smithi149.stdout:6/75: dread d9/fd [0,4194304] 0 2024-03-22T07:08:48.110 INFO:tasks.workunit.client.1.smithi149.stdout:6/76: fsync f6 0 2024-03-22T07:08:48.111 INFO:tasks.workunit.client.1.smithi149.stdout:6/77: link f1 d9/f19 0 2024-03-22T07:08:48.111 INFO:tasks.workunit.client.1.smithi149.stdout:6/78: creat d9/df/f1a x:0 0 0 2024-03-22T07:08:48.111 INFO:tasks.workunit.client.1.smithi149.stdout:6/79: fsync d9/f17 0 2024-03-22T07:08:48.113 INFO:tasks.workunit.client.1.smithi149.stdout:1/110: dread d4/d10/f13 [0,4194304] 0 2024-03-22T07:08:48.115 INFO:tasks.workunit.client.1.smithi149.stdout:5/78: truncate d7/da/fe 2894952 0 2024-03-22T07:08:48.116 INFO:tasks.workunit.client.1.smithi149.stdout:5/79: rmdir d7 39 2024-03-22T07:08:48.124 INFO:tasks.workunit.client.1.smithi149.stdout:8/115: sync 2024-03-22T07:08:48.124 INFO:tasks.workunit.client.1.smithi149.stdout:2/147: sync 2024-03-22T07:08:48.124 INFO:tasks.workunit.client.1.smithi149.stdout:2/148: truncate d1/d3/f25 484096 0 2024-03-22T07:08:48.125 INFO:tasks.workunit.client.1.smithi149.stdout:8/116: creat d1/f24 x:0 0 0 2024-03-22T07:08:48.127 INFO:tasks.workunit.client.1.smithi149.stdout:2/149: mkdir d1/d8/d34 0 2024-03-22T07:08:48.129 INFO:tasks.workunit.client.1.smithi149.stdout:4/63: dwrite d0/d8/dc/fe [0,4194304] 0 2024-03-22T07:08:48.129 INFO:tasks.workunit.client.1.smithi149.stdout:8/117: dread d1/f16 [0,4194304] 0 2024-03-22T07:08:48.131 INFO:tasks.workunit.client.1.smithi149.stdout:8/118: symlink d1/d2/d3/l25 0 2024-03-22T07:08:48.132 INFO:tasks.workunit.client.1.smithi149.stdout:8/119: mknod d1/d2/d9/df/c26 0 2024-03-22T07:08:48.133 INFO:tasks.workunit.client.1.smithi149.stdout:8/120: link d1/d2/d9/lc d1/d2/d9/l27 0 2024-03-22T07:08:48.133 INFO:tasks.workunit.client.1.smithi149.stdout:8/121: mkdir d1/d2/d9/d11/d28 0 2024-03-22T07:08:48.135 INFO:tasks.workunit.client.1.smithi149.stdout:8/122: read d1/d2/d9/da/fb [1042861,83609] 0 2024-03-22T07:08:48.144 INFO:tasks.workunit.client.1.smithi149.stdout:2/150: dread d1/f27 [0,4194304] 0 2024-03-22T07:08:48.146 INFO:tasks.workunit.client.1.smithi149.stdout:4/64: write d0/f3 [2772228,90962] 0 2024-03-22T07:08:48.147 INFO:tasks.workunit.client.1.smithi149.stdout:6/80: dwrite d9/f17 [0,4194304] 0 2024-03-22T07:08:48.149 INFO:tasks.workunit.client.1.smithi149.stdout:1/111: dwrite d4/d10/f13 [0,4194304] 0 2024-03-22T07:08:48.151 INFO:tasks.workunit.client.1.smithi149.stdout:1/112: unlink d4/d10/f13 0 2024-03-22T07:08:48.152 INFO:tasks.workunit.client.1.smithi149.stdout:1/113: write d4/f6 [2272738,97196] 0 2024-03-22T07:08:48.152 INFO:tasks.workunit.client.1.smithi149.stdout:1/114: mknod d4/d20/d23/c26 0 2024-03-22T07:08:48.153 INFO:tasks.workunit.client.1.smithi149.stdout:1/115: creat d4/d20/d23/f27 x:0 0 0 2024-03-22T07:08:48.153 INFO:tasks.workunit.client.1.smithi149.stdout:1/116: write d4/d10/f12 [617810,67222] 0 2024-03-22T07:08:48.154 INFO:tasks.workunit.client.1.smithi149.stdout:1/117: mkdir d4/d10/d28 0 2024-03-22T07:08:48.155 INFO:tasks.workunit.client.1.smithi149.stdout:1/118: symlink d4/d20/d23/l29 0 2024-03-22T07:08:48.160 INFO:tasks.workunit.client.1.smithi149.stdout:8/123: dwrite d1/d2/d9/df/f19 [0,4194304] 0 2024-03-22T07:08:48.162 INFO:tasks.workunit.client.1.smithi149.stdout:6/81: write f6 [2873635,57082] 0 2024-03-22T07:08:48.165 INFO:tasks.workunit.client.1.smithi149.stdout:6/82: rmdir d9 39 2024-03-22T07:08:48.169 INFO:tasks.workunit.client.1.smithi149.stdout:4/65: dread d0/f3 [0,4194304] 0 2024-03-22T07:08:48.170 INFO:tasks.workunit.client.1.smithi149.stdout:5/80: read d7/da/fe [331641,100080] 0 2024-03-22T07:08:48.171 INFO:tasks.workunit.client.1.smithi149.stdout:4/66: creat d0/d8/dc/f13 x:0 0 0 2024-03-22T07:08:48.171 INFO:tasks.workunit.client.1.smithi149.stdout:4/67: fdatasync d0/d8/dc/fe 0 2024-03-22T07:08:48.173 INFO:tasks.workunit.client.1.smithi149.stdout:5/81: creat d7/f17 x:0 0 0 2024-03-22T07:08:48.173 INFO:tasks.workunit.client.1.smithi149.stdout:5/82: chown d7/da/f12 4354 1 2024-03-22T07:08:48.174 INFO:tasks.workunit.client.1.smithi149.stdout:4/68: mkdir d0/d8/dc/d14 0 2024-03-22T07:08:48.176 INFO:tasks.workunit.client.1.smithi149.stdout:5/83: rename d7/c10 to d7/db/c18 0 2024-03-22T07:08:48.178 INFO:tasks.workunit.client.1.smithi149.stdout:5/84: mknod d7/da/c19 0 2024-03-22T07:08:48.178 INFO:tasks.workunit.client.1.smithi149.stdout:5/85: truncate d7/da/fe 3409862 0 2024-03-22T07:08:48.178 INFO:tasks.workunit.client.1.smithi149.stdout:1/119: rmdir d4/d10 39 2024-03-22T07:08:48.178 INFO:tasks.workunit.client.1.smithi149.stdout:1/120: write d4/f6 [945291,2454] 0 2024-03-22T07:08:48.181 INFO:tasks.workunit.client.1.smithi149.stdout:1/121: creat d4/d10/d28/f2a x:0 0 0 2024-03-22T07:08:48.184 INFO:tasks.workunit.client.1.smithi149.stdout:1/122: symlink d4/d7/l2b 0 2024-03-22T07:08:48.184 INFO:tasks.workunit.client.1.smithi149.stdout:1/123: fsync d4/d20/d23/f27 0 2024-03-22T07:08:48.185 INFO:tasks.workunit.client.1.smithi149.stdout:1/124: mkdir d4/d10/d28/d2c 0 2024-03-22T07:08:48.187 INFO:tasks.workunit.client.1.smithi149.stdout:1/125: dread d4/f11 [0,4194304] 0 2024-03-22T07:08:48.188 INFO:tasks.workunit.client.1.smithi149.stdout:5/86: getdents d7 0 2024-03-22T07:08:48.190 INFO:tasks.workunit.client.1.smithi149.stdout:5/87: mknod d7/c1a 0 2024-03-22T07:08:48.193 INFO:tasks.workunit.client.1.smithi149.stdout:5/88: dread d7/f9 [0,4194304] 0 2024-03-22T07:08:48.195 INFO:tasks.workunit.client.1.smithi149.stdout:5/89: write d7/da/fd [845309,129114] 0 2024-03-22T07:08:48.195 INFO:tasks.workunit.client.1.smithi149.stdout:1/126: dread d4/fb [0,4194304] 0 2024-03-22T07:08:48.195 INFO:tasks.workunit.client.1.smithi149.stdout:1/127: chown d4/d7/l2b 449 1 2024-03-22T07:08:48.196 INFO:tasks.workunit.client.1.smithi149.stdout:1/128: mkdir d4/d20/d2d 0 2024-03-22T07:08:48.196 INFO:tasks.workunit.client.1.smithi149.stdout:1/129: creat d4/d10/d28/f2e x:0 0 0 2024-03-22T07:08:48.198 INFO:tasks.workunit.client.1.smithi149.stdout:1/130: rename d4/d10/d28/d2c to d4/d20/d23/d2f 0 2024-03-22T07:08:48.199 INFO:tasks.workunit.client.1.smithi149.stdout:1/131: link d4/f15 d4/d10/f30 0 2024-03-22T07:08:48.199 INFO:tasks.workunit.client.1.smithi149.stdout:1/132: write d4/d20/d23/f27 [535430,5954] 0 2024-03-22T07:08:48.204 INFO:tasks.workunit.client.1.smithi149.stdout:8/124: dwrite d1/d2/d3/f6 [0,4194304] 0 2024-03-22T07:08:48.205 INFO:tasks.workunit.client.1.smithi149.stdout:8/125: stat d1/d2/d9/l27 0 2024-03-22T07:08:48.205 INFO:tasks.workunit.client.1.smithi149.stdout:8/126: dread - d1/d2/d9/f1b zero size 2024-03-22T07:08:48.205 INFO:tasks.workunit.client.1.smithi149.stdout:8/127: readlink d1/d2/d3/l25 0 2024-03-22T07:08:48.206 INFO:tasks.workunit.client.1.smithi149.stdout:4/69: dwrite d0/d8/dc/f13 [0,4194304] 0 2024-03-22T07:08:48.206 INFO:tasks.workunit.client.1.smithi149.stdout:4/70: chown d0/d8/dc/f13 14509631 1 2024-03-22T07:08:48.207 INFO:tasks.workunit.client.1.smithi149.stdout:8/128: link d1/d2/d9/df/f15 d1/d2/d9/f29 0 2024-03-22T07:08:48.207 INFO:tasks.workunit.client.1.smithi149.stdout:8/129: chown d1/d2/f1c 1719 1 2024-03-22T07:08:48.207 INFO:tasks.workunit.client.1.smithi149.stdout:6/83: dwrite d9/f17 [0,4194304] 0 2024-03-22T07:08:48.208 INFO:tasks.workunit.client.1.smithi149.stdout:1/133: rmdir d4/d7 39 2024-03-22T07:08:48.208 INFO:tasks.workunit.client.1.smithi149.stdout:1/134: stat d4/d10/d28/f2a 0 2024-03-22T07:08:48.209 INFO:tasks.workunit.client.1.smithi149.stdout:1/135: write d4/d10/f30 [949392,86865] 0 2024-03-22T07:08:48.210 INFO:tasks.workunit.client.1.smithi149.stdout:4/71: symlink d0/d8/dc/l15 0 2024-03-22T07:08:48.211 INFO:tasks.workunit.client.1.smithi149.stdout:8/130: mknod d1/d2/d9/d11/c2a 0 2024-03-22T07:08:48.212 INFO:tasks.workunit.client.1.smithi149.stdout:6/84: creat d9/da/f1b x:0 0 0 2024-03-22T07:08:48.213 INFO:tasks.workunit.client.1.smithi149.stdout:4/72: rmdir d0 39 2024-03-22T07:08:48.213 INFO:tasks.workunit.client.1.smithi149.stdout:4/73: fdatasync d0/d8/fd 0 2024-03-22T07:08:48.214 INFO:tasks.workunit.client.1.smithi149.stdout:4/74: creat d0/d8/dc/d14/f16 x:0 0 0 2024-03-22T07:08:48.215 INFO:tasks.workunit.client.1.smithi149.stdout:5/90: dwrite d7/da/fe [0,4194304] 0 2024-03-22T07:08:48.215 INFO:tasks.workunit.client.1.smithi149.stdout:5/91: write d7/da/f12 [80288,13684] 0 2024-03-22T07:08:48.217 INFO:tasks.workunit.client.1.smithi149.stdout:5/92: write d7/da/fd [193031,84097] 0 2024-03-22T07:08:48.219 INFO:tasks.workunit.client.1.smithi149.stdout:1/136: dread d4/d10/f30 [0,4194304] 0 2024-03-22T07:08:48.222 INFO:tasks.workunit.client.1.smithi149.stdout:5/93: link d7/db/fc d7/da/f1b 0 2024-03-22T07:08:48.222 INFO:tasks.workunit.client.1.smithi149.stdout:5/94: stat d7/da/fe 0 2024-03-22T07:08:48.224 INFO:tasks.workunit.client.1.smithi149.stdout:5/95: symlink d7/db/l1c 0 2024-03-22T07:08:48.230 INFO:tasks.workunit.client.1.smithi149.stdout:4/75: fsync d0/d8/dc/d14/f16 0 2024-03-22T07:08:48.233 INFO:tasks.workunit.client.1.smithi149.stdout:4/76: symlink d0/d8/dc/l17 0 2024-03-22T07:08:48.236 INFO:tasks.workunit.client.1.smithi149.stdout:4/77: rmdir d0/d8 39 2024-03-22T07:08:48.252 INFO:tasks.workunit.client.1.smithi149.stdout:4/78: dread d0/d8/dc/fe [0,4194304] 0 2024-03-22T07:08:48.256 INFO:tasks.workunit.client.1.smithi149.stdout:5/96: dwrite d7/db/fc [0,4194304] 0 2024-03-22T07:08:48.257 INFO:tasks.workunit.client.1.smithi149.stdout:6/85: dwrite f8 [0,4194304] 0 2024-03-22T07:08:48.257 INFO:tasks.workunit.client.1.smithi149.stdout:6/86: creat d9/da/f1c x:0 0 0 2024-03-22T07:08:48.257 INFO:tasks.workunit.client.1.smithi149.stdout:8/131: dwrite d1/d2/d3/fd [0,4194304] 0 2024-03-22T07:08:48.258 INFO:tasks.workunit.client.1.smithi149.stdout:5/97: write d7/f17 [432503,101876] 0 2024-03-22T07:08:48.258 INFO:tasks.workunit.client.1.smithi149.stdout:5/98: stat d7/da/fd 0 2024-03-22T07:08:48.261 INFO:tasks.workunit.client.1.smithi149.stdout:1/137: dwrite d4/f17 [0,4194304] 0 2024-03-22T07:08:48.261 INFO:tasks.workunit.client.1.smithi149.stdout:6/87: creat d9/df/f1d x:0 0 0 2024-03-22T07:08:48.261 INFO:tasks.workunit.client.1.smithi149.stdout:6/88: write f5 [1318183,113420] 0 2024-03-22T07:08:48.262 INFO:tasks.workunit.client.1.smithi149.stdout:6/89: truncate d9/df/f1d 918373 0 2024-03-22T07:08:48.262 INFO:tasks.workunit.client.1.smithi149.stdout:6/90: readlink d9/df/l18 0 2024-03-22T07:08:48.262 INFO:tasks.workunit.client.1.smithi149.stdout:6/91: dread - d9/da/f1b zero size 2024-03-22T07:08:48.262 INFO:tasks.workunit.client.1.smithi149.stdout:6/92: readlink l4 0 2024-03-22T07:08:48.262 INFO:tasks.workunit.client.1.smithi149.stdout:6/93: chown l4 19155137 1 2024-03-22T07:08:48.262 INFO:tasks.workunit.client.1.smithi149.stdout:6/94: creat d9/df/f1e x:0 0 0 2024-03-22T07:08:48.262 INFO:tasks.workunit.client.1.smithi149.stdout:6/95: dread - d9/df/f11 zero size 2024-03-22T07:08:48.262 INFO:tasks.workunit.client.1.smithi149.stdout:1/138: write d4/f15 [264576,76164] 0 2024-03-22T07:08:48.263 INFO:tasks.workunit.client.1.smithi149.stdout:5/99: rename d7/db/c18 to d7/da/c1d 0 2024-03-22T07:08:48.269 INFO:tasks.workunit.client.1.smithi149.stdout:6/96: unlink d9/df/f11 0 2024-03-22T07:08:48.270 INFO:tasks.workunit.client.1.smithi149.stdout:5/100: mknod d7/db/c1e 0 2024-03-22T07:08:48.272 INFO:tasks.workunit.client.1.smithi149.stdout:1/139: rename d4/cd to d4/d20/c31 0 2024-03-22T07:08:48.274 INFO:tasks.workunit.client.1.smithi149.stdout:6/97: truncate d9/f19 4886200 0 2024-03-22T07:08:48.275 INFO:tasks.workunit.client.1.smithi149.stdout:5/101: fsync d7/f8 0 2024-03-22T07:08:48.275 INFO:tasks.workunit.client.1.smithi149.stdout:4/79: dwrite d0/d8/dc/fe [0,4194304] 0 2024-03-22T07:08:48.276 INFO:tasks.workunit.client.1.smithi149.stdout:1/140: creat d4/d1f/f32 x:0 0 0 2024-03-22T07:08:48.277 INFO:tasks.workunit.client.1.smithi149.stdout:6/98: mkdir d9/d1f 0 2024-03-22T07:08:48.283 INFO:tasks.workunit.client.1.smithi149.stdout:5/102: rename d7/da/c1d to d7/db/c1f 0 2024-03-22T07:08:48.283 INFO:tasks.workunit.client.1.smithi149.stdout:5/103: chown d7/da/f1b 4061347 1 2024-03-22T07:08:48.283 INFO:tasks.workunit.client.1.smithi149.stdout:5/104: chown d7/db/c1e 25684220 1 2024-03-22T07:08:48.283 INFO:tasks.workunit.client.1.smithi149.stdout:5/105: read d7/f9 [196769,121052] 0 2024-03-22T07:08:48.283 INFO:tasks.workunit.client.1.smithi149.stdout:5/106: chown d7/da/c14 31201 1 2024-03-22T07:08:48.284 INFO:tasks.workunit.client.1.smithi149.stdout:1/141: rename d4/fb to d4/d7/f33 0 2024-03-22T07:08:48.284 INFO:tasks.workunit.client.1.smithi149.stdout:1/142: chown d4/d1f/l21 421 1 2024-03-22T07:08:48.286 INFO:tasks.workunit.client.1.smithi149.stdout:1/143: rename d4/d1f/f32 to d4/d10/d28/f34 0 2024-03-22T07:08:48.292 INFO:tasks.workunit.client.1.smithi149.stdout:4/80: write d0/d8/dc/f13 [3593188,82600] 0 2024-03-22T07:08:48.292 INFO:tasks.workunit.client.1.smithi149.stdout:4/81: creat d0/d8/dc/f18 x:0 0 0 2024-03-22T07:08:48.296 INFO:tasks.workunit.client.1.smithi149.stdout:5/107: dread d7/da/fe [0,4194304] 0 2024-03-22T07:08:48.301 INFO:tasks.workunit.client.1.smithi149.stdout:8/132: dwrite d1/d2/d3/fd [0,4194304] 0 2024-03-22T07:08:48.304 INFO:tasks.workunit.client.1.smithi149.stdout:6/99: dwrite f8 [0,4194304] 0 2024-03-22T07:08:48.304 INFO:tasks.workunit.client.1.smithi149.stdout:6/100: write d9/df/f1e [890293,48261] 0 2024-03-22T07:08:48.304 INFO:tasks.workunit.client.1.smithi149.stdout:6/101: dread - d9/df/f16 zero size 2024-03-22T07:08:48.317 INFO:tasks.workunit.client.1.smithi149.stdout:6/102: write f6 [3384994,32312] 0 2024-03-22T07:08:48.317 INFO:tasks.workunit.client.1.smithi149.stdout:6/103: creat d9/da/f20 x:0 0 0 2024-03-22T07:08:48.319 INFO:tasks.workunit.client.1.smithi149.stdout:1/144: dwrite d4/f1b [0,4194304] 0 2024-03-22T07:08:48.322 INFO:tasks.workunit.client.1.smithi149.stdout:1/145: dread d4/d20/d23/f27 [0,4194304] 0 2024-03-22T07:08:48.322 INFO:tasks.workunit.client.1.smithi149.stdout:1/146: stat d4/d10/d28 0 2024-03-22T07:08:48.322 INFO:tasks.workunit.client.1.smithi149.stdout:1/147: write d4/d10/f12 [1157729,65025] 0 2024-03-22T07:08:48.329 INFO:tasks.workunit.client.1.smithi149.stdout:1/148: write d4/f17 [2599181,19731] 0 2024-03-22T07:08:48.330 INFO:tasks.workunit.client.1.smithi149.stdout:5/108: dwrite d7/f17 [0,4194304] 0 2024-03-22T07:08:48.633 INFO:tasks.workunit.client.1.smithi149.stdout:4/82: mknod d0/d8/dc/d14/c19 0 2024-03-22T07:08:48.633 INFO:tasks.workunit.client.1.smithi149.stdout:4/83: dread - d0/d8/dc/f18 zero size 2024-03-22T07:08:48.635 INFO:tasks.workunit.client.1.smithi149.stdout:8/133: symlink d1/d2/d3/l2b 0 2024-03-22T07:08:48.635 INFO:tasks.workunit.client.1.smithi149.stdout:8/134: creat d1/f2c x:0 0 0 2024-03-22T07:08:48.636 INFO:tasks.workunit.client.1.smithi149.stdout:8/135: read d1/f21 [762719,88090] 0 2024-03-22T07:08:48.645 INFO:tasks.workunit.client.1.smithi149.stdout:6/104: unlink l4 0 2024-03-22T07:08:48.648 INFO:tasks.workunit.client.1.smithi149.stdout:1/149: creat d4/d20/d23/d2f/f35 x:0 0 0 2024-03-22T07:08:48.648 INFO:tasks.workunit.client.1.smithi149.stdout:1/150: chown d4/d7/f33 289195 1 2024-03-22T07:08:48.648 INFO:tasks.workunit.client.1.smithi149.stdout:1/151: readlink d4/d20/l25 0 2024-03-22T07:08:48.648 INFO:tasks.workunit.client.1.smithi149.stdout:1/152: write d4/f17 [1725464,95964] 0 2024-03-22T07:08:48.648 INFO:tasks.workunit.client.1.smithi149.stdout:1/153: chown d4/d20/c31 1 1 2024-03-22T07:08:48.648 INFO:tasks.workunit.client.1.smithi149.stdout:5/109: mkdir d7/da/d20 0 2024-03-22T07:08:48.652 INFO:tasks.workunit.client.1.smithi149.stdout:4/84: truncate d0/f3 958077 0 2024-03-22T07:08:48.652 INFO:tasks.workunit.client.1.smithi149.stdout:8/136: creat d1/d2/d9/df/f2d x:0 0 0 2024-03-22T07:08:48.653 INFO:tasks.workunit.client.1.smithi149.stdout:6/105: creat d9/d1f/f21 x:0 0 0 2024-03-22T07:08:48.655 INFO:tasks.workunit.client.1.smithi149.stdout:1/154: rename d4/d20/d23/c24 to d4/d20/d2d/c36 0 2024-03-22T07:08:48.655 INFO:tasks.workunit.client.1.smithi149.stdout:1/155: creat d4/d10/d28/f37 x:0 0 0 2024-03-22T07:08:48.659 INFO:tasks.workunit.client.1.smithi149.stdout:6/106: mkdir d9/df/d22 0 2024-03-22T07:08:48.659 INFO:tasks.workunit.client.1.smithi149.stdout:6/107: truncate d9/da/f20 908303 0 2024-03-22T07:08:48.659 INFO:tasks.workunit.client.1.smithi149.stdout:6/108: creat d9/da/f23 x:0 0 0 2024-03-22T07:08:48.660 INFO:tasks.workunit.client.1.smithi149.stdout:1/156: truncate d4/d7/f33 319078 0 2024-03-22T07:08:48.661 INFO:tasks.workunit.client.1.smithi149.stdout:1/157: stat d4/d10/l1e 0 2024-03-22T07:08:48.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/109: mkdir d9/d1f/d24 0 2024-03-22T07:08:48.663 INFO:tasks.workunit.client.1.smithi149.stdout:1/158: symlink d4/d20/d2d/l38 0 2024-03-22T07:08:48.663 INFO:tasks.workunit.client.1.smithi149.stdout:1/159: read - d4/d20/f22 zero size 2024-03-22T07:08:48.666 INFO:tasks.workunit.client.1.smithi149.stdout:1/160: creat d4/d7/f39 x:0 0 0 2024-03-22T07:08:48.666 INFO:tasks.workunit.client.1.smithi149.stdout:1/161: fsync d4/d20/f22 0 2024-03-22T07:08:48.677 INFO:tasks.workunit.client.1.smithi149.stdout:7/110: sync 2024-03-22T07:08:48.677 INFO:tasks.workunit.client.1.smithi149.stdout:7/111: chown d1/d18/d1b/c20 7 1 2024-03-22T07:08:48.677 INFO:tasks.workunit.client.1.smithi149.stdout:9/154: sync 2024-03-22T07:08:48.677 INFO:tasks.workunit.client.1.smithi149.stdout:3/69: sync 2024-03-22T07:08:48.677 INFO:tasks.workunit.client.1.smithi149.stdout:7/112: mknod d1/c24 0 2024-03-22T07:08:48.677 INFO:tasks.workunit.client.1.smithi149.stdout:2/151: sync 2024-03-22T07:08:48.678 INFO:tasks.workunit.client.1.smithi149.stdout:9/155: mknod db/d22/c2a 0 2024-03-22T07:08:48.678 INFO:tasks.workunit.client.1.smithi149.stdout:7/113: symlink d1/d18/d1b/l25 0 2024-03-22T07:08:48.678 INFO:tasks.workunit.client.1.smithi149.stdout:7/114: truncate d1/f21 375352 0 2024-03-22T07:08:48.680 INFO:tasks.workunit.client.1.smithi149.stdout:0/62: sync 2024-03-22T07:08:48.680 INFO:tasks.workunit.client.1.smithi149.stdout:0/63: fsync d1/d9/ff 0 2024-03-22T07:08:48.680 INFO:tasks.workunit.client.1.smithi149.stdout:7/115: creat d1/d18/d1b/d1d/f26 x:0 0 0 2024-03-22T07:08:48.683 INFO:tasks.workunit.client.1.smithi149.stdout:5/110: dwrite d7/da/fd [0,4194304] 0 2024-03-22T07:08:48.683 INFO:tasks.workunit.client.1.smithi149.stdout:5/111: fdatasync d7/da/fe 0 2024-03-22T07:08:48.683 INFO:tasks.workunit.client.1.smithi149.stdout:5/112: chown d7/c1a 64387217 1 2024-03-22T07:08:48.685 INFO:tasks.workunit.client.1.smithi149.stdout:0/64: dread d1/fb [0,4194304] 0 2024-03-22T07:08:48.685 INFO:tasks.workunit.client.1.smithi149.stdout:0/65: write d1/d9/ff [298176,13953] 0 2024-03-22T07:08:48.691 INFO:tasks.workunit.client.1.smithi149.stdout:3/70: write d0/d4/fe [7100210,26495] 0 2024-03-22T07:08:48.691 INFO:tasks.workunit.client.1.smithi149.stdout:3/71: creat d0/d4/f1f x:0 0 0 2024-03-22T07:08:48.691 INFO:tasks.workunit.client.1.smithi149.stdout:3/72: stat d0/d4/d8 0 2024-03-22T07:08:48.691 INFO:tasks.workunit.client.1.smithi149.stdout:6/110: getdents d9/df 0 2024-03-22T07:08:48.694 INFO:tasks.workunit.client.1.smithi149.stdout:9/156: dread db/d19/f1f [0,4194304] 0 2024-03-22T07:08:48.694 INFO:tasks.workunit.client.1.smithi149.stdout:9/157: stat db/l27 0 2024-03-22T07:08:48.694 INFO:tasks.workunit.client.1.smithi149.stdout:9/158: chown db/d21/f26 5602533 1 2024-03-22T07:08:48.695 INFO:tasks.workunit.client.1.smithi149.stdout:3/73: rename d0/f13 to d0/d4/d8/d1c/f20 0 2024-03-22T07:08:48.695 INFO:tasks.workunit.client.1.smithi149.stdout:3/74: chown d0/c1 1519440 1 2024-03-22T07:08:48.695 INFO:tasks.workunit.client.1.smithi149.stdout:3/75: creat d0/d4/d8/f21 x:0 0 0 2024-03-22T07:08:48.696 INFO:tasks.workunit.client.1.smithi149.stdout:1/162: dread d4/d10/f12 [0,4194304] 0 2024-03-22T07:08:48.699 INFO:tasks.workunit.client.1.smithi149.stdout:4/85: dwrite d0/d8/dc/f13 [0,4194304] 0 2024-03-22T07:08:48.699 INFO:tasks.workunit.client.1.smithi149.stdout:9/159: creat db/d22/f2b x:0 0 0 2024-03-22T07:08:48.699 INFO:tasks.workunit.client.1.smithi149.stdout:9/160: truncate db/d12/f17 2601925 0 2024-03-22T07:08:48.700 INFO:tasks.workunit.client.1.smithi149.stdout:7/116: dread d1/f19 [0,4194304] 0 2024-03-22T07:08:48.700 INFO:tasks.workunit.client.1.smithi149.stdout:7/117: chown d1/d18/d1b/d1d/fd 14555 1 2024-03-22T07:08:48.701 INFO:tasks.workunit.client.1.smithi149.stdout:3/76: creat d0/d4/d8/d1c/f22 x:0 0 0 2024-03-22T07:08:48.701 INFO:tasks.workunit.client.1.smithi149.stdout:3/77: dread - d0/d4/d8/d1c/f22 zero size 2024-03-22T07:08:48.705 INFO:tasks.workunit.client.1.smithi149.stdout:7/118: rename d1/d18/d1b/d1d/l1a to d1/d18/l27 0 2024-03-22T07:08:48.708 INFO:tasks.workunit.client.1.smithi149.stdout:7/119: mkdir d1/d18/d1b/d1d/d28 0 2024-03-22T07:08:48.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/111: write d9/fd [7374993,69347] 0 2024-03-22T07:08:48.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/112: fsync d9/df/f1e 0 2024-03-22T07:08:48.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/113: truncate d9/df/f1d 1884650 0 2024-03-22T07:08:48.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/114: creat d9/d1f/f25 x:0 0 0 2024-03-22T07:08:48.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/115: dread - d9/da/f23 zero size 2024-03-22T07:08:48.719 INFO:tasks.workunit.client.1.smithi149.stdout:4/86: dread d0/d8/dc/fe [0,4194304] 0 2024-03-22T07:08:48.719 INFO:tasks.workunit.client.1.smithi149.stdout:7/120: dread d1/f19 [0,4194304] 0 2024-03-22T07:08:48.719 INFO:tasks.workunit.client.1.smithi149.stdout:7/121: creat d1/f29 x:0 0 0 2024-03-22T07:08:48.722 INFO:tasks.workunit.client.1.smithi149.stdout:4/87: creat d0/f1a x:0 0 0 2024-03-22T07:08:48.725 INFO:tasks.workunit.client.1.smithi149.stdout:7/122: mkdir d1/d18/d1b/d1d/d2a 0 2024-03-22T07:08:48.725 INFO:tasks.workunit.client.1.smithi149.stdout:7/123: chown d1/d18/d1b/d1d/d2a 8330 1 2024-03-22T07:08:48.726 INFO:tasks.workunit.client.1.smithi149.stdout:7/124: symlink d1/d18/d1b/d1d/d28/l2b 0 2024-03-22T07:08:48.734 INFO:tasks.workunit.client.1.smithi149.stdout:7/125: creat d1/d18/d1b/f2c x:0 0 0 2024-03-22T07:08:48.737 INFO:tasks.workunit.client.1.smithi149.stdout:8/137: rmdir d1/d2/d9/d11/d28 0 2024-03-22T07:08:48.739 INFO:tasks.workunit.client.1.smithi149.stdout:5/113: dwrite d7/da/fd [0,4194304] 0 2024-03-22T07:08:48.739 INFO:tasks.workunit.client.1.smithi149.stdout:8/138: mknod d1/d2/d9/c2e 0 2024-03-22T07:08:48.740 INFO:tasks.workunit.client.1.smithi149.stdout:5/114: dread d7/f8 [0,4194304] 0 2024-03-22T07:08:48.742 INFO:tasks.workunit.client.1.smithi149.stdout:3/78: fsync d0/d4/d8/f21 0 2024-03-22T07:08:48.742 INFO:tasks.workunit.client.1.smithi149.stdout:3/79: chown d0/d4/f1e 350 1 2024-03-22T07:08:48.742 INFO:tasks.workunit.client.1.smithi149.stdout:3/80: chown d0/d4/d8/lb 11705 1 2024-03-22T07:08:48.742 INFO:tasks.workunit.client.1.smithi149.stdout:3/81: read d0/d4/fe [1262911,81813] 0 2024-03-22T07:08:48.742 INFO:tasks.workunit.client.1.smithi149.stdout:3/82: write d0/d4/d8/f1d [873258,123121] 0 2024-03-22T07:08:48.742 INFO:tasks.workunit.client.1.smithi149.stdout:8/139: symlink d1/d2/d9/df/l2f 0 2024-03-22T07:08:48.744 INFO:tasks.workunit.client.1.smithi149.stdout:3/83: read d0/d4/d8/f19 [273189,84757] 0 2024-03-22T07:08:48.746 INFO:tasks.workunit.client.1.smithi149.stdout:5/115: symlink d7/l21 0 2024-03-22T07:08:48.746 INFO:tasks.workunit.client.1.smithi149.stdout:5/116: creat d7/db/f22 x:0 0 0 2024-03-22T07:08:48.752 INFO:tasks.workunit.client.1.smithi149.stdout:5/117: symlink d7/db/l23 0 2024-03-22T07:08:48.752 INFO:tasks.workunit.client.1.smithi149.stdout:3/84: rmdir d0 39 2024-03-22T07:08:48.752 INFO:tasks.workunit.client.1.smithi149.stdout:3/85: readlink d0/d4/l16 0 2024-03-22T07:08:48.753 INFO:tasks.workunit.client.1.smithi149.stdout:5/118: dread d7/f9 [0,4194304] 0 2024-03-22T07:08:48.754 INFO:tasks.workunit.client.1.smithi149.stdout:6/116: dwrite d9/df/f16 [0,4194304] 0 2024-03-22T07:08:48.755 INFO:tasks.workunit.client.1.smithi149.stdout:9/161: dwrite db/d21/f26 [0,4194304] 0 2024-03-22T07:08:48.758 INFO:tasks.workunit.client.1.smithi149.stdout:9/162: mknod db/d19/c2c 0 2024-03-22T07:08:48.758 INFO:tasks.workunit.client.1.smithi149.stdout:9/163: read db/d12/f17 [1260048,84484] 0 2024-03-22T07:08:48.760 INFO:tasks.workunit.client.1.smithi149.stdout:4/88: dwrite d0/f1a [0,4194304] 0 2024-03-22T07:08:48.760 INFO:tasks.workunit.client.1.smithi149.stdout:4/89: fdatasync d0/d8/dc/f13 0 2024-03-22T07:08:48.760 INFO:tasks.workunit.client.1.smithi149.stdout:4/90: chown d0/d8/dc/d14/c19 16580821 1 2024-03-22T07:08:48.761 INFO:tasks.workunit.client.1.smithi149.stdout:3/86: truncate d0/d4/fe 5162759 0 2024-03-22T07:08:48.761 INFO:tasks.workunit.client.1.smithi149.stdout:3/87: chown d0/c14 2149 1 2024-03-22T07:08:48.761 INFO:tasks.workunit.client.1.smithi149.stdout:3/88: fdatasync d0/d4/d8/d1c/f22 0 2024-03-22T07:08:48.761 INFO:tasks.workunit.client.1.smithi149.stdout:3/89: truncate d0/d4/d8/d1c/f20 1441803 0 2024-03-22T07:08:48.764 INFO:tasks.workunit.client.1.smithi149.stdout:9/164: dread db/fc [0,4194304] 0 2024-03-22T07:08:48.768 INFO:tasks.workunit.client.1.smithi149.stdout:5/119: dread d7/da/fd [0,4194304] 0 2024-03-22T07:08:48.768 INFO:tasks.workunit.client.1.smithi149.stdout:5/120: stat d7/da/c19 0 2024-03-22T07:08:48.768 INFO:tasks.workunit.client.1.smithi149.stdout:7/126: dwrite d1/f19 [0,4194304] 0 2024-03-22T07:08:48.768 INFO:tasks.workunit.client.1.smithi149.stdout:7/127: dread - d1/d18/f1e zero size 2024-03-22T07:08:48.769 INFO:tasks.workunit.client.1.smithi149.stdout:3/90: read d0/d4/d8/f1d [550394,8260] 0 2024-03-22T07:08:48.769 INFO:tasks.workunit.client.1.smithi149.stdout:8/140: dwrite d1/f2c [0,4194304] 0 2024-03-22T07:08:48.769 INFO:tasks.workunit.client.1.smithi149.stdout:8/141: read - d1/d2/d3/f1f zero size 2024-03-22T07:08:48.769 INFO:tasks.workunit.client.1.smithi149.stdout:5/121: rename d7/da/c14 to d7/da/d20/c24 0 2024-03-22T07:08:48.771 INFO:tasks.workunit.client.1.smithi149.stdout:7/128: mkdir d1/d18/d1b/d1d/d2a/d2d 0 2024-03-22T07:08:48.771 INFO:tasks.workunit.client.1.smithi149.stdout:7/129: write d1/f21 [1062433,106577] 0 2024-03-22T07:08:48.771 INFO:tasks.workunit.client.1.smithi149.stdout:7/130: write d1/d18/d1b/d1d/fd [1260929,105616] 0 2024-03-22T07:08:48.773 INFO:tasks.workunit.client.1.smithi149.stdout:9/165: dread db/fd [0,4194304] 0 2024-03-22T07:08:48.774 INFO:tasks.workunit.client.1.smithi149.stdout:3/91: dread d0/d4/d8/f19 [0,4194304] 0 2024-03-22T07:08:48.775 INFO:tasks.workunit.client.1.smithi149.stdout:5/122: rename d7/db/c1e to d7/db/c25 0 2024-03-22T07:08:48.775 INFO:tasks.workunit.client.1.smithi149.stdout:7/131: mknod d1/d18/d1b/d1d/d2a/c2e 0 2024-03-22T07:08:48.777 INFO:tasks.workunit.client.1.smithi149.stdout:9/166: unlink db/d19/d28/c1e 0 2024-03-22T07:08:48.779 INFO:tasks.workunit.client.1.smithi149.stdout:3/92: creat d0/d4/d8/f23 x:0 0 0 2024-03-22T07:08:48.779 INFO:tasks.workunit.client.1.smithi149.stdout:5/123: rename d7 to d7/d26 22 2024-03-22T07:08:48.781 INFO:tasks.workunit.client.1.smithi149.stdout:3/93: mkdir d0/d4/d8/d24 0 2024-03-22T07:08:48.782 INFO:tasks.workunit.client.1.smithi149.stdout:3/94: creat d0/d4/d8/f25 x:0 0 0 2024-03-22T07:08:48.784 INFO:tasks.workunit.client.1.smithi149.stdout:3/95: symlink d0/d4/d8/l26 0 2024-03-22T07:08:48.784 INFO:tasks.workunit.client.1.smithi149.stdout:3/96: write d0/d4/fe [5146989,19774] 0 2024-03-22T07:08:48.784 INFO:tasks.workunit.client.1.smithi149.stdout:3/97: write d0/d4/f1e [1037296,43393] 0 2024-03-22T07:08:48.786 INFO:tasks.workunit.client.1.smithi149.stdout:3/98: unlink d0/c18 0 2024-03-22T07:08:48.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:48 smithi149 ceph-mon[36068]: pgmap v7: 65 pgs: 65 active+clean; 869 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 1.06k op/s 2024-03-22T07:08:48.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:48 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.81:0/3007820561' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:08:48.787 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:48 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.81:0/3007820561' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi081.bcxarw/key"}]: dispatch 2024-03-22T07:08:48.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:48 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.81:0/3007820561' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:08:48.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:48 smithi149 ceph-mon[36068]: mgrmap e25: smithi149.sgmtoa(active, since 8s), standbys: smithi081.bcxarw 2024-03-22T07:08:48.788 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:48 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:08:48.788 INFO:tasks.workunit.client.1.smithi149.stdout:7/132: getdents d1/d18/d1b/d1d/d2a 0 2024-03-22T07:08:48.788 INFO:tasks.workunit.client.1.smithi149.stdout:3/99: dread d0/d4/d8/d1c/f20 [0,4194304] 0 2024-03-22T07:08:48.791 INFO:tasks.workunit.client.1.smithi149.stdout:3/100: fsync d0/d4/f1e 0 2024-03-22T07:08:48.791 INFO:tasks.workunit.client.1.smithi149.stdout:5/124: read d7/f17 [3729196,90963] 0 2024-03-22T07:08:48.791 INFO:tasks.workunit.client.1.smithi149.stdout:8/142: dwrite d1/d2/d3/f1f [0,4194304] 0 2024-03-22T07:08:48.791 INFO:tasks.workunit.client.1.smithi149.stdout:8/143: fdatasync d1/f21 0 2024-03-22T07:08:48.792 INFO:tasks.workunit.client.1.smithi149.stdout:7/133: rmdir d1 39 2024-03-22T07:08:48.793 INFO:tasks.workunit.client.1.smithi149.stdout:7/134: write d1/d18/d1b/d1d/f13 [127584,26445] 0 2024-03-22T07:08:48.793 INFO:tasks.workunit.client.1.smithi149.stdout:7/135: read - d1/f29 zero size 2024-03-22T07:08:48.793 INFO:tasks.workunit.client.1.smithi149.stdout:5/125: dread d7/da/f12 [0,4194304] 0 2024-03-22T07:08:48.794 INFO:tasks.workunit.client.1.smithi149.stdout:3/101: mknod d0/d4/c27 0 2024-03-22T07:08:48.796 INFO:tasks.workunit.client.1.smithi149.stdout:8/144: mknod d1/d2/d9/c30 0 2024-03-22T07:08:48.796 INFO:tasks.workunit.client.1.smithi149.stdout:7/136: mkdir d1/d1c/d2f 0 2024-03-22T07:08:48.797 INFO:tasks.workunit.client.1.smithi149.stdout:3/102: symlink d0/d4/d8/d1c/l28 0 2024-03-22T07:08:48.797 INFO:tasks.workunit.client.1.smithi149.stdout:3/103: chown d0/d2 532738 1 2024-03-22T07:08:48.798 INFO:tasks.workunit.client.1.smithi149.stdout:3/104: write d0/d4/d8/f25 [773811,25517] 0 2024-03-22T07:08:48.798 INFO:tasks.workunit.client.1.smithi149.stdout:3/105: fdatasync d0/d4/d8/d1c/f22 0 2024-03-22T07:08:48.798 INFO:tasks.workunit.client.1.smithi149.stdout:8/145: rename d1/d2/f1c to d1/d20/f31 0 2024-03-22T07:08:48.801 INFO:tasks.workunit.client.1.smithi149.stdout:5/126: mknod d7/c27 0 2024-03-22T07:08:48.801 INFO:tasks.workunit.client.1.smithi149.stdout:5/127: dread - d7/db/f22 zero size 2024-03-22T07:08:48.801 INFO:tasks.workunit.client.1.smithi149.stdout:5/128: read - d7/db/f22 zero size 2024-03-22T07:08:48.801 INFO:tasks.workunit.client.1.smithi149.stdout:5/129: dread - d7/db/f22 zero size 2024-03-22T07:08:48.801 INFO:tasks.workunit.client.1.smithi149.stdout:7/137: unlink d1/f29 0 2024-03-22T07:08:48.803 INFO:tasks.workunit.client.1.smithi149.stdout:8/146: unlink d1/d2/d9/df/c26 0 2024-03-22T07:08:48.803 INFO:tasks.workunit.client.1.smithi149.stdout:5/130: rmdir d7 39 2024-03-22T07:08:48.803 INFO:tasks.workunit.client.1.smithi149.stdout:5/131: chown d7/da/f12 135162 1 2024-03-22T07:08:48.803 INFO:tasks.workunit.client.1.smithi149.stdout:5/132: chown d7 38932 1 2024-03-22T07:08:48.804 INFO:tasks.workunit.client.1.smithi149.stdout:5/133: fdatasync d7/da/f1b 0 2024-03-22T07:08:48.804 INFO:tasks.workunit.client.1.smithi149.stdout:5/134: truncate d7/da/f12 297923 0 2024-03-22T07:08:48.804 INFO:tasks.workunit.client.1.smithi149.stdout:7/138: mkdir d1/d18/d30 0 2024-03-22T07:08:48.812 INFO:tasks.workunit.client.1.smithi149.stdout:8/147: dread d1/d2/d9/d11/f17 [0,4194304] 0 2024-03-22T07:08:48.812 INFO:tasks.workunit.client.1.smithi149.stdout:8/148: chown d1/d2/d3 5504373 1 2024-03-22T07:08:48.812 INFO:tasks.workunit.client.1.smithi149.stdout:8/149: write d1/d2/d9/f1d [879194,116826] 0 2024-03-22T07:08:48.812 INFO:tasks.workunit.client.1.smithi149.stdout:8/150: chown d1/f21 2 1 2024-03-22T07:08:48.815 INFO:tasks.workunit.client.1.smithi149.stdout:8/151: link d1/d2/d9/l27 d1/d2/d9/da/l32 0 2024-03-22T07:08:48.815 INFO:tasks.workunit.client.1.smithi149.stdout:7/139: write d1/d18/d1b/d1d/f8 [3251363,79370] 0 2024-03-22T07:08:48.815 INFO:tasks.workunit.client.1.smithi149.stdout:7/140: chown d1/f19 32373576 1 2024-03-22T07:08:48.816 INFO:tasks.workunit.client.1.smithi149.stdout:8/152: mknod d1/d2/d9/da/c33 0 2024-03-22T07:08:48.818 INFO:tasks.workunit.client.1.smithi149.stdout:7/141: rename d1/l9 to d1/d1c/l31 0 2024-03-22T07:08:48.818 INFO:tasks.workunit.client.1.smithi149.stdout:7/142: write d1/f5 [1835376,48102] 0 2024-03-22T07:08:48.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:48 smithi081 ceph-mon[29245]: pgmap v7: 65 pgs: 65 active+clean; 869 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 5.4 MiB/s rd, 11 MiB/s wr, 1.06k op/s 2024-03-22T07:08:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:48 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.81:0/3007820561' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:08:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:48 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.81:0/3007820561' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi081.bcxarw/key"}]: dispatch 2024-03-22T07:08:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:48 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.81:0/3007820561' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:08:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:48 smithi081 ceph-mon[29245]: mgrmap e25: smithi149.sgmtoa(active, since 8s), standbys: smithi081.bcxarw 2024-03-22T07:08:48.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:48 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:08:48.819 INFO:tasks.workunit.client.1.smithi149.stdout:7/143: write d1/f21 [703765,126326] 0 2024-03-22T07:08:48.821 INFO:tasks.workunit.client.1.smithi149.stdout:7/144: symlink d1/d1c/d2f/l32 0 2024-03-22T07:08:48.825 INFO:tasks.workunit.client.1.smithi149.stdout:8/153: write d1/d2/d9/d11/f17 [1333282,21814] 0 2024-03-22T07:08:48.825 INFO:tasks.workunit.client.1.smithi149.stdout:8/154: chown d1/d2/d9/da/l14 30 1 2024-03-22T07:08:48.825 INFO:tasks.workunit.client.1.smithi149.stdout:8/155: truncate d1/d2/d9/f1b 744506 0 2024-03-22T07:08:48.825 INFO:tasks.workunit.client.1.smithi149.stdout:3/106: dwrite d0/d4/fe [0,4194304] 0 2024-03-22T07:08:48.825 INFO:tasks.workunit.client.1.smithi149.stdout:5/135: dwrite d7/db/fc [0,4194304] 0 2024-03-22T07:08:48.826 INFO:tasks.workunit.client.1.smithi149.stdout:5/136: truncate d7/db/f22 934601 0 2024-03-22T07:08:48.828 INFO:tasks.workunit.client.1.smithi149.stdout:7/145: write d1/f10 [22750,67287] 0 2024-03-22T07:08:48.828 INFO:tasks.workunit.client.1.smithi149.stdout:7/146: truncate d1/d18/d1b/d1d/f16 885480 0 2024-03-22T07:08:48.869 INFO:tasks.workunit.client.1.smithi149.stdout:8/156: dwrite d1/f2c [0,4194304] 0 2024-03-22T07:08:48.869 INFO:tasks.workunit.client.1.smithi149.stdout:3/107: dwrite d0/d4/f1e [0,4194304] 0 2024-03-22T07:08:48.870 INFO:tasks.workunit.client.1.smithi149.stdout:7/147: dwrite d1/d18/d1b/d1d/f1f [0,4194304] 0 2024-03-22T07:08:48.870 INFO:tasks.workunit.client.1.smithi149.stdout:5/137: dwrite d7/db/fc [4194304,4194304] 0 2024-03-22T07:08:48.873 INFO:tasks.workunit.client.1.smithi149.stdout:7/148: creat d1/f33 x:0 0 0 2024-03-22T07:08:48.878 INFO:tasks.workunit.client.1.smithi149.stdout:5/138: link d7/c27 d7/db/c28 0 2024-03-22T07:08:48.887 INFO:tasks.workunit.client.1.smithi149.stdout:8/157: getdents d1/d2/d9/d11 0 2024-03-22T07:08:48.888 INFO:tasks.workunit.client.1.smithi149.stdout:8/158: fdatasync d1/d2/d9/df/f2d 0 2024-03-22T07:08:48.889 INFO:tasks.workunit.client.1.smithi149.stdout:7/149: creat d1/d18/f34 x:0 0 0 2024-03-22T07:08:48.889 INFO:tasks.workunit.client.1.smithi149.stdout:7/150: chown d1/d18/d1b/c20 418869 1 2024-03-22T07:08:48.889 INFO:tasks.workunit.client.1.smithi149.stdout:7/151: chown d1/d18/d1b/d1d/la 221833 1 2024-03-22T07:08:48.889 INFO:tasks.workunit.client.1.smithi149.stdout:7/152: dread - d1/d18/d1b/f2c zero size 2024-03-22T07:08:48.889 INFO:tasks.workunit.client.1.smithi149.stdout:7/153: write d1/f5 [645476,118156] 0 2024-03-22T07:08:48.890 INFO:tasks.workunit.client.1.smithi149.stdout:7/154: rename d1/d18/d1b/d1d/l7 to d1/d18/d1b/d1d/d2a/d2d/l35 0 2024-03-22T07:08:48.891 INFO:tasks.workunit.client.1.smithi149.stdout:5/139: mkdir d7/d29 0 2024-03-22T07:08:48.893 INFO:tasks.workunit.client.1.smithi149.stdout:7/155: mkdir d1/d18/d1b/d1d/d2a/d2d/d36 0 2024-03-22T07:08:48.893 INFO:tasks.workunit.client.1.smithi149.stdout:5/140: mkdir d7/d29/d2a 0 2024-03-22T07:08:48.896 INFO:tasks.workunit.client.1.smithi149.stdout:7/156: dread d1/d18/d1b/d1d/fd [0,4194304] 0 2024-03-22T07:08:48.896 INFO:tasks.workunit.client.1.smithi149.stdout:7/157: readlink d1/l15 0 2024-03-22T07:08:48.896 INFO:tasks.workunit.client.1.smithi149.stdout:7/158: creat d1/d18/f37 x:0 0 0 2024-03-22T07:08:48.901 INFO:tasks.workunit.client.1.smithi149.stdout:5/141: dread d7/da/fe [0,4194304] 0 2024-03-22T07:08:48.903 INFO:tasks.workunit.client.1.smithi149.stdout:5/142: mkdir d7/db/d2b 0 2024-03-22T07:08:48.918 INFO:tasks.workunit.client.1.smithi149.stdout:7/159: dwrite d1/d18/f1e [0,4194304] 0 2024-03-22T07:08:48.921 INFO:tasks.workunit.client.1.smithi149.stdout:7/160: link d1/f33 d1/d18/d1b/d1d/f38 0 2024-03-22T07:08:48.921 INFO:tasks.workunit.client.1.smithi149.stdout:7/161: chown d1/d18/d1b/c20 425 1 2024-03-22T07:08:48.921 INFO:tasks.workunit.client.1.smithi149.stdout:7/162: symlink d1/d18/d1b/d1d/d28/l39 0 2024-03-22T07:08:48.921 INFO:tasks.workunit.client.1.smithi149.stdout:7/163: creat d1/f3a x:0 0 0 2024-03-22T07:08:48.921 INFO:tasks.workunit.client.1.smithi149.stdout:7/164: chown d1/d18/d1b/l25 5291206 1 2024-03-22T07:08:48.921 INFO:tasks.workunit.client.1.smithi149.stdout:7/165: getdents d1/d18/d1b/d1d/d2a 0 2024-03-22T07:08:48.921 INFO:tasks.workunit.client.1.smithi149.stdout:7/166: write d1/d18/d1b/f2c [524431,108160] 0 2024-03-22T07:08:48.921 INFO:tasks.workunit.client.1.smithi149.stdout:7/167: getdents d1/d18/d1b/d1d/d2a/d2d/d36 0 2024-03-22T07:08:48.924 INFO:tasks.workunit.client.1.smithi149.stdout:7/168: rmdir d1/d18/d1b/d1d/d2a/d2d/d36 0 2024-03-22T07:08:48.925 INFO:tasks.workunit.client.1.smithi149.stdout:7/169: creat d1/d18/d1b/d1d/d2a/f3b x:0 0 0 2024-03-22T07:08:48.926 INFO:tasks.workunit.client.1.smithi149.stdout:7/170: mkdir d1/d18/d1b/d1d/d3c 0 2024-03-22T07:08:48.929 INFO:tasks.workunit.client.1.smithi149.stdout:5/143: dwrite d7/da/fd [0,4194304] 0 2024-03-22T07:08:48.929 INFO:tasks.workunit.client.1.smithi149.stdout:5/144: readlink d7/l21 0 2024-03-22T07:08:48.929 INFO:tasks.workunit.client.1.smithi149.stdout:5/145: write d7/da/fe [4986170,107853] 0 2024-03-22T07:08:48.930 INFO:tasks.workunit.client.1.smithi149.stdout:5/146: truncate d7/db/fc 590414 0 2024-03-22T07:08:48.954 INFO:tasks.workunit.client.1.smithi149.stdout:5/147: dwrite d7/f9 [0,4194304] 0 2024-03-22T07:08:48.954 INFO:tasks.workunit.client.1.smithi149.stdout:7/171: dwrite d1/d18/d1b/d1d/fd [0,4194304] 0 2024-03-22T07:08:48.955 INFO:tasks.workunit.client.1.smithi149.stdout:7/172: fsync d1/d18/d1b/d1d/f26 0 2024-03-22T07:08:48.980 INFO:tasks.workunit.client.1.smithi149.stdout:5/148: dwrite d7/f9 [0,4194304] 0 2024-03-22T07:08:48.980 INFO:tasks.workunit.client.1.smithi149.stdout:7/173: dwrite d1/d18/d1b/d1d/f8 [0,4194304] 0 2024-03-22T07:08:48.981 INFO:tasks.workunit.client.1.smithi149.stdout:5/149: truncate d7/f17 1214106 0 2024-03-22T07:08:48.981 INFO:tasks.workunit.client.1.smithi149.stdout:5/150: chown d7/da 353 1 2024-03-22T07:08:48.993 INFO:tasks.workunit.client.1.smithi149.stdout:5/151: write d7/da/fd [1533729,17597] 0 2024-03-22T07:08:49.004 INFO:tasks.workunit.client.1.smithi149.stdout:7/174: dwrite d1/d18/d1b/d1d/f1f [0,4194304] 0 2024-03-22T07:08:49.007 INFO:tasks.workunit.client.1.smithi149.stdout:5/152: dread d7/da/fe [0,4194304] 0 2024-03-22T07:08:49.015 INFO:tasks.workunit.client.1.smithi149.stdout:7/175: read d1/d18/d1b/d1d/f8 [713895,26777] 0 2024-03-22T07:08:49.015 INFO:tasks.workunit.client.1.smithi149.stdout:7/176: creat d1/d18/d1b/d1d/d2a/f3d x:0 0 0 2024-03-22T07:08:49.017 INFO:tasks.workunit.client.1.smithi149.stdout:7/177: creat d1/d18/d1b/d1d/d2a/d2d/f3e x:0 0 0 2024-03-22T07:08:49.019 INFO:tasks.workunit.client.1.smithi149.stdout:7/178: link d1/d18/d1b/d1d/d2a/c2e d1/d18/d1b/d1d/d2a/c3f 0 2024-03-22T07:08:49.019 INFO:tasks.workunit.client.1.smithi149.stdout:7/179: write d1/d18/d1b/d1d/d2a/f3b [450048,49110] 0 2024-03-22T07:08:49.019 INFO:tasks.workunit.client.1.smithi149.stdout:7/180: truncate d1/f33 715199 0 2024-03-22T07:08:49.019 INFO:tasks.workunit.client.1.smithi149.stdout:7/181: write d1/d18/d1b/f2c [523721,67468] 0 2024-03-22T07:08:49.026 INFO:tasks.workunit.client.1.smithi149.stdout:5/153: dwrite d7/da/f12 [0,4194304] 0 2024-03-22T07:08:49.026 INFO:tasks.workunit.client.1.smithi149.stdout:5/154: write d7/f8 [359370,75670] 0 2024-03-22T07:08:49.041 INFO:tasks.workunit.client.1.smithi149.stdout:7/182: dwrite d1/f33 [0,4194304] 0 2024-03-22T07:08:49.041 INFO:tasks.workunit.client.1.smithi149.stdout:7/183: dread - d1/d18/f37 zero size 2024-03-22T07:08:49.041 INFO:tasks.workunit.client.1.smithi149.stdout:7/184: truncate d1/d18/d1b/d1d/f1f 4266138 0 2024-03-22T07:08:49.041 INFO:tasks.workunit.client.1.smithi149.stdout:7/185: dread - d1/d18/d1b/d1d/f26 zero size 2024-03-22T07:08:49.041 INFO:tasks.workunit.client.1.smithi149.stdout:5/155: write d7/da/f12 [3890143,115691] 0 2024-03-22T07:08:49.043 INFO:tasks.workunit.client.1.smithi149.stdout:7/186: creat d1/d18/d1b/d1d/f40 x:0 0 0 2024-03-22T07:08:49.043 INFO:tasks.workunit.client.1.smithi149.stdout:7/187: creat d1/f41 x:0 0 0 2024-03-22T07:08:49.043 INFO:tasks.workunit.client.1.smithi149.stdout:7/188: fdatasync d1/d18/d1b/d1d/f16 0 2024-03-22T07:08:49.043 INFO:tasks.workunit.client.1.smithi149.stdout:7/189: chown d1/d18/d1b/d1d/d2a/c2e 177 1 2024-03-22T07:08:49.043 INFO:tasks.workunit.client.1.smithi149.stdout:7/190: getdents d1/d18/d1b/d1d/d3c 0 2024-03-22T07:08:49.049 INFO:tasks.workunit.client.1.smithi149.stdout:5/156: truncate d7/da/fd 187706 0 2024-03-22T07:08:49.049 INFO:tasks.workunit.client.1.smithi149.stdout:5/157: chown d7/db/f22 0 1 2024-03-22T07:08:49.058 INFO:tasks.workunit.client.1.smithi149.stdout:5/158: dread d7/da/fd [0,4194304] 0 2024-03-22T07:08:49.100 INFO:tasks.workunit.client.1.smithi149.stdout:5/159: mkdir d7/d29/d2c 0 2024-03-22T07:08:49.100 INFO:tasks.workunit.client.1.smithi149.stdout:5/160: dread d7/db/f22 [0,4194304] 0 2024-03-22T07:08:49.101 INFO:tasks.workunit.client.1.smithi149.stdout:7/191: dwrite d1/d18/d1b/d1d/f16 [0,4194304] 0 2024-03-22T07:08:49.101 INFO:tasks.workunit.client.1.smithi149.stdout:7/192: creat d1/d18/d1b/f42 x:0 0 0 2024-03-22T07:08:49.101 INFO:tasks.workunit.client.1.smithi149.stdout:7/193: dwrite d1/d18/d1b/d1d/f26 [0,4194304] 0 2024-03-22T07:08:49.101 INFO:tasks.workunit.client.1.smithi149.stdout:7/194: truncate d1/d18/d1b/d1d/d2a/d2d/f3e 595793 0 2024-03-22T07:08:49.101 INFO:tasks.workunit.client.1.smithi149.stdout:7/195: read d1/d18/f1e [189618,28199] 0 2024-03-22T07:08:49.101 INFO:tasks.workunit.client.1.smithi149.stdout:7/196: creat d1/d1c/d2f/f43 x:0 0 0 2024-03-22T07:08:49.101 INFO:tasks.workunit.client.1.smithi149.stdout:7/197: getdents d1/d18/d1b/d1d/d2a/d2d 0 2024-03-22T07:08:49.188 INFO:tasks.workunit.client.1.smithi149.stdout:2/152: sync 2024-03-22T07:08:49.188 INFO:tasks.workunit.client.1.smithi149.stdout:6/117: sync 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:1/163: sync 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:9/167: sync 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:0/66: sync 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:3/108: sync 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:6/118: readlink d9/df/l18 0 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:3/109: read d0/d4/d8/f19 [87074,42438] 0 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:9/168: write db/d22/f2b [542878,66464] 0 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:4/91: sync 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:9/169: chown db/l27 1159 1 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:9/170: readlink db/d12/l1b 0 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:9/171: readlink l6 0 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:9/172: chown db/d12/f17 405347619 1 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:4/92: getdents d0/d8 0 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:9/173: chown db/d19/d1c/f20 19750 1 2024-03-22T07:08:49.189 INFO:tasks.workunit.client.1.smithi149.stdout:9/174: chown db/cf 720187780 1 2024-03-22T07:08:49.190 INFO:tasks.workunit.client.1.smithi149.stdout:0/67: read d1/d9/ff [140759,70867] 0 2024-03-22T07:08:49.190 INFO:tasks.workunit.client.1.smithi149.stdout:2/153: rename d1/d3/le to d1/d3/d5/d1f/l35 0 2024-03-22T07:08:49.190 INFO:tasks.workunit.client.1.smithi149.stdout:6/119: truncate f5 505088 0 2024-03-22T07:08:49.190 INFO:tasks.workunit.client.1.smithi149.stdout:3/110: truncate d0/d4/f1e 4073215 0 2024-03-22T07:08:49.191 INFO:tasks.workunit.client.1.smithi149.stdout:9/175: truncate db/fd 1237531 0 2024-03-22T07:08:49.191 INFO:tasks.workunit.client.1.smithi149.stdout:9/176: dread - db/d19/d1c/f20 zero size 2024-03-22T07:08:49.191 INFO:tasks.workunit.client.1.smithi149.stdout:6/120: dread d9/da/f20 [0,4194304] 0 2024-03-22T07:08:49.192 INFO:tasks.workunit.client.1.smithi149.stdout:4/93: symlink d0/d8/l1b 0 2024-03-22T07:08:49.193 INFO:tasks.workunit.client.1.smithi149.stdout:4/94: stat d0/d8/dc/f18 0 2024-03-22T07:08:49.193 INFO:tasks.workunit.client.1.smithi149.stdout:2/154: creat d1/d3/f36 x:0 0 0 2024-03-22T07:08:49.197 INFO:tasks.workunit.client.1.smithi149.stdout:3/111: rename d0/d2/c15 to d0/d2/c29 0 2024-03-22T07:08:49.197 INFO:tasks.workunit.client.1.smithi149.stdout:9/177: dread db/d12/f17 [0,4194304] 0 2024-03-22T07:08:49.197 INFO:tasks.workunit.client.1.smithi149.stdout:6/121: creat d9/d1f/d24/f26 x:0 0 0 2024-03-22T07:08:49.198 INFO:tasks.workunit.client.1.smithi149.stdout:2/155: creat d1/d3/d26/f37 x:0 0 0 2024-03-22T07:08:49.198 INFO:tasks.workunit.client.1.smithi149.stdout:2/156: read - d1/d3/d26/f37 zero size 2024-03-22T07:08:49.198 INFO:tasks.workunit.client.1.smithi149.stdout:6/122: dread - d9/d1f/f25 zero size 2024-03-22T07:08:49.198 INFO:tasks.workunit.client.1.smithi149.stdout:2/157: fdatasync d1/f1a 0 2024-03-22T07:08:49.198 INFO:tasks.workunit.client.1.smithi149.stdout:6/123: chown d9/da/f12 7621066 1 2024-03-22T07:08:49.198 INFO:tasks.workunit.client.1.smithi149.stdout:6/124: creat d9/da/f27 x:0 0 0 2024-03-22T07:08:49.200 INFO:tasks.workunit.client.1.smithi149.stdout:9/178: mknod db/d19/d1c/c2d 0 2024-03-22T07:08:49.200 INFO:tasks.workunit.client.1.smithi149.stdout:1/164: write f1 [2079090,93767] 0 2024-03-22T07:08:49.200 INFO:tasks.workunit.client.1.smithi149.stdout:2/158: symlink d1/d3/d6/l38 0 2024-03-22T07:08:49.202 INFO:tasks.workunit.client.1.smithi149.stdout:1/165: dread d4/d20/d23/f27 [0,4194304] 0 2024-03-22T07:08:49.202 INFO:tasks.workunit.client.1.smithi149.stdout:9/179: mkdir db/d21/d2e 0 2024-03-22T07:08:49.202 INFO:tasks.workunit.client.1.smithi149.stdout:9/180: truncate db/d19/d1c/f20 908075 0 2024-03-22T07:08:49.204 INFO:tasks.workunit.client.1.smithi149.stdout:9/181: creat db/f2f x:0 0 0 2024-03-22T07:08:49.205 INFO:tasks.workunit.client.1.smithi149.stdout:4/95: write d0/d8/dc/fe [220497,78693] 0 2024-03-22T07:08:49.207 INFO:tasks.workunit.client.1.smithi149.stdout:9/182: truncate f9 4559473 0 2024-03-22T07:08:49.209 INFO:tasks.workunit.client.1.smithi149.stdout:6/125: dread d9/da/f20 [0,4194304] 0 2024-03-22T07:08:49.211 INFO:tasks.workunit.client.1.smithi149.stdout:4/96: creat d0/d8/f1c x:0 0 0 2024-03-22T07:08:49.212 INFO:tasks.workunit.client.1.smithi149.stdout:9/183: truncate db/d19/f1f 2376440 0 2024-03-22T07:08:49.216 INFO:tasks.workunit.client.1.smithi149.stdout:9/184: dread db/fc [0,4194304] 0 2024-03-22T07:08:49.217 INFO:tasks.workunit.client.1.smithi149.stdout:0/68: dwrite d1/fb [0,4194304] 0 2024-03-22T07:08:49.220 INFO:tasks.workunit.client.1.smithi149.stdout:1/166: read d4/f1b [1831822,130197] 0 2024-03-22T07:08:49.220 INFO:tasks.workunit.client.1.smithi149.stdout:6/126: symlink d9/df/d22/l28 0 2024-03-22T07:08:49.222 INFO:tasks.workunit.client.1.smithi149.stdout:6/127: mknod d9/df/c29 0 2024-03-22T07:08:49.222 INFO:tasks.workunit.client.1.smithi149.stdout:6/128: creat d9/d1f/f2a x:0 0 0 2024-03-22T07:08:49.222 INFO:tasks.workunit.client.1.smithi149.stdout:6/129: write d9/d1f/d24/f26 [774840,104477] 0 2024-03-22T07:08:49.224 INFO:tasks.workunit.client.1.smithi149.stdout:6/130: rename d9/df/f1a to d9/df/d22/f2b 0 2024-03-22T07:08:49.228 INFO:tasks.workunit.client.1.smithi149.stdout:1/167: dread d4/f1b [0,4194304] 0 2024-03-22T07:08:49.228 INFO:tasks.workunit.client.1.smithi149.stdout:1/168: truncate d4/d7/f39 93973 0 2024-03-22T07:08:49.228 INFO:tasks.workunit.client.1.smithi149.stdout:4/97: dread d0/d8/dc/f13 [0,4194304] 0 2024-03-22T07:08:49.228 INFO:tasks.workunit.client.1.smithi149.stdout:3/112: dwrite d0/d4/d8/f19 [0,4194304] 0 2024-03-22T07:08:49.228 INFO:tasks.workunit.client.1.smithi149.stdout:3/113: write d0/d4/d8/f21 [435854,79040] 0 2024-03-22T07:08:49.231 INFO:tasks.workunit.client.1.smithi149.stdout:4/98: mkdir d0/d8/d1d 0 2024-03-22T07:08:49.231 INFO:tasks.workunit.client.1.smithi149.stdout:4/99: write d0/f3 [1868312,53819] 0 2024-03-22T07:08:49.231 INFO:tasks.workunit.client.1.smithi149.stdout:4/100: stat d0 0 2024-03-22T07:08:49.232 INFO:tasks.workunit.client.1.smithi149.stdout:9/185: write db/d21/f26 [213046,113596] 0 2024-03-22T07:08:49.232 INFO:tasks.workunit.client.1.smithi149.stdout:9/186: fsync f9 0 2024-03-22T07:08:49.232 INFO:tasks.workunit.client.1.smithi149.stdout:3/114: getdents d0 0 2024-03-22T07:08:49.232 INFO:tasks.workunit.client.1.smithi149.stdout:3/115: dread - d0/f1a zero size 2024-03-22T07:08:49.233 INFO:tasks.workunit.client.1.smithi149.stdout:2/159: dwrite d1/f10 [4194304,4194304] 0 2024-03-22T07:08:49.233 INFO:tasks.workunit.client.1.smithi149.stdout:2/160: write d1/d3/d26/f37 [64434,2298] 0 2024-03-22T07:08:49.233 INFO:tasks.workunit.client.1.smithi149.stdout:2/161: readlink d1/d3/d6/l38 0 2024-03-22T07:08:49.234 INFO:tasks.workunit.client.1.smithi149.stdout:4/101: mknod d0/d8/dc/c1e 0 2024-03-22T07:08:49.235 INFO:tasks.workunit.client.1.smithi149.stdout:9/187: mknod db/d12/c30 0 2024-03-22T07:08:49.235 INFO:tasks.workunit.client.1.smithi149.stdout:0/69: read d1/fb [456179,37386] 0 2024-03-22T07:08:49.235 INFO:tasks.workunit.client.1.smithi149.stdout:0/70: chown d1/d9/c10 130 1 2024-03-22T07:08:49.237 INFO:tasks.workunit.client.1.smithi149.stdout:2/162: creat d1/d3/d5/d1f/f39 x:0 0 0 2024-03-22T07:08:49.238 INFO:tasks.workunit.client.1.smithi149.stdout:2/163: creat d1/d3/d5/f3a x:0 0 0 2024-03-22T07:08:49.240 INFO:tasks.workunit.client.1.smithi149.stdout:0/71: stat d1/c3 0 2024-03-22T07:08:49.242 INFO:tasks.workunit.client.1.smithi149.stdout:2/164: truncate d1/d3/d5/d1f/f1e 682509 0 2024-03-22T07:08:49.247 INFO:tasks.workunit.client.1.smithi149.stdout:0/72: readlink d1/l2 0 2024-03-22T07:08:49.255 INFO:tasks.workunit.client.1.smithi149.stdout:0/73: dread d1/d9/ff [0,4194304] 0 2024-03-22T07:08:49.257 INFO:tasks.workunit.client.1.smithi149.stdout:0/74: mknod d1/d9/c11 0 2024-03-22T07:08:49.257 INFO:tasks.workunit.client.1.smithi149.stdout:0/75: write d1/fb [3304922,10593] 0 2024-03-22T07:08:49.280 INFO:tasks.workunit.client.1.smithi149.stdout:1/169: dwrite d4/d7/fe [0,4194304] 0 2024-03-22T07:08:49.280 INFO:tasks.workunit.client.1.smithi149.stdout:6/131: dwrite d9/df/f1d [0,4194304] 0 2024-03-22T07:08:49.280 INFO:tasks.workunit.client.1.smithi149.stdout:6/132: truncate d9/da/f20 1165926 0 2024-03-22T07:08:49.281 INFO:tasks.workunit.client.1.smithi149.stdout:4/102: dwrite d0/f1a [0,4194304] 0 2024-03-22T07:08:49.281 INFO:tasks.workunit.client.1.smithi149.stdout:2/165: dwrite d1/d3/f13 [0,4194304] 0 2024-03-22T07:08:49.281 INFO:tasks.workunit.client.1.smithi149.stdout:2/166: truncate d1/d3/f25 960427 0 2024-03-22T07:08:49.281 INFO:tasks.workunit.client.1.smithi149.stdout:3/116: dwrite d0/d4/fe [4194304,4194304] 0 2024-03-22T07:08:49.283 INFO:tasks.workunit.client.1.smithi149.stdout:2/167: dread d1/d8/f19 [0,4194304] 0 2024-03-22T07:08:49.286 INFO:tasks.workunit.client.1.smithi149.stdout:1/170: creat d4/d20/d2d/f3a x:0 0 0 2024-03-22T07:08:49.286 INFO:tasks.workunit.client.1.smithi149.stdout:1/171: write d4/f6 [4762467,94863] 0 2024-03-22T07:08:49.287 INFO:tasks.workunit.client.1.smithi149.stdout:9/188: truncate db/d21/f26 243094 0 2024-03-22T07:08:49.287 INFO:tasks.workunit.client.1.smithi149.stdout:6/133: creat d9/f2c x:0 0 0 2024-03-22T07:08:49.287 INFO:tasks.workunit.client.1.smithi149.stdout:6/134: truncate d9/f17 4965622 0 2024-03-22T07:08:49.292 INFO:tasks.workunit.client.1.smithi149.stdout:0/76: dread d1/fb [0,4194304] 0 2024-03-22T07:08:49.292 INFO:tasks.workunit.client.1.smithi149.stdout:2/168: write d1/d3/d6/f16 [2959499,87355] 0 2024-03-22T07:08:49.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/189: dread db/d21/f26 [0,4194304] 0 2024-03-22T07:08:49.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/190: fsync db/d12/f14 0 2024-03-22T07:08:49.305 INFO:tasks.workunit.client.1.smithi149.stdout:3/117: mknod d0/d4/d8/c2a 0 2024-03-22T07:08:49.305 INFO:tasks.workunit.client.1.smithi149.stdout:0/77: write d1/d9/fd [3636803,115654] 0 2024-03-22T07:08:49.305 INFO:tasks.workunit.client.1.smithi149.stdout:0/78: creat d1/d9/f12 x:0 0 0 2024-03-22T07:08:49.305 INFO:tasks.workunit.client.1.smithi149.stdout:0/79: write d1/fb [3123367,40862] 0 2024-03-22T07:08:49.305 INFO:tasks.workunit.client.1.smithi149.stdout:0/80: creat d1/d9/f13 x:0 0 0 2024-03-22T07:08:49.307 INFO:tasks.workunit.client.1.smithi149.stdout:2/169: symlink d1/d8/l3b 0 2024-03-22T07:08:49.310 INFO:tasks.workunit.client.1.smithi149.stdout:9/191: mknod db/d21/d2e/c31 0 2024-03-22T07:08:49.311 INFO:tasks.workunit.client.1.smithi149.stdout:9/192: stat db/d12/f17 0 2024-03-22T07:08:49.315 INFO:tasks.workunit.client.1.smithi149.stdout:1/172: getdents d4 0 2024-03-22T07:08:49.318 INFO:tasks.workunit.client.1.smithi149.stdout:9/193: symlink db/d12/l32 0 2024-03-22T07:08:49.319 INFO:tasks.workunit.client.1.smithi149.stdout:8/159: sync 2024-03-22T07:08:49.319 INFO:tasks.workunit.client.1.smithi149.stdout:5/161: sync 2024-03-22T07:08:49.319 INFO:tasks.workunit.client.1.smithi149.stdout:5/162: chown d7/db/c25 781350306 1 2024-03-22T07:08:49.319 INFO:tasks.workunit.client.1.smithi149.stdout:5/163: chown d7/db/l23 3917 1 2024-03-22T07:08:49.319 INFO:tasks.workunit.client.1.smithi149.stdout:8/160: fsync d1/d2/d9/df/f19 0 2024-03-22T07:08:49.321 INFO:tasks.workunit.client.1.smithi149.stdout:9/194: mkdir db/d33 0 2024-03-22T07:08:49.321 INFO:tasks.workunit.client.1.smithi149.stdout:6/135: dwrite d9/df/f1d [0,4194304] 0 2024-03-22T07:08:49.323 INFO:tasks.workunit.client.1.smithi149.stdout:8/161: rename d1/d2/d9/da/c33 to d1/d2/d9/da/c34 0 2024-03-22T07:08:49.323 INFO:tasks.workunit.client.1.smithi149.stdout:8/162: write d1/d2/d9/da/fb [51995,8677] 0 2024-03-22T07:08:49.326 INFO:tasks.workunit.client.1.smithi149.stdout:2/170: dread d1/d8/f24 [0,4194304] 0 2024-03-22T07:08:49.326 INFO:tasks.workunit.client.1.smithi149.stdout:2/171: chown d1/f27 0 1 2024-03-22T07:08:49.330 INFO:tasks.workunit.client.1.smithi149.stdout:7/198: sync 2024-03-22T07:08:49.330 INFO:tasks.workunit.client.1.smithi149.stdout:7/199: stat d1/d18/d1b/d1d/d3c 0 2024-03-22T07:08:49.333 INFO:tasks.workunit.client.1.smithi149.stdout:7/200: symlink d1/d1c/d2f/l44 0 2024-03-22T07:08:49.333 INFO:tasks.workunit.client.1.smithi149.stdout:7/201: truncate d1/d18/f37 677554 0 2024-03-22T07:08:49.333 INFO:tasks.workunit.client.1.smithi149.stdout:7/202: write d1/d18/d1b/f2c [1020390,32129] 0 2024-03-22T07:08:49.337 INFO:tasks.workunit.client.1.smithi149.stdout:7/203: creat d1/d18/d30/f45 x:0 0 0 2024-03-22T07:08:49.339 INFO:tasks.workunit.client.1.smithi149.stdout:7/204: mkdir d1/d18/d1b/d1d/d46 0 2024-03-22T07:08:49.341 INFO:tasks.workunit.client.1.smithi149.stdout:2/172: write d1/d3/f13 [2341822,78012] 0 2024-03-22T07:08:49.343 INFO:tasks.workunit.client.1.smithi149.stdout:2/173: symlink d1/d3/d26/d2f/l3c 0 2024-03-22T07:08:49.343 INFO:tasks.workunit.client.1.smithi149.stdout:2/174: chown d1/d3/dd 1218245 1 2024-03-22T07:08:49.343 INFO:tasks.workunit.client.1.smithi149.stdout:0/81: dwrite d1/fb [0,4194304] 0 2024-03-22T07:08:49.343 INFO:tasks.workunit.client.1.smithi149.stdout:0/82: write d1/d9/ff [1165535,25307] 0 2024-03-22T07:08:49.344 INFO:tasks.workunit.client.1.smithi149.stdout:3/118: dwrite d0/d4/d8/f19 [0,4194304] 0 2024-03-22T07:08:49.344 INFO:tasks.workunit.client.1.smithi149.stdout:2/175: mkdir d1/d8/d34/d3d 0 2024-03-22T07:08:49.344 INFO:tasks.workunit.client.1.smithi149.stdout:2/176: read - d1/d3/d26/d2f/f33 zero size 2024-03-22T07:08:49.344 INFO:tasks.workunit.client.1.smithi149.stdout:2/177: creat d1/d3/f3e x:0 0 0 2024-03-22T07:08:49.344 INFO:tasks.workunit.client.1.smithi149.stdout:2/178: write d1/f1a [5180927,56049] 0 2024-03-22T07:08:49.349 INFO:tasks.workunit.client.1.smithi149.stdout:0/83: rmdir d1 39 2024-03-22T07:08:49.349 INFO:tasks.workunit.client.1.smithi149.stdout:0/84: fdatasync d1/fb 0 2024-03-22T07:08:49.351 INFO:tasks.workunit.client.1.smithi149.stdout:8/163: rename d1/d2/d9/da to d1/d20/d35 0 2024-03-22T07:08:49.351 INFO:tasks.workunit.client.1.smithi149.stdout:7/205: rename d1/d18/d1b/d1d/d28 to d1/d18/d1b/d1d/d28/d47 22 2024-03-22T07:08:49.351 INFO:tasks.workunit.client.1.smithi149.stdout:8/164: truncate d1/d20/f31 733015 0 2024-03-22T07:08:49.356 INFO:tasks.workunit.client.1.smithi149.stdout:8/165: getdents d1/d2/d3 0 2024-03-22T07:08:49.367 INFO:tasks.workunit.client.1.smithi149.stdout:3/119: write d0/d4/fe [3843236,101608] 0 2024-03-22T07:08:49.367 INFO:tasks.workunit.client.1.smithi149.stdout:3/120: chown d0/d4/d8/lb 152824748 1 2024-03-22T07:08:49.369 INFO:tasks.workunit.client.1.smithi149.stdout:3/121: mkdir d0/d4/d8/d2b 0 2024-03-22T07:08:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:1/173: dwrite d4/d10/d28/f34 [0,4194304] 0 2024-03-22T07:08:49.370 INFO:tasks.workunit.client.1.smithi149.stdout:1/174: stat d4/d10/d28/f2e 0 2024-03-22T07:08:49.371 INFO:tasks.workunit.client.1.smithi149.stdout:3/122: rename d0/lc to d0/d2/l2c 0 2024-03-22T07:08:49.371 INFO:tasks.workunit.client.1.smithi149.stdout:3/123: readlink d0/d4/d8/lb 0 2024-03-22T07:08:49.372 INFO:tasks.workunit.client.1.smithi149.stdout:1/175: creat d4/d20/d23/d2f/f3b x:0 0 0 2024-03-22T07:08:49.372 INFO:tasks.workunit.client.1.smithi149.stdout:1/176: chown d4/d20/d23/f27 286 1 2024-03-22T07:08:49.373 INFO:tasks.workunit.client.1.smithi149.stdout:7/206: read d1/d18/d1b/d1d/fd [2530656,119429] 0 2024-03-22T07:08:49.373 INFO:tasks.workunit.client.1.smithi149.stdout:7/207: read - d1/d1c/d2f/f43 zero size 2024-03-22T07:08:49.374 INFO:tasks.workunit.client.1.smithi149.stdout:7/208: creat d1/d18/d1b/d1d/d2a/d2d/f48 x:0 0 0 2024-03-22T07:08:49.376 INFO:tasks.workunit.client.1.smithi149.stdout:7/209: rmdir d1/d18/d1b/d1d/d46 0 2024-03-22T07:08:49.378 INFO:tasks.workunit.client.1.smithi149.stdout:7/210: mkdir d1/d1c/d49 0 2024-03-22T07:08:49.382 INFO:tasks.workunit.client.1.smithi149.stdout:9/195: dwrite db/d22/f2b [0,4194304] 0 2024-03-22T07:08:49.382 INFO:tasks.workunit.client.1.smithi149.stdout:5/164: dwrite d7/f8 [0,4194304] 0 2024-03-22T07:08:49.382 INFO:tasks.workunit.client.1.smithi149.stdout:6/136: dwrite d9/da/f20 [0,4194304] 0 2024-03-22T07:08:49.383 INFO:tasks.workunit.client.1.smithi149.stdout:7/211: rmdir d1/d18/d1b/d1d/d3c 0 2024-03-22T07:08:49.383 INFO:tasks.workunit.client.1.smithi149.stdout:7/212: chown d1/d18/d1b/d1d/f13 298 1 2024-03-22T07:08:49.383 INFO:tasks.workunit.client.1.smithi149.stdout:0/85: dwrite d1/fb [0,4194304] 0 2024-03-22T07:08:49.383 INFO:tasks.workunit.client.1.smithi149.stdout:0/86: chown d1/c8 2 1 2024-03-22T07:08:49.383 INFO:tasks.workunit.client.1.smithi149.stdout:2/179: dwrite d1/f4 [0,4194304] 0 2024-03-22T07:08:49.383 INFO:tasks.workunit.client.1.smithi149.stdout:8/166: dwrite d1/d2/d9/f1b [0,4194304] 0 2024-03-22T07:08:49.388 INFO:tasks.workunit.client.1.smithi149.stdout:8/167: dread d1/d2/d9/f1d [0,4194304] 0 2024-03-22T07:08:49.391 INFO:tasks.workunit.client.1.smithi149.stdout:9/196: truncate f9 3864326 0 2024-03-22T07:08:49.395 INFO:tasks.workunit.client.1.smithi149.stdout:6/137: unlink d9/df/d22/f2b 0 2024-03-22T07:08:49.397 INFO:tasks.workunit.client.1.smithi149.stdout:0/87: mkdir d1/d9/d14 0 2024-03-22T07:08:49.400 INFO:tasks.workunit.client.1.smithi149.stdout:5/165: dread d7/f9 [0,4194304] 0 2024-03-22T07:08:49.400 INFO:tasks.workunit.client.1.smithi149.stdout:6/138: dread d9/d1f/d24/f26 [0,4194304] 0 2024-03-22T07:08:49.400 INFO:tasks.workunit.client.1.smithi149.stdout:6/139: dread - d9/d1f/f21 zero size 2024-03-22T07:08:49.400 INFO:tasks.workunit.client.1.smithi149.stdout:6/140: readlink d9/df/l18 0 2024-03-22T07:08:49.400 INFO:tasks.workunit.client.1.smithi149.stdout:1/177: dwrite d4/d20/d23/d2f/f3b [0,4194304] 0 2024-03-22T07:08:49.401 INFO:tasks.workunit.client.1.smithi149.stdout:8/168: symlink d1/d2/l36 0 2024-03-22T07:08:49.401 INFO:tasks.workunit.client.1.smithi149.stdout:8/169: fdatasync d1/f21 0 2024-03-22T07:08:49.401 INFO:tasks.workunit.client.1.smithi149.stdout:8/170: write d1/d2/d9/df/f19 [4699864,99996] 0 2024-03-22T07:08:49.401 INFO:tasks.workunit.client.1.smithi149.stdout:9/197: symlink db/d19/l34 0 2024-03-22T07:08:49.402 INFO:tasks.workunit.client.1.smithi149.stdout:0/88: link d1/d9/ff d1/d9/f15 0 2024-03-22T07:08:49.404 INFO:tasks.workunit.client.1.smithi149.stdout:1/178: mkdir d4/d20/d23/d3c 0 2024-03-22T07:08:49.404 INFO:tasks.workunit.client.1.smithi149.stdout:1/179: readlink d4/d20/d2d/l38 0 2024-03-22T07:08:49.405 INFO:tasks.workunit.client.1.smithi149.stdout:2/180: dread d1/f4 [0,4194304] 0 2024-03-22T07:08:49.405 INFO:tasks.workunit.client.1.smithi149.stdout:2/181: fdatasync d1/f10 0 2024-03-22T07:08:49.405 INFO:tasks.workunit.client.1.smithi149.stdout:2/182: chown d1/d3/d26 12033 1 2024-03-22T07:08:49.406 INFO:tasks.workunit.client.1.smithi149.stdout:9/198: truncate db/d12/f17 2477600 0 2024-03-22T07:08:49.408 INFO:tasks.workunit.client.1.smithi149.stdout:0/89: mknod d1/d9/c16 0 2024-03-22T07:08:49.409 INFO:tasks.workunit.client.1.smithi149.stdout:1/180: mknod d4/d20/c3d 0 2024-03-22T07:08:49.411 INFO:tasks.workunit.client.1.smithi149.stdout:2/183: rename d1/d8/d34/d3d to d1/d3/d6/d3f 0 2024-03-22T07:08:49.413 INFO:tasks.workunit.client.1.smithi149.stdout:1/181: unlink d4/f17 0 2024-03-22T07:08:49.415 INFO:tasks.workunit.client.1.smithi149.stdout:0/90: rename d1/l7 to d1/d9/d14/l17 0 2024-03-22T07:08:49.415 INFO:tasks.workunit.client.1.smithi149.stdout:0/91: chown d1 13497892 1 2024-03-22T07:08:49.417 INFO:tasks.workunit.client.1.smithi149.stdout:1/182: creat d4/f3e x:0 0 0 2024-03-22T07:08:49.431 INFO:tasks.workunit.client.1.smithi149.stdout:7/213: dwrite d1/d18/d1b/d1d/f26 [0,4194304] 0 2024-03-22T07:08:49.431 INFO:tasks.workunit.client.1.smithi149.stdout:7/214: readlink d1/l12 0 2024-03-22T07:08:49.431 INFO:tasks.workunit.client.1.smithi149.stdout:7/215: fdatasync d1/f5 0 2024-03-22T07:08:49.431 INFO:tasks.workunit.client.1.smithi149.stdout:7/216: write d1/d18/d1b/d1d/f13 [175186,126755] 0 2024-03-22T07:08:49.435 INFO:tasks.workunit.client.1.smithi149.stdout:1/183: dread f1 [0,4194304] 0 2024-03-22T07:08:49.447 INFO:tasks.workunit.client.1.smithi149.stdout:4/103: sync 2024-03-22T07:08:49.447 INFO:tasks.workunit.client.1.smithi149.stdout:4/104: creat d0/d8/f1f x:0 0 0 2024-03-22T07:08:49.447 INFO:tasks.workunit.client.1.smithi149.stdout:4/105: rename d0/d8 to d0/d8/d1d/d20 22 2024-03-22T07:08:49.451 INFO:tasks.workunit.client.1.smithi149.stdout:5/166: dwrite d7/f17 [0,4194304] 0 2024-03-22T07:08:49.452 INFO:tasks.workunit.client.1.smithi149.stdout:8/171: dwrite d1/f16 [0,4194304] 0 2024-03-22T07:08:49.455 INFO:tasks.workunit.client.1.smithi149.stdout:8/172: rename d1/d2/d9/df/f19 to d1/d2/d9/f37 0 2024-03-22T07:08:49.456 INFO:tasks.workunit.client.1.smithi149.stdout:8/173: creat d1/d20/d35/d22/f38 x:0 0 0 2024-03-22T07:08:49.458 INFO:tasks.workunit.client.1.smithi149.stdout:8/174: rename d1/d2/d9/d11/c13 to d1/d2/d3/c39 0 2024-03-22T07:08:49.458 INFO:tasks.workunit.client.1.smithi149.stdout:8/175: fdatasync d1/d20/f31 0 2024-03-22T07:08:49.459 INFO:tasks.workunit.client.1.smithi149.stdout:8/176: creat d1/d2/d3/f3a x:0 0 0 2024-03-22T07:08:49.469 INFO:tasks.workunit.client.1.smithi149.stdout:5/167: dread d7/f17 [0,4194304] 0 2024-03-22T07:08:49.471 INFO:tasks.workunit.client.1.smithi149.stdout:5/168: rename d7/da/c19 to d7/db/c2d 0 2024-03-22T07:08:49.473 INFO:tasks.workunit.client.1.smithi149.stdout:5/169: symlink d7/l2e 0 2024-03-22T07:08:49.473 INFO:tasks.workunit.client.1.smithi149.stdout:5/170: chown d7/d29 0 1 2024-03-22T07:08:49.478 INFO:tasks.workunit.client.1.smithi149.stdout:6/141: dwrite d9/fd [0,4194304] 0 2024-03-22T07:08:49.478 INFO:tasks.workunit.client.1.smithi149.stdout:6/142: dread - d9/da/f27 zero size 2024-03-22T07:08:49.478 INFO:tasks.workunit.client.1.smithi149.stdout:7/217: dwrite d1/d18/d1b/d1d/f40 [0,4194304] 0 2024-03-22T07:08:49.481 INFO:tasks.workunit.client.1.smithi149.stdout:4/106: dwrite d0/d8/dc/f18 [0,4194304] 0 2024-03-22T07:08:49.481 INFO:tasks.workunit.client.1.smithi149.stdout:4/107: truncate d0/d8/dc/d14/f16 81550 0 2024-03-22T07:08:49.481 INFO:tasks.workunit.client.1.smithi149.stdout:4/108: creat d0/d8/f21 x:0 0 0 2024-03-22T07:08:49.481 INFO:tasks.workunit.client.1.smithi149.stdout:4/109: write d0/f3 [2633250,80897] 0 2024-03-22T07:08:49.482 INFO:tasks.workunit.client.1.smithi149.stdout:2/184: dwrite d1/f10 [0,4194304] 0 2024-03-22T07:08:49.482 INFO:tasks.workunit.client.1.smithi149.stdout:1/184: dwrite d4/d20/d23/d2f/f35 [0,4194304] 0 2024-03-22T07:08:49.482 INFO:tasks.workunit.client.1.smithi149.stdout:1/185: stat d4/d20/f22 0 2024-03-22T07:08:49.487 INFO:tasks.workunit.client.1.smithi149.stdout:0/92: dwrite d1/d9/f12 [0,4194304] 0 2024-03-22T07:08:49.487 INFO:tasks.workunit.client.1.smithi149.stdout:0/93: chown d1/d9/d14 360340113 1 2024-03-22T07:08:49.488 INFO:tasks.workunit.client.1.smithi149.stdout:2/185: dread d1/d3/f9 [0,4194304] 0 2024-03-22T07:08:49.493 INFO:tasks.workunit.client.1.smithi149.stdout:0/94: creat d1/d9/d14/f18 x:0 0 0 2024-03-22T07:08:49.493 INFO:tasks.workunit.client.1.smithi149.stdout:6/143: read d9/df/f16 [375129,11377] 0 2024-03-22T07:08:49.495 INFO:tasks.workunit.client.1.smithi149.stdout:6/144: read d9/df/f1e [483204,33493] 0 2024-03-22T07:08:49.495 INFO:tasks.workunit.client.1.smithi149.stdout:6/145: creat d9/d1f/d24/f2d x:0 0 0 2024-03-22T07:08:49.496 INFO:tasks.workunit.client.1.smithi149.stdout:0/95: mknod d1/c19 0 2024-03-22T07:08:49.499 INFO:tasks.workunit.client.1.smithi149.stdout:8/177: dwrite d1/d2/d3/f6 [4194304,4194304] 0 2024-03-22T07:08:49.499 INFO:tasks.workunit.client.1.smithi149.stdout:8/178: creat d1/d2/d9/f3b x:0 0 0 2024-03-22T07:08:49.499 INFO:tasks.workunit.client.1.smithi149.stdout:5/171: dwrite d7/f9 [4194304,4194304] 0 2024-03-22T07:08:49.503 INFO:tasks.workunit.client.1.smithi149.stdout:6/146: rename d9/da to d9/df/d2e 0 2024-03-22T07:08:49.504 INFO:tasks.workunit.client.1.smithi149.stdout:5/172: read d7/da/f1b [441960,125853] 0 2024-03-22T07:08:49.508 INFO:tasks.workunit.client.1.smithi149.stdout:0/96: write d1/fb [371736,109220] 0 2024-03-22T07:08:49.509 INFO:tasks.workunit.client.1.smithi149.stdout:0/97: write d1/d9/fd [2717644,129562] 0 2024-03-22T07:08:49.509 INFO:tasks.workunit.client.1.smithi149.stdout:7/218: dwrite d1/d18/d1b/d1d/f38 [0,4194304] 0 2024-03-22T07:08:49.510 INFO:tasks.workunit.client.1.smithi149.stdout:2/186: dwrite d1/d3/f9 [0,4194304] 0 2024-03-22T07:08:49.510 INFO:tasks.workunit.client.1.smithi149.stdout:9/199: truncate f9 1739584 0 2024-03-22T07:08:49.510 INFO:tasks.workunit.client.1.smithi149.stdout:9/200: creat db/d12/f35 x:0 0 0 2024-03-22T07:08:49.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/173: symlink d7/da/d20/l2f 0 2024-03-22T07:08:49.513 INFO:tasks.workunit.client.1.smithi149.stdout:6/147: unlink d9/df/f16 0 2024-03-22T07:08:49.513 INFO:tasks.workunit.client.1.smithi149.stdout:8/179: write d1/d2/d3/f6 [7420375,36041] 0 2024-03-22T07:08:49.513 INFO:tasks.workunit.client.1.smithi149.stdout:8/180: chown d1/d20/d35/d22 48702188 1 2024-03-22T07:08:49.514 INFO:tasks.workunit.client.1.smithi149.stdout:7/219: mknod d1/d1c/d2f/c4a 0 2024-03-22T07:08:49.514 INFO:tasks.workunit.client.1.smithi149.stdout:7/220: readlink d1/d18/d1b/d1d/la 0 2024-03-22T07:08:49.516 INFO:tasks.workunit.client.1.smithi149.stdout:2/187: rename d1/f2a to d1/d3/dd/f40 0 2024-03-22T07:08:49.516 INFO:tasks.workunit.client.1.smithi149.stdout:6/148: rename d9 to d9/df/d22/d2f 22 2024-03-22T07:08:49.516 INFO:tasks.workunit.client.1.smithi149.stdout:2/188: chown d1/d3/dd/f21 3236 1 2024-03-22T07:08:49.516 INFO:tasks.workunit.client.1.smithi149.stdout:2/189: chown d1/d3/d5/d1f/l35 48344 1 2024-03-22T07:08:49.517 INFO:tasks.workunit.client.1.smithi149.stdout:2/190: write d1/d3/dd/f21 [903622,48635] 0 2024-03-22T07:08:49.522 INFO:tasks.workunit.client.1.smithi149.stdout:9/201: unlink db/d21/f26 0 2024-03-22T07:08:49.522 INFO:tasks.workunit.client.1.smithi149.stdout:5/174: mkdir d7/da/d20/d30 0 2024-03-22T07:08:49.522 INFO:tasks.workunit.client.1.smithi149.stdout:5/175: chown d7/db/l1c 15749 1 2024-03-22T07:08:49.523 INFO:tasks.workunit.client.1.smithi149.stdout:0/98: mkdir d1/d1a 0 2024-03-22T07:08:49.523 INFO:tasks.workunit.client.1.smithi149.stdout:8/181: read d1/d2/d3/f1a [1372131,122346] 0 2024-03-22T07:08:49.523 INFO:tasks.workunit.client.1.smithi149.stdout:8/182: fdatasync d1/d20/f31 0 2024-03-22T07:08:49.523 INFO:tasks.workunit.client.1.smithi149.stdout:8/183: fdatasync d1/d2/d3/f3a 0 2024-03-22T07:08:49.523 INFO:tasks.workunit.client.1.smithi149.stdout:7/221: rename d1/d18/d1b/d1d/d28/l2b to d1/d18/d1b/l4b 0 2024-03-22T07:08:49.523 INFO:tasks.workunit.client.1.smithi149.stdout:6/149: mknod d9/df/c30 0 2024-03-22T07:08:49.523 INFO:tasks.workunit.client.1.smithi149.stdout:6/150: chown d9/df/d2e/f12 0 1 2024-03-22T07:08:49.524 INFO:tasks.workunit.client.1.smithi149.stdout:6/151: fdatasync d9/f2c 0 2024-03-22T07:08:49.524 INFO:tasks.workunit.client.1.smithi149.stdout:9/202: truncate db/d22/f2b 3195886 0 2024-03-22T07:08:49.525 INFO:tasks.workunit.client.1.smithi149.stdout:2/191: read d1/f27 [3984163,10754] 0 2024-03-22T07:08:49.525 INFO:tasks.workunit.client.1.smithi149.stdout:9/203: stat db/d22 0 2024-03-22T07:08:49.525 INFO:tasks.workunit.client.1.smithi149.stdout:8/184: dread d1/d2/d9/f37 [4194304,4194304] 0 2024-03-22T07:08:49.525 INFO:tasks.workunit.client.1.smithi149.stdout:8/185: stat d1/d2/f23 0 2024-03-22T07:08:49.525 INFO:tasks.workunit.client.1.smithi149.stdout:8/186: chown d1/d2/d9/f3b 0 1 2024-03-22T07:08:49.525 INFO:tasks.workunit.client.1.smithi149.stdout:5/176: link d7/db/c2d d7/db/c31 0 2024-03-22T07:08:49.525 INFO:tasks.workunit.client.1.smithi149.stdout:5/177: chown d7/da/fe 10794703 1 2024-03-22T07:08:49.525 INFO:tasks.workunit.client.1.smithi149.stdout:5/178: chown d7/da 965804220 1 2024-03-22T07:08:49.526 INFO:tasks.workunit.client.1.smithi149.stdout:5/179: chown d7/db/fc 19395 1 2024-03-22T07:08:49.528 INFO:tasks.workunit.client.1.smithi149.stdout:0/99: unlink d1/d9/c11 0 2024-03-22T07:08:49.528 INFO:tasks.workunit.client.1.smithi149.stdout:0/100: chown d1/d9/f13 126213052 1 2024-03-22T07:08:49.529 INFO:tasks.workunit.client.1.smithi149.stdout:7/222: rename d1/d18/d1b/d1d/l23 to d1/d1c/d49/l4c 0 2024-03-22T07:08:49.529 INFO:tasks.workunit.client.1.smithi149.stdout:7/223: chown d1/d18/d1b/d1d/f38 115 1 2024-03-22T07:08:49.532 INFO:tasks.workunit.client.1.smithi149.stdout:6/152: creat d9/df/d22/f31 x:0 0 0 2024-03-22T07:08:49.532 INFO:tasks.workunit.client.1.smithi149.stdout:6/153: creat d9/f32 x:0 0 0 2024-03-22T07:08:49.532 INFO:tasks.workunit.client.1.smithi149.stdout:6/154: chown d9/f19 7431369 1 2024-03-22T07:08:49.532 INFO:tasks.workunit.client.1.smithi149.stdout:6/155: chown d9/d1f/d24/f26 1278155 1 2024-03-22T07:08:49.540 INFO:tasks.workunit.client.1.smithi149.stdout:5/180: rmdir d7/d29/d2a 0 2024-03-22T07:08:49.540 INFO:tasks.workunit.client.1.smithi149.stdout:7/224: rename d1/d18/d1b/d1d/d2a/d2d to d1/d1c/d4d 0 2024-03-22T07:08:49.541 INFO:tasks.workunit.client.1.smithi149.stdout:5/181: rmdir d7/da/d20 39 2024-03-22T07:08:49.541 INFO:tasks.workunit.client.1.smithi149.stdout:5/182: readlink d7/da/lf 0 2024-03-22T07:08:49.541 INFO:tasks.workunit.client.1.smithi149.stdout:5/183: chown d7/da/fe 3728974 1 2024-03-22T07:08:49.543 INFO:tasks.workunit.client.1.smithi149.stdout:2/192: write d1/f4 [3090704,109303] 0 2024-03-22T07:08:49.546 INFO:tasks.workunit.client.1.smithi149.stdout:5/184: read d7/da/fe [4290824,58391] 0 2024-03-22T07:08:49.548 INFO:tasks.workunit.client.1.smithi149.stdout:8/187: dread d1/f21 [0,4194304] 0 2024-03-22T07:08:49.549 INFO:tasks.workunit.client.1.smithi149.stdout:5/185: mkdir d7/da/d20/d30/d32 0 2024-03-22T07:08:49.549 INFO:tasks.workunit.client.1.smithi149.stdout:5/186: write d7/da/f1b [326643,130311] 0 2024-03-22T07:08:49.550 INFO:tasks.workunit.client.1.smithi149.stdout:8/188: unlink d1/f2c 0 2024-03-22T07:08:49.554 INFO:tasks.workunit.client.1.smithi149.stdout:2/193: dread d1/f1a [0,4194304] 0 2024-03-22T07:08:49.565 INFO:tasks.workunit.client.1.smithi149.stdout:6/156: dwrite d9/f17 [0,4194304] 0 2024-03-22T07:08:49.566 INFO:tasks.workunit.client.1.smithi149.stdout:0/101: dwrite d1/d9/f15 [0,4194304] 0 2024-03-22T07:08:49.566 INFO:tasks.workunit.client.1.smithi149.stdout:6/157: truncate d9/df/d2e/f12 2951323 0 2024-03-22T07:08:49.566 INFO:tasks.workunit.client.1.smithi149.stdout:6/158: read - d9/df/d2e/f1c zero size 2024-03-22T07:08:49.568 INFO:tasks.workunit.client.1.smithi149.stdout:5/187: write d7/f9 [4243338,120843] 0 2024-03-22T07:08:49.568 INFO:tasks.workunit.client.1.smithi149.stdout:9/204: dwrite f3 [0,4194304] 0 2024-03-22T07:08:49.569 INFO:tasks.workunit.client.1.smithi149.stdout:9/205: fdatasync db/fc 0 2024-03-22T07:08:49.569 INFO:tasks.workunit.client.1.smithi149.stdout:9/206: write db/f2f [71646,87298] 0 2024-03-22T07:08:49.572 INFO:tasks.workunit.client.1.smithi149.stdout:5/188: creat d7/f33 x:0 0 0 2024-03-22T07:08:49.572 INFO:tasks.workunit.client.1.smithi149.stdout:5/189: fsync d7/f17 0 2024-03-22T07:08:49.573 INFO:tasks.workunit.client.1.smithi149.stdout:8/189: dread d1/d2/d9/f37 [0,4194304] 0 2024-03-22T07:08:49.577 INFO:tasks.workunit.client.1.smithi149.stdout:5/190: rename d7/f17 to d7/d29/f34 0 2024-03-22T07:08:49.577 INFO:tasks.workunit.client.1.smithi149.stdout:8/190: creat d1/d2/d18/f3c x:0 0 0 2024-03-22T07:08:49.577 INFO:tasks.workunit.client.1.smithi149.stdout:8/191: dread - d1/f24 zero size 2024-03-22T07:08:49.577 INFO:tasks.workunit.client.1.smithi149.stdout:8/192: fdatasync d1/d2/d9/d11/f17 0 2024-03-22T07:08:49.577 INFO:tasks.workunit.client.1.smithi149.stdout:8/193: creat d1/d2/d18/f3d x:0 0 0 2024-03-22T07:08:49.579 INFO:tasks.workunit.client.1.smithi149.stdout:9/207: rmdir db/d12 39 2024-03-22T07:08:49.581 INFO:tasks.workunit.client.1.smithi149.stdout:8/194: link d1/d2/d9/l27 d1/d2/d9/l3e 0 2024-03-22T07:08:49.583 INFO:tasks.workunit.client.1.smithi149.stdout:2/194: dwrite d1/d3/fb [0,4194304] 0 2024-03-22T07:08:49.584 INFO:tasks.workunit.client.1.smithi149.stdout:8/195: rename d1/d2/d3/l2b to d1/l3f 0 2024-03-22T07:08:49.589 INFO:tasks.workunit.client.1.smithi149.stdout:0/102: rmdir d1/d9 39 2024-03-22T07:08:49.589 INFO:tasks.workunit.client.1.smithi149.stdout:0/103: fdatasync d1/d9/f12 0 2024-03-22T07:08:49.590 INFO:tasks.workunit.client.1.smithi149.stdout:7/225: getdents d1/d18/d1b/d1d 0 2024-03-22T07:08:49.590 INFO:tasks.workunit.client.1.smithi149.stdout:7/226: truncate d1/d1c/d4d/f48 139027 0 2024-03-22T07:08:49.592 INFO:tasks.workunit.client.1.smithi149.stdout:8/196: unlink d1/d20/f31 0 2024-03-22T07:08:49.592 INFO:tasks.workunit.client.1.smithi149.stdout:8/197: creat d1/d20/d35/d22/f40 x:0 0 0 2024-03-22T07:08:49.594 INFO:tasks.workunit.client.1.smithi149.stdout:7/227: symlink d1/d18/d1b/l4e 0 2024-03-22T07:08:49.598 INFO:tasks.workunit.client.1.smithi149.stdout:8/198: mkdir d1/d20/d41 0 2024-03-22T07:08:49.599 INFO:tasks.workunit.client.1.smithi149.stdout:0/104: dread d1/d9/fd [0,4194304] 0 2024-03-22T07:08:49.599 INFO:tasks.workunit.client.1.smithi149.stdout:0/105: chown d1/d9/fd 28057 1 2024-03-22T07:08:49.600 INFO:tasks.workunit.client.1.smithi149.stdout:0/106: symlink d1/d1a/l1b 0 2024-03-22T07:08:49.600 INFO:tasks.workunit.client.1.smithi149.stdout:0/107: chown d1/d9/f13 1109961 1 2024-03-22T07:08:49.600 INFO:tasks.workunit.client.1.smithi149.stdout:0/108: dread - d1/d9/f13 zero size 2024-03-22T07:08:49.601 INFO:tasks.workunit.client.1.smithi149.stdout:0/109: rmdir d1/d9 39 2024-03-22T07:08:49.601 INFO:tasks.workunit.client.1.smithi149.stdout:2/195: dread d1/d3/d5/ff [0,4194304] 0 2024-03-22T07:08:49.603 INFO:tasks.workunit.client.1.smithi149.stdout:2/196: mkdir d1/d3/d5/d41 0 2024-03-22T07:08:49.603 INFO:tasks.workunit.client.1.smithi149.stdout:2/197: chown d1/d3/d26/f37 24644 1 2024-03-22T07:08:49.604 INFO:tasks.workunit.client.1.smithi149.stdout:8/199: dread d1/d2/d9/f29 [0,4194304] 0 2024-03-22T07:08:49.604 INFO:tasks.workunit.client.1.smithi149.stdout:8/200: readlink d1/d2/l12 0 2024-03-22T07:08:49.604 INFO:tasks.workunit.client.1.smithi149.stdout:8/201: chown d1/d2/d9/f29 1 1 2024-03-22T07:08:49.604 INFO:tasks.workunit.client.1.smithi149.stdout:8/202: chown d1/d2/d9/d11 2821041 1 2024-03-22T07:08:49.604 INFO:tasks.workunit.client.1.smithi149.stdout:2/198: creat d1/f42 x:0 0 0 2024-03-22T07:08:49.612 INFO:tasks.workunit.client.1.smithi149.stdout:8/203: dread d1/d2/d3/f1a [0,4194304] 0 2024-03-22T07:08:49.613 INFO:tasks.workunit.client.1.smithi149.stdout:8/204: stat d1/d2/d3/c39 0 2024-03-22T07:08:49.613 INFO:tasks.workunit.client.1.smithi149.stdout:8/205: creat d1/d2/d9/f42 x:0 0 0 2024-03-22T07:08:49.613 INFO:tasks.workunit.client.1.smithi149.stdout:8/206: creat d1/d2/f43 x:0 0 0 2024-03-22T07:08:49.613 INFO:tasks.workunit.client.1.smithi149.stdout:8/207: write d1/d2/d18/f3c [665233,104058] 0 2024-03-22T07:08:49.615 INFO:tasks.workunit.client.1.smithi149.stdout:8/208: mkdir d1/d20/d35/d22/d44 0 2024-03-22T07:08:49.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/199: dread d1/d3/d6/f7 [0,4194304] 0 2024-03-22T07:08:49.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/200: readlink d1/d3/d26/l31 0 2024-03-22T07:08:49.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/201: fsync d1/d3/d6/f16 0 2024-03-22T07:08:49.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/202: chown d1/d3/d26/d2f/l3c 0 1 2024-03-22T07:08:49.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/203: fdatasync d1/d3/fb 0 2024-03-22T07:08:49.617 INFO:tasks.workunit.client.1.smithi149.stdout:8/209: mkdir d1/d2/d9/d11/d45 0 2024-03-22T07:08:49.617 INFO:tasks.workunit.client.1.smithi149.stdout:0/110: dread d1/d9/f12 [0,4194304] 0 2024-03-22T07:08:49.617 INFO:tasks.workunit.client.1.smithi149.stdout:5/191: dwrite d7/da/fe [4194304,4194304] 0 2024-03-22T07:08:49.617 INFO:tasks.workunit.client.1.smithi149.stdout:5/192: chown d7/f33 0 1 2024-03-22T07:08:49.617 INFO:tasks.workunit.client.1.smithi149.stdout:2/204: link d1/d3/d26/c2d d1/d8/c43 0 2024-03-22T07:08:49.618 INFO:tasks.workunit.client.1.smithi149.stdout:8/210: truncate d1/d2/d3/fd 3474326 0 2024-03-22T07:08:49.619 INFO:tasks.workunit.client.1.smithi149.stdout:0/111: rename d1/c3 to d1/d1a/c1c 0 2024-03-22T07:08:49.620 INFO:tasks.workunit.client.1.smithi149.stdout:8/211: symlink d1/d2/d9/d11/d45/l46 0 2024-03-22T07:08:49.620 INFO:tasks.workunit.client.1.smithi149.stdout:0/112: getdents d1/d9 0 2024-03-22T07:08:49.620 INFO:tasks.workunit.client.1.smithi149.stdout:0/113: write d1/fb [2365245,77024] 0 2024-03-22T07:08:49.620 INFO:tasks.workunit.client.1.smithi149.stdout:0/114: truncate d1/d9/f13 947005 0 2024-03-22T07:08:49.621 INFO:tasks.workunit.client.1.smithi149.stdout:5/193: mkdir d7/d35 0 2024-03-22T07:08:49.622 INFO:tasks.workunit.client.1.smithi149.stdout:5/194: mkdir d7/da/d20/d36 0 2024-03-22T07:08:49.622 INFO:tasks.workunit.client.1.smithi149.stdout:5/195: chown d7/d29/f34 126964514 1 2024-03-22T07:08:49.623 INFO:tasks.workunit.client.1.smithi149.stdout:8/212: mknod d1/d20/d35/d22/c47 0 2024-03-22T07:08:49.625 INFO:tasks.workunit.client.1.smithi149.stdout:5/196: creat d7/da/f37 x:0 0 0 2024-03-22T07:08:49.625 INFO:tasks.workunit.client.1.smithi149.stdout:9/208: dwrite f3 [4194304,4194304] 0 2024-03-22T07:08:49.625 INFO:tasks.workunit.client.1.smithi149.stdout:8/213: truncate d1/d2/d9/df/f15 428542 0 2024-03-22T07:08:49.627 INFO:tasks.workunit.client.1.smithi149.stdout:8/214: truncate d1/d2/d9/f1b 3931415 0 2024-03-22T07:08:49.628 INFO:tasks.workunit.client.1.smithi149.stdout:8/215: creat d1/d20/d41/f48 x:0 0 0 2024-03-22T07:08:49.630 INFO:tasks.workunit.client.1.smithi149.stdout:8/216: creat d1/d20/f49 x:0 0 0 2024-03-22T07:08:49.634 INFO:tasks.workunit.client.1.smithi149.stdout:5/197: write d7/da/fe [5538995,44923] 0 2024-03-22T07:08:49.635 INFO:tasks.workunit.client.1.smithi149.stdout:9/209: dread db/d22/f2b [0,4194304] 0 2024-03-22T07:08:49.635 INFO:tasks.workunit.client.1.smithi149.stdout:9/210: creat db/d22/f36 x:0 0 0 2024-03-22T07:08:49.635 INFO:tasks.workunit.client.1.smithi149.stdout:4/110: sync 2024-03-22T07:08:49.635 INFO:tasks.workunit.client.1.smithi149.stdout:3/124: sync 2024-03-22T07:08:49.635 INFO:tasks.workunit.client.1.smithi149.stdout:5/198: mknod d7/db/d2b/c38 0 2024-03-22T07:08:49.636 INFO:tasks.workunit.client.1.smithi149.stdout:9/211: truncate db/d19/f1f 1472765 0 2024-03-22T07:08:49.637 INFO:tasks.workunit.client.1.smithi149.stdout:4/111: mknod d0/d8/d1d/c22 0 2024-03-22T07:08:49.637 INFO:tasks.workunit.client.1.smithi149.stdout:4/112: creat d0/d8/dc/d14/f23 x:0 0 0 2024-03-22T07:08:49.638 INFO:tasks.workunit.client.1.smithi149.stdout:4/113: read - d0/d8/f1c zero size 2024-03-22T07:08:49.639 INFO:tasks.workunit.client.1.smithi149.stdout:3/125: creat d0/d2/f2d x:0 0 0 2024-03-22T07:08:49.640 INFO:tasks.workunit.client.1.smithi149.stdout:5/199: rename d7/l2e to d7/d35/l39 0 2024-03-22T07:08:49.640 INFO:tasks.workunit.client.1.smithi149.stdout:8/217: write d1/f16 [449064,9390] 0 2024-03-22T07:08:49.641 INFO:tasks.workunit.client.1.smithi149.stdout:9/212: mkdir db/d21/d37 0 2024-03-22T07:08:49.641 INFO:tasks.workunit.client.1.smithi149.stdout:9/213: dread - db/d22/f36 zero size 2024-03-22T07:08:49.641 INFO:tasks.workunit.client.1.smithi149.stdout:9/214: readlink l6 0 2024-03-22T07:08:49.643 INFO:tasks.workunit.client.1.smithi149.stdout:4/114: mkdir d0/d8/d1d/d24 0 2024-03-22T07:08:49.643 INFO:tasks.workunit.client.1.smithi149.stdout:4/115: write d0/d8/f21 [854907,111374] 0 2024-03-22T07:08:49.644 INFO:tasks.workunit.client.1.smithi149.stdout:3/126: creat d0/d4/d8/d1c/f2e x:0 0 0 2024-03-22T07:08:49.644 INFO:tasks.workunit.client.1.smithi149.stdout:3/127: chown d0/d4/f1f 311 1 2024-03-22T07:08:49.645 INFO:tasks.workunit.client.1.smithi149.stdout:5/200: mkdir d7/da/d20/d30/d3a 0 2024-03-22T07:08:49.647 INFO:tasks.workunit.client.1.smithi149.stdout:2/205: dwrite d1/f27 [0,4194304] 0 2024-03-22T07:08:49.647 INFO:tasks.workunit.client.1.smithi149.stdout:0/115: dwrite d1/d9/fd [4194304,4194304] 0 2024-03-22T07:08:49.648 INFO:tasks.workunit.client.1.smithi149.stdout:8/218: link d1/d20/f49 d1/d2/d9/f4a 0 2024-03-22T07:08:49.648 INFO:tasks.workunit.client.1.smithi149.stdout:8/219: write d1/d2/d9/f37 [5264280,21614] 0 2024-03-22T07:08:49.648 INFO:tasks.workunit.client.1.smithi149.stdout:9/215: symlink db/d21/l38 0 2024-03-22T07:08:49.649 INFO:tasks.workunit.client.1.smithi149.stdout:4/116: mknod d0/d8/dc/c25 0 2024-03-22T07:08:49.650 INFO:tasks.workunit.client.1.smithi149.stdout:4/117: getdents d0/d8/d1d/d24 0 2024-03-22T07:08:49.650 INFO:tasks.workunit.client.1.smithi149.stdout:4/118: chown d0/c2 88828 1 2024-03-22T07:08:49.650 INFO:tasks.workunit.client.1.smithi149.stdout:4/119: read d0/d8/dc/d14/f16 [67427,910] 0 2024-03-22T07:08:49.651 INFO:tasks.workunit.client.1.smithi149.stdout:3/128: creat d0/d4/f2f x:0 0 0 2024-03-22T07:08:49.651 INFO:tasks.workunit.client.1.smithi149.stdout:3/129: dread - d0/d4/d8/d1c/f22 zero size 2024-03-22T07:08:49.653 INFO:tasks.workunit.client.1.smithi149.stdout:5/201: mknod d7/d35/c3b 0 2024-03-22T07:08:49.653 INFO:tasks.workunit.client.1.smithi149.stdout:5/202: truncate d7/db/f22 1366301 0 2024-03-22T07:08:49.653 INFO:tasks.workunit.client.1.smithi149.stdout:5/203: write d7/da/f1b [770450,22664] 0 2024-03-22T07:08:49.658 INFO:tasks.workunit.client.1.smithi149.stdout:2/206: unlink d1/f1a 0 2024-03-22T07:08:49.658 INFO:tasks.workunit.client.1.smithi149.stdout:2/207: stat d1/d3/d5/d1f/l30 0 2024-03-22T07:08:49.660 INFO:tasks.workunit.client.1.smithi149.stdout:0/116: symlink d1/d1a/l1d 0 2024-03-22T07:08:49.660 INFO:tasks.workunit.client.1.smithi149.stdout:0/117: chown d1/d9 215840282 1 2024-03-22T07:08:49.660 INFO:tasks.workunit.client.1.smithi149.stdout:0/118: fdatasync d1/d9/f15 0 2024-03-22T07:08:49.664 INFO:tasks.workunit.client.1.smithi149.stdout:3/130: dread d0/d4/fe [4194304,4194304] 0 2024-03-22T07:08:49.666 INFO:tasks.workunit.client.1.smithi149.stdout:8/220: mknod d1/d20/d35/c4b 0 2024-03-22T07:08:49.667 INFO:tasks.workunit.client.1.smithi149.stdout:9/216: mkdir db/d19/d39 0 2024-03-22T07:08:49.668 INFO:tasks.workunit.client.1.smithi149.stdout:5/204: mknod d7/d35/c3c 0 2024-03-22T07:08:49.669 INFO:tasks.workunit.client.1.smithi149.stdout:2/208: link d1/d8/l14 d1/d3/d26/d2f/l44 0 2024-03-22T07:08:49.670 INFO:tasks.workunit.client.1.smithi149.stdout:0/119: creat d1/d9/f1e x:0 0 0 2024-03-22T07:08:49.671 INFO:tasks.workunit.client.1.smithi149.stdout:4/120: dwrite d0/d8/dc/fe [0,4194304] 0 2024-03-22T07:08:49.671 INFO:tasks.workunit.client.1.smithi149.stdout:4/121: chown d0/c1 336480 1 2024-03-22T07:08:49.671 INFO:tasks.workunit.client.1.smithi149.stdout:4/122: chown d0/lf 36579162 1 2024-03-22T07:08:49.671 INFO:tasks.workunit.client.1.smithi149.stdout:3/131: mkdir d0/d2/d30 0 2024-03-22T07:08:49.672 INFO:tasks.workunit.client.1.smithi149.stdout:8/221: rename d1/d20/d35/d22/c47 to d1/d2/d9/d11/c4c 0 2024-03-22T07:08:49.673 INFO:tasks.workunit.client.1.smithi149.stdout:9/217: mkdir db/d21/d37/d3a 0 2024-03-22T07:08:49.677 INFO:tasks.workunit.client.1.smithi149.stdout:4/123: mknod d0/c26 0 2024-03-22T07:08:49.679 INFO:tasks.workunit.client.1.smithi149.stdout:3/132: mkdir d0/d2/d30/d31 0 2024-03-22T07:08:49.680 INFO:tasks.workunit.client.1.smithi149.stdout:5/205: rename d7/d29 to d7/da/d20/d30/d3a/d3d 0 2024-03-22T07:08:49.681 INFO:tasks.workunit.client.1.smithi149.stdout:8/222: getdents d1/d2/d9/d11/d45 0 2024-03-22T07:08:49.681 INFO:tasks.workunit.client.1.smithi149.stdout:8/223: truncate d1/d2/d9/f4a 47889 0 2024-03-22T07:08:49.682 INFO:tasks.workunit.client.1.smithi149.stdout:2/209: dread d1/f27 [0,4194304] 0 2024-03-22T07:08:49.682 INFO:tasks.workunit.client.1.smithi149.stdout:2/210: fdatasync d1/f42 0 2024-03-22T07:08:49.682 INFO:tasks.workunit.client.1.smithi149.stdout:9/218: mkdir db/d19/d39/d3b 0 2024-03-22T07:08:49.682 INFO:tasks.workunit.client.1.smithi149.stdout:2/211: fsync d1/d3/f25 0 2024-03-22T07:08:49.682 INFO:tasks.workunit.client.1.smithi149.stdout:2/212: truncate d1/d3/f3e 408164 0 2024-03-22T07:08:49.682 INFO:tasks.workunit.client.1.smithi149.stdout:4/124: symlink d0/d8/d1d/d24/l27 0 2024-03-22T07:08:49.682 INFO:tasks.workunit.client.1.smithi149.stdout:4/125: chown d0/c2 8 1 2024-03-22T07:08:49.685 INFO:tasks.workunit.client.1.smithi149.stdout:3/133: link d0/d4/d8/d1c/f22 d0/d4/f32 0 2024-03-22T07:08:49.686 INFO:tasks.workunit.client.1.smithi149.stdout:0/120: dwrite d1/d9/f1e [0,4194304] 0 2024-03-22T07:08:49.687 INFO:tasks.workunit.client.1.smithi149.stdout:5/206: rename d7/db/l1c to d7/da/d20/d30/d3a/d3d/d2c/l3e 0 2024-03-22T07:08:49.688 INFO:tasks.workunit.client.1.smithi149.stdout:8/224: unlink d1/d20/f49 0 2024-03-22T07:08:49.689 INFO:tasks.workunit.client.1.smithi149.stdout:9/219: creat db/d19/f3c x:0 0 0 2024-03-22T07:08:49.691 INFO:tasks.workunit.client.1.smithi149.stdout:2/213: link d1/d3/d5/f28 d1/d3/dd/f45 0 2024-03-22T07:08:49.691 INFO:tasks.workunit.client.1.smithi149.stdout:2/214: rename d1 to d1/d3/d6/d3f/d46 22 2024-03-22T07:08:49.691 INFO:tasks.workunit.client.1.smithi149.stdout:2/215: chown d1/d3/l1d 2390423 1 2024-03-22T07:08:49.692 INFO:tasks.workunit.client.1.smithi149.stdout:4/126: creat d0/d8/d1d/f28 x:0 0 0 2024-03-22T07:08:49.693 INFO:tasks.workunit.client.1.smithi149.stdout:3/134: truncate d0/d4/f1e 257285 0 2024-03-22T07:08:49.693 INFO:tasks.workunit.client.1.smithi149.stdout:3/135: dread - d0/f1a zero size 2024-03-22T07:08:49.694 INFO:tasks.workunit.client.1.smithi149.stdout:3/136: chown d0/d4/d8/f23 2844 1 2024-03-22T07:08:49.694 INFO:tasks.workunit.client.1.smithi149.stdout:0/121: dread d1/d9/f1e [0,4194304] 0 2024-03-22T07:08:49.702 INFO:tasks.workunit.client.1.smithi149.stdout:5/207: mknod d7/d35/c3f 0 2024-03-22T07:08:49.705 INFO:tasks.workunit.client.1.smithi149.stdout:9/220: unlink db/c10 0 2024-03-22T07:08:49.705 INFO:tasks.workunit.client.1.smithi149.stdout:3/137: dread d0/d4/d8/f19 [0,4194304] 0 2024-03-22T07:08:49.705 INFO:tasks.workunit.client.1.smithi149.stdout:3/138: truncate d0/f1a 38079 0 2024-03-22T07:08:49.706 INFO:tasks.workunit.client.1.smithi149.stdout:2/216: link d1/l20 d1/d3/d5/d41/l47 0 2024-03-22T07:08:49.706 INFO:tasks.workunit.client.1.smithi149.stdout:2/217: write d1/d3/d5/d1f/f1e [4596295,14374] 0 2024-03-22T07:08:49.707 INFO:tasks.workunit.client.1.smithi149.stdout:2/218: fdatasync d1/d3/d5/d1f/f1b 0 2024-03-22T07:08:49.709 INFO:tasks.workunit.client.1.smithi149.stdout:4/127: mkdir d0/d29 0 2024-03-22T07:08:49.709 INFO:tasks.workunit.client.1.smithi149.stdout:4/128: chown d0/d8/dc/f13 210658 1 2024-03-22T07:08:49.710 INFO:tasks.workunit.client.1.smithi149.stdout:5/208: creat d7/d35/f40 x:0 0 0 2024-03-22T07:08:49.711 INFO:tasks.workunit.client.1.smithi149.stdout:0/122: dwrite d1/d9/f13 [0,4194304] 0 2024-03-22T07:08:49.712 INFO:tasks.workunit.client.1.smithi149.stdout:8/225: rmdir d1/d2/d3 39 2024-03-22T07:08:49.714 INFO:tasks.workunit.client.1.smithi149.stdout:3/139: creat d0/d2/d30/d31/f33 x:0 0 0 2024-03-22T07:08:49.719 INFO:tasks.workunit.client.1.smithi149.stdout:2/219: creat d1/d3/d5/d41/f48 x:0 0 0 2024-03-22T07:08:49.721 INFO:tasks.workunit.client.1.smithi149.stdout:4/129: rename d0/lb to d0/d29/l2a 0 2024-03-22T07:08:49.721 INFO:tasks.workunit.client.1.smithi149.stdout:4/130: creat d0/d8/dc/d14/f2b x:0 0 0 2024-03-22T07:08:49.724 INFO:tasks.workunit.client.1.smithi149.stdout:0/123: mkdir d1/d9/d14/d1f 0 2024-03-22T07:08:49.725 INFO:tasks.workunit.client.1.smithi149.stdout:8/226: mkdir d1/d2/d9/d11/d4d 0 2024-03-22T07:08:49.726 INFO:tasks.workunit.client.1.smithi149.stdout:3/140: mknod d0/d4/d8/d1c/c34 0 2024-03-22T07:08:49.730 INFO:tasks.workunit.client.1.smithi149.stdout:4/131: mkdir d0/d8/dc/d14/d2c 0 2024-03-22T07:08:49.731 INFO:tasks.workunit.client.1.smithi149.stdout:4/132: read d0/f3 [1557640,72551] 0 2024-03-22T07:08:49.731 INFO:tasks.workunit.client.1.smithi149.stdout:4/133: truncate d0/d8/f1f 705168 0 2024-03-22T07:08:49.731 INFO:tasks.workunit.client.1.smithi149.stdout:3/141: mknod d0/d2/c35 0 2024-03-22T07:08:49.732 INFO:tasks.workunit.client.1.smithi149.stdout:2/220: symlink d1/d8/d34/l49 0 2024-03-22T07:08:49.732 INFO:tasks.workunit.client.1.smithi149.stdout:2/221: creat d1/d3/d26/f4a x:0 0 0 2024-03-22T07:08:49.732 INFO:tasks.workunit.client.1.smithi149.stdout:2/222: creat d1/d3/f4b x:0 0 0 2024-03-22T07:08:49.732 INFO:tasks.workunit.client.1.smithi149.stdout:2/223: truncate d1/d3/f9 5071575 0 2024-03-22T07:08:49.733 INFO:tasks.workunit.client.1.smithi149.stdout:2/224: stat d1/d3/d6/f7 0 2024-03-22T07:08:49.733 INFO:tasks.workunit.client.1.smithi149.stdout:0/124: mkdir d1/d20 0 2024-03-22T07:08:49.734 INFO:tasks.workunit.client.1.smithi149.stdout:2/225: truncate d1/d3/d6/f16 2683653 0 2024-03-22T07:08:49.735 INFO:tasks.workunit.client.1.smithi149.stdout:2/226: write d1/d8/f19 [1037876,124941] 0 2024-03-22T07:08:49.737 INFO:tasks.workunit.client.1.smithi149.stdout:2/227: creat d1/d3/dd/f4c x:0 0 0 2024-03-22T07:08:49.748 INFO:tasks.workunit.client.1.smithi149.stdout:5/209: dwrite d7/d35/f40 [0,4194304] 0 2024-03-22T07:08:49.748 INFO:tasks.workunit.client.1.smithi149.stdout:9/221: dwrite db/d12/f35 [0,4194304] 0 2024-03-22T07:08:49.754 INFO:tasks.workunit.client.1.smithi149.stdout:8/227: rename d1/d2/d9/lc to d1/d2/l4e 0 2024-03-22T07:08:49.756 INFO:tasks.workunit.client.1.smithi149.stdout:8/228: mkdir d1/d20/d35/d22/d44/d4f 0 2024-03-22T07:08:49.756 INFO:tasks.workunit.client.1.smithi149.stdout:8/229: write d1/d20/d41/f48 [1012833,17499] 0 2024-03-22T07:08:49.756 INFO:tasks.workunit.client.1.smithi149.stdout:8/230: dread - d1/f24 zero size 2024-03-22T07:08:49.756 INFO:tasks.workunit.client.1.smithi149.stdout:8/231: stat d1/d2 0 2024-03-22T07:08:49.760 INFO:tasks.workunit.client.1.smithi149.stdout:3/142: rename d0/d4/d8/d1c to d0/d2/d36 0 2024-03-22T07:08:49.761 INFO:tasks.workunit.client.1.smithi149.stdout:9/222: rename db/fd to db/d33/f3d 0 2024-03-22T07:08:49.762 INFO:tasks.workunit.client.1.smithi149.stdout:2/228: read d1/d3/d6/f16 [1483207,54403] 0 2024-03-22T07:08:49.763 INFO:tasks.workunit.client.1.smithi149.stdout:3/143: creat d0/d2/d36/f37 x:0 0 0 2024-03-22T07:08:49.764 INFO:tasks.workunit.client.1.smithi149.stdout:9/223: link db/d21/d2e/c31 db/d21/d37/c3e 0 2024-03-22T07:08:49.765 INFO:tasks.workunit.client.1.smithi149.stdout:2/229: mkdir d1/d4d 0 2024-03-22T07:08:49.766 INFO:tasks.workunit.client.1.smithi149.stdout:9/224: symlink db/d21/d2e/l3f 0 2024-03-22T07:08:49.766 INFO:tasks.workunit.client.1.smithi149.stdout:9/225: stat db/d12/l1b 0 2024-03-22T07:08:49.766 INFO:tasks.workunit.client.1.smithi149.stdout:3/144: rmdir d0/d4/d8/d2b 0 2024-03-22T07:08:49.767 INFO:tasks.workunit.client.1.smithi149.stdout:3/145: truncate d0/d2/f2d 428942 0 2024-03-22T07:08:49.767 INFO:tasks.workunit.client.1.smithi149.stdout:3/146: creat d0/f38 x:0 0 0 2024-03-22T07:08:49.767 INFO:tasks.workunit.client.1.smithi149.stdout:3/147: dread - d0/d2/d36/f37 zero size 2024-03-22T07:08:49.797 INFO:tasks.workunit.client.1.smithi149.stdout:1/186: sync 2024-03-22T07:08:49.797 INFO:tasks.workunit.client.1.smithi149.stdout:1/187: write d4/f11 [1185748,113413] 0 2024-03-22T07:08:49.798 INFO:tasks.workunit.client.1.smithi149.stdout:1/188: creat d4/d20/d23/f3f x:0 0 0 2024-03-22T07:08:49.799 INFO:tasks.workunit.client.1.smithi149.stdout:7/228: sync 2024-03-22T07:08:49.800 INFO:tasks.workunit.client.1.smithi149.stdout:7/229: chown d1/d1c/d2f/l44 216 1 2024-03-22T07:08:49.800 INFO:tasks.workunit.client.1.smithi149.stdout:6/159: sync 2024-03-22T07:08:49.800 INFO:tasks.workunit.client.1.smithi149.stdout:4/134: dwrite d0/d8/fd [0,4194304] 0 2024-03-22T07:08:49.800 INFO:tasks.workunit.client.1.smithi149.stdout:4/135: chown d0/f1a 1600 1 2024-03-22T07:08:49.800 INFO:tasks.workunit.client.1.smithi149.stdout:4/136: creat d0/f2d x:0 0 0 2024-03-22T07:08:49.800 INFO:tasks.workunit.client.1.smithi149.stdout:7/230: symlink d1/d1c/d4d/l4f 0 2024-03-22T07:08:49.801 INFO:tasks.workunit.client.1.smithi149.stdout:1/189: read d4/f11 [524608,58394] 0 2024-03-22T07:08:49.801 INFO:tasks.workunit.client.1.smithi149.stdout:1/190: creat d4/d20/d2d/f40 x:0 0 0 2024-03-22T07:08:49.801 INFO:tasks.workunit.client.1.smithi149.stdout:4/137: mkdir d0/d8/dc/d2e 0 2024-03-22T07:08:49.803 INFO:tasks.workunit.client.1.smithi149.stdout:7/231: creat d1/d18/d1b/d1d/d28/f50 x:0 0 0 2024-03-22T07:08:49.804 INFO:tasks.workunit.client.1.smithi149.stdout:1/191: creat d4/d20/d23/d3c/f41 x:0 0 0 2024-03-22T07:08:49.804 INFO:tasks.workunit.client.1.smithi149.stdout:1/192: creat d4/f42 x:0 0 0 2024-03-22T07:08:49.805 INFO:tasks.workunit.client.1.smithi149.stdout:2/230: dread d1/d3/dd/f21 [0,4194304] 0 2024-03-22T07:08:49.806 INFO:tasks.workunit.client.1.smithi149.stdout:2/231: fdatasync d1/d3/f9 0 2024-03-22T07:08:49.806 INFO:tasks.workunit.client.1.smithi149.stdout:0/125: dwrite d1/d9/f1e [0,4194304] 0 2024-03-22T07:08:49.806 INFO:tasks.workunit.client.1.smithi149.stdout:0/126: stat d1/d20 0 2024-03-22T07:08:49.806 INFO:tasks.workunit.client.1.smithi149.stdout:4/138: rmdir d0/d8/dc/d14/d2c 0 2024-03-22T07:08:49.806 INFO:tasks.workunit.client.1.smithi149.stdout:4/139: chown d0/d8/dc/f13 526806 1 2024-03-22T07:08:49.807 INFO:tasks.workunit.client.1.smithi149.stdout:7/232: mkdir d1/d18/d1b/d51 0 2024-03-22T07:08:49.808 INFO:tasks.workunit.client.1.smithi149.stdout:2/232: mknod d1/d4d/c4e 0 2024-03-22T07:08:49.808 INFO:tasks.workunit.client.1.smithi149.stdout:2/233: dread - d1/d3/f36 zero size 2024-03-22T07:08:49.809 INFO:tasks.workunit.client.1.smithi149.stdout:0/127: stat d1/d1a/c1c 0 2024-03-22T07:08:49.810 INFO:tasks.workunit.client.1.smithi149.stdout:4/140: creat d0/d8/dc/d2e/f2f x:0 0 0 2024-03-22T07:08:49.811 INFO:tasks.workunit.client.1.smithi149.stdout:4/141: readlink d0/l7 0 2024-03-22T07:08:49.811 INFO:tasks.workunit.client.1.smithi149.stdout:4/142: truncate d0/d8/dc/d2e/f2f 892014 0 2024-03-22T07:08:49.812 INFO:tasks.workunit.client.1.smithi149.stdout:8/232: write d1/d2/d3/f6 [1490709,75859] 0 2024-03-22T07:08:49.813 INFO:tasks.workunit.client.1.smithi149.stdout:4/143: link d0/f3 d0/d8/dc/f30 0 2024-03-22T07:08:49.813 INFO:tasks.workunit.client.1.smithi149.stdout:4/144: chown d0 67031 1 2024-03-22T07:08:49.813 INFO:tasks.workunit.client.1.smithi149.stdout:4/145: fdatasync d0/d8/dc/fe 0 2024-03-22T07:08:49.814 INFO:tasks.workunit.client.1.smithi149.stdout:6/160: write d9/df/d2e/f12 [2511432,37511] 0 2024-03-22T07:08:49.816 INFO:tasks.workunit.client.1.smithi149.stdout:8/233: link d1/d2/l36 d1/d20/d35/d22/d44/d4f/l50 0 2024-03-22T07:08:49.818 INFO:tasks.workunit.client.1.smithi149.stdout:9/226: dwrite db/d19/f3c [0,4194304] 0 2024-03-22T07:08:49.818 INFO:tasks.workunit.client.1.smithi149.stdout:1/193: write d4/d7/fe [3450008,78291] 0 2024-03-22T07:08:49.818 INFO:tasks.workunit.client.1.smithi149.stdout:6/161: rmdir d9/df/d22 39 2024-03-22T07:08:49.818 INFO:tasks.workunit.client.1.smithi149.stdout:6/162: fdatasync f6 0 2024-03-22T07:08:49.818 INFO:tasks.workunit.client.1.smithi149.stdout:6/163: write d9/f19 [2331601,67814] 0 2024-03-22T07:08:49.818 INFO:tasks.workunit.client.1.smithi149.stdout:6/164: write d9/df/f15 [979654,104261] 0 2024-03-22T07:08:49.821 INFO:tasks.workunit.client.1.smithi149.stdout:9/227: mknod db/d21/d37/d3a/c40 0 2024-03-22T07:08:49.822 INFO:tasks.workunit.client.1.smithi149.stdout:9/228: mknod db/d21/d37/d3a/c41 0 2024-03-22T07:08:49.824 INFO:tasks.workunit.client.1.smithi149.stdout:9/229: mkdir db/d42 0 2024-03-22T07:08:49.824 INFO:tasks.workunit.client.1.smithi149.stdout:9/230: fdatasync db/d19/f3c 0 2024-03-22T07:08:49.825 INFO:tasks.workunit.client.1.smithi149.stdout:4/146: read d0/d8/f1f [281664,97070] 0 2024-03-22T07:08:49.836 INFO:tasks.workunit.client.1.smithi149.stdout:6/165: read d9/df/f1d [3471980,69171] 0 2024-03-22T07:08:49.836 INFO:tasks.workunit.client.1.smithi149.stdout:6/166: chown d9/df/l18 792270 1 2024-03-22T07:08:49.838 INFO:tasks.workunit.client.1.smithi149.stdout:6/167: symlink d9/d1f/l33 0 2024-03-22T07:08:49.838 INFO:tasks.workunit.client.1.smithi149.stdout:6/168: stat d9 0 2024-03-22T07:08:49.838 INFO:tasks.workunit.client.1.smithi149.stdout:6/169: fdatasync d9/df/d2e/f1b 0 2024-03-22T07:08:49.840 INFO:tasks.workunit.client.1.smithi149.stdout:6/170: mkdir d9/d1f/d24/d34 0 2024-03-22T07:08:49.842 INFO:tasks.workunit.client.1.smithi149.stdout:6/171: mknod d9/df/c35 0 2024-03-22T07:08:49.847 INFO:tasks.workunit.client.1.smithi149.stdout:9/231: dread f3 [0,4194304] 0 2024-03-22T07:08:49.847 INFO:tasks.workunit.client.1.smithi149.stdout:9/232: stat db/d19/d39 0 2024-03-22T07:08:49.849 INFO:tasks.workunit.client.1.smithi149.stdout:9/233: mknod db/d42/c43 0 2024-03-22T07:08:49.850 INFO:tasks.workunit.client.1.smithi149.stdout:9/234: unlink f9 0 2024-03-22T07:08:49.850 INFO:tasks.workunit.client.1.smithi149.stdout:9/235: readlink db/d12/l1b 0 2024-03-22T07:08:49.851 INFO:tasks.workunit.client.1.smithi149.stdout:1/194: getdents d4/d20/d2d 0 2024-03-22T07:08:49.851 INFO:tasks.workunit.client.1.smithi149.stdout:1/195: fsync d4/d10/f12 0 2024-03-22T07:08:49.853 INFO:tasks.workunit.client.1.smithi149.stdout:1/196: mknod d4/c43 0 2024-03-22T07:08:49.853 INFO:tasks.workunit.client.1.smithi149.stdout:1/197: write d4/f11 [779575,10656] 0 2024-03-22T07:08:49.853 INFO:tasks.workunit.client.1.smithi149.stdout:1/198: creat d4/d20/d23/f44 x:0 0 0 2024-03-22T07:08:49.853 INFO:tasks.workunit.client.1.smithi149.stdout:1/199: dread - d4/f42 zero size 2024-03-22T07:08:49.853 INFO:tasks.workunit.client.1.smithi149.stdout:1/200: chown d4/f15 49459 1 2024-03-22T07:08:49.856 INFO:tasks.workunit.client.1.smithi149.stdout:1/201: mknod d4/d10/d28/c45 0 2024-03-22T07:08:49.856 INFO:tasks.workunit.client.1.smithi149.stdout:1/202: chown d4/c16 1173 1 2024-03-22T07:08:49.857 INFO:tasks.workunit.client.1.smithi149.stdout:1/203: getdents d4/d20/d23/d2f 0 2024-03-22T07:08:49.858 INFO:tasks.workunit.client.1.smithi149.stdout:1/204: write d4/d10/d28/f2a [288044,105613] 0 2024-03-22T07:08:49.859 INFO:tasks.workunit.client.1.smithi149.stdout:1/205: link d4/lf d4/l46 0 2024-03-22T07:08:49.860 INFO:tasks.workunit.client.1.smithi149.stdout:1/206: stat d4/d7/c18 0 2024-03-22T07:08:49.860 INFO:tasks.workunit.client.1.smithi149.stdout:1/207: dread - d4/d10/d28/f2e zero size 2024-03-22T07:08:49.862 INFO:tasks.workunit.client.1.smithi149.stdout:1/208: link d4/f3e d4/d20/d23/d3c/f47 0 2024-03-22T07:08:49.862 INFO:tasks.workunit.client.1.smithi149.stdout:1/209: readlink d4/d20/d23/l29 0 2024-03-22T07:08:49.862 INFO:tasks.workunit.client.1.smithi149.stdout:1/210: truncate d4/f42 348358 0 2024-03-22T07:08:49.862 INFO:tasks.workunit.client.1.smithi149.stdout:1/211: chown d4/d10/d28/f34 1374 1 2024-03-22T07:08:49.862 INFO:tasks.workunit.client.1.smithi149.stdout:1/212: write d4/d20/d23/f44 [624896,21426] 0 2024-03-22T07:08:49.862 INFO:tasks.workunit.client.1.smithi149.stdout:9/236: dread db/d12/f14 [0,4194304] 0 2024-03-22T07:08:49.863 INFO:tasks.workunit.client.1.smithi149.stdout:1/213: mknod d4/d10/d28/c48 0 2024-03-22T07:08:49.865 INFO:tasks.workunit.client.1.smithi149.stdout:9/237: mknod db/d12/c44 0 2024-03-22T07:08:49.865 INFO:tasks.workunit.client.1.smithi149.stdout:9/238: fdatasync db/d33/f3d 0 2024-03-22T07:08:49.865 INFO:tasks.workunit.client.1.smithi149.stdout:9/239: write db/d22/f36 [870811,96915] 0 2024-03-22T07:08:49.865 INFO:tasks.workunit.client.1.smithi149.stdout:4/147: dwrite d0/d8/f1f [0,4194304] 0 2024-03-22T07:08:49.867 INFO:tasks.workunit.client.1.smithi149.stdout:4/148: creat d0/d29/f31 x:0 0 0 2024-03-22T07:08:49.868 INFO:tasks.workunit.client.1.smithi149.stdout:2/234: dwrite d1/d3/d6/f7 [0,4194304] 0 2024-03-22T07:08:49.868 INFO:tasks.workunit.client.1.smithi149.stdout:2/235: fsync d1/d3/d5/d1f/f1e 0 2024-03-22T07:08:49.868 INFO:tasks.workunit.client.1.smithi149.stdout:7/233: dwrite d1/d18/d1b/d1d/d2a/f3d [0,4194304] 0 2024-03-22T07:08:49.868 INFO:tasks.workunit.client.1.smithi149.stdout:7/234: truncate d1/d18/d1b/f42 893654 0 2024-03-22T07:08:49.868 INFO:tasks.workunit.client.1.smithi149.stdout:7/235: dread - d1/d18/d1b/d1d/d28/f50 zero size 2024-03-22T07:08:49.868 INFO:tasks.workunit.client.1.smithi149.stdout:7/236: dread - d1/d18/f34 zero size 2024-03-22T07:08:49.869 INFO:tasks.workunit.client.1.smithi149.stdout:8/234: dwrite d1/d2/d18/f3d [0,4194304] 0 2024-03-22T07:08:49.869 INFO:tasks.workunit.client.1.smithi149.stdout:6/172: dwrite d9/df/d2e/f1b [0,4194304] 0 2024-03-22T07:08:49.869 INFO:tasks.workunit.client.1.smithi149.stdout:0/128: dwrite d1/d9/fd [0,4194304] 0 2024-03-22T07:08:49.871 INFO:tasks.workunit.client.1.smithi149.stdout:2/236: symlink d1/d3/dd/l4f 0 2024-03-22T07:08:49.871 INFO:tasks.workunit.client.1.smithi149.stdout:2/237: dread - d1/d3/d5/f3a zero size 2024-03-22T07:08:49.874 INFO:tasks.workunit.client.1.smithi149.stdout:8/235: dread d1/d20/d41/f48 [0,4194304] 0 2024-03-22T07:08:49.878 INFO:tasks.workunit.client.1.smithi149.stdout:7/237: creat d1/d18/d1b/d51/f52 x:0 0 0 2024-03-22T07:08:49.878 INFO:tasks.workunit.client.1.smithi149.stdout:7/238: write d1/d18/d1b/d1d/f38 [4337937,110015] 0 2024-03-22T07:08:49.881 INFO:tasks.workunit.client.1.smithi149.stdout:9/240: dread db/d12/f35 [0,4194304] 0 2024-03-22T07:08:49.882 INFO:tasks.workunit.client.1.smithi149.stdout:9/241: creat db/d22/f45 x:0 0 0 2024-03-22T07:08:49.882 INFO:tasks.workunit.client.1.smithi149.stdout:9/242: dread - db/d21/f23 zero size 2024-03-22T07:08:49.883 INFO:tasks.workunit.client.1.smithi149.stdout:6/173: symlink d9/l36 0 2024-03-22T07:08:49.885 INFO:tasks.workunit.client.1.smithi149.stdout:0/129: mknod d1/d20/c21 0 2024-03-22T07:08:49.885 INFO:tasks.workunit.client.1.smithi149.stdout:0/130: stat d1/fb 0 2024-03-22T07:08:49.886 INFO:tasks.workunit.client.1.smithi149.stdout:4/149: getdents d0/d8 0 2024-03-22T07:08:49.886 INFO:tasks.workunit.client.1.smithi149.stdout:4/150: creat d0/f32 x:0 0 0 2024-03-22T07:08:49.886 INFO:tasks.workunit.client.1.smithi149.stdout:2/238: mkdir d1/d8/d34/d50 0 2024-03-22T07:08:49.894 INFO:tasks.workunit.client.1.smithi149.stdout:6/174: mkdir d9/d1f/d24/d34/d37 0 2024-03-22T07:08:49.894 INFO:tasks.workunit.client.1.smithi149.stdout:6/175: dread - d9/df/d22/f31 zero size 2024-03-22T07:08:49.894 INFO:tasks.workunit.client.1.smithi149.stdout:6/176: write d9/df/d2e/f20 [4589689,43414] 0 2024-03-22T07:08:49.897 INFO:tasks.workunit.client.1.smithi149.stdout:4/151: dread d0/d8/dc/f30 [0,4194304] 0 2024-03-22T07:08:49.897 INFO:tasks.workunit.client.1.smithi149.stdout:4/152: stat d0/d8/dc/d2e/f2f 0 2024-03-22T07:08:49.898 INFO:tasks.workunit.client.1.smithi149.stdout:4/153: truncate d0/d8/f1c 1002459 0 2024-03-22T07:08:49.903 INFO:tasks.workunit.client.1.smithi149.stdout:0/131: dread d1/d9/ff [0,4194304] 0 2024-03-22T07:08:49.903 INFO:tasks.workunit.client.1.smithi149.stdout:1/214: dwrite d4/f1b [0,4194304] 0 2024-03-22T07:08:49.908 INFO:tasks.workunit.client.1.smithi149.stdout:8/236: getdents d1 0 2024-03-22T07:08:49.908 INFO:tasks.workunit.client.1.smithi149.stdout:6/177: mkdir d9/d1f/d24/d34/d37/d38 0 2024-03-22T07:08:49.911 INFO:tasks.workunit.client.1.smithi149.stdout:8/237: mkdir d1/d20/d51 0 2024-03-22T07:08:49.912 INFO:tasks.workunit.client.1.smithi149.stdout:4/154: read d0/d8/dc/fe [2796210,123330] 0 2024-03-22T07:08:49.913 INFO:tasks.workunit.client.1.smithi149.stdout:0/132: rename d1/d9/ff to d1/f22 0 2024-03-22T07:08:49.913 INFO:tasks.workunit.client.1.smithi149.stdout:7/239: dwrite d1/d18/f1e [0,4194304] 0 2024-03-22T07:08:49.913 INFO:tasks.workunit.client.1.smithi149.stdout:7/240: fdatasync d1/d18/d1b/d1d/f8 0 2024-03-22T07:08:49.915 INFO:tasks.workunit.client.1.smithi149.stdout:8/238: creat d1/d20/d35/d22/d44/f52 x:0 0 0 2024-03-22T07:08:49.915 INFO:tasks.workunit.client.1.smithi149.stdout:8/239: chown d1/d20/d35/d22/f40 1 1 2024-03-22T07:08:49.916 INFO:tasks.workunit.client.1.smithi149.stdout:9/243: dwrite f3 [4194304,4194304] 0 2024-03-22T07:08:49.916 INFO:tasks.workunit.client.1.smithi149.stdout:2/239: rmdir d1 39 2024-03-22T07:08:49.918 INFO:tasks.workunit.client.1.smithi149.stdout:4/155: rename d0/d8/c11 to d0/d8/dc/c33 0 2024-03-22T07:08:49.920 INFO:tasks.workunit.client.1.smithi149.stdout:7/241: truncate d1/f19 2254132 0 2024-03-22T07:08:49.922 INFO:tasks.workunit.client.1.smithi149.stdout:9/244: symlink db/d21/d37/l46 0 2024-03-22T07:08:49.922 INFO:tasks.workunit.client.1.smithi149.stdout:9/245: stat db/d22/c25 0 2024-03-22T07:08:49.922 INFO:tasks.workunit.client.1.smithi149.stdout:7/242: dread d1/d18/d1b/d1d/d2a/f3b [0,4194304] 0 2024-03-22T07:08:49.923 INFO:tasks.workunit.client.1.smithi149.stdout:2/240: link d1/d3/dd/f45 d1/f51 0 2024-03-22T07:08:49.924 INFO:tasks.workunit.client.1.smithi149.stdout:1/215: dwrite d4/d20/d23/f44 [0,4194304] 0 2024-03-22T07:08:49.925 INFO:tasks.workunit.client.1.smithi149.stdout:1/216: chown d4/d20/d2d 4896592 1 2024-03-22T07:08:49.925 INFO:tasks.workunit.client.1.smithi149.stdout:0/133: mkdir d1/d23 0 2024-03-22T07:08:49.926 INFO:tasks.workunit.client.1.smithi149.stdout:8/240: mknod d1/d2/d9/c53 0 2024-03-22T07:08:49.927 INFO:tasks.workunit.client.1.smithi149.stdout:7/243: mkdir d1/d18/d1b/d1d/d28/d53 0 2024-03-22T07:08:49.929 INFO:tasks.workunit.client.1.smithi149.stdout:2/241: link d1/d3/d6/l38 d1/d3/d6/l52 0 2024-03-22T07:08:49.930 INFO:tasks.workunit.client.1.smithi149.stdout:4/156: getdents d0 0 2024-03-22T07:08:49.930 INFO:tasks.workunit.client.1.smithi149.stdout:4/157: dread - d0/d8/dc/d14/f2b zero size 2024-03-22T07:08:49.930 INFO:tasks.workunit.client.1.smithi149.stdout:4/158: chown d0/f2d 483392 1 2024-03-22T07:08:49.933 INFO:tasks.workunit.client.1.smithi149.stdout:1/217: mkdir d4/d10/d28/d49 0 2024-03-22T07:08:49.934 INFO:tasks.workunit.client.1.smithi149.stdout:1/218: write d4/f11 [2113049,128531] 0 2024-03-22T07:08:49.934 INFO:tasks.workunit.client.1.smithi149.stdout:0/134: unlink d1/d9/f15 0 2024-03-22T07:08:49.936 INFO:tasks.workunit.client.1.smithi149.stdout:8/241: rmdir d1/d20/d35 39 2024-03-22T07:08:49.936 INFO:tasks.workunit.client.1.smithi149.stdout:8/242: dread - d1/d2/f23 zero size 2024-03-22T07:08:49.936 INFO:tasks.workunit.client.1.smithi149.stdout:8/243: chown d1/d20/d35/d22/d44 448033 1 2024-03-22T07:08:49.936 INFO:tasks.workunit.client.1.smithi149.stdout:8/244: creat d1/d20/f54 x:0 0 0 2024-03-22T07:08:49.936 INFO:tasks.workunit.client.1.smithi149.stdout:7/244: rmdir d1/d1c/d4d 39 2024-03-22T07:08:49.937 INFO:tasks.workunit.client.1.smithi149.stdout:9/246: dwrite db/f2f [0,4194304] 0 2024-03-22T07:08:49.938 INFO:tasks.workunit.client.1.smithi149.stdout:4/159: creat d0/d8/f34 x:0 0 0 2024-03-22T07:08:49.939 INFO:tasks.workunit.client.1.smithi149.stdout:1/219: rename d4/d20/c3d to d4/d10/c4a 0 2024-03-22T07:08:49.941 INFO:tasks.workunit.client.1.smithi149.stdout:0/135: creat d1/d9/d14/d1f/f24 x:0 0 0 2024-03-22T07:08:49.943 INFO:tasks.workunit.client.1.smithi149.stdout:7/245: symlink d1/d18/d30/l54 0 2024-03-22T07:08:49.944 INFO:tasks.workunit.client.1.smithi149.stdout:7/246: write d1/d1c/d2f/f43 [776684,76516] 0 2024-03-22T07:08:49.944 INFO:tasks.workunit.client.1.smithi149.stdout:7/247: readlink d1/d1c/d2f/l44 0 2024-03-22T07:08:49.944 INFO:tasks.workunit.client.1.smithi149.stdout:2/242: symlink d1/d8/l53 0 2024-03-22T07:08:49.948 INFO:tasks.workunit.client.1.smithi149.stdout:4/160: unlink d0/d8/dc/d14/c19 0 2024-03-22T07:08:49.948 INFO:tasks.workunit.client.1.smithi149.stdout:7/248: write d1/f19 [2583100,68888] 0 2024-03-22T07:08:49.951 INFO:tasks.workunit.client.1.smithi149.stdout:1/220: mknod d4/c4b 0 2024-03-22T07:08:49.952 INFO:tasks.workunit.client.1.smithi149.stdout:0/136: rmdir d1/d23 0 2024-03-22T07:08:49.953 INFO:tasks.workunit.client.1.smithi149.stdout:2/243: creat d1/d3/f54 x:0 0 0 2024-03-22T07:08:49.958 INFO:tasks.workunit.client.1.smithi149.stdout:1/221: creat d4/d10/f4c x:0 0 0 2024-03-22T07:08:49.958 INFO:tasks.workunit.client.1.smithi149.stdout:1/222: truncate d4/d20/d2d/f3a 216956 0 2024-03-22T07:08:49.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/137: creat d1/d1a/f25 x:0 0 0 2024-03-22T07:08:49.960 INFO:tasks.workunit.client.1.smithi149.stdout:8/245: dwrite d1/d2/d9/f29 [0,4194304] 0 2024-03-22T07:08:49.967 INFO:tasks.workunit.client.1.smithi149.stdout:7/249: getdents d1/d18/d30 0 2024-03-22T07:08:49.969 INFO:tasks.workunit.client.1.smithi149.stdout:7/250: rename d1/d18/d1b/d1d/d2a/c2e to d1/c55 0 2024-03-22T07:08:49.973 INFO:tasks.workunit.client.1.smithi149.stdout:7/251: dread d1/f10 [0,4194304] 0 2024-03-22T07:08:49.974 INFO:tasks.workunit.client.1.smithi149.stdout:7/252: mkdir d1/d1c/d4d/d56 0 2024-03-22T07:08:49.975 INFO:tasks.workunit.client.1.smithi149.stdout:7/253: mknod d1/d1c/c57 0 2024-03-22T07:08:49.976 INFO:tasks.workunit.client.1.smithi149.stdout:4/161: dwrite d0/f3 [0,4194304] 0 2024-03-22T07:08:49.976 INFO:tasks.workunit.client.1.smithi149.stdout:4/162: write d0/d8/dc/d14/f23 [512982,25488] 0 2024-03-22T07:08:49.976 INFO:tasks.workunit.client.1.smithi149.stdout:4/163: creat d0/d29/f35 x:0 0 0 2024-03-22T07:08:49.976 INFO:tasks.workunit.client.1.smithi149.stdout:4/164: write d0/d8/dc/f18 [4545294,21106] 0 2024-03-22T07:08:49.977 INFO:tasks.workunit.client.1.smithi149.stdout:7/254: rename d1/f3a to d1/d1c/d4d/f58 0 2024-03-22T07:08:49.978 INFO:tasks.workunit.client.1.smithi149.stdout:8/246: dread d1/d2/d9/f29 [0,4194304] 0 2024-03-22T07:08:49.979 INFO:tasks.workunit.client.1.smithi149.stdout:4/165: mknod d0/c36 0 2024-03-22T07:08:49.980 INFO:tasks.workunit.client.1.smithi149.stdout:4/166: write d0/d8/dc/fe [1879609,72724] 0 2024-03-22T07:08:49.980 INFO:tasks.workunit.client.1.smithi149.stdout:1/223: dwrite d4/d7/fe [0,4194304] 0 2024-03-22T07:08:49.980 INFO:tasks.workunit.client.1.smithi149.stdout:1/224: read - d4/d10/d28/f37 zero size 2024-03-22T07:08:49.981 INFO:tasks.workunit.client.1.smithi149.stdout:4/167: mkdir d0/d8/dc/d37 0 2024-03-22T07:08:49.982 INFO:tasks.workunit.client.1.smithi149.stdout:8/247: creat d1/d2/d9/d11/f55 x:0 0 0 2024-03-22T07:08:49.983 INFO:tasks.workunit.client.1.smithi149.stdout:4/168: mknod d0/d8/dc/d14/c38 0 2024-03-22T07:08:49.983 INFO:tasks.workunit.client.1.smithi149.stdout:4/169: dread - d0/d8/dc/d14/f2b zero size 2024-03-22T07:08:49.984 INFO:tasks.workunit.client.1.smithi149.stdout:8/248: creat d1/d20/d35/d22/d44/d4f/f56 x:0 0 0 2024-03-22T07:08:49.985 INFO:tasks.workunit.client.1.smithi149.stdout:8/249: mknod d1/d20/d35/d22/c57 0 2024-03-22T07:08:49.985 INFO:tasks.workunit.client.1.smithi149.stdout:8/250: creat d1/d2/d9/d11/f58 x:0 0 0 2024-03-22T07:08:49.985 INFO:tasks.workunit.client.1.smithi149.stdout:8/251: write d1/d2/d3/f3a [94441,74787] 0 2024-03-22T07:08:49.985 INFO:tasks.workunit.client.1.smithi149.stdout:8/252: dread - d1/d20/d35/d22/d44/f52 zero size 2024-03-22T07:08:49.996 INFO:tasks.workunit.client.1.smithi149.stdout:7/255: dread d1/f33 [0,4194304] 0 2024-03-22T07:08:49.999 INFO:tasks.workunit.client.1.smithi149.stdout:1/225: dwrite d4/d20/d23/d3c/f41 [0,4194304] 0 2024-03-22T07:08:49.999 INFO:tasks.workunit.client.1.smithi149.stdout:1/226: creat d4/d10/f4d x:0 0 0 2024-03-22T07:08:50.001 INFO:tasks.workunit.client.1.smithi149.stdout:7/256: dread d1/d18/d1b/d1d/f13 [0,4194304] 0 2024-03-22T07:08:50.002 INFO:tasks.workunit.client.1.smithi149.stdout:7/257: rmdir d1/d1c/d4d 39 2024-03-22T07:08:50.003 INFO:tasks.workunit.client.1.smithi149.stdout:7/258: readlink d1/l12 0 2024-03-22T07:08:50.006 INFO:tasks.workunit.client.1.smithi149.stdout:4/170: dwrite d0/d8/dc/d14/f2b [0,4194304] 0 2024-03-22T07:08:50.006 INFO:tasks.workunit.client.1.smithi149.stdout:4/171: chown d0/c1 454 1 2024-03-22T07:08:50.007 INFO:tasks.workunit.client.1.smithi149.stdout:4/172: chown d0/d8/dc/d14/c38 21 1 2024-03-22T07:08:50.009 INFO:tasks.workunit.client.1.smithi149.stdout:4/173: unlink d0/d8/fd 0 2024-03-22T07:08:50.011 INFO:tasks.workunit.client.1.smithi149.stdout:4/174: mknod d0/d8/d1d/c39 0 2024-03-22T07:08:50.011 INFO:tasks.workunit.client.1.smithi149.stdout:4/175: creat d0/d8/d1d/f3a x:0 0 0 2024-03-22T07:08:50.012 INFO:tasks.workunit.client.1.smithi149.stdout:8/253: dwrite d1/d2/d9/df/f15 [0,4194304] 0 2024-03-22T07:08:50.013 INFO:tasks.workunit.client.1.smithi149.stdout:4/176: rename d0/c9 to d0/d8/dc/d37/c3b 0 2024-03-22T07:08:50.017 INFO:tasks.workunit.client.1.smithi149.stdout:4/177: link d0/d29/f31 d0/d29/f3c 0 2024-03-22T07:08:50.017 INFO:tasks.workunit.client.1.smithi149.stdout:4/178: stat d0/d8/d1d/f28 0 2024-03-22T07:08:50.018 INFO:tasks.workunit.client.1.smithi149.stdout:4/179: creat d0/d8/dc/d37/f3d x:0 0 0 2024-03-22T07:08:50.024 INFO:tasks.workunit.client.1.smithi149.stdout:1/227: dwrite d4/d10/d28/f37 [0,4194304] 0 2024-03-22T07:08:50.025 INFO:tasks.workunit.client.1.smithi149.stdout:7/259: dwrite d1/d18/d1b/d1d/d28/f50 [0,4194304] 0 2024-03-22T07:08:50.025 INFO:tasks.workunit.client.1.smithi149.stdout:1/228: symlink d4/d10/d28/l4e 0 2024-03-22T07:08:50.026 INFO:tasks.workunit.client.1.smithi149.stdout:1/229: truncate d4/d20/d23/d3c/f47 875952 0 2024-03-22T07:08:50.034 INFO:tasks.workunit.client.1.smithi149.stdout:7/260: mknod d1/d1c/d2f/c59 0 2024-03-22T07:08:50.034 INFO:tasks.workunit.client.1.smithi149.stdout:7/261: dread - d1/d18/f34 zero size 2024-03-22T07:08:50.035 INFO:tasks.workunit.client.1.smithi149.stdout:1/230: unlink d4/d10/c1a 0 2024-03-22T07:08:50.035 INFO:tasks.workunit.client.1.smithi149.stdout:1/231: chown d4/d10 0 1 2024-03-22T07:08:50.035 INFO:tasks.workunit.client.1.smithi149.stdout:8/254: dwrite d1/d2/d9/df/f2d [0,4194304] 0 2024-03-22T07:08:50.037 INFO:tasks.workunit.client.1.smithi149.stdout:7/262: mkdir d1/d18/d1b/d51/d5a 0 2024-03-22T07:08:50.040 INFO:tasks.workunit.client.1.smithi149.stdout:7/263: mkdir d1/d18/d1b/d1d/d2a/d5b 0 2024-03-22T07:08:50.042 INFO:tasks.workunit.client.1.smithi149.stdout:7/264: mkdir d1/d18/d1b/d1d/d28/d53/d5c 0 2024-03-22T07:08:50.042 INFO:tasks.workunit.client.1.smithi149.stdout:5/210: sync 2024-03-22T07:08:50.044 INFO:tasks.workunit.client.1.smithi149.stdout:5/211: unlink d7/c15 0 2024-03-22T07:08:50.046 INFO:tasks.workunit.client.1.smithi149.stdout:5/212: creat d7/da/d20/d36/f41 x:0 0 0 2024-03-22T07:08:50.051 INFO:tasks.workunit.client.1.smithi149.stdout:7/265: write d1/d18/d1b/d1d/f16 [979909,567] 0 2024-03-22T07:08:50.053 INFO:tasks.workunit.client.1.smithi149.stdout:7/266: truncate d1/d18/d1b/d1d/d2a/f3d 3601534 0 2024-03-22T07:08:50.054 INFO:tasks.workunit.client.1.smithi149.stdout:7/267: rmdir d1/d1c/d2f 39 2024-03-22T07:08:50.055 INFO:tasks.workunit.client.1.smithi149.stdout:5/213: write d7/d35/f40 [3924353,103434] 0 2024-03-22T07:08:50.055 INFO:tasks.workunit.client.1.smithi149.stdout:7/268: mkdir d1/d18/d1b/d1d/d2a/d5d 0 2024-03-22T07:08:50.057 INFO:tasks.workunit.client.1.smithi149.stdout:7/269: rename d1/f19 to d1/d1c/d49/f5e 0 2024-03-22T07:08:50.057 INFO:tasks.workunit.client.1.smithi149.stdout:7/270: chown d1/d1c/d4d/f3e 0 1 2024-03-22T07:08:50.060 INFO:tasks.workunit.client.1.smithi149.stdout:1/232: dwrite d4/d20/d23/d3c/f47 [0,4194304] 0 2024-03-22T07:08:50.060 INFO:tasks.workunit.client.1.smithi149.stdout:8/255: dwrite d1/f16 [0,4194304] 0 2024-03-22T07:08:50.065 INFO:tasks.workunit.client.1.smithi149.stdout:1/233: truncate d4/f1b 621893 0 2024-03-22T07:08:50.065 INFO:tasks.workunit.client.1.smithi149.stdout:1/234: readlink d4/lf 0 2024-03-22T07:08:50.065 INFO:tasks.workunit.client.1.smithi149.stdout:1/235: creat d4/d1f/f4f x:0 0 0 2024-03-22T07:08:50.065 INFO:tasks.workunit.client.1.smithi149.stdout:1/236: read d4/f6 [1531243,61718] 0 2024-03-22T07:08:50.065 INFO:tasks.workunit.client.1.smithi149.stdout:1/237: fdatasync d4/d20/f22 0 2024-03-22T07:08:50.065 INFO:tasks.workunit.client.1.smithi149.stdout:1/238: write d4/f6 [2339618,103541] 0 2024-03-22T07:08:50.065 INFO:tasks.workunit.client.1.smithi149.stdout:1/239: write d4/d20/d23/d3c/f41 [5188292,110409] 0 2024-03-22T07:08:50.065 INFO:tasks.workunit.client.1.smithi149.stdout:1/240: creat d4/d7/f50 x:0 0 0 2024-03-22T07:08:50.065 INFO:tasks.workunit.client.1.smithi149.stdout:1/241: write d4/f15 [977118,17252] 0 2024-03-22T07:08:50.072 INFO:tasks.workunit.client.1.smithi149.stdout:1/242: link d4/d10/l1e d4/d1f/l51 0 2024-03-22T07:08:50.072 INFO:tasks.workunit.client.1.smithi149.stdout:1/243: creat d4/d20/f52 x:0 0 0 2024-03-22T07:08:50.072 INFO:tasks.workunit.client.1.smithi149.stdout:1/244: chown d4/f42 237395456 1 2024-03-22T07:08:50.072 INFO:tasks.workunit.client.1.smithi149.stdout:1/245: creat d4/d10/d28/f53 x:0 0 0 2024-03-22T07:08:50.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/256: getdents d1/d20 0 2024-03-22T07:08:50.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/257: chown d1/d2/d9/d11/d4d 0 1 2024-03-22T07:08:50.076 INFO:tasks.workunit.client.1.smithi149.stdout:1/246: getdents d4/d10 0 2024-03-22T07:08:50.077 INFO:tasks.workunit.client.1.smithi149.stdout:5/214: dwrite d7/db/f22 [0,4194304] 0 2024-03-22T07:08:50.077 INFO:tasks.workunit.client.1.smithi149.stdout:5/215: truncate d7/da/f37 82311 0 2024-03-22T07:08:50.078 INFO:tasks.workunit.client.1.smithi149.stdout:8/258: getdents d1/d2/d9/d11 0 2024-03-22T07:08:50.078 INFO:tasks.workunit.client.1.smithi149.stdout:8/259: chown d1/d20/d35/fb 1 1 2024-03-22T07:08:50.079 INFO:tasks.workunit.client.1.smithi149.stdout:1/247: unlink d4/d20/d23/f3f 0 2024-03-22T07:08:50.083 INFO:tasks.workunit.client.1.smithi149.stdout:7/271: dwrite d1/d18/d1b/f42 [0,4194304] 0 2024-03-22T07:08:50.084 INFO:tasks.workunit.client.1.smithi149.stdout:8/260: rmdir d1/d2/d3 39 2024-03-22T07:08:50.084 INFO:tasks.workunit.client.1.smithi149.stdout:8/261: read d1/d2/d3/f3a [142433,115335] 0 2024-03-22T07:08:50.084 INFO:tasks.workunit.client.1.smithi149.stdout:8/262: write d1/d20/f54 [1036533,77797] 0 2024-03-22T07:08:50.088 INFO:tasks.workunit.client.1.smithi149.stdout:7/272: symlink d1/d18/d1b/d1d/d2a/d5d/l5f 0 2024-03-22T07:08:50.089 INFO:tasks.workunit.client.1.smithi149.stdout:8/263: symlink d1/d20/d51/l59 0 2024-03-22T07:08:50.089 INFO:tasks.workunit.client.1.smithi149.stdout:1/248: getdents d4/d20/d23 0 2024-03-22T07:08:50.090 INFO:tasks.workunit.client.1.smithi149.stdout:7/273: mknod d1/d18/d1b/d1d/d28/d53/c60 0 2024-03-22T07:08:50.091 INFO:tasks.workunit.client.1.smithi149.stdout:8/264: symlink d1/d20/d35/d22/l5a 0 2024-03-22T07:08:50.093 INFO:tasks.workunit.client.1.smithi149.stdout:3/148: sync 2024-03-22T07:08:50.094 INFO:tasks.workunit.client.1.smithi149.stdout:3/149: mkdir d0/d4/d8/d24/d39 0 2024-03-22T07:08:50.095 INFO:tasks.workunit.client.1.smithi149.stdout:3/150: truncate d0/d4/f1e 56969 0 2024-03-22T07:08:50.095 INFO:tasks.workunit.client.1.smithi149.stdout:3/151: read - d0/d2/d36/f37 zero size 2024-03-22T07:08:50.095 INFO:tasks.workunit.client.1.smithi149.stdout:3/152: write d0/d4/f32 [360177,5672] 0 2024-03-22T07:08:50.097 INFO:tasks.workunit.client.1.smithi149.stdout:3/153: rmdir d0/d2/d30 39 2024-03-22T07:08:50.098 INFO:tasks.workunit.client.1.smithi149.stdout:1/249: dread d4/f3e [0,4194304] 0 2024-03-22T07:08:50.098 INFO:tasks.workunit.client.1.smithi149.stdout:1/250: write d4/f15 [1630300,84598] 0 2024-03-22T07:08:50.099 INFO:tasks.workunit.client.1.smithi149.stdout:8/265: getdents d1/d20/d35/d22 0 2024-03-22T07:08:50.099 INFO:tasks.workunit.client.1.smithi149.stdout:8/266: rename d1 to d1/d2/d9/d11/d45/d5b 22 2024-03-22T07:08:50.099 INFO:tasks.workunit.client.1.smithi149.stdout:8/267: truncate d1/d2/d9/f4a 785612 0 2024-03-22T07:08:50.100 INFO:tasks.workunit.client.1.smithi149.stdout:8/268: mknod d1/d20/d35/d22/d44/c5c 0 2024-03-22T07:08:50.101 INFO:tasks.workunit.client.1.smithi149.stdout:3/154: dread d0/d4/d8/f25 [0,4194304] 0 2024-03-22T07:08:50.101 INFO:tasks.workunit.client.1.smithi149.stdout:8/269: truncate d1/d2/d3/f3a 160252 0 2024-03-22T07:08:50.102 INFO:tasks.workunit.client.1.smithi149.stdout:3/155: read d0/d4/f32 [107273,118485] 0 2024-03-22T07:08:50.103 INFO:tasks.workunit.client.1.smithi149.stdout:8/270: mknod d1/d2/d18/c5d 0 2024-03-22T07:08:50.105 INFO:tasks.workunit.client.1.smithi149.stdout:3/156: symlink d0/d2/d30/l3a 0 2024-03-22T07:08:50.105 INFO:tasks.workunit.client.1.smithi149.stdout:3/157: write d0/d4/fe [1045233,123986] 0 2024-03-22T07:08:50.111 INFO:tasks.workunit.client.1.smithi149.stdout:3/158: write d0/d4/d8/f19 [4068875,58796] 0 2024-03-22T07:08:50.112 INFO:tasks.workunit.client.1.smithi149.stdout:3/159: symlink d0/d2/d30/d31/l3b 0 2024-03-22T07:08:50.113 INFO:tasks.workunit.client.1.smithi149.stdout:3/160: rename d0/d4/f1f to d0/d2/d30/f3c 0 2024-03-22T07:08:50.114 INFO:tasks.workunit.client.1.smithi149.stdout:8/271: write d1/d2/d3/f3a [374701,55384] 0 2024-03-22T07:08:50.116 INFO:tasks.workunit.client.1.smithi149.stdout:8/272: dread d1/d20/d41/f48 [0,4194304] 0 2024-03-22T07:08:50.118 INFO:tasks.workunit.client.1.smithi149.stdout:8/273: link d1/d2/d9/f1b d1/d2/d9/d11/d4d/f5e 0 2024-03-22T07:08:50.119 INFO:tasks.workunit.client.1.smithi149.stdout:8/274: rename d1/d2/l12 to d1/d2/l5f 0 2024-03-22T07:08:50.119 INFO:tasks.workunit.client.1.smithi149.stdout:8/275: chown d1/d2/d9/f1b 34190 1 2024-03-22T07:08:50.120 INFO:tasks.workunit.client.1.smithi149.stdout:8/276: readlink d1/d20/d35/d22/l5a 0 2024-03-22T07:08:50.120 INFO:tasks.workunit.client.1.smithi149.stdout:8/277: stat d1/d2/d3/f3a 0 2024-03-22T07:08:50.139 INFO:tasks.workunit.client.1.smithi149.stdout:1/251: dwrite d4/d20/d23/f44 [0,4194304] 0 2024-03-22T07:08:50.139 INFO:tasks.workunit.client.1.smithi149.stdout:7/274: dwrite d1/d1c/d4d/f3e [0,4194304] 0 2024-03-22T07:08:50.139 INFO:tasks.workunit.client.1.smithi149.stdout:7/275: write d1/f5 [2230689,15064] 0 2024-03-22T07:08:50.139 INFO:tasks.workunit.client.1.smithi149.stdout:3/161: dwrite d0/d2/d30/d31/f33 [0,4194304] 0 2024-03-22T07:08:50.140 INFO:tasks.workunit.client.1.smithi149.stdout:8/278: dwrite d1/f24 [0,4194304] 0 2024-03-22T07:08:50.146 INFO:tasks.workunit.client.1.smithi149.stdout:3/162: rename d0/d4/f32 to d0/d2/d30/d31/f3d 0 2024-03-22T07:08:50.150 INFO:tasks.workunit.client.1.smithi149.stdout:8/279: link d1/d2/d18/c5d d1/d2/d3/c60 0 2024-03-22T07:08:50.150 INFO:tasks.workunit.client.1.smithi149.stdout:8/280: chown d1/d20/d35 1 1 2024-03-22T07:08:50.151 INFO:tasks.workunit.client.1.smithi149.stdout:3/163: mknod d0/d4/d8/d24/c3e 0 2024-03-22T07:08:50.152 INFO:tasks.workunit.client.1.smithi149.stdout:1/252: write d4/d20/d23/d2f/f35 [1552965,92123] 0 2024-03-22T07:08:50.153 INFO:tasks.workunit.client.1.smithi149.stdout:3/164: creat d0/d2/d30/f3f x:0 0 0 2024-03-22T07:08:50.153 INFO:tasks.workunit.client.1.smithi149.stdout:3/165: dread - d0/d2/d30/f3f zero size 2024-03-22T07:08:50.155 INFO:tasks.workunit.client.1.smithi149.stdout:3/166: mkdir d0/d4/d8/d24/d40 0 2024-03-22T07:08:50.155 INFO:tasks.workunit.client.1.smithi149.stdout:3/167: write d0/d2/d30/f3f [753371,73734] 0 2024-03-22T07:08:50.155 INFO:tasks.workunit.client.1.smithi149.stdout:3/168: chown d0/d2/l11 5453329 1 2024-03-22T07:08:50.163 INFO:tasks.workunit.client.1.smithi149.stdout:7/276: dwrite d1/f10 [0,4194304] 0 2024-03-22T07:08:50.174 INFO:tasks.workunit.client.1.smithi149.stdout:1/253: dread d4/f15 [0,4194304] 0 2024-03-22T07:08:50.176 INFO:tasks.workunit.client.1.smithi149.stdout:1/254: mknod d4/c54 0 2024-03-22T07:08:50.176 INFO:tasks.workunit.client.1.smithi149.stdout:1/255: getdents d4/d20/d23/d2f 0 2024-03-22T07:08:50.176 INFO:tasks.workunit.client.1.smithi149.stdout:1/256: chown d4/d7/f33 262 1 2024-03-22T07:08:50.176 INFO:tasks.workunit.client.1.smithi149.stdout:1/257: creat d4/d10/f55 x:0 0 0 2024-03-22T07:08:50.188 INFO:tasks.workunit.client.1.smithi149.stdout:1/258: write d4/d10/d28/f37 [2117795,89918] 0 2024-03-22T07:08:50.188 INFO:tasks.workunit.client.1.smithi149.stdout:3/169: dwrite d0/d4/d8/f21 [0,4194304] 0 2024-03-22T07:08:50.189 INFO:tasks.workunit.client.1.smithi149.stdout:7/277: dwrite d1/d18/f37 [0,4194304] 0 2024-03-22T07:08:50.189 INFO:tasks.workunit.client.1.smithi149.stdout:7/278: write d1/f21 [207637,109876] 0 2024-03-22T07:08:50.189 INFO:tasks.workunit.client.1.smithi149.stdout:7/279: chown d1/d1c/d4d/f58 11654 1 2024-03-22T07:08:50.189 INFO:tasks.workunit.client.1.smithi149.stdout:1/259: mknod d4/d20/d23/c56 0 2024-03-22T07:08:50.197 INFO:tasks.workunit.client.1.smithi149.stdout:7/280: dread d1/f33 [0,4194304] 0 2024-03-22T07:08:50.198 INFO:tasks.workunit.client.1.smithi149.stdout:1/260: dread d4/d20/d23/d3c/f41 [4194304,4194304] 0 2024-03-22T07:08:50.198 INFO:tasks.workunit.client.1.smithi149.stdout:1/261: creat d4/d7/f57 x:0 0 0 2024-03-22T07:08:50.198 INFO:tasks.workunit.client.1.smithi149.stdout:1/262: fdatasync d4/d20/d23/d3c/f41 0 2024-03-22T07:08:50.199 INFO:tasks.workunit.client.1.smithi149.stdout:7/281: dread d1/d18/d1b/d1d/d2a/f3b [0,4194304] 0 2024-03-22T07:08:50.201 INFO:tasks.workunit.client.1.smithi149.stdout:9/247: sync 2024-03-22T07:08:50.201 INFO:tasks.workunit.client.1.smithi149.stdout:6/178: sync 2024-03-22T07:08:50.201 INFO:tasks.workunit.client.1.smithi149.stdout:6/179: chown d9/d1f/f25 1344474 1 2024-03-22T07:08:50.202 INFO:tasks.workunit.client.1.smithi149.stdout:1/263: mkdir d4/d10/d58 0 2024-03-22T07:08:50.204 INFO:tasks.workunit.client.1.smithi149.stdout:9/248: rename l1 to db/d19/l47 0 2024-03-22T07:08:50.204 INFO:tasks.workunit.client.1.smithi149.stdout:9/249: truncate db/d19/d1c/f20 1762952 0 2024-03-22T07:08:50.205 INFO:tasks.workunit.client.1.smithi149.stdout:6/180: symlink d9/d1f/d24/d34/d37/d38/l39 0 2024-03-22T07:08:50.205 INFO:tasks.workunit.client.1.smithi149.stdout:6/181: read d9/f19 [993366,78471] 0 2024-03-22T07:08:50.206 INFO:tasks.workunit.client.1.smithi149.stdout:9/250: creat db/d19/d39/d3b/f48 x:0 0 0 2024-03-22T07:08:50.208 INFO:tasks.workunit.client.1.smithi149.stdout:9/251: dread db/fc [0,4194304] 0 2024-03-22T07:08:50.214 INFO:tasks.workunit.client.1.smithi149.stdout:9/252: write db/d19/f1f [39122,688] 0 2024-03-22T07:08:50.214 INFO:tasks.workunit.client.1.smithi149.stdout:1/264: write d4/d20/d23/f44 [2079079,14781] 0 2024-03-22T07:08:50.214 INFO:tasks.workunit.client.1.smithi149.stdout:3/170: dwrite d0/d2/d36/f20 [0,4194304] 0 2024-03-22T07:08:50.215 INFO:tasks.workunit.client.1.smithi149.stdout:3/171: write d0/d4/d8/f21 [4715986,43717] 0 2024-03-22T07:08:50.215 INFO:tasks.workunit.client.1.smithi149.stdout:3/172: write d0/d2/d36/f2e [4392,92036] 0 2024-03-22T07:08:50.215 INFO:tasks.workunit.client.1.smithi149.stdout:9/253: mkdir db/d42/d49 0 2024-03-22T07:08:50.217 INFO:tasks.workunit.client.1.smithi149.stdout:1/265: mkdir d4/d20/d23/d59 0 2024-03-22T07:08:50.220 INFO:tasks.workunit.client.1.smithi149.stdout:9/254: truncate db/f2f 3130849 0 2024-03-22T07:08:50.221 INFO:tasks.workunit.client.1.smithi149.stdout:3/173: getdents d0/d4/d8 0 2024-03-22T07:08:50.223 INFO:tasks.workunit.client.1.smithi149.stdout:1/266: write d4/f1b [431584,91992] 0 2024-03-22T07:08:50.223 INFO:tasks.workunit.client.1.smithi149.stdout:1/267: fsync d4/d10/d28/f37 0 2024-03-22T07:08:50.224 INFO:tasks.workunit.client.1.smithi149.stdout:0/138: sync 2024-03-22T07:08:50.224 INFO:tasks.workunit.client.1.smithi149.stdout:2/244: sync 2024-03-22T07:08:50.224 INFO:tasks.workunit.client.1.smithi149.stdout:4/180: sync 2024-03-22T07:08:50.224 INFO:tasks.workunit.client.1.smithi149.stdout:4/181: fsync d0/d8/dc/d14/f16 0 2024-03-22T07:08:50.225 INFO:tasks.workunit.client.1.smithi149.stdout:1/268: mkdir d4/d7/d5a 0 2024-03-22T07:08:50.226 INFO:tasks.workunit.client.1.smithi149.stdout:0/139: truncate d1/fb 3429268 0 2024-03-22T07:08:50.226 INFO:tasks.workunit.client.1.smithi149.stdout:0/140: chown d1/d9/fd 1 1 2024-03-22T07:08:50.226 INFO:tasks.workunit.client.1.smithi149.stdout:0/141: write d1/d9/d14/d1f/f24 [118382,93340] 0 2024-03-22T07:08:50.227 INFO:tasks.workunit.client.1.smithi149.stdout:6/182: dwrite d9/df/f15 [0,4194304] 0 2024-03-22T07:08:50.228 INFO:tasks.workunit.client.1.smithi149.stdout:2/245: creat d1/d3/d5/d1f/f55 x:0 0 0 2024-03-22T07:08:50.228 INFO:tasks.workunit.client.1.smithi149.stdout:2/246: truncate d1/d3/d26/d2f/f33 19445 0 2024-03-22T07:08:50.228 INFO:tasks.workunit.client.1.smithi149.stdout:5/216: sync 2024-03-22T07:08:50.232 INFO:tasks.workunit.client.1.smithi149.stdout:1/269: link d4/d20/d23/d3c/f41 d4/d10/d58/f5b 0 2024-03-22T07:08:50.232 INFO:tasks.workunit.client.1.smithi149.stdout:1/270: chown d4/d10/d28/l4e 77421 1 2024-03-22T07:08:50.234 INFO:tasks.workunit.client.1.smithi149.stdout:0/142: creat d1/d9/d14/f26 x:0 0 0 2024-03-22T07:08:50.239 INFO:tasks.workunit.client.1.smithi149.stdout:2/247: creat d1/d8/f56 x:0 0 0 2024-03-22T07:08:50.241 INFO:tasks.workunit.client.1.smithi149.stdout:5/217: symlink d7/db/l42 0 2024-03-22T07:08:50.244 INFO:tasks.workunit.client.1.smithi149.stdout:1/271: creat d4/d20/d23/d59/f5c x:0 0 0 2024-03-22T07:08:50.246 INFO:tasks.workunit.client.1.smithi149.stdout:2/248: rmdir d1/d4d 39 2024-03-22T07:08:50.248 INFO:tasks.workunit.client.1.smithi149.stdout:5/218: rename d7/da/d20/c24 to d7/da/d20/c43 0 2024-03-22T07:08:50.248 INFO:tasks.workunit.client.1.smithi149.stdout:5/219: write d7/f33 [807904,31632] 0 2024-03-22T07:08:50.249 INFO:tasks.workunit.client.1.smithi149.stdout:4/182: rename d0/d8/dc/d37 to d0/d8/dc/d14/d3e 0 2024-03-22T07:08:50.249 INFO:tasks.workunit.client.1.smithi149.stdout:4/183: fsync d0/d29/f3c 0 2024-03-22T07:08:50.249 INFO:tasks.workunit.client.1.smithi149.stdout:1/272: mkdir d4/d7/d5d 0 2024-03-22T07:08:50.251 INFO:tasks.workunit.client.1.smithi149.stdout:0/143: mkdir d1/d27 0 2024-03-22T07:08:50.251 INFO:tasks.workunit.client.1.smithi149.stdout:0/144: creat d1/d9/d14/f28 x:0 0 0 2024-03-22T07:08:50.251 INFO:tasks.workunit.client.1.smithi149.stdout:0/145: write d1/d9/d14/f28 [680914,3456] 0 2024-03-22T07:08:50.251 INFO:tasks.workunit.client.1.smithi149.stdout:0/146: creat d1/d9/f29 x:0 0 0 2024-03-22T07:08:50.251 INFO:tasks.workunit.client.1.smithi149.stdout:0/147: chown d1/d9/c10 68 1 2024-03-22T07:08:50.251 INFO:tasks.workunit.client.1.smithi149.stdout:2/249: creat d1/d8/d34/f57 x:0 0 0 2024-03-22T07:08:50.251 INFO:tasks.workunit.client.1.smithi149.stdout:5/220: mkdir d7/da/d20/d36/d44 0 2024-03-22T07:08:50.252 INFO:tasks.workunit.client.1.smithi149.stdout:4/184: symlink d0/d29/l3f 0 2024-03-22T07:08:50.253 INFO:tasks.workunit.client.1.smithi149.stdout:1/273: creat d4/d7/d5a/f5e x:0 0 0 2024-03-22T07:08:50.254 INFO:tasks.workunit.client.1.smithi149.stdout:9/255: dwrite db/d22/f2b [0,4194304] 0 2024-03-22T07:08:50.254 INFO:tasks.workunit.client.1.smithi149.stdout:9/256: chown db/d19/f3c 0 1 2024-03-22T07:08:50.255 INFO:tasks.workunit.client.1.smithi149.stdout:3/174: dwrite d0/d4/d8/f19 [0,4194304] 0 2024-03-22T07:08:50.258 INFO:tasks.workunit.client.1.smithi149.stdout:0/148: unlink d1/d9/f12 0 2024-03-22T07:08:50.260 INFO:tasks.workunit.client.1.smithi149.stdout:2/250: symlink d1/d3/d6/d3f/l58 0 2024-03-22T07:08:50.260 INFO:tasks.workunit.client.1.smithi149.stdout:2/251: dread - d1/d3/f29 zero size 2024-03-22T07:08:50.261 INFO:tasks.workunit.client.1.smithi149.stdout:5/221: truncate d7/da/d20/d30/d3a/d3d/f34 1026092 0 2024-03-22T07:08:50.262 INFO:tasks.workunit.client.1.smithi149.stdout:4/185: symlink d0/d8/dc/d14/l40 0 2024-03-22T07:08:50.262 INFO:tasks.workunit.client.1.smithi149.stdout:4/186: readlink d0/d8/dc/l15 0 2024-03-22T07:08:50.264 INFO:tasks.workunit.client.1.smithi149.stdout:3/175: dread d0/d2/d30/d31/f33 [0,4194304] 0 2024-03-22T07:08:50.266 INFO:tasks.workunit.client.1.smithi149.stdout:5/222: mkdir d7/da/d20/d45 0 2024-03-22T07:08:50.266 INFO:tasks.workunit.client.1.smithi149.stdout:5/223: fsync d7/da/f12 0 2024-03-22T07:08:50.269 INFO:tasks.workunit.client.1.smithi149.stdout:4/187: rename d0/c2 to d0/d8/dc/d2e/c41 0 2024-03-22T07:08:50.270 INFO:tasks.workunit.client.1.smithi149.stdout:0/149: write d1/d9/f1e [36674,110791] 0 2024-03-22T07:08:50.270 INFO:tasks.workunit.client.1.smithi149.stdout:0/150: creat d1/d9/d14/d1f/f2a x:0 0 0 2024-03-22T07:08:50.270 INFO:tasks.workunit.client.1.smithi149.stdout:0/151: dread - d1/d9/d14/f26 zero size 2024-03-22T07:08:50.270 INFO:tasks.workunit.client.1.smithi149.stdout:0/152: creat d1/d9/f2b x:0 0 0 2024-03-22T07:08:50.270 INFO:tasks.workunit.client.1.smithi149.stdout:0/153: write d1/d9/d14/f26 [266688,112664] 0 2024-03-22T07:08:50.271 INFO:tasks.workunit.client.1.smithi149.stdout:4/188: symlink d0/d8/d1d/d24/l42 0 2024-03-22T07:08:50.271 INFO:tasks.workunit.client.1.smithi149.stdout:4/189: creat d0/d8/dc/d14/f43 x:0 0 0 2024-03-22T07:08:50.278 INFO:tasks.workunit.client.1.smithi149.stdout:0/154: mknod d1/d27/c2c 0 2024-03-22T07:08:50.278 INFO:tasks.workunit.client.1.smithi149.stdout:0/155: stat d1/d9/cc 0 2024-03-22T07:08:50.281 INFO:tasks.workunit.client.1.smithi149.stdout:0/156: rename d1/d9/d14/f26 to d1/d1a/f2d 0 2024-03-22T07:08:50.283 INFO:tasks.workunit.client.1.smithi149.stdout:0/157: symlink d1/d9/d14/l2e 0 2024-03-22T07:08:50.283 INFO:tasks.workunit.client.1.smithi149.stdout:8/281: sync 2024-03-22T07:08:50.285 INFO:tasks.workunit.client.1.smithi149.stdout:8/282: mknod d1/d20/c61 0 2024-03-22T07:08:50.287 INFO:tasks.workunit.client.1.smithi149.stdout:8/283: truncate d1/d2/d3/f6 6964839 0 2024-03-22T07:08:50.287 INFO:tasks.workunit.client.1.smithi149.stdout:8/284: creat d1/d20/d35/d22/d44/f62 x:0 0 0 2024-03-22T07:08:50.287 INFO:tasks.workunit.client.1.smithi149.stdout:8/285: read - d1/d2/f43 zero size 2024-03-22T07:08:50.287 INFO:tasks.workunit.client.1.smithi149.stdout:8/286: chown d1/d2/d9/d11/f17 358 1 2024-03-22T07:08:50.291 INFO:tasks.workunit.client.1.smithi149.stdout:8/287: dread d1/d2/d9/f1d [0,4194304] 0 2024-03-22T07:08:50.291 INFO:tasks.workunit.client.1.smithi149.stdout:8/288: creat d1/d20/d41/f63 x:0 0 0 2024-03-22T07:08:50.292 INFO:tasks.workunit.client.1.smithi149.stdout:8/289: symlink d1/d20/d35/d22/d44/d4f/l64 0 2024-03-22T07:08:50.293 INFO:tasks.workunit.client.1.smithi149.stdout:8/290: write d1/d2/d9/d11/f17 [3908780,65083] 0 2024-03-22T07:08:50.293 INFO:tasks.workunit.client.1.smithi149.stdout:8/291: read - d1/d2/f43 zero size 2024-03-22T07:08:50.294 INFO:tasks.workunit.client.1.smithi149.stdout:8/292: creat d1/d2/d9/f65 x:0 0 0 2024-03-22T07:08:50.294 INFO:tasks.workunit.client.1.smithi149.stdout:8/293: truncate d1/d2/d9/df/f2d 1588431 0 2024-03-22T07:08:50.296 INFO:tasks.workunit.client.1.smithi149.stdout:8/294: rename d1/d2/d9/f42 to d1/d20/f66 0 2024-03-22T07:08:50.298 INFO:tasks.workunit.client.1.smithi149.stdout:9/257: dwrite db/d22/f45 [0,4194304] 0 2024-03-22T07:08:50.298 INFO:tasks.workunit.client.1.smithi149.stdout:9/258: fdatasync db/d33/f3d 0 2024-03-22T07:08:50.298 INFO:tasks.workunit.client.1.smithi149.stdout:9/259: read db/d19/f1f [1143817,70791] 0 2024-03-22T07:08:50.300 INFO:tasks.workunit.client.1.smithi149.stdout:9/260: symlink db/l4a 0 2024-03-22T07:08:50.301 INFO:tasks.workunit.client.1.smithi149.stdout:9/261: mkdir db/d42/d49/d4b 0 2024-03-22T07:08:50.301 INFO:tasks.workunit.client.1.smithi149.stdout:9/262: write db/fc [2403038,2973] 0 2024-03-22T07:08:50.304 INFO:tasks.workunit.client.1.smithi149.stdout:9/263: link db/d19/d39/d3b/f48 db/d42/f4c 0 2024-03-22T07:08:50.311 INFO:tasks.workunit.client.1.smithi149.stdout:3/176: dwrite d0/d4/d8/f21 [0,4194304] 0 2024-03-22T07:08:50.311 INFO:tasks.workunit.client.1.smithi149.stdout:3/177: fsync d0/d4/d8/f25 0 2024-03-22T07:08:50.311 INFO:tasks.workunit.client.1.smithi149.stdout:1/274: dwrite d4/d10/d28/f37 [0,4194304] 0 2024-03-22T07:08:50.312 INFO:tasks.workunit.client.1.smithi149.stdout:4/190: dwrite d0/f1a [0,4194304] 0 2024-03-22T07:08:50.312 INFO:tasks.workunit.client.1.smithi149.stdout:4/191: creat d0/d8/dc/d14/f44 x:0 0 0 2024-03-22T07:08:50.312 INFO:tasks.workunit.client.1.smithi149.stdout:5/224: dwrite d7/da/f37 [0,4194304] 0 2024-03-22T07:08:50.313 INFO:tasks.workunit.client.1.smithi149.stdout:3/178: mknod d0/d4/d8/c41 0 2024-03-22T07:08:50.313 INFO:tasks.workunit.client.1.smithi149.stdout:3/179: fsync d0/d2/d36/f22 0 2024-03-22T07:08:50.315 INFO:tasks.workunit.client.1.smithi149.stdout:1/275: getdents d4/d10/d58 0 2024-03-22T07:08:50.316 INFO:tasks.workunit.client.1.smithi149.stdout:9/264: dread db/d22/f2b [0,4194304] 0 2024-03-22T07:08:50.323 INFO:tasks.workunit.client.1.smithi149.stdout:4/192: mknod d0/d8/dc/c45 0 2024-03-22T07:08:50.323 INFO:tasks.workunit.client.1.smithi149.stdout:0/158: rmdir d1/d9 39 2024-03-22T07:08:50.326 INFO:tasks.workunit.client.1.smithi149.stdout:3/180: mkdir d0/d2/d42 0 2024-03-22T07:08:50.327 INFO:tasks.workunit.client.1.smithi149.stdout:3/181: truncate d0/f1a 405413 0 2024-03-22T07:08:50.327 INFO:tasks.workunit.client.1.smithi149.stdout:3/182: dread d0/d2/f2d [0,4194304] 0 2024-03-22T07:08:50.327 INFO:tasks.workunit.client.1.smithi149.stdout:3/183: chown d0/f38 1471 1 2024-03-22T07:08:50.328 INFO:tasks.workunit.client.1.smithi149.stdout:3/184: write d0/d2/d36/f2e [943847,36115] 0 2024-03-22T07:08:50.328 INFO:tasks.workunit.client.1.smithi149.stdout:1/276: getdents d4/d20/d23 0 2024-03-22T07:08:50.329 INFO:tasks.workunit.client.1.smithi149.stdout:9/265: mkdir db/d42/d49/d4b/d4d 0 2024-03-22T07:08:50.329 INFO:tasks.workunit.client.1.smithi149.stdout:5/225: dwrite d7/da/f1b [0,4194304] 0 2024-03-22T07:08:50.329 INFO:tasks.workunit.client.1.smithi149.stdout:5/226: fdatasync d7/da/fd 0 2024-03-22T07:08:50.329 INFO:tasks.workunit.client.1.smithi149.stdout:4/193: stat d0/d8/dc/d2e/c41 0 2024-03-22T07:08:50.331 INFO:tasks.workunit.client.1.smithi149.stdout:0/159: rename d1/d9/fd to d1/d1a/f2f 0 2024-03-22T07:08:50.332 INFO:tasks.workunit.client.1.smithi149.stdout:3/185: link d0/d4/d8/c41 d0/d4/c43 0 2024-03-22T07:08:50.334 INFO:tasks.workunit.client.1.smithi149.stdout:1/277: mknod d4/d20/d23/c5f 0 2024-03-22T07:08:50.334 INFO:tasks.workunit.client.1.smithi149.stdout:1/278: chown d4/d10/f4c 552 1 2024-03-22T07:08:50.338 INFO:tasks.workunit.client.1.smithi149.stdout:5/227: mknod d7/da/d20/d45/c46 0 2024-03-22T07:08:50.338 INFO:tasks.workunit.client.1.smithi149.stdout:4/194: link d0/f32 d0/d8/f46 0 2024-03-22T07:08:50.338 INFO:tasks.workunit.client.1.smithi149.stdout:4/195: chown d0/lf 3 1 2024-03-22T07:08:50.338 INFO:tasks.workunit.client.1.smithi149.stdout:4/196: stat d0/d8/dc 0 2024-03-22T07:08:50.339 INFO:tasks.workunit.client.1.smithi149.stdout:0/160: mknod d1/d9/d14/c30 0 2024-03-22T07:08:50.340 INFO:tasks.workunit.client.1.smithi149.stdout:3/186: symlink d0/d4/d8/l44 0 2024-03-22T07:08:50.341 INFO:tasks.workunit.client.1.smithi149.stdout:5/228: mkdir d7/da/d20/d30/d3a/d3d/d2c/d47 0 2024-03-22T07:08:50.343 INFO:tasks.workunit.client.1.smithi149.stdout:4/197: rename d0/d8/l10 to d0/d8/dc/d14/l47 0 2024-03-22T07:08:50.350 INFO:tasks.workunit.client.1.smithi149.stdout:5/229: rmdir d7/da/d20/d45 39 2024-03-22T07:08:50.350 INFO:tasks.workunit.client.1.smithi149.stdout:5/230: readlink d7/db/l42 0 2024-03-22T07:08:50.351 INFO:tasks.workunit.client.1.smithi149.stdout:3/187: write d0/d4/d8/f21 [3127015,37125] 0 2024-03-22T07:08:50.351 INFO:tasks.workunit.client.1.smithi149.stdout:5/231: chown d7/da/lf 219645 1 2024-03-22T07:08:50.351 INFO:tasks.workunit.client.1.smithi149.stdout:0/161: mkdir d1/d31 0 2024-03-22T07:08:50.351 INFO:tasks.workunit.client.1.smithi149.stdout:0/162: stat d1/c19 0 2024-03-22T07:08:50.352 INFO:tasks.workunit.client.1.smithi149.stdout:5/232: creat d7/da/d20/d45/f48 x:0 0 0 2024-03-22T07:08:50.353 INFO:tasks.workunit.client.1.smithi149.stdout:5/233: write d7/f33 [162036,45721] 0 2024-03-22T07:08:50.355 INFO:tasks.workunit.client.1.smithi149.stdout:0/163: dread d1/d9/d14/f28 [0,4194304] 0 2024-03-22T07:08:50.355 INFO:tasks.workunit.client.1.smithi149.stdout:3/188: dread d0/d4/d8/f1d [0,4194304] 0 2024-03-22T07:08:50.356 INFO:tasks.workunit.client.1.smithi149.stdout:3/189: stat d0/d4/d8/c41 0 2024-03-22T07:08:50.357 INFO:tasks.workunit.client.1.smithi149.stdout:0/164: symlink d1/l32 0 2024-03-22T07:08:50.357 INFO:tasks.workunit.client.1.smithi149.stdout:0/165: write d1/d9/f29 [425076,75742] 0 2024-03-22T07:08:50.358 INFO:tasks.workunit.client.1.smithi149.stdout:3/190: symlink d0/d4/d8/d24/l45 0 2024-03-22T07:08:50.359 INFO:tasks.workunit.client.1.smithi149.stdout:3/191: truncate d0/d4/d8/f1d 1359271 0 2024-03-22T07:08:50.359 INFO:tasks.workunit.client.1.smithi149.stdout:3/192: stat d0/d4/d8/f1d 0 2024-03-22T07:08:50.359 INFO:tasks.workunit.client.1.smithi149.stdout:3/193: creat d0/d2/d30/f46 x:0 0 0 2024-03-22T07:08:50.359 INFO:tasks.workunit.client.1.smithi149.stdout:7/282: sync 2024-03-22T07:08:50.359 INFO:tasks.workunit.client.1.smithi149.stdout:7/283: dread - d1/d1c/d4d/f58 zero size 2024-03-22T07:08:50.359 INFO:tasks.workunit.client.1.smithi149.stdout:0/166: unlink d1/l32 0 2024-03-22T07:08:50.363 INFO:tasks.workunit.client.1.smithi149.stdout:3/194: rmdir d0/d2/d36 39 2024-03-22T07:08:50.364 INFO:tasks.workunit.client.1.smithi149.stdout:7/284: symlink d1/d18/d1b/d1d/d2a/l61 0 2024-03-22T07:08:50.365 INFO:tasks.workunit.client.1.smithi149.stdout:9/266: dwrite db/d22/f2b [0,4194304] 0 2024-03-22T07:08:50.366 INFO:tasks.workunit.client.1.smithi149.stdout:3/195: creat d0/d4/d8/d24/f47 x:0 0 0 2024-03-22T07:08:50.366 INFO:tasks.workunit.client.1.smithi149.stdout:3/196: write d0/d2/f2d [1183616,70896] 0 2024-03-22T07:08:50.367 INFO:tasks.workunit.client.1.smithi149.stdout:1/279: dwrite d4/d10/d28/f53 [0,4194304] 0 2024-03-22T07:08:50.367 INFO:tasks.workunit.client.1.smithi149.stdout:1/280: creat d4/d10/d28/f60 x:0 0 0 2024-03-22T07:08:50.367 INFO:tasks.workunit.client.1.smithi149.stdout:4/198: dwrite d0/d8/dc/d14/f44 [0,4194304] 0 2024-03-22T07:08:50.367 INFO:tasks.workunit.client.1.smithi149.stdout:4/199: dread - d0/d29/f3c zero size 2024-03-22T07:08:50.367 INFO:tasks.workunit.client.1.smithi149.stdout:4/200: write d0/d8/dc/f30 [4842987,61032] 0 2024-03-22T07:08:50.368 INFO:tasks.workunit.client.1.smithi149.stdout:7/285: creat d1/d18/d1b/d1d/f62 x:0 0 0 2024-03-22T07:08:50.371 INFO:tasks.workunit.client.1.smithi149.stdout:9/267: link db/d33/f3d db/d21/d37/f4e 0 2024-03-22T07:08:50.371 INFO:tasks.workunit.client.1.smithi149.stdout:9/268: creat db/d19/d1c/f4f x:0 0 0 2024-03-22T07:08:50.372 INFO:tasks.workunit.client.1.smithi149.stdout:0/167: dwrite d1/d9/d14/d1f/f2a [0,4194304] 0 2024-03-22T07:08:50.372 INFO:tasks.workunit.client.1.smithi149.stdout:0/168: write d1/d1a/f25 [473930,11326] 0 2024-03-22T07:08:50.372 INFO:tasks.workunit.client.1.smithi149.stdout:3/197: mknod d0/d2/c48 0 2024-03-22T07:08:50.377 INFO:tasks.workunit.client.1.smithi149.stdout:1/281: mkdir d4/d7/d61 0 2024-03-22T07:08:50.377 INFO:tasks.workunit.client.1.smithi149.stdout:1/282: readlink d4/d20/d2d/l38 0 2024-03-22T07:08:50.379 INFO:tasks.workunit.client.1.smithi149.stdout:5/234: truncate d7/da/f1b 4144365 0 2024-03-22T07:08:50.381 INFO:tasks.workunit.client.1.smithi149.stdout:4/201: rename d0/d8/dc/d14/f23 to d0/d8/d1d/f48 0 2024-03-22T07:08:50.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/202: dread d0/d8/dc/d2e/f2f [0,4194304] 0 2024-03-22T07:08:50.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/203: write d0/d8/f34 [1003676,67371] 0 2024-03-22T07:08:50.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/204: dread - d0/d29/f35 zero size 2024-03-22T07:08:50.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/205: creat d0/d29/f49 x:0 0 0 2024-03-22T07:08:50.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/206: creat d0/d29/f4a x:0 0 0 2024-03-22T07:08:50.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/207: write d0/d29/f3c [480579,26684] 0 2024-03-22T07:08:50.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/208: stat d0/d8/d1d/f48 0 2024-03-22T07:08:50.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/209: stat d0/d8/dc/l17 0 2024-03-22T07:08:50.384 INFO:tasks.workunit.client.1.smithi149.stdout:0/169: rmdir d1/d9 39 2024-03-22T07:08:50.385 INFO:tasks.workunit.client.1.smithi149.stdout:0/170: chown d1/d1a/f25 126 1 2024-03-22T07:08:50.386 INFO:tasks.workunit.client.1.smithi149.stdout:5/235: creat d7/da/d20/f49 x:0 0 0 2024-03-22T07:08:50.391 INFO:tasks.workunit.client.1.smithi149.stdout:4/210: mkdir d0/d8/dc/d4b 0 2024-03-22T07:08:50.392 INFO:tasks.workunit.client.1.smithi149.stdout:7/286: dwrite d1/d1c/d4d/f58 [0,4194304] 0 2024-03-22T07:08:50.393 INFO:tasks.workunit.client.1.smithi149.stdout:1/283: rename d4/d20/d2d/f40 to d4/d20/d23/d2f/f62 0 2024-03-22T07:08:50.397 INFO:tasks.workunit.client.1.smithi149.stdout:1/284: link d4/d20/d23/d2f/f3b d4/d10/f63 0 2024-03-22T07:08:50.397 INFO:tasks.workunit.client.1.smithi149.stdout:1/285: chown d4/d20/l25 2134293918 1 2024-03-22T07:08:50.397 INFO:tasks.workunit.client.1.smithi149.stdout:0/171: dread d1/d9/f13 [0,4194304] 0 2024-03-22T07:08:50.397 INFO:tasks.workunit.client.1.smithi149.stdout:0/172: chown d1/d9/d14/f28 24 1 2024-03-22T07:08:50.397 INFO:tasks.workunit.client.1.smithi149.stdout:0/173: fsync d1/fb 0 2024-03-22T07:08:50.399 INFO:tasks.workunit.client.1.smithi149.stdout:0/174: creat d1/d31/f33 x:0 0 0 2024-03-22T07:08:50.401 INFO:tasks.workunit.client.1.smithi149.stdout:5/236: dread d7/f8 [0,4194304] 0 2024-03-22T07:08:50.402 INFO:tasks.workunit.client.1.smithi149.stdout:0/175: dread d1/d1a/f2d [0,4194304] 0 2024-03-22T07:08:50.403 INFO:tasks.workunit.client.1.smithi149.stdout:0/176: creat d1/d31/f34 x:0 0 0 2024-03-22T07:08:50.403 INFO:tasks.workunit.client.1.smithi149.stdout:5/237: rmdir d7/da/d20/d30/d3a/d3d 39 2024-03-22T07:08:50.403 INFO:tasks.workunit.client.1.smithi149.stdout:9/269: dwrite db/d22/f2b [0,4194304] 0 2024-03-22T07:08:50.405 INFO:tasks.workunit.client.1.smithi149.stdout:7/287: write d1/d1c/d4d/f3e [1496461,70283] 0 2024-03-22T07:08:50.406 INFO:tasks.workunit.client.1.smithi149.stdout:0/177: symlink d1/d1a/l35 0 2024-03-22T07:08:50.406 INFO:tasks.workunit.client.1.smithi149.stdout:9/270: mknod db/d19/c50 0 2024-03-22T07:08:50.406 INFO:tasks.workunit.client.1.smithi149.stdout:0/178: chown d1/d1a/l35 38 1 2024-03-22T07:08:50.406 INFO:tasks.workunit.client.1.smithi149.stdout:7/288: dread - d1/d18/d1b/d51/f52 zero size 2024-03-22T07:08:50.406 INFO:tasks.workunit.client.1.smithi149.stdout:9/271: creat db/d19/d1c/f51 x:0 0 0 2024-03-22T07:08:50.408 INFO:tasks.workunit.client.1.smithi149.stdout:1/286: write d4/d10/d58/f5b [577165,59421] 0 2024-03-22T07:08:50.409 INFO:tasks.workunit.client.1.smithi149.stdout:1/287: mkdir d4/d20/d2d/d64 0 2024-03-22T07:08:50.421 INFO:tasks.workunit.client.1.smithi149.stdout:7/289: dread d1/f10 [0,4194304] 0 2024-03-22T07:08:50.422 INFO:tasks.workunit.client.1.smithi149.stdout:9/272: dread db/f2f [0,4194304] 0 2024-03-22T07:08:50.423 INFO:tasks.workunit.client.1.smithi149.stdout:7/290: symlink d1/d18/d1b/d51/l63 0 2024-03-22T07:08:50.423 INFO:tasks.workunit.client.1.smithi149.stdout:7/291: chown d1/d1c/d4d/f3e 217991 1 2024-03-22T07:08:50.423 INFO:tasks.workunit.client.1.smithi149.stdout:7/292: write d1/d1c/d4d/f48 [965737,31758] 0 2024-03-22T07:08:50.423 INFO:tasks.workunit.client.1.smithi149.stdout:3/198: dwrite d0/d4/d8/f21 [0,4194304] 0 2024-03-22T07:08:50.423 INFO:tasks.workunit.client.1.smithi149.stdout:9/273: truncate db/d19/f3c 625877 0 2024-03-22T07:08:50.424 INFO:tasks.workunit.client.1.smithi149.stdout:5/238: write d7/f9 [4070747,46152] 0 2024-03-22T07:08:50.424 INFO:tasks.workunit.client.1.smithi149.stdout:5/239: write d7/da/fd [1045120,44392] 0 2024-03-22T07:08:50.424 INFO:tasks.workunit.client.1.smithi149.stdout:5/240: dread - d7/da/d20/f49 zero size 2024-03-22T07:08:50.426 INFO:tasks.workunit.client.1.smithi149.stdout:0/179: write d1/f22 [3368201,28352] 0 2024-03-22T07:08:50.430 INFO:tasks.workunit.client.1.smithi149.stdout:4/211: dwrite d0/f3 [0,4194304] 0 2024-03-22T07:08:50.430 INFO:tasks.workunit.client.1.smithi149.stdout:4/212: truncate d0/d8/f1f 5217584 0 2024-03-22T07:08:50.431 INFO:tasks.workunit.client.1.smithi149.stdout:4/213: readlink d0/l6 0 2024-03-22T07:08:50.431 INFO:tasks.workunit.client.1.smithi149.stdout:7/293: rename d1/d1c/d4d/l35 to d1/d1c/l64 0 2024-03-22T07:08:50.431 INFO:tasks.workunit.client.1.smithi149.stdout:7/294: write d1/d1c/d4d/f3e [2829081,104802] 0 2024-03-22T07:08:50.431 INFO:tasks.workunit.client.1.smithi149.stdout:7/295: stat d1/d18/d1b/d1d/d28/d53 0 2024-03-22T07:08:50.432 INFO:tasks.workunit.client.1.smithi149.stdout:0/180: mkdir d1/d31/d36 0 2024-03-22T07:08:50.432 INFO:tasks.workunit.client.1.smithi149.stdout:4/214: mkdir d0/d8/d4c 0 2024-03-22T07:08:50.432 INFO:tasks.workunit.client.1.smithi149.stdout:4/215: write d0/d29/f3c [1306111,22770] 0 2024-03-22T07:08:50.434 INFO:tasks.workunit.client.1.smithi149.stdout:3/199: rename d0/c14 to d0/d4/d8/d24/d39/c49 0 2024-03-22T07:08:50.434 INFO:tasks.workunit.client.1.smithi149.stdout:3/200: write d0/d2/d30/f46 [200324,79309] 0 2024-03-22T07:08:50.435 INFO:tasks.workunit.client.1.smithi149.stdout:7/296: symlink d1/d18/d1b/d1d/d28/l65 0 2024-03-22T07:08:50.436 INFO:tasks.workunit.client.1.smithi149.stdout:9/274: rmdir db/d21 39 2024-03-22T07:08:50.438 INFO:tasks.workunit.client.1.smithi149.stdout:0/181: creat d1/d9/d14/d1f/f37 x:0 0 0 2024-03-22T07:08:50.438 INFO:tasks.workunit.client.1.smithi149.stdout:0/182: creat d1/d9/d14/d1f/f38 x:0 0 0 2024-03-22T07:08:50.438 INFO:tasks.workunit.client.1.smithi149.stdout:5/241: dread d7/db/fc [0,4194304] 0 2024-03-22T07:08:50.438 INFO:tasks.workunit.client.1.smithi149.stdout:5/242: chown d7/db/f22 6111 1 2024-03-22T07:08:50.439 INFO:tasks.workunit.client.1.smithi149.stdout:4/216: creat d0/d8/dc/d4b/f4d x:0 0 0 2024-03-22T07:08:50.439 INFO:tasks.workunit.client.1.smithi149.stdout:4/217: chown d0/d8/dc/d14/f44 97539386 1 2024-03-22T07:08:50.442 INFO:tasks.workunit.client.1.smithi149.stdout:9/275: mkdir db/d42/d49/d4b/d4d/d52 0 2024-03-22T07:08:50.445 INFO:tasks.workunit.client.1.smithi149.stdout:4/218: mknod d0/d8/c4e 0 2024-03-22T07:08:50.446 INFO:tasks.workunit.client.1.smithi149.stdout:4/219: read d0/d8/d1d/f48 [364011,28448] 0 2024-03-22T07:08:50.447 INFO:tasks.workunit.client.1.smithi149.stdout:0/183: rename d1/d1a/f2f to d1/f39 0 2024-03-22T07:08:50.447 INFO:tasks.workunit.client.1.smithi149.stdout:0/184: chown d1/f22 0 1 2024-03-22T07:08:50.449 INFO:tasks.workunit.client.1.smithi149.stdout:0/185: link d1/d9/f1e d1/d9/f3a 0 2024-03-22T07:08:50.449 INFO:tasks.workunit.client.1.smithi149.stdout:6/183: sync 2024-03-22T07:08:50.449 INFO:tasks.workunit.client.1.smithi149.stdout:2/252: sync 2024-03-22T07:08:50.449 INFO:tasks.workunit.client.1.smithi149.stdout:8/295: sync 2024-03-22T07:08:50.451 INFO:tasks.workunit.client.1.smithi149.stdout:5/243: read d7/da/f1b [295526,3653] 0 2024-03-22T07:08:50.452 INFO:tasks.workunit.client.1.smithi149.stdout:8/296: unlink d1/d2/d9/f1b 0 2024-03-22T07:08:50.452 INFO:tasks.workunit.client.1.smithi149.stdout:8/297: write d1/d20/d35/d22/f40 [315765,116671] 0 2024-03-22T07:08:50.454 INFO:tasks.workunit.client.1.smithi149.stdout:2/253: mkdir d1/d8/d59 0 2024-03-22T07:08:50.454 INFO:tasks.workunit.client.1.smithi149.stdout:2/254: write d1/d3/d5/f2b [14521,69596] 0 2024-03-22T07:08:50.456 INFO:tasks.workunit.client.1.smithi149.stdout:8/298: creat d1/d2/d9/d11/d4d/f67 x:0 0 0 2024-03-22T07:08:50.459 INFO:tasks.workunit.client.1.smithi149.stdout:2/255: rename d1/f51 to d1/d3/d5/f5a 0 2024-03-22T07:08:50.462 INFO:tasks.workunit.client.1.smithi149.stdout:8/299: unlink d1/d2/d3/f3a 0 2024-03-22T07:08:50.462 INFO:tasks.workunit.client.1.smithi149.stdout:8/300: stat d1/d20/d35/c10 0 2024-03-22T07:08:50.464 INFO:tasks.workunit.client.1.smithi149.stdout:2/256: link d1/l20 d1/d3/d6/l5b 0 2024-03-22T07:08:50.466 INFO:tasks.workunit.client.1.smithi149.stdout:8/301: mknod d1/d20/d35/d22/c68 0 2024-03-22T07:08:50.466 INFO:tasks.workunit.client.1.smithi149.stdout:8/302: fsync d1/d20/d35/d22/d44/f52 0 2024-03-22T07:08:50.468 INFO:tasks.workunit.client.1.smithi149.stdout:2/257: symlink d1/d3/dd/l5c 0 2024-03-22T07:08:50.469 INFO:tasks.workunit.client.1.smithi149.stdout:8/303: creat d1/d20/f69 x:0 0 0 2024-03-22T07:08:50.471 INFO:tasks.workunit.client.1.smithi149.stdout:2/258: unlink d1/d3/d6/l5b 0 2024-03-22T07:08:50.472 INFO:tasks.workunit.client.1.smithi149.stdout:7/297: truncate d1/d1c/d4d/f48 282518 0 2024-03-22T07:08:50.472 INFO:tasks.workunit.client.1.smithi149.stdout:7/298: fdatasync d1/d18/f37 0 2024-03-22T07:08:50.473 INFO:tasks.workunit.client.1.smithi149.stdout:2/259: mknod d1/d3/d6/c5d 0 2024-03-22T07:08:50.474 INFO:tasks.workunit.client.1.smithi149.stdout:7/299: truncate d1/d18/d1b/d1d/f16 2927219 0 2024-03-22T07:08:50.474 INFO:tasks.workunit.client.1.smithi149.stdout:7/300: fsync d1/d18/d1b/f2c 0 2024-03-22T07:08:50.474 INFO:tasks.workunit.client.1.smithi149.stdout:7/301: read d1/d18/d1b/d1d/f8 [107182,36238] 0 2024-03-22T07:08:50.474 INFO:tasks.workunit.client.1.smithi149.stdout:2/260: truncate d1/d3/fb 3725067 0 2024-03-22T07:08:50.476 INFO:tasks.workunit.client.1.smithi149.stdout:7/302: symlink d1/d18/d1b/d1d/d28/l66 0 2024-03-22T07:08:50.477 INFO:tasks.workunit.client.1.smithi149.stdout:2/261: rename d1/d3/d26 to d1/d3/d5/d1f/d5e 0 2024-03-22T07:08:50.477 INFO:tasks.workunit.client.1.smithi149.stdout:2/262: write d1/d3/f25 [1485437,15104] 0 2024-03-22T07:08:50.477 INFO:tasks.workunit.client.1.smithi149.stdout:2/263: read - d1/d3/d5/f5a zero size 2024-03-22T07:08:50.480 INFO:tasks.workunit.client.1.smithi149.stdout:3/201: dwrite d0/d4/f1e [0,4194304] 0 2024-03-22T07:08:50.481 INFO:tasks.workunit.client.1.smithi149.stdout:3/202: rename d0/c1 to d0/c4a 0 2024-03-22T07:08:50.482 INFO:tasks.workunit.client.1.smithi149.stdout:3/203: stat d0/d4/fe 0 2024-03-22T07:08:50.482 INFO:tasks.workunit.client.1.smithi149.stdout:3/204: rename d0/d2/c29 to d0/d2/d30/d31/c4b 0 2024-03-22T07:08:50.490 INFO:tasks.workunit.client.1.smithi149.stdout:7/303: write d1/d18/d1b/d1d/f40 [1031581,44726] 0 2024-03-22T07:08:50.495 INFO:tasks.workunit.client.1.smithi149.stdout:0/186: dwrite d1/d31/f34 [0,4194304] 0 2024-03-22T07:08:50.495 INFO:tasks.workunit.client.1.smithi149.stdout:0/187: fdatasync d1/d9/d14/d1f/f24 0 2024-03-22T07:08:50.495 INFO:tasks.workunit.client.1.smithi149.stdout:0/188: fdatasync d1/fb 0 2024-03-22T07:08:50.499 INFO:tasks.workunit.client.1.smithi149.stdout:4/220: dwrite d0/d29/f4a [0,4194304] 0 2024-03-22T07:08:50.505 INFO:tasks.workunit.client.1.smithi149.stdout:6/184: dwrite f6 [4194304,4194304] 0 2024-03-22T07:08:50.509 INFO:tasks.workunit.client.1.smithi149.stdout:4/221: write d0/d8/dc/d14/f2b [2010015,98478] 0 2024-03-22T07:08:50.509 INFO:tasks.workunit.client.1.smithi149.stdout:4/222: fdatasync d0/d8/dc/f30 0 2024-03-22T07:08:50.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/244: dwrite d7/f9 [8388608,4194304] 0 2024-03-22T07:08:50.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/245: truncate d7/da/f12 4539286 0 2024-03-22T07:08:50.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/246: dread - d7/da/d20/f49 zero size 2024-03-22T07:08:50.511 INFO:tasks.workunit.client.1.smithi149.stdout:4/223: truncate d0/d8/f21 260534 0 2024-03-22T07:08:50.511 INFO:tasks.workunit.client.1.smithi149.stdout:4/224: truncate d0/d8/d1d/f28 1047544 0 2024-03-22T07:08:50.512 INFO:tasks.workunit.client.1.smithi149.stdout:4/225: mknod d0/d8/d4c/c4f 0 2024-03-22T07:08:50.513 INFO:tasks.workunit.client.1.smithi149.stdout:8/304: dwrite d1/d20/d41/f63 [0,4194304] 0 2024-03-22T07:08:50.517 INFO:tasks.workunit.client.1.smithi149.stdout:0/189: dwrite d1/d9/d14/f18 [0,4194304] 0 2024-03-22T07:08:50.517 INFO:tasks.workunit.client.1.smithi149.stdout:0/190: stat d1/d9/f13 0 2024-03-22T07:08:50.519 INFO:tasks.workunit.client.1.smithi149.stdout:8/305: link d1/f24 d1/d20/d35/d22/d44/f6a 0 2024-03-22T07:08:50.520 INFO:tasks.workunit.client.1.smithi149.stdout:9/276: dwrite db/d22/f45 [0,4194304] 0 2024-03-22T07:08:50.520 INFO:tasks.workunit.client.1.smithi149.stdout:6/185: dread d9/f19 [0,4194304] 0 2024-03-22T07:08:50.521 INFO:tasks.workunit.client.1.smithi149.stdout:7/304: dwrite d1/d18/d1b/d1d/d2a/f3b [0,4194304] 0 2024-03-22T07:08:50.521 INFO:tasks.workunit.client.1.smithi149.stdout:9/277: truncate db/d19/d39/d3b/f48 732060 0 2024-03-22T07:08:50.521 INFO:tasks.workunit.client.1.smithi149.stdout:9/278: chown db/d19/f3c 4667272 1 2024-03-22T07:08:50.521 INFO:tasks.workunit.client.1.smithi149.stdout:7/305: write d1/d18/d1b/d1d/fd [2425788,18656] 0 2024-03-22T07:08:50.522 INFO:tasks.workunit.client.1.smithi149.stdout:9/279: dread db/d19/d1c/f20 [0,4194304] 0 2024-03-22T07:08:50.522 INFO:tasks.workunit.client.1.smithi149.stdout:9/280: creat db/d19/d39/d3b/f53 x:0 0 0 2024-03-22T07:08:50.522 INFO:tasks.workunit.client.1.smithi149.stdout:9/281: chown db/d21/f23 0 1 2024-03-22T07:08:50.523 INFO:tasks.workunit.client.1.smithi149.stdout:2/264: fsync d1/d3/d5/f5a 0 2024-03-22T07:08:50.524 INFO:tasks.workunit.client.1.smithi149.stdout:0/191: dread d1/d9/f13 [0,4194304] 0 2024-03-22T07:08:50.532 INFO:tasks.workunit.client.1.smithi149.stdout:4/226: dwrite d0/d8/dc/fe [0,4194304] 0 2024-03-22T07:08:50.534 INFO:tasks.workunit.client.1.smithi149.stdout:6/186: symlink d9/l3a 0 2024-03-22T07:08:50.534 INFO:tasks.workunit.client.1.smithi149.stdout:6/187: truncate d9/df/d2e/f27 613659 0 2024-03-22T07:08:50.535 INFO:tasks.workunit.client.1.smithi149.stdout:7/306: creat d1/d18/d1b/d1d/d28/d53/f67 x:0 0 0 2024-03-22T07:08:50.537 INFO:tasks.workunit.client.1.smithi149.stdout:9/282: symlink db/d21/d37/l54 0 2024-03-22T07:08:50.537 INFO:tasks.workunit.client.1.smithi149.stdout:9/283: dread - db/d19/d1c/f4f zero size 2024-03-22T07:08:50.537 INFO:tasks.workunit.client.1.smithi149.stdout:9/284: write db/d19/d39/d3b/f53 [751496,8151] 0 2024-03-22T07:08:50.537 INFO:tasks.workunit.client.1.smithi149.stdout:9/285: chown db/d19/d39 0 1 2024-03-22T07:08:50.540 INFO:tasks.workunit.client.1.smithi149.stdout:2/265: rename d1/d8/f56 to d1/d3/dd/f5f 0 2024-03-22T07:08:50.542 INFO:tasks.workunit.client.1.smithi149.stdout:2/266: dread d1/d3/f25 [0,4194304] 0 2024-03-22T07:08:50.543 INFO:tasks.workunit.client.1.smithi149.stdout:0/192: symlink d1/d31/l3b 0 2024-03-22T07:08:50.544 INFO:tasks.workunit.client.1.smithi149.stdout:4/227: mknod d0/d8/dc/d4b/c50 0 2024-03-22T07:08:50.544 INFO:tasks.workunit.client.1.smithi149.stdout:4/228: readlink d0/la 0 2024-03-22T07:08:50.545 INFO:tasks.workunit.client.1.smithi149.stdout:7/307: creat d1/d1c/d4d/f68 x:0 0 0 2024-03-22T07:08:50.547 INFO:tasks.workunit.client.1.smithi149.stdout:9/286: creat db/d42/d49/d4b/d4d/f55 x:0 0 0 2024-03-22T07:08:50.548 INFO:tasks.workunit.client.1.smithi149.stdout:6/188: rename d9/df/d2e/cb to d9/d1f/d24/c3b 0 2024-03-22T07:08:50.548 INFO:tasks.workunit.client.1.smithi149.stdout:6/189: creat d9/f3c x:0 0 0 2024-03-22T07:08:50.549 INFO:tasks.workunit.client.1.smithi149.stdout:4/229: dread d0/d8/dc/f13 [0,4194304] 0 2024-03-22T07:08:50.551 INFO:tasks.workunit.client.1.smithi149.stdout:7/308: read d1/d1c/d49/f5e [1753534,20109] 0 2024-03-22T07:08:50.551 INFO:tasks.workunit.client.1.smithi149.stdout:7/309: stat d1/d18/d1b/d1d/d28/l39 0 2024-03-22T07:08:50.553 INFO:tasks.workunit.client.1.smithi149.stdout:2/267: creat d1/f60 x:0 0 0 2024-03-22T07:08:50.553 INFO:tasks.workunit.client.1.smithi149.stdout:2/268: dread - d1/d3/dd/f5f zero size 2024-03-22T07:08:50.553 INFO:tasks.workunit.client.1.smithi149.stdout:2/269: truncate d1/d3/d5/f2b 204315 0 2024-03-22T07:08:50.553 INFO:tasks.workunit.client.1.smithi149.stdout:0/193: symlink d1/d1a/l3c 0 2024-03-22T07:08:50.554 INFO:tasks.workunit.client.1.smithi149.stdout:9/287: symlink db/d19/d1c/l56 0 2024-03-22T07:08:50.555 INFO:tasks.workunit.client.1.smithi149.stdout:6/190: mknod d9/df/d22/c3d 0 2024-03-22T07:08:50.557 INFO:tasks.workunit.client.1.smithi149.stdout:4/230: unlink d0/d8/d1d/f3a 0 2024-03-22T07:08:50.558 INFO:tasks.workunit.client.1.smithi149.stdout:7/310: rename d1/d18/d1b/d1d/la to d1/d18/d1b/l69 0 2024-03-22T07:08:50.558 INFO:tasks.workunit.client.1.smithi149.stdout:2/270: rmdir d1/d4d 39 2024-03-22T07:08:50.559 INFO:tasks.workunit.client.1.smithi149.stdout:2/271: fsync d1/d3/d5/ff 0 2024-03-22T07:08:50.560 INFO:tasks.workunit.client.1.smithi149.stdout:9/288: symlink db/d22/l57 0 2024-03-22T07:08:50.562 INFO:tasks.workunit.client.1.smithi149.stdout:4/231: unlink d0/f1a 0 2024-03-22T07:08:50.563 INFO:tasks.workunit.client.1.smithi149.stdout:4/232: readlink d0/d29/l3f 0 2024-03-22T07:08:50.563 INFO:tasks.workunit.client.1.smithi149.stdout:7/311: getdents d1/d1c/d2f 0 2024-03-22T07:08:50.564 INFO:tasks.workunit.client.1.smithi149.stdout:2/272: mkdir d1/d8/d34/d61 0 2024-03-22T07:08:50.564 INFO:tasks.workunit.client.1.smithi149.stdout:2/273: chown d1/d3/d5/d1f/f1e 7381 1 2024-03-22T07:08:50.566 INFO:tasks.workunit.client.1.smithi149.stdout:7/312: dread d1/f21 [0,4194304] 0 2024-03-22T07:08:50.566 INFO:tasks.workunit.client.1.smithi149.stdout:9/289: rmdir db/d42/d49/d4b/d4d/d52 0 2024-03-22T07:08:50.567 INFO:tasks.workunit.client.1.smithi149.stdout:6/191: getdents d9/d1f 0 2024-03-22T07:08:50.568 INFO:tasks.workunit.client.1.smithi149.stdout:7/313: mkdir d1/d18/d1b/d51/d5a/d6a 0 2024-03-22T07:08:50.568 INFO:tasks.workunit.client.1.smithi149.stdout:7/314: write d1/d18/d1b/d51/f52 [575131,51128] 0 2024-03-22T07:08:50.569 INFO:tasks.workunit.client.1.smithi149.stdout:9/290: mknod db/d21/d37/d3a/c58 0 2024-03-22T07:08:50.569 INFO:tasks.workunit.client.1.smithi149.stdout:9/291: readlink db/l27 0 2024-03-22T07:08:50.569 INFO:tasks.workunit.client.1.smithi149.stdout:9/292: dread - db/d19/d1c/f4f zero size 2024-03-22T07:08:50.570 INFO:tasks.workunit.client.1.smithi149.stdout:9/293: creat db/d19/d39/f59 x:0 0 0 2024-03-22T07:08:50.572 INFO:tasks.workunit.client.1.smithi149.stdout:9/294: creat db/d42/f5a x:0 0 0 2024-03-22T07:08:50.573 INFO:tasks.workunit.client.1.smithi149.stdout:9/295: write db/fc [290990,98663] 0 2024-03-22T07:08:50.580 INFO:tasks.workunit.client.1.smithi149.stdout:0/194: rmdir d1/d1a 39 2024-03-22T07:08:50.581 INFO:tasks.workunit.client.1.smithi149.stdout:6/192: dread d9/df/f1d [0,4194304] 0 2024-03-22T07:08:50.583 INFO:tasks.workunit.client.1.smithi149.stdout:0/195: creat d1/d20/f3d x:0 0 0 2024-03-22T07:08:50.583 INFO:tasks.workunit.client.1.smithi149.stdout:6/193: getdents d9/d1f/d24/d34/d37/d38 0 2024-03-22T07:08:50.583 INFO:tasks.workunit.client.1.smithi149.stdout:6/194: chown d9/df/f1e 161 1 2024-03-22T07:08:50.583 INFO:tasks.workunit.client.1.smithi149.stdout:6/195: write d9/f19 [120567,128041] 0 2024-03-22T07:08:50.585 INFO:tasks.workunit.client.1.smithi149.stdout:0/196: mknod d1/d9/d14/c3e 0 2024-03-22T07:08:50.585 INFO:tasks.workunit.client.1.smithi149.stdout:7/315: dread d1/d18/f1e [0,4194304] 0 2024-03-22T07:08:50.587 INFO:tasks.workunit.client.1.smithi149.stdout:7/316: creat d1/d18/d1b/d1d/d2a/d5d/f6b x:0 0 0 2024-03-22T07:08:50.591 INFO:tasks.workunit.client.1.smithi149.stdout:2/274: dwrite d1/d3/d5/f3a [0,4194304] 0 2024-03-22T07:08:50.596 INFO:tasks.workunit.client.1.smithi149.stdout:4/233: dwrite d0/d8/dc/d4b/f4d [0,4194304] 0 2024-03-22T07:08:50.600 INFO:tasks.workunit.client.1.smithi149.stdout:4/234: unlink d0/d8/dc/c45 0 2024-03-22T07:08:50.602 INFO:tasks.workunit.client.1.smithi149.stdout:4/235: creat d0/d8/dc/d14/d3e/f51 x:0 0 0 2024-03-22T07:08:50.605 INFO:tasks.workunit.client.1.smithi149.stdout:4/236: write d0/d29/f3c [616445,69346] 0 2024-03-22T07:08:50.605 INFO:tasks.workunit.client.1.smithi149.stdout:4/237: chown d0/f32 1072 1 2024-03-22T07:08:50.606 INFO:tasks.workunit.client.1.smithi149.stdout:4/238: truncate d0/d8/dc/d14/f2b 2736182 0 2024-03-22T07:08:50.607 INFO:tasks.workunit.client.1.smithi149.stdout:4/239: rename d0/c26 to d0/d8/d1d/c52 0 2024-03-22T07:08:50.607 INFO:tasks.workunit.client.1.smithi149.stdout:9/296: dwrite db/d19/d1c/f4f [0,4194304] 0 2024-03-22T07:08:50.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/240: mknod d0/d8/dc/d2e/c53 0 2024-03-22T07:08:50.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/241: dread - d0/d8/dc/d14/f43 zero size 2024-03-22T07:08:50.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/242: fdatasync d0/f3 0 2024-03-22T07:08:50.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/243: chown d0/d29/f31 29 1 2024-03-22T07:08:50.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/244: chown d0/d8/d1d/c52 17818473 1 2024-03-22T07:08:50.620 INFO:tasks.workunit.client.1.smithi149.stdout:4/245: dread d0/d29/f4a [0,4194304] 0 2024-03-22T07:08:50.620 INFO:tasks.workunit.client.1.smithi149.stdout:4/246: stat d0/d8/d4c 0 2024-03-22T07:08:50.621 INFO:tasks.workunit.client.1.smithi149.stdout:6/196: dwrite d9/f2c [0,4194304] 0 2024-03-22T07:08:50.621 INFO:tasks.workunit.client.1.smithi149.stdout:6/197: write d9/df/f1e [1396329,35450] 0 2024-03-22T07:08:50.621 INFO:tasks.workunit.client.1.smithi149.stdout:6/198: read - d9/d1f/f21 zero size 2024-03-22T07:08:50.621 INFO:tasks.workunit.client.1.smithi149.stdout:6/199: write d9/d1f/f21 [598096,24920] 0 2024-03-22T07:08:50.621 INFO:tasks.workunit.client.1.smithi149.stdout:0/197: dwrite d1/d9/f3a [0,4194304] 0 2024-03-22T07:08:50.621 INFO:tasks.workunit.client.1.smithi149.stdout:0/198: creat d1/d31/f3f x:0 0 0 2024-03-22T07:08:50.622 INFO:tasks.workunit.client.1.smithi149.stdout:4/247: creat d0/d8/d1d/d24/f54 x:0 0 0 2024-03-22T07:08:50.622 INFO:tasks.workunit.client.1.smithi149.stdout:4/248: fsync d0/d8/dc/d4b/f4d 0 2024-03-22T07:08:50.624 INFO:tasks.workunit.client.1.smithi149.stdout:7/317: dwrite d1/f33 [0,4194304] 0 2024-03-22T07:08:50.625 INFO:tasks.workunit.client.1.smithi149.stdout:6/200: creat d9/d1f/d24/d34/d37/f3e x:0 0 0 2024-03-22T07:08:50.628 INFO:tasks.workunit.client.1.smithi149.stdout:4/249: dread d0/d8/dc/f13 [0,4194304] 0 2024-03-22T07:08:50.628 INFO:tasks.workunit.client.1.smithi149.stdout:7/318: symlink d1/d1c/d4d/d56/l6c 0 2024-03-22T07:08:50.629 INFO:tasks.workunit.client.1.smithi149.stdout:7/319: truncate d1/f10 343011 0 2024-03-22T07:08:50.629 INFO:tasks.workunit.client.1.smithi149.stdout:7/320: write d1/d1c/d4d/f58 [4757553,77635] 0 2024-03-22T07:08:50.631 INFO:tasks.workunit.client.1.smithi149.stdout:7/321: symlink d1/d18/d30/l6d 0 2024-03-22T07:08:50.632 INFO:tasks.workunit.client.1.smithi149.stdout:7/322: truncate d1/d1c/d49/f5e 190509 0 2024-03-22T07:08:50.632 INFO:tasks.workunit.client.1.smithi149.stdout:7/323: chown d1/d18/d1b/d1d/f40 3743 1 2024-03-22T07:08:50.633 INFO:tasks.workunit.client.1.smithi149.stdout:9/297: dwrite db/d19/d1c/f20 [0,4194304] 0 2024-03-22T07:08:50.633 INFO:tasks.workunit.client.1.smithi149.stdout:9/298: stat db/d19/d1c/f4f 0 2024-03-22T07:08:50.634 INFO:tasks.workunit.client.1.smithi149.stdout:4/250: write d0/d8/dc/f13 [1740442,68791] 0 2024-03-22T07:08:50.634 INFO:tasks.workunit.client.1.smithi149.stdout:4/251: dread - d0/d8/dc/d14/f43 zero size 2024-03-22T07:08:50.634 INFO:tasks.workunit.client.1.smithi149.stdout:4/252: truncate d0/d29/f49 343334 0 2024-03-22T07:08:50.634 INFO:tasks.workunit.client.1.smithi149.stdout:7/324: rename d1/d18/d1b/d1d/d2a/f3d to d1/d18/d1b/d51/d5a/d6a/f6e 0 2024-03-22T07:08:50.638 INFO:tasks.workunit.client.1.smithi149.stdout:6/201: write d9/f2c [1314264,48585] 0 2024-03-22T07:08:50.643 INFO:tasks.workunit.client.1.smithi149.stdout:6/202: mknod d9/df/c3f 0 2024-03-22T07:08:50.643 INFO:tasks.workunit.client.1.smithi149.stdout:9/299: read db/d12/f35 [2695921,57585] 0 2024-03-22T07:08:50.643 INFO:tasks.workunit.client.1.smithi149.stdout:9/300: truncate db/d19/d1c/f4f 4546133 0 2024-03-22T07:08:50.644 INFO:tasks.workunit.client.1.smithi149.stdout:4/253: getdents d0/d29 0 2024-03-22T07:08:50.651 INFO:tasks.workunit.client.1.smithi149.stdout:7/325: rmdir d1/d18/d1b/d1d 39 2024-03-22T07:08:50.651 INFO:tasks.workunit.client.1.smithi149.stdout:6/203: rename d9/df/l18 to d9/df/l40 0 2024-03-22T07:08:50.653 INFO:tasks.workunit.client.1.smithi149.stdout:6/204: creat d9/d1f/f41 x:0 0 0 2024-03-22T07:08:50.653 INFO:tasks.workunit.client.1.smithi149.stdout:7/326: dread d1/d18/d1b/d1d/f38 [4194304,4194304] 0 2024-03-22T07:08:50.653 INFO:tasks.workunit.client.1.smithi149.stdout:7/327: dread - d1/d18/f34 zero size 2024-03-22T07:08:50.655 INFO:tasks.workunit.client.1.smithi149.stdout:6/205: dread d9/df/d2e/f27 [0,4194304] 0 2024-03-22T07:08:50.656 INFO:tasks.workunit.client.1.smithi149.stdout:7/328: mkdir d1/d18/d1b/d1d/d2a/d6f 0 2024-03-22T07:08:50.656 INFO:tasks.workunit.client.1.smithi149.stdout:7/329: rename d1/d18/d1b/d1d/d28/d53/d5c to d1/d18/d1b/d1d/d28/d53/d5c/d70 22 2024-03-22T07:08:50.657 INFO:tasks.workunit.client.1.smithi149.stdout:6/206: mkdir d9/d1f/d24/d42 0 2024-03-22T07:08:50.657 INFO:tasks.workunit.client.1.smithi149.stdout:6/207: chown d9/df/f15 0 1 2024-03-22T07:08:50.660 INFO:tasks.workunit.client.1.smithi149.stdout:6/208: rename d9/d1f/d24/f26 to d9/d1f/d24/d34/d37/d38/f43 0 2024-03-22T07:08:50.660 INFO:tasks.workunit.client.1.smithi149.stdout:6/209: fdatasync d9/d1f/d24/d34/d37/f3e 0 2024-03-22T07:08:50.660 INFO:tasks.workunit.client.1.smithi149.stdout:6/210: fsync d9/df/d2e/f23 0 2024-03-22T07:08:50.660 INFO:tasks.workunit.client.1.smithi149.stdout:6/211: readlink d9/df/d22/l28 0 2024-03-22T07:08:50.661 INFO:tasks.workunit.client.1.smithi149.stdout:6/212: symlink d9/d1f/d24/d34/d37/l44 0 2024-03-22T07:08:50.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/213: mkdir d9/d1f/d24/d34/d45 0 2024-03-22T07:08:50.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/214: dread - d9/f32 zero size 2024-03-22T07:08:50.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/215: chown d9/df/f1d 1 1 2024-03-22T07:08:50.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/216: creat d9/d1f/d24/d34/d37/f46 x:0 0 0 2024-03-22T07:08:50.664 INFO:tasks.workunit.client.1.smithi149.stdout:6/217: mkdir d9/d47 0 2024-03-22T07:08:50.664 INFO:tasks.workunit.client.1.smithi149.stdout:6/218: stat d9/f3c 0 2024-03-22T07:08:50.664 INFO:tasks.workunit.client.1.smithi149.stdout:6/219: write d9/df/d2e/f1c [24587,57951] 0 2024-03-22T07:08:50.664 INFO:tasks.workunit.client.1.smithi149.stdout:6/220: getdents d9/d1f/d24/d34/d45 0 2024-03-22T07:08:50.664 INFO:tasks.workunit.client.1.smithi149.stdout:6/221: write d9/df/d2e/f23 [422626,22269] 0 2024-03-22T07:08:50.669 INFO:tasks.workunit.client.1.smithi149.stdout:7/330: write d1/d18/d1b/f42 [3731722,24266] 0 2024-03-22T07:08:50.669 INFO:tasks.workunit.client.1.smithi149.stdout:7/331: chown d1/d18/d1b/d1d/d2a/d5d/f6b 930859620 1 2024-03-22T07:08:50.672 INFO:tasks.workunit.client.1.smithi149.stdout:9/301: dwrite db/f2f [0,4194304] 0 2024-03-22T07:08:50.673 INFO:tasks.workunit.client.1.smithi149.stdout:4/254: dwrite d0/d8/dc/d2e/f2f [0,4194304] 0 2024-03-22T07:08:50.674 INFO:tasks.workunit.client.1.smithi149.stdout:4/255: dread d0/d8/dc/d14/f16 [0,4194304] 0 2024-03-22T07:08:50.674 INFO:tasks.workunit.client.1.smithi149.stdout:4/256: write d0/d29/f31 [1505660,56337] 0 2024-03-22T07:08:50.675 INFO:tasks.workunit.client.1.smithi149.stdout:9/302: getdents db/d21/d37/d3a 0 2024-03-22T07:08:50.678 INFO:tasks.workunit.client.1.smithi149.stdout:9/303: dread db/d21/d37/f4e [0,4194304] 0 2024-03-22T07:08:50.678 INFO:tasks.workunit.client.1.smithi149.stdout:9/304: chown db/d19/d1c/l56 8055336 1 2024-03-22T07:08:50.678 INFO:tasks.workunit.client.1.smithi149.stdout:4/257: mkdir d0/d29/d55 0 2024-03-22T07:08:50.679 INFO:tasks.workunit.client.1.smithi149.stdout:9/305: symlink db/d42/d49/l5b 0 2024-03-22T07:08:50.680 INFO:tasks.workunit.client.1.smithi149.stdout:4/258: truncate d0/d8/dc/f18 4412454 0 2024-03-22T07:08:50.683 INFO:tasks.workunit.client.1.smithi149.stdout:7/332: dread d1/d18/d1b/f42 [0,4194304] 0 2024-03-22T07:08:50.683 INFO:tasks.workunit.client.1.smithi149.stdout:6/222: dwrite f5 [0,4194304] 0 2024-03-22T07:08:50.691 INFO:tasks.workunit.client.1.smithi149.stdout:6/223: rmdir d9/df 39 2024-03-22T07:08:50.692 INFO:tasks.workunit.client.1.smithi149.stdout:6/224: chown d9/df/d2e/f20 7983166 1 2024-03-22T07:08:50.692 INFO:tasks.workunit.client.1.smithi149.stdout:6/225: chown d9/f32 14765890 1 2024-03-22T07:08:50.692 INFO:tasks.workunit.client.1.smithi149.stdout:6/226: creat d9/d1f/f48 x:0 0 0 2024-03-22T07:08:50.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/333: write d1/f33 [4077092,81473] 0 2024-03-22T07:08:50.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/334: chown d1/d1c 13 1 2024-03-22T07:08:50.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/335: chown d1/d1c/l31 4 1 2024-03-22T07:08:50.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/336: write d1/d18/d1b/d1d/f13 [468076,130851] 0 2024-03-22T07:08:50.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/337: write d1/f41 [340391,80426] 0 2024-03-22T07:08:50.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/338: write d1/d18/d1b/f2c [1315315,51490] 0 2024-03-22T07:08:50.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/339: fdatasync d1/d1c/d4d/f3e 0 2024-03-22T07:08:50.696 INFO:tasks.workunit.client.1.smithi149.stdout:7/340: symlink d1/d1c/d2f/l71 0 2024-03-22T07:08:50.700 INFO:tasks.workunit.client.1.smithi149.stdout:4/259: link d0/d8/dc/f18 d0/d8/f56 0 2024-03-22T07:08:50.700 INFO:tasks.workunit.client.1.smithi149.stdout:4/260: write d0/d8/dc/d14/f16 [871967,103413] 0 2024-03-22T07:08:50.700 INFO:tasks.workunit.client.1.smithi149.stdout:7/341: mkdir d1/d18/d1b/d1d/d72 0 2024-03-22T07:08:50.702 INFO:tasks.workunit.client.1.smithi149.stdout:4/261: creat d0/d8/d4c/f57 x:0 0 0 2024-03-22T07:08:50.702 INFO:tasks.workunit.client.1.smithi149.stdout:6/227: rmdir d9 39 2024-03-22T07:08:50.704 INFO:tasks.workunit.client.1.smithi149.stdout:6/228: creat d9/d1f/d24/d34/d37/f49 x:0 0 0 2024-03-22T07:08:50.705 INFO:tasks.workunit.client.1.smithi149.stdout:6/229: write f5 [4576806,6196] 0 2024-03-22T07:08:50.705 INFO:tasks.workunit.client.1.smithi149.stdout:6/230: fsync f8 0 2024-03-22T07:08:50.705 INFO:tasks.workunit.client.1.smithi149.stdout:6/231: chown d9/d1f/f25 44958839 1 2024-03-22T07:08:50.708 INFO:tasks.workunit.client.1.smithi149.stdout:4/262: dread d0/d8/dc/f30 [0,4194304] 0 2024-03-22T07:08:50.710 INFO:tasks.workunit.client.1.smithi149.stdout:4/263: creat d0/d29/f58 x:0 0 0 2024-03-22T07:08:50.710 INFO:tasks.workunit.client.1.smithi149.stdout:4/264: chown d0/d8/d1d/d24/l27 11524426 1 2024-03-22T07:08:50.710 INFO:tasks.workunit.client.1.smithi149.stdout:7/342: dread d1/d1c/d4d/f58 [0,4194304] 0 2024-03-22T07:08:50.711 INFO:tasks.workunit.client.1.smithi149.stdout:7/343: rmdir d1/d18/d1b/d51/d5a 39 2024-03-22T07:08:50.717 INFO:tasks.workunit.client.1.smithi149.stdout:7/344: write d1/d18/d1b/d51/f52 [306086,85689] 0 2024-03-22T07:08:50.718 INFO:tasks.workunit.client.1.smithi149.stdout:7/345: creat d1/d1c/f73 x:0 0 0 2024-03-22T07:08:50.723 INFO:tasks.workunit.client.1.smithi149.stdout:7/346: dread d1/d18/d1b/f42 [0,4194304] 0 2024-03-22T07:08:50.724 INFO:tasks.workunit.client.1.smithi149.stdout:7/347: mkdir d1/d1c/d2f/d74 0 2024-03-22T07:08:50.727 INFO:tasks.workunit.client.1.smithi149.stdout:6/232: dwrite d9/f17 [0,4194304] 0 2024-03-22T07:08:50.728 INFO:tasks.workunit.client.1.smithi149.stdout:4/265: dwrite d0/d8/d1d/f28 [0,4194304] 0 2024-03-22T07:08:50.729 INFO:tasks.workunit.client.1.smithi149.stdout:6/233: mknod d9/d47/c4a 0 2024-03-22T07:08:50.729 INFO:tasks.workunit.client.1.smithi149.stdout:6/234: write d9/df/f1d [406235,65760] 0 2024-03-22T07:08:50.732 INFO:tasks.workunit.client.1.smithi149.stdout:7/348: rmdir d1/d18/d1b 39 2024-03-22T07:08:50.734 INFO:tasks.workunit.client.1.smithi149.stdout:6/235: creat d9/df/d22/f4b x:0 0 0 2024-03-22T07:08:50.734 INFO:tasks.workunit.client.1.smithi149.stdout:6/236: read - d9/d1f/d24/d34/d37/f46 zero size 2024-03-22T07:08:50.737 INFO:tasks.workunit.client.1.smithi149.stdout:7/349: dread d1/d18/d1b/d1d/f38 [0,4194304] 0 2024-03-22T07:08:50.738 INFO:tasks.workunit.client.1.smithi149.stdout:7/350: creat d1/d1c/d4d/f75 x:0 0 0 2024-03-22T07:08:50.739 INFO:tasks.workunit.client.1.smithi149.stdout:4/266: write d0/d8/f56 [2812982,56197] 0 2024-03-22T07:08:50.740 INFO:tasks.workunit.client.1.smithi149.stdout:4/267: dread d0/d29/f49 [0,4194304] 0 2024-03-22T07:08:50.742 INFO:tasks.workunit.client.1.smithi149.stdout:7/351: rename d1/d1c/d4d/f48 to d1/d1c/d2f/f76 0 2024-03-22T07:08:50.742 INFO:tasks.workunit.client.1.smithi149.stdout:7/352: readlink d1/d18/d1b/d1d/d28/l65 0 2024-03-22T07:08:50.743 INFO:tasks.workunit.client.1.smithi149.stdout:4/268: symlink d0/d8/dc/d4b/l59 0 2024-03-22T07:08:50.743 INFO:tasks.workunit.client.1.smithi149.stdout:4/269: read - d0/d8/d1d/d24/f54 zero size 2024-03-22T07:08:50.746 INFO:tasks.workunit.client.1.smithi149.stdout:7/353: creat d1/d18/d1b/d1d/d2a/f77 x:0 0 0 2024-03-22T07:08:50.746 INFO:tasks.workunit.client.1.smithi149.stdout:1/288: sync 2024-03-22T07:08:50.746 INFO:tasks.workunit.client.1.smithi149.stdout:3/205: sync 2024-03-22T07:08:50.748 INFO:tasks.workunit.client.1.smithi149.stdout:7/354: creat d1/d18/d1b/d51/d5a/f78 x:0 0 0 2024-03-22T07:08:50.749 INFO:tasks.workunit.client.1.smithi149.stdout:1/289: link d4/d7/fe d4/d10/d28/f65 0 2024-03-22T07:08:50.749 INFO:tasks.workunit.client.1.smithi149.stdout:1/290: readlink d4/d20/d2d/l38 0 2024-03-22T07:08:50.751 INFO:tasks.workunit.client.1.smithi149.stdout:7/355: rmdir d1/d18/d1b/d1d/d2a/d6f 0 2024-03-22T07:08:50.758 INFO:tasks.workunit.client.1.smithi149.stdout:6/237: dwrite d9/d1f/f2a [0,4194304] 0 2024-03-22T07:08:50.761 INFO:tasks.workunit.client.1.smithi149.stdout:4/270: write d0/d8/dc/fe [2975219,50587] 0 2024-03-22T07:08:50.764 INFO:tasks.workunit.client.1.smithi149.stdout:6/238: creat d9/d1f/d24/d34/f4c x:0 0 0 2024-03-22T07:08:50.779 INFO:tasks.workunit.client.1.smithi149.stdout:6/239: dread f6 [4194304,4194304] 0 2024-03-22T07:08:50.779 INFO:tasks.workunit.client.1.smithi149.stdout:3/206: dwrite d0/f1a [0,4194304] 0 2024-03-22T07:08:50.779 INFO:tasks.workunit.client.1.smithi149.stdout:3/207: readlink d0/d2/d30/d31/l3b 0 2024-03-22T07:08:50.779 INFO:tasks.workunit.client.1.smithi149.stdout:6/240: readlink d9/d1f/d24/d34/d37/d38/l39 0 2024-03-22T07:08:50.779 INFO:tasks.workunit.client.1.smithi149.stdout:3/208: stat d0/d4/d8/lb 0 2024-03-22T07:08:50.781 INFO:tasks.workunit.client.1.smithi149.stdout:6/241: unlink c3 0 2024-03-22T07:08:50.781 INFO:tasks.workunit.client.1.smithi149.stdout:6/242: fsync f5 0 2024-03-22T07:08:50.781 INFO:tasks.workunit.client.1.smithi149.stdout:6/243: truncate d9/df/d22/f4b 285995 0 2024-03-22T07:08:50.781 INFO:tasks.workunit.client.1.smithi149.stdout:6/244: readlink d9/l36 0 2024-03-22T07:08:50.781 INFO:tasks.workunit.client.1.smithi149.stdout:5/247: sync 2024-03-22T07:08:50.781 INFO:tasks.workunit.client.1.smithi149.stdout:5/248: write d7/da/d20/d36/f41 [320969,74578] 0 2024-03-22T07:08:50.781 INFO:tasks.workunit.client.1.smithi149.stdout:3/209: mknod d0/d4/d8/d24/d40/c4c 0 2024-03-22T07:08:50.782 INFO:tasks.workunit.client.1.smithi149.stdout:1/291: dwrite d4/d7/f39 [0,4194304] 0 2024-03-22T07:08:50.782 INFO:tasks.workunit.client.1.smithi149.stdout:7/356: dwrite d1/f10 [0,4194304] 0 2024-03-22T07:08:50.783 INFO:tasks.workunit.client.1.smithi149.stdout:4/271: dwrite d0/d8/dc/f30 [4194304,4194304] 0 2024-03-22T07:08:50.783 INFO:tasks.workunit.client.1.smithi149.stdout:4/272: chown d0/d29/f3c 476582 1 2024-03-22T07:08:50.783 INFO:tasks.workunit.client.1.smithi149.stdout:4/273: chown d0/d8/d1d/c52 0 1 2024-03-22T07:08:50.783 INFO:tasks.workunit.client.1.smithi149.stdout:4/274: readlink d0/d8/dc/d14/l47 0 2024-03-22T07:08:50.783 INFO:tasks.workunit.client.1.smithi149.stdout:5/249: unlink d7/db/c25 0 2024-03-22T07:08:50.784 INFO:tasks.workunit.client.1.smithi149.stdout:3/210: dread d0/d2/d30/f46 [0,4194304] 0 2024-03-22T07:08:50.784 INFO:tasks.workunit.client.1.smithi149.stdout:3/211: creat d0/d2/f4d x:0 0 0 2024-03-22T07:08:50.786 INFO:tasks.workunit.client.1.smithi149.stdout:1/292: symlink d4/d20/d2d/l66 0 2024-03-22T07:08:50.786 INFO:tasks.workunit.client.1.smithi149.stdout:1/293: read d4/f1b [324809,25842] 0 2024-03-22T07:08:50.791 INFO:tasks.workunit.client.1.smithi149.stdout:6/245: write d9/df/d2e/f1b [3710298,59521] 0 2024-03-22T07:08:50.791 INFO:tasks.workunit.client.1.smithi149.stdout:6/246: chown d9/d1f/d24/d34/f4c 52710 1 2024-03-22T07:08:50.792 INFO:tasks.workunit.client.1.smithi149.stdout:6/247: fdatasync d9/d1f/d24/d34/d37/f46 0 2024-03-22T07:08:50.796 INFO:tasks.workunit.client.1.smithi149.stdout:4/275: rename d0/d8/dc/d14/f2b to d0/f5a 0 2024-03-22T07:08:50.798 INFO:tasks.workunit.client.1.smithi149.stdout:1/294: write d4/d20/d23/f27 [743404,21165] 0 2024-03-22T07:08:50.798 INFO:tasks.workunit.client.1.smithi149.stdout:1/295: read d4/d20/d23/d2f/f35 [1661715,97766] 0 2024-03-22T07:08:50.802 INFO:tasks.workunit.client.1.smithi149.stdout:5/250: rename d7/da/d20/d30/d32 to d7/da/d20/d36/d44/d4a 0 2024-03-22T07:08:50.805 INFO:tasks.workunit.client.1.smithi149.stdout:1/296: link d4/c43 d4/d20/d2d/d64/c67 0 2024-03-22T07:08:50.805 INFO:tasks.workunit.client.1.smithi149.stdout:1/297: write d4/d7/d5a/f5e [902689,62773] 0 2024-03-22T07:08:50.805 INFO:tasks.workunit.client.1.smithi149.stdout:1/298: dread - d4/d10/f4d zero size 2024-03-22T07:08:50.806 INFO:tasks.workunit.client.1.smithi149.stdout:4/276: getdents d0/d8 0 2024-03-22T07:08:50.806 INFO:tasks.workunit.client.1.smithi149.stdout:4/277: creat d0/d8/dc/f5b x:0 0 0 2024-03-22T07:08:50.806 INFO:tasks.workunit.client.1.smithi149.stdout:4/278: chown d0/d8/dc/f30 3902317 1 2024-03-22T07:08:50.807 INFO:tasks.workunit.client.1.smithi149.stdout:0/199: sync 2024-03-22T07:08:50.807 INFO:tasks.workunit.client.1.smithi149.stdout:9/306: sync 2024-03-22T07:08:50.807 INFO:tasks.workunit.client.1.smithi149.stdout:8/306: sync 2024-03-22T07:08:50.807 INFO:tasks.workunit.client.1.smithi149.stdout:2/275: sync 2024-03-22T07:08:50.807 INFO:tasks.workunit.client.1.smithi149.stdout:0/200: fsync d1/d31/f3f 0 2024-03-22T07:08:50.807 INFO:tasks.workunit.client.1.smithi149.stdout:9/307: stat db/d42 0 2024-03-22T07:08:50.807 INFO:tasks.workunit.client.1.smithi149.stdout:0/201: chown d1/d1a/f2d 959 1 2024-03-22T07:08:50.808 INFO:tasks.workunit.client.1.smithi149.stdout:4/279: mkdir d0/d8/d5c 0 2024-03-22T07:08:50.808 INFO:tasks.workunit.client.1.smithi149.stdout:2/276: mknod d1/d3/d5/d41/c62 0 2024-03-22T07:08:50.808 INFO:tasks.workunit.client.1.smithi149.stdout:9/308: write db/d19/d39/d3b/f53 [710173,84248] 0 2024-03-22T07:08:50.809 INFO:tasks.workunit.client.1.smithi149.stdout:8/307: symlink d1/d2/d9/df/l6b 0 2024-03-22T07:08:50.810 INFO:tasks.workunit.client.1.smithi149.stdout:0/202: symlink d1/d27/l40 0 2024-03-22T07:08:50.812 INFO:tasks.workunit.client.1.smithi149.stdout:2/277: unlink d1/d3/d6/f16 0 2024-03-22T07:08:50.813 INFO:tasks.workunit.client.1.smithi149.stdout:9/309: rmdir db/d42/d49 39 2024-03-22T07:08:50.814 INFO:tasks.workunit.client.1.smithi149.stdout:8/308: symlink d1/d20/l6c 0 2024-03-22T07:08:50.815 INFO:tasks.workunit.client.1.smithi149.stdout:0/203: mknod d1/d1a/c41 0 2024-03-22T07:08:50.817 INFO:tasks.workunit.client.1.smithi149.stdout:2/278: unlink d1/d3/d5/f3a 0 2024-03-22T07:08:50.817 INFO:tasks.workunit.client.1.smithi149.stdout:2/279: chown d1/d3/d6/d3f 53923 1 2024-03-22T07:08:50.818 INFO:tasks.workunit.client.1.smithi149.stdout:2/280: truncate d1/d3/d5/d1f/f1b 218291 0 2024-03-22T07:08:50.818 INFO:tasks.workunit.client.1.smithi149.stdout:2/281: write d1/d3/d5/d41/f48 [28641,34764] 0 2024-03-22T07:08:50.818 INFO:tasks.workunit.client.1.smithi149.stdout:2/282: chown d1/d8/l53 3594211 1 2024-03-22T07:08:50.818 INFO:tasks.workunit.client.1.smithi149.stdout:2/283: creat d1/d8/d34/f63 x:0 0 0 2024-03-22T07:08:50.820 INFO:tasks.workunit.client.1.smithi149.stdout:9/310: creat db/d21/d37/f5c x:0 0 0 2024-03-22T07:08:50.821 INFO:tasks.workunit.client.1.smithi149.stdout:5/251: read d7/da/f37 [2121255,61366] 0 2024-03-22T07:08:50.821 INFO:tasks.workunit.client.1.smithi149.stdout:9/311: dread db/d19/d39/d3b/f48 [0,4194304] 0 2024-03-22T07:08:50.821 INFO:tasks.workunit.client.1.smithi149.stdout:9/312: chown db/d12/l32 3 1 2024-03-22T07:08:50.822 INFO:tasks.workunit.client.1.smithi149.stdout:5/252: write d7/da/fd [304410,92173] 0 2024-03-22T07:08:50.824 INFO:tasks.workunit.client.1.smithi149.stdout:2/284: symlink d1/d8/d34/d50/l64 0 2024-03-22T07:08:50.824 INFO:tasks.workunit.client.1.smithi149.stdout:2/285: write d1/d8/d34/f57 [985203,125192] 0 2024-03-22T07:08:50.825 INFO:tasks.workunit.client.1.smithi149.stdout:8/309: write d1/f24 [2202429,19804] 0 2024-03-22T07:08:50.825 INFO:tasks.workunit.client.1.smithi149.stdout:8/310: dread - d1/d2/f23 zero size 2024-03-22T07:08:50.825 INFO:tasks.workunit.client.1.smithi149.stdout:0/204: mknod d1/c42 0 2024-03-22T07:08:50.826 INFO:tasks.workunit.client.1.smithi149.stdout:9/313: mknod db/d19/d28/c5d 0 2024-03-22T07:08:50.826 INFO:tasks.workunit.client.1.smithi149.stdout:9/314: creat db/d19/d39/f5e x:0 0 0 2024-03-22T07:08:50.828 INFO:tasks.workunit.client.1.smithi149.stdout:2/286: rmdir d1/d8/d34/d61 0 2024-03-22T07:08:50.828 INFO:tasks.workunit.client.1.smithi149.stdout:2/287: fdatasync d1/d3/d5/f5a 0 2024-03-22T07:08:50.829 INFO:tasks.workunit.client.1.smithi149.stdout:2/288: fdatasync d1/d8/f19 0 2024-03-22T07:08:50.829 INFO:tasks.workunit.client.1.smithi149.stdout:2/289: chown d1/d3/d5/d1f/f55 7583 1 2024-03-22T07:08:50.829 INFO:tasks.workunit.client.1.smithi149.stdout:2/290: write d1/d3/dd/f45 [509335,62938] 0 2024-03-22T07:08:50.829 INFO:tasks.workunit.client.1.smithi149.stdout:2/291: readlink d1/d3/d6/l52 0 2024-03-22T07:08:50.830 INFO:tasks.workunit.client.1.smithi149.stdout:8/311: truncate d1/d2/d3/f1a 2532491 0 2024-03-22T07:08:50.833 INFO:tasks.workunit.client.1.smithi149.stdout:0/205: link d1/d9/d14/d1f/f2a d1/d31/d36/f43 0 2024-03-22T07:08:50.837 INFO:tasks.workunit.client.1.smithi149.stdout:3/212: dwrite d0/d4/d8/f1d [0,4194304] 0 2024-03-22T07:08:50.838 INFO:tasks.workunit.client.1.smithi149.stdout:1/299: dwrite d4/d20/f52 [0,4194304] 0 2024-03-22T07:08:50.838 INFO:tasks.workunit.client.1.smithi149.stdout:5/253: rmdir d7/da/d20 39 2024-03-22T07:08:50.839 INFO:tasks.workunit.client.1.smithi149.stdout:7/357: dwrite d1/d1c/d2f/f43 [0,4194304] 0 2024-03-22T07:08:50.839 INFO:tasks.workunit.client.1.smithi149.stdout:7/358: write d1/d1c/d2f/f76 [202103,57873] 0 2024-03-22T07:08:50.841 INFO:tasks.workunit.client.1.smithi149.stdout:8/312: rmdir d1/d2/d9/d11/d4d 39 2024-03-22T07:08:50.844 INFO:tasks.workunit.client.1.smithi149.stdout:0/206: unlink d1/d9/f2b 0 2024-03-22T07:08:50.848 INFO:tasks.workunit.client.1.smithi149.stdout:3/213: rename d0/d2/d36/f22 to d0/d2/d42/f4e 0 2024-03-22T07:08:50.849 INFO:tasks.workunit.client.1.smithi149.stdout:4/280: dwrite d0/d29/f3c [0,4194304] 0 2024-03-22T07:08:50.849 INFO:tasks.workunit.client.1.smithi149.stdout:6/248: dwrite d9/d1f/d24/d34/d37/f46 [0,4194304] 0 2024-03-22T07:08:50.850 INFO:tasks.workunit.client.1.smithi149.stdout:6/249: chown d9/df/d22/f31 38 1 2024-03-22T07:08:50.851 INFO:tasks.workunit.client.1.smithi149.stdout:9/315: dwrite f3 [4194304,4194304] 0 2024-03-22T07:08:50.853 INFO:tasks.workunit.client.1.smithi149.stdout:5/254: mkdir d7/da/d20/d30/d3a/d3d/d2c/d4b 0 2024-03-22T07:08:50.853 INFO:tasks.workunit.client.1.smithi149.stdout:5/255: truncate d7/db/fc 4847567 0 2024-03-22T07:08:50.856 INFO:tasks.workunit.client.1.smithi149.stdout:7/359: write d1/d18/f37 [1912646,105910] 0 2024-03-22T07:08:50.856 INFO:tasks.workunit.client.1.smithi149.stdout:7/360: fdatasync d1/d18/d1b/d1d/d2a/f3b 0 2024-03-22T07:08:50.856 INFO:tasks.workunit.client.1.smithi149.stdout:7/361: fdatasync d1/f21 0 2024-03-22T07:08:50.858 INFO:tasks.workunit.client.1.smithi149.stdout:0/207: getdents d1/d9/d14 0 2024-03-22T07:08:50.858 INFO:tasks.workunit.client.1.smithi149.stdout:1/300: dread d4/d10/d28/f37 [0,4194304] 0 2024-03-22T07:08:50.858 INFO:tasks.workunit.client.1.smithi149.stdout:1/301: write d4/d10/f12 [557927,116225] 0 2024-03-22T07:08:50.858 INFO:tasks.workunit.client.1.smithi149.stdout:1/302: chown l3 117882909 1 2024-03-22T07:08:50.861 INFO:tasks.workunit.client.1.smithi149.stdout:4/281: rename d0/d8/dc/d14/c38 to d0/d8/dc/d2e/c5d 0 2024-03-22T07:08:50.863 INFO:tasks.workunit.client.1.smithi149.stdout:6/250: link d9/df/d22/f31 d9/df/d2e/f4d 0 2024-03-22T07:08:50.863 INFO:tasks.workunit.client.1.smithi149.stdout:6/251: readlink d9/l3a 0 2024-03-22T07:08:50.863 INFO:tasks.workunit.client.1.smithi149.stdout:2/292: dwrite d1/d8/f24 [0,4194304] 0 2024-03-22T07:08:50.866 INFO:tasks.workunit.client.1.smithi149.stdout:9/316: mkdir db/d42/d49/d4b/d5f 0 2024-03-22T07:08:50.871 INFO:tasks.workunit.client.1.smithi149.stdout:7/362: link d1/d1c/d49/f5e d1/d18/d1b/d1d/d2a/d5b/f79 0 2024-03-22T07:08:50.871 INFO:tasks.workunit.client.1.smithi149.stdout:7/363: readlink d1/d18/d1b/d1d/d2a/l61 0 2024-03-22T07:08:50.872 INFO:tasks.workunit.client.1.smithi149.stdout:8/313: mknod d1/d2/d9/d11/d4d/c6d 0 2024-03-22T07:08:50.874 INFO:tasks.workunit.client.1.smithi149.stdout:3/214: dwrite d0/d4/d8/f23 [0,4194304] 0 2024-03-22T07:08:50.874 INFO:tasks.workunit.client.1.smithi149.stdout:3/215: chown d0/d2/c35 161239 1 2024-03-22T07:08:50.874 INFO:tasks.workunit.client.1.smithi149.stdout:3/216: creat d0/d4/d8/d24/f4f x:0 0 0 2024-03-22T07:08:50.874 INFO:tasks.workunit.client.1.smithi149.stdout:0/208: mkdir d1/d31/d44 0 2024-03-22T07:08:50.874 INFO:tasks.workunit.client.1.smithi149.stdout:8/314: dread d1/d20/d41/f48 [0,4194304] 0 2024-03-22T07:08:50.875 INFO:tasks.workunit.client.1.smithi149.stdout:1/303: mkdir d4/d10/d58/d68 0 2024-03-22T07:08:50.878 INFO:tasks.workunit.client.1.smithi149.stdout:6/252: rename d9/d1f/d24/d34/d45 to d9/df/d2e/d4e 0 2024-03-22T07:08:50.879 INFO:tasks.workunit.client.1.smithi149.stdout:6/253: truncate d9/df/d22/f31 312229 0 2024-03-22T07:08:50.880 INFO:tasks.workunit.client.1.smithi149.stdout:2/293: dwrite d1/d3/dd/f21 [0,4194304] 0 2024-03-22T07:08:50.881 INFO:tasks.workunit.client.1.smithi149.stdout:9/317: symlink db/d21/l60 0 2024-03-22T07:08:50.885 INFO:tasks.workunit.client.1.smithi149.stdout:7/364: mkdir d1/d18/d1b/d51/d7a 0 2024-03-22T07:08:50.885 INFO:tasks.workunit.client.1.smithi149.stdout:7/365: fdatasync d1/d18/d1b/d1d/f26 0 2024-03-22T07:08:50.895 INFO:tasks.workunit.client.1.smithi149.stdout:6/254: dread d9/df/f15 [0,4194304] 0 2024-03-22T07:08:50.895 INFO:tasks.workunit.client.1.smithi149.stdout:3/217: stat d0/d2/d30/d31/c4b 0 2024-03-22T07:08:50.898 INFO:tasks.workunit.client.1.smithi149.stdout:2/294: dwrite d1/d3/d5/d1f/f55 [0,4194304] 0 2024-03-22T07:08:50.903 INFO:tasks.workunit.client.1.smithi149.stdout:0/209: creat d1/d31/d36/f45 x:0 0 0 2024-03-22T07:08:50.903 INFO:tasks.workunit.client.1.smithi149.stdout:0/210: chown d1/d31/d36/f45 94292501 1 2024-03-22T07:08:50.903 INFO:tasks.workunit.client.1.smithi149.stdout:1/304: rmdir d4/d20/d2d 39 2024-03-22T07:08:50.904 INFO:tasks.workunit.client.1.smithi149.stdout:4/282: rmdir d0/d8/d1d/d24 39 2024-03-22T07:08:50.906 INFO:tasks.workunit.client.1.smithi149.stdout:1/305: dread d4/d10/f30 [0,4194304] 0 2024-03-22T07:08:50.906 INFO:tasks.workunit.client.1.smithi149.stdout:7/366: dwrite d1/d1c/d2f/f76 [0,4194304] 0 2024-03-22T07:08:50.907 INFO:tasks.workunit.client.1.smithi149.stdout:9/318: link db/d21/d37/d3a/c41 db/d19/d39/c61 0 2024-03-22T07:08:50.909 INFO:tasks.workunit.client.1.smithi149.stdout:6/255: mkdir d9/d1f/d24/d34/d4f 0 2024-03-22T07:08:50.910 INFO:tasks.workunit.client.1.smithi149.stdout:3/218: creat d0/d2/d36/f50 x:0 0 0 2024-03-22T07:08:50.913 INFO:tasks.workunit.client.1.smithi149.stdout:3/219: dread d0/d2/d30/f46 [0,4194304] 0 2024-03-22T07:08:50.913 INFO:tasks.workunit.client.1.smithi149.stdout:3/220: stat d0/d2/d30/f46 0 2024-03-22T07:08:50.914 INFO:tasks.workunit.client.1.smithi149.stdout:2/295: creat d1/d3/dd/f65 x:0 0 0 2024-03-22T07:08:50.914 INFO:tasks.workunit.client.1.smithi149.stdout:2/296: write d1/d3/dd/f45 [1367253,53708] 0 2024-03-22T07:08:50.914 INFO:tasks.workunit.client.1.smithi149.stdout:2/297: readlink d1/d3/dd/l4f 0 2024-03-22T07:08:50.915 INFO:tasks.workunit.client.1.smithi149.stdout:6/256: dread f6 [4194304,4194304] 0 2024-03-22T07:08:50.915 INFO:tasks.workunit.client.1.smithi149.stdout:6/257: stat d9/df/d2e/f27 0 2024-03-22T07:08:50.919 INFO:tasks.workunit.client.1.smithi149.stdout:8/315: rmdir d1/d2 39 2024-03-22T07:08:50.919 INFO:tasks.workunit.client.1.smithi149.stdout:8/316: creat d1/d20/d35/d22/d44/d4f/f6e x:0 0 0 2024-03-22T07:08:50.923 INFO:tasks.workunit.client.1.smithi149.stdout:2/298: dread d1/f10 [4194304,4194304] 0 2024-03-22T07:08:50.923 INFO:tasks.workunit.client.1.smithi149.stdout:2/299: write d1/d3/d5/d1f/f1b [280441,45725] 0 2024-03-22T07:08:50.923 INFO:tasks.workunit.client.1.smithi149.stdout:9/319: link db/d21/d37/l54 db/d21/d2e/l62 0 2024-03-22T07:08:50.923 INFO:tasks.workunit.client.1.smithi149.stdout:9/320: dread - db/d21/d37/f5c zero size 2024-03-22T07:08:50.924 INFO:tasks.workunit.client.1.smithi149.stdout:8/317: read d1/d2/d18/f3c [156621,65806] 0 2024-03-22T07:08:50.924 INFO:tasks.workunit.client.1.smithi149.stdout:3/221: mknod d0/c51 0 2024-03-22T07:08:50.924 INFO:tasks.workunit.client.1.smithi149.stdout:3/222: dread - d0/d2/f4d zero size 2024-03-22T07:08:50.925 INFO:tasks.workunit.client.1.smithi149.stdout:6/258: unlink d9/df/c14 0 2024-03-22T07:08:50.925 INFO:tasks.workunit.client.1.smithi149.stdout:7/367: dwrite d1/d18/d1b/d1d/f1f [0,4194304] 0 2024-03-22T07:08:50.927 INFO:tasks.workunit.client.1.smithi149.stdout:1/306: link d4/d20/d23/d3c/f41 d4/f69 0 2024-03-22T07:08:50.929 INFO:tasks.workunit.client.1.smithi149.stdout:8/318: rename d1/f21 to d1/d20/d35/f6f 0 2024-03-22T07:08:50.930 INFO:tasks.workunit.client.1.smithi149.stdout:3/223: mknod d0/d4/d8/d24/d40/c52 0 2024-03-22T07:08:50.931 INFO:tasks.workunit.client.1.smithi149.stdout:9/321: dread db/d22/f2b [0,4194304] 0 2024-03-22T07:08:50.931 INFO:tasks.workunit.client.1.smithi149.stdout:6/259: mknod d9/df/d22/c50 0 2024-03-22T07:08:50.933 INFO:tasks.workunit.client.1.smithi149.stdout:7/368: link d1/d18/d1b/d51/f52 d1/d1c/f7b 0 2024-03-22T07:08:50.933 INFO:tasks.workunit.client.1.smithi149.stdout:7/369: creat d1/d18/d1b/d51/d5a/f7c x:0 0 0 2024-03-22T07:08:50.933 INFO:tasks.workunit.client.1.smithi149.stdout:7/370: truncate d1/d18/d1b/d1d/f13 1604947 0 2024-03-22T07:08:50.933 INFO:tasks.workunit.client.1.smithi149.stdout:1/307: truncate d4/d20/d23/d59/f5c 927555 0 2024-03-22T07:08:50.936 INFO:tasks.workunit.client.1.smithi149.stdout:3/224: rename d0/d2/l11 to d0/d4/d8/d24/d39/l53 0 2024-03-22T07:08:50.936 INFO:tasks.workunit.client.1.smithi149.stdout:6/260: truncate d9/f19 4288258 0 2024-03-22T07:08:50.937 INFO:tasks.workunit.client.1.smithi149.stdout:8/319: write d1/f16 [1312709,30868] 0 2024-03-22T07:08:50.938 INFO:tasks.workunit.client.1.smithi149.stdout:7/371: unlink d1/d1c/l31 0 2024-03-22T07:08:50.938 INFO:tasks.workunit.client.1.smithi149.stdout:7/372: write d1/d18/f37 [3982042,58100] 0 2024-03-22T07:08:50.938 INFO:tasks.workunit.client.1.smithi149.stdout:7/373: write d1/d18/d1b/d1d/d2a/d5b/f79 [908293,73342] 0 2024-03-22T07:08:50.938 INFO:tasks.workunit.client.1.smithi149.stdout:1/308: mkdir d4/d1f/d6a 0 2024-03-22T07:08:50.940 INFO:tasks.workunit.client.1.smithi149.stdout:9/322: rename db/d21/d2e/l62 to db/d19/d1c/l63 0 2024-03-22T07:08:50.945 INFO:tasks.workunit.client.1.smithi149.stdout:8/320: creat d1/d2/d9/d11/d45/f70 x:0 0 0 2024-03-22T07:08:50.945 INFO:tasks.workunit.client.1.smithi149.stdout:8/321: write d1/f24 [450799,2574] 0 2024-03-22T07:08:50.945 INFO:tasks.workunit.client.1.smithi149.stdout:8/322: creat d1/d2/d9/d11/d45/f71 x:0 0 0 2024-03-22T07:08:50.945 INFO:tasks.workunit.client.1.smithi149.stdout:8/323: write d1/d2/d9/f4a [1410608,72017] 0 2024-03-22T07:08:50.945 INFO:tasks.workunit.client.1.smithi149.stdout:8/324: chown d1/d20/d35/d22/d44/d4f/l64 9 1 2024-03-22T07:08:50.946 INFO:tasks.workunit.client.1.smithi149.stdout:2/300: dwrite d1/d3/d5/f28 [0,4194304] 0 2024-03-22T07:08:50.946 INFO:tasks.workunit.client.1.smithi149.stdout:2/301: fsync d1/d3/f25 0 2024-03-22T07:08:50.951 INFO:tasks.workunit.client.1.smithi149.stdout:3/225: rmdir d0/d2/d30/d31 39 2024-03-22T07:08:50.953 INFO:tasks.workunit.client.1.smithi149.stdout:2/302: mknod d1/d3/d6/d32/c66 0 2024-03-22T07:08:50.956 INFO:tasks.workunit.client.1.smithi149.stdout:3/226: truncate d0/d2/f2d 809150 0 2024-03-22T07:08:50.961 INFO:tasks.workunit.client.1.smithi149.stdout:7/374: rename d1/d18/d1b/d1d/d2a to d1/d18/d1b/d51/d7a/d7d 0 2024-03-22T07:08:50.963 INFO:tasks.workunit.client.1.smithi149.stdout:2/303: write d1/d3/d5/f28 [1334437,89171] 0 2024-03-22T07:08:50.963 INFO:tasks.workunit.client.1.smithi149.stdout:3/227: unlink d0/d4/l16 0 2024-03-22T07:08:50.964 INFO:tasks.workunit.client.1.smithi149.stdout:6/261: dwrite d9/df/f1d [0,4194304] 0 2024-03-22T07:08:50.964 INFO:tasks.workunit.client.1.smithi149.stdout:6/262: truncate d9/f3c 603109 0 2024-03-22T07:08:50.967 INFO:tasks.workunit.client.1.smithi149.stdout:8/325: rename d1/d20/d35/d22/d44/f62 to d1/d20/f72 0 2024-03-22T07:08:50.970 INFO:tasks.workunit.client.1.smithi149.stdout:2/304: mknod d1/d8/d34/c67 0 2024-03-22T07:08:50.972 INFO:tasks.workunit.client.1.smithi149.stdout:6/263: unlink d9/df/c29 0 2024-03-22T07:08:50.979 INFO:tasks.workunit.client.1.smithi149.stdout:7/375: write d1/d1c/d2f/f43 [812296,46820] 0 2024-03-22T07:08:50.981 INFO:tasks.workunit.client.1.smithi149.stdout:2/305: mkdir d1/d3/d5/d1f/d68 0 2024-03-22T07:08:50.983 INFO:tasks.workunit.client.1.smithi149.stdout:7/376: mknod d1/d18/d1b/d51/d7a/d7d/d5d/c7e 0 2024-03-22T07:08:50.983 INFO:tasks.workunit.client.1.smithi149.stdout:7/377: chown d1/l15 1021950 1 2024-03-22T07:08:50.984 INFO:tasks.workunit.client.1.smithi149.stdout:2/306: chown d1/d8/c43 0 1 2024-03-22T07:08:50.984 INFO:tasks.workunit.client.1.smithi149.stdout:2/307: fsync d1/d3/d5/d1f/d5e/f4a 0 2024-03-22T07:08:50.986 INFO:tasks.workunit.client.1.smithi149.stdout:6/264: rmdir d9/df 39 2024-03-22T07:08:50.988 INFO:tasks.workunit.client.1.smithi149.stdout:2/308: getdents d1/d3/dd 0 2024-03-22T07:08:50.988 INFO:tasks.workunit.client.1.smithi149.stdout:6/265: rmdir d9/d1f/d24/d34/d37 39 2024-03-22T07:08:50.990 INFO:tasks.workunit.client.1.smithi149.stdout:2/309: unlink d1/d3/dd/f45 0 2024-03-22T07:08:50.991 INFO:tasks.workunit.client.1.smithi149.stdout:2/310: mkdir d1/d4d/d69 0 2024-03-22T07:08:50.992 INFO:tasks.workunit.client.1.smithi149.stdout:2/311: rename d1/d8/d34/d50 to d1/d4d/d6a 0 2024-03-22T07:08:50.993 INFO:tasks.workunit.client.1.smithi149.stdout:2/312: symlink d1/d3/d6/d3f/l6b 0 2024-03-22T07:08:50.998 INFO:tasks.workunit.client.1.smithi149.stdout:6/266: write d9/d1f/d24/d34/d37/f46 [3687197,94982] 0 2024-03-22T07:08:51.000 INFO:tasks.workunit.client.1.smithi149.stdout:2/313: dread d1/d3/f13 [0,4194304] 0 2024-03-22T07:08:51.000 INFO:tasks.workunit.client.1.smithi149.stdout:2/314: dread - d1/d3/f54 zero size 2024-03-22T07:08:51.000 INFO:tasks.workunit.client.1.smithi149.stdout:2/315: chown d1/d4d/d6a 288802071 1 2024-03-22T07:08:51.000 INFO:tasks.workunit.client.1.smithi149.stdout:6/267: rename f6 to d9/d47/f51 0 2024-03-22T07:08:51.001 INFO:tasks.workunit.client.1.smithi149.stdout:2/316: unlink d1/d3/f9 0 2024-03-22T07:08:51.002 INFO:tasks.workunit.client.1.smithi149.stdout:2/317: symlink d1/d3/l6c 0 2024-03-22T07:08:51.002 INFO:tasks.workunit.client.1.smithi149.stdout:2/318: stat d1/d8/d34/f63 0 2024-03-22T07:08:51.009 INFO:tasks.workunit.client.1.smithi149.stdout:6/268: dread f1 [0,4194304] 0 2024-03-22T07:08:51.011 INFO:tasks.workunit.client.1.smithi149.stdout:6/269: mkdir d9/d47/d52 0 2024-03-22T07:08:51.011 INFO:tasks.workunit.client.1.smithi149.stdout:2/319: dread d1/d3/dd/f40 [0,4194304] 0 2024-03-22T07:08:51.011 INFO:tasks.workunit.client.1.smithi149.stdout:6/270: getdents d9/df/d22 0 2024-03-22T07:08:51.012 INFO:tasks.workunit.client.1.smithi149.stdout:6/271: creat d9/d1f/d24/d34/d37/d38/f53 x:0 0 0 2024-03-22T07:08:51.012 INFO:tasks.workunit.client.1.smithi149.stdout:6/272: dread - d9/d1f/d24/d34/d37/f49 zero size 2024-03-22T07:08:51.012 INFO:tasks.workunit.client.1.smithi149.stdout:6/273: write d9/df/f1e [614448,118176] 0 2024-03-22T07:08:51.013 INFO:tasks.workunit.client.1.smithi149.stdout:2/320: rename d1/d8/l3b to d1/d3/l6d 0 2024-03-22T07:08:51.013 INFO:tasks.workunit.client.1.smithi149.stdout:2/321: dread - d1/d8/d34/f63 zero size 2024-03-22T07:08:51.013 INFO:tasks.workunit.client.1.smithi149.stdout:2/322: readlink d1/d3/d5/d1f/d5e/l31 0 2024-03-22T07:08:51.014 INFO:tasks.workunit.client.1.smithi149.stdout:6/274: dread d9/d1f/f21 [0,4194304] 0 2024-03-22T07:08:51.014 INFO:tasks.workunit.client.1.smithi149.stdout:2/323: creat d1/d3/dd/f6e x:0 0 0 2024-03-22T07:08:51.015 INFO:tasks.workunit.client.1.smithi149.stdout:6/275: link d9/f17 d9/d1f/d24/d34/d37/d38/f54 0 2024-03-22T07:08:51.016 INFO:tasks.workunit.client.1.smithi149.stdout:6/276: mkdir d9/d1f/d24/d34/d4f/d55 0 2024-03-22T07:08:51.016 INFO:tasks.workunit.client.1.smithi149.stdout:6/277: truncate d9/df/d2e/f1c 911336 0 2024-03-22T07:08:51.024 INFO:tasks.workunit.client.1.smithi149.stdout:6/278: dread d9/d1f/d24/d34/d37/d38/f54 [0,4194304] 0 2024-03-22T07:08:51.025 INFO:tasks.workunit.client.1.smithi149.stdout:6/279: chown d9/df/d2e 3807265 1 2024-03-22T07:08:51.041 INFO:tasks.workunit.client.1.smithi149.stdout:6/280: dwrite d9/d1f/d24/d34/d37/f49 [0,4194304] 0 2024-03-22T07:08:51.041 INFO:tasks.workunit.client.1.smithi149.stdout:6/281: stat d9/d1f/d24/d34/d37/l44 0 2024-03-22T07:08:51.043 INFO:tasks.workunit.client.1.smithi149.stdout:6/282: link d9/df/d2e/f27 d9/df/f56 0 2024-03-22T07:08:51.043 INFO:tasks.workunit.client.1.smithi149.stdout:6/283: chown d9/d1f/d24/d34 777740320 1 2024-03-22T07:08:51.044 INFO:tasks.workunit.client.1.smithi149.stdout:6/284: write d9/d1f/d24/f2d [378182,103898] 0 2024-03-22T07:08:51.044 INFO:tasks.workunit.client.1.smithi149.stdout:6/285: getdents d9/df/d2e/d4e 0 2024-03-22T07:08:51.050 INFO:tasks.workunit.client.1.smithi149.stdout:6/286: dread d9/d47/f51 [0,4194304] 0 2024-03-22T07:08:51.050 INFO:tasks.workunit.client.1.smithi149.stdout:6/287: mknod d9/d1f/d24/d34/d37/d38/c57 0 2024-03-22T07:08:51.052 INFO:tasks.workunit.client.1.smithi149.stdout:6/288: rename d9/fd to d9/df/d22/f58 0 2024-03-22T07:08:51.052 INFO:tasks.workunit.client.1.smithi149.stdout:6/289: chown d9/d1f/d24/d34/d37/f46 208 1 2024-03-22T07:08:51.053 INFO:tasks.workunit.client.1.smithi149.stdout:6/290: symlink d9/d1f/d24/d42/l59 0 2024-03-22T07:08:51.053 INFO:tasks.workunit.client.1.smithi149.stdout:6/291: write f8 [9290586,95087] 0 2024-03-22T07:08:51.054 INFO:tasks.workunit.client.1.smithi149.stdout:6/292: creat d9/d47/f5a x:0 0 0 2024-03-22T07:08:51.054 INFO:tasks.workunit.client.1.smithi149.stdout:6/293: rename d9/d1f/d24 to d9/d1f/d24/d42/d5b 22 2024-03-22T07:08:51.054 INFO:tasks.workunit.client.1.smithi149.stdout:6/294: mkdir d9/d1f/d5c 0 2024-03-22T07:08:51.055 INFO:tasks.workunit.client.1.smithi149.stdout:6/295: link d9/d1f/d24/d34/d37/d38/c57 d9/d1f/d24/d34/d37/d38/c5d 0 2024-03-22T07:08:51.057 INFO:tasks.workunit.client.1.smithi149.stdout:6/296: unlink d9/l36 0 2024-03-22T07:08:51.058 INFO:tasks.workunit.client.1.smithi149.stdout:6/297: creat d9/df/d2e/d4e/f5e x:0 0 0 2024-03-22T07:08:51.058 INFO:tasks.workunit.client.1.smithi149.stdout:6/298: mkdir d9/d1f/d24/d5f 0 2024-03-22T07:08:51.065 INFO:tasks.workunit.client.1.smithi149.stdout:6/299: rmdir d9 39 2024-03-22T07:08:51.067 INFO:tasks.workunit.client.1.smithi149.stdout:5/256: sync 2024-03-22T07:08:51.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:50 smithi081 ceph-mon[29245]: pgmap v8: 65 pgs: 65 active+clean; 869 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 4.1 MiB/s rd, 8.1 MiB/s wr, 814 op/s 2024-03-22T07:08:51.069 INFO:tasks.workunit.client.1.smithi149.stdout:5/257: rename d7/da/d20/d45/f48 to d7/da/d20/f4c 0 2024-03-22T07:08:51.071 INFO:tasks.workunit.client.1.smithi149.stdout:5/258: creat d7/da/d20/d30/d3a/d3d/d2c/f4d x:0 0 0 2024-03-22T07:08:51.076 INFO:tasks.workunit.client.1.smithi149.stdout:5/259: dread d7/f33 [0,4194304] 0 2024-03-22T07:08:51.076 INFO:tasks.workunit.client.1.smithi149.stdout:5/260: read d7/f33 [484811,9086] 0 2024-03-22T07:08:51.083 INFO:tasks.workunit.client.1.smithi149.stdout:6/300: dwrite d9/f3c [0,4194304] 0 2024-03-22T07:08:51.085 INFO:tasks.workunit.client.1.smithi149.stdout:6/301: creat d9/d47/d52/f60 x:0 0 0 2024-03-22T07:08:51.100 INFO:tasks.workunit.client.1.smithi149.stdout:5/261: dwrite d7/da/d20/f4c [0,4194304] 0 2024-03-22T07:08:51.100 INFO:tasks.workunit.client.1.smithi149.stdout:5/262: chown d7/da/lf 437222797 1 2024-03-22T07:08:51.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:50 smithi149 ceph-mon[36068]: pgmap v8: 65 pgs: 65 active+clean; 869 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 4.1 MiB/s rd, 8.1 MiB/s wr, 814 op/s 2024-03-22T07:08:51.103 INFO:tasks.workunit.client.1.smithi149.stdout:5/263: mknod d7/da/c4e 0 2024-03-22T07:08:51.105 INFO:tasks.workunit.client.1.smithi149.stdout:6/302: dwrite d9/d1f/d24/f2d [0,4194304] 0 2024-03-22T07:08:51.113 INFO:tasks.workunit.client.1.smithi149.stdout:4/283: sync 2024-03-22T07:08:51.113 INFO:tasks.workunit.client.1.smithi149.stdout:1/309: sync 2024-03-22T07:08:51.113 INFO:tasks.workunit.client.1.smithi149.stdout:0/211: sync 2024-03-22T07:08:51.113 INFO:tasks.workunit.client.1.smithi149.stdout:0/212: chown d1/fb 43769 1 2024-03-22T07:08:51.115 INFO:tasks.workunit.client.1.smithi149.stdout:4/284: creat d0/d8/d5c/f5e x:0 0 0 2024-03-22T07:08:51.115 INFO:tasks.workunit.client.1.smithi149.stdout:5/264: dread d7/da/d20/f4c [0,4194304] 0 2024-03-22T07:08:51.115 INFO:tasks.workunit.client.1.smithi149.stdout:4/285: write d0/d8/d1d/d24/f54 [274338,9999] 0 2024-03-22T07:08:51.115 INFO:tasks.workunit.client.1.smithi149.stdout:0/213: creat d1/d27/f46 x:0 0 0 2024-03-22T07:08:51.115 INFO:tasks.workunit.client.1.smithi149.stdout:0/214: creat d1/d31/d36/f47 x:0 0 0 2024-03-22T07:08:51.117 INFO:tasks.workunit.client.1.smithi149.stdout:4/286: truncate d0/d29/f31 3894113 0 2024-03-22T07:08:51.123 INFO:tasks.workunit.client.1.smithi149.stdout:4/287: symlink d0/d8/dc/d4b/l5f 0 2024-03-22T07:08:51.130 INFO:tasks.workunit.client.1.smithi149.stdout:5/265: read d7/da/d20/d30/d3a/d3d/f34 [66883,100945] 0 2024-03-22T07:08:51.133 INFO:tasks.workunit.client.1.smithi149.stdout:5/266: creat d7/da/d20/d45/f4f x:0 0 0 2024-03-22T07:08:51.133 INFO:tasks.workunit.client.1.smithi149.stdout:5/267: chown d7/da/d20/d36/d44 0 1 2024-03-22T07:08:51.133 INFO:tasks.workunit.client.1.smithi149.stdout:5/268: readlink d7/da/d20/l2f 0 2024-03-22T07:08:51.134 INFO:tasks.workunit.client.1.smithi149.stdout:6/303: dwrite d9/d1f/d24/d34/d37/d38/f53 [0,4194304] 0 2024-03-22T07:08:51.137 INFO:tasks.workunit.client.1.smithi149.stdout:6/304: getdents d9/d1f/d24/d42 0 2024-03-22T07:08:51.137 INFO:tasks.workunit.client.1.smithi149.stdout:6/305: stat d9 0 2024-03-22T07:08:51.144 INFO:tasks.workunit.client.1.smithi149.stdout:1/310: dwrite d4/d10/f4d [0,4194304] 0 2024-03-22T07:08:51.144 INFO:tasks.workunit.client.1.smithi149.stdout:1/311: stat d4/d10/f12 0 2024-03-22T07:08:51.148 INFO:tasks.workunit.client.1.smithi149.stdout:4/288: dwrite d0/d29/f35 [0,4194304] 0 2024-03-22T07:08:51.151 INFO:tasks.workunit.client.1.smithi149.stdout:2/324: sync 2024-03-22T07:08:51.151 INFO:tasks.workunit.client.1.smithi149.stdout:2/325: dread - d1/d3/f4b zero size 2024-03-22T07:08:51.152 INFO:tasks.workunit.client.1.smithi149.stdout:8/326: sync 2024-03-22T07:08:51.152 INFO:tasks.workunit.client.1.smithi149.stdout:4/289: rmdir d0/d8/dc/d14 39 2024-03-22T07:08:51.157 INFO:tasks.workunit.client.1.smithi149.stdout:4/290: dread d0/d8/f21 [0,4194304] 0 2024-03-22T07:08:51.158 INFO:tasks.workunit.client.1.smithi149.stdout:3/228: sync 2024-03-22T07:08:51.158 INFO:tasks.workunit.client.1.smithi149.stdout:3/229: chown d0/d4/d8/f23 4886096 1 2024-03-22T07:08:51.158 INFO:tasks.workunit.client.1.smithi149.stdout:7/378: sync 2024-03-22T07:08:51.158 INFO:tasks.workunit.client.1.smithi149.stdout:7/379: write d1/d18/d1b/d1d/fd [1072318,51566] 0 2024-03-22T07:08:51.158 INFO:tasks.workunit.client.1.smithi149.stdout:9/323: sync 2024-03-22T07:08:51.158 INFO:tasks.workunit.client.1.smithi149.stdout:9/324: dread - db/d19/d39/f59 zero size 2024-03-22T07:08:51.159 INFO:tasks.workunit.client.1.smithi149.stdout:9/325: write db/d19/d39/d3b/f48 [1576166,72365] 0 2024-03-22T07:08:51.159 INFO:tasks.workunit.client.1.smithi149.stdout:9/326: fdatasync db/d19/d1c/f4f 0 2024-03-22T07:08:51.159 INFO:tasks.workunit.client.1.smithi149.stdout:3/230: mkdir d0/d4/d8/d24/d40/d54 0 2024-03-22T07:08:51.159 INFO:tasks.workunit.client.1.smithi149.stdout:7/380: creat d1/d18/d1b/d51/d7a/f7f x:0 0 0 2024-03-22T07:08:51.159 INFO:tasks.workunit.client.1.smithi149.stdout:3/231: unlink d0/d2/d30/f46 0 2024-03-22T07:08:51.159 INFO:tasks.workunit.client.1.smithi149.stdout:3/232: creat d0/d2/f55 x:0 0 0 2024-03-22T07:08:51.162 INFO:tasks.workunit.client.1.smithi149.stdout:3/233: mknod d0/d4/d8/d24/d39/c56 0 2024-03-22T07:08:51.162 INFO:tasks.workunit.client.1.smithi149.stdout:2/326: dread d1/d3/d5/d1f/f1e [0,4194304] 0 2024-03-22T07:08:51.166 INFO:tasks.workunit.client.1.smithi149.stdout:2/327: mkdir d1/d3/d6/d32/d6f 0 2024-03-22T07:08:51.167 INFO:tasks.workunit.client.1.smithi149.stdout:2/328: write d1/d3/dd/f65 [754649,90822] 0 2024-03-22T07:08:51.168 INFO:tasks.workunit.client.1.smithi149.stdout:0/215: dwrite d1/d1a/f25 [0,4194304] 0 2024-03-22T07:08:51.168 INFO:tasks.workunit.client.1.smithi149.stdout:8/327: dread d1/d20/d41/f63 [0,4194304] 0 2024-03-22T07:08:51.168 INFO:tasks.workunit.client.1.smithi149.stdout:1/312: dwrite d4/d7/f57 [0,4194304] 0 2024-03-22T07:08:51.171 INFO:tasks.workunit.client.1.smithi149.stdout:5/269: dwrite d7/f8 [0,4194304] 0 2024-03-22T07:08:51.171 INFO:tasks.workunit.client.1.smithi149.stdout:5/270: write d7/f33 [651050,77209] 0 2024-03-22T07:08:51.172 INFO:tasks.workunit.client.1.smithi149.stdout:2/329: stat d1/d3/d5/f28 0 2024-03-22T07:08:51.172 INFO:tasks.workunit.client.1.smithi149.stdout:2/330: read - d1/d3/dd/f6e zero size 2024-03-22T07:08:51.173 INFO:tasks.workunit.client.1.smithi149.stdout:4/291: read d0/d8/d1d/f28 [3504938,15042] 0 2024-03-22T07:08:51.174 INFO:tasks.workunit.client.1.smithi149.stdout:0/216: link d1/d1a/c1c d1/d27/c48 0 2024-03-22T07:08:51.174 INFO:tasks.workunit.client.1.smithi149.stdout:3/234: write d0/d4/d8/f21 [370720,12235] 0 2024-03-22T07:08:51.176 INFO:tasks.workunit.client.1.smithi149.stdout:1/313: mkdir d4/d1f/d6a/d6b 0 2024-03-22T07:08:51.177 INFO:tasks.workunit.client.1.smithi149.stdout:8/328: symlink d1/d20/d35/d22/l73 0 2024-03-22T07:08:51.177 INFO:tasks.workunit.client.1.smithi149.stdout:8/329: write d1/d2/d3/f1a [2976575,57672] 0 2024-03-22T07:08:51.178 INFO:tasks.workunit.client.1.smithi149.stdout:8/330: chown d1/d20/d41/f48 8 1 2024-03-22T07:08:51.178 INFO:tasks.workunit.client.1.smithi149.stdout:8/331: write d1/d2/d9/d11/d45/f70 [418402,57019] 0 2024-03-22T07:08:51.178 INFO:tasks.workunit.client.1.smithi149.stdout:5/271: link d7/l21 d7/d35/l50 0 2024-03-22T07:08:51.179 INFO:tasks.workunit.client.1.smithi149.stdout:2/331: mknod d1/d3/d5/d1f/d68/c70 0 2024-03-22T07:08:51.182 INFO:tasks.workunit.client.1.smithi149.stdout:4/292: symlink d0/d29/l60 0 2024-03-22T07:08:51.184 INFO:tasks.workunit.client.1.smithi149.stdout:3/235: mknod d0/d2/d42/c57 0 2024-03-22T07:08:51.185 INFO:tasks.workunit.client.1.smithi149.stdout:7/381: dwrite d1/f33 [0,4194304] 0 2024-03-22T07:08:51.185 INFO:tasks.workunit.client.1.smithi149.stdout:1/314: mknod d4/c6c 0 2024-03-22T07:08:51.185 INFO:tasks.workunit.client.1.smithi149.stdout:1/315: chown d4 5505 1 2024-03-22T07:08:51.187 INFO:tasks.workunit.client.1.smithi149.stdout:9/327: dwrite db/d19/f3c [0,4194304] 0 2024-03-22T07:08:51.187 INFO:tasks.workunit.client.1.smithi149.stdout:9/328: creat db/d21/d37/f64 x:0 0 0 2024-03-22T07:08:51.190 INFO:tasks.workunit.client.1.smithi149.stdout:5/272: rename d7/da/f12 to d7/da/d20/d45/f51 0 2024-03-22T07:08:51.190 INFO:tasks.workunit.client.1.smithi149.stdout:5/273: fdatasync d7/da/d20/f4c 0 2024-03-22T07:08:51.190 INFO:tasks.workunit.client.1.smithi149.stdout:0/217: mknod d1/c49 0 2024-03-22T07:08:51.193 INFO:tasks.workunit.client.1.smithi149.stdout:1/316: write d4/f69 [4303133,60746] 0 2024-03-22T07:08:51.193 INFO:tasks.workunit.client.1.smithi149.stdout:1/317: creat d4/d7/d5a/f6d x:0 0 0 2024-03-22T07:08:51.195 INFO:tasks.workunit.client.1.smithi149.stdout:9/329: symlink db/d19/d28/l65 0 2024-03-22T07:08:51.195 INFO:tasks.workunit.client.1.smithi149.stdout:9/330: chown db/d21/d37/d3a/c40 1 1 2024-03-22T07:08:51.196 INFO:tasks.workunit.client.1.smithi149.stdout:9/331: stat db/l4a 0 2024-03-22T07:08:51.196 INFO:tasks.workunit.client.1.smithi149.stdout:8/332: link d1/d2/l36 d1/d2/d9/l74 0 2024-03-22T07:08:51.196 INFO:tasks.workunit.client.1.smithi149.stdout:8/333: stat d1/d20/d35/d22/f40 0 2024-03-22T07:08:51.196 INFO:tasks.workunit.client.1.smithi149.stdout:8/334: chown d1/d2/d3/fd 0 1 2024-03-22T07:08:51.196 INFO:tasks.workunit.client.1.smithi149.stdout:8/335: read - d1/d20/d35/d22/f38 zero size 2024-03-22T07:08:51.198 INFO:tasks.workunit.client.1.smithi149.stdout:3/236: rename d0/d4/d8/f1d to d0/d4/f58 0 2024-03-22T07:08:51.199 INFO:tasks.workunit.client.1.smithi149.stdout:0/218: creat d1/f4a x:0 0 0 2024-03-22T07:08:51.201 INFO:tasks.workunit.client.1.smithi149.stdout:9/332: rename db/d19/d39/f59 to db/d42/d49/d4b/d4d/f66 0 2024-03-22T07:08:51.202 INFO:tasks.workunit.client.1.smithi149.stdout:9/333: truncate db/d42/d49/d4b/d4d/f55 655027 0 2024-03-22T07:08:51.202 INFO:tasks.workunit.client.1.smithi149.stdout:4/293: dwrite d0/d29/f4a [0,4194304] 0 2024-03-22T07:08:51.203 INFO:tasks.workunit.client.1.smithi149.stdout:9/334: creat db/d33/f67 x:0 0 0 2024-03-22T07:08:51.203 INFO:tasks.workunit.client.1.smithi149.stdout:9/335: write db/d21/d37/f4e [1732845,76964] 0 2024-03-22T07:08:51.206 INFO:tasks.workunit.client.1.smithi149.stdout:1/318: dread d4/f6 [0,4194304] 0 2024-03-22T07:08:51.206 INFO:tasks.workunit.client.1.smithi149.stdout:1/319: chown d4/d20/d23/d59 45762557 1 2024-03-22T07:08:51.206 INFO:tasks.workunit.client.1.smithi149.stdout:8/336: rename d1/d2/d9/l3e to d1/d2/d3/l75 0 2024-03-22T07:08:51.207 INFO:tasks.workunit.client.1.smithi149.stdout:4/294: truncate d0/f5a 1649250 0 2024-03-22T07:08:51.207 INFO:tasks.workunit.client.1.smithi149.stdout:3/237: dread d0/d4/f1e [0,4194304] 0 2024-03-22T07:08:51.208 INFO:tasks.workunit.client.1.smithi149.stdout:3/238: fsync d0/d4/d8/d24/f47 0 2024-03-22T07:08:51.208 INFO:tasks.workunit.client.1.smithi149.stdout:3/239: fsync d0/d2/d30/d31/f3d 0 2024-03-22T07:08:51.208 INFO:tasks.workunit.client.1.smithi149.stdout:5/274: getdents d7/d35 0 2024-03-22T07:08:51.210 INFO:tasks.workunit.client.1.smithi149.stdout:9/336: mkdir db/d19/d39/d68 0 2024-03-22T07:08:51.210 INFO:tasks.workunit.client.1.smithi149.stdout:9/337: fsync db/d22/f36 0 2024-03-22T07:08:51.214 INFO:tasks.workunit.client.1.smithi149.stdout:8/337: creat d1/d2/d3/f76 x:0 0 0 2024-03-22T07:08:51.214 INFO:tasks.workunit.client.1.smithi149.stdout:8/338: fdatasync d1/d20/d35/fb 0 2024-03-22T07:08:51.215 INFO:tasks.workunit.client.1.smithi149.stdout:8/339: chown d1/d20/d51 18517454 1 2024-03-22T07:08:51.215 INFO:tasks.workunit.client.1.smithi149.stdout:8/340: stat d1/d2/d9/d11/c4c 0 2024-03-22T07:08:51.215 INFO:tasks.workunit.client.1.smithi149.stdout:4/295: creat d0/d8/dc/d14/d3e/f61 x:0 0 0 2024-03-22T07:08:51.220 INFO:tasks.workunit.client.1.smithi149.stdout:5/275: creat d7/da/d20/d30/d3a/d3d/d2c/d47/f52 x:0 0 0 2024-03-22T07:08:51.225 INFO:tasks.workunit.client.1.smithi149.stdout:0/219: rename d1/d9/c10 to d1/c4b 0 2024-03-22T07:08:51.225 INFO:tasks.workunit.client.1.smithi149.stdout:8/341: creat d1/d2/d9/d11/f77 x:0 0 0 2024-03-22T07:08:51.225 INFO:tasks.workunit.client.1.smithi149.stdout:8/342: stat d1/d2/d9/d11/d4d 0 2024-03-22T07:08:51.226 INFO:tasks.workunit.client.1.smithi149.stdout:5/276: truncate d7/da/d20/f4c 2362806 0 2024-03-22T07:08:51.227 INFO:tasks.workunit.client.1.smithi149.stdout:7/382: rename d1/d1c/d4d/f3e to d1/d1c/d49/f80 0 2024-03-22T07:08:51.227 INFO:tasks.workunit.client.1.smithi149.stdout:7/383: chown d1/d1c/d49 8 1 2024-03-22T07:08:51.229 INFO:tasks.workunit.client.1.smithi149.stdout:3/240: rename d0/d2/d30/f3c to d0/d2/d36/f59 0 2024-03-22T07:08:51.230 INFO:tasks.workunit.client.1.smithi149.stdout:7/384: read d1/d1c/f7b [478793,81388] 0 2024-03-22T07:08:51.233 INFO:tasks.workunit.client.1.smithi149.stdout:5/277: rename d7/d35/c3f to d7/da/d20/d30/d3a/d3d/d2c/d47/c53 0 2024-03-22T07:08:51.233 INFO:tasks.workunit.client.1.smithi149.stdout:5/278: chown d7/da/d20/d36/f41 2141652 1 2024-03-22T07:08:51.233 INFO:tasks.workunit.client.1.smithi149.stdout:5/279: truncate d7/da/d20/f49 579252 0 2024-03-22T07:08:51.233 INFO:tasks.workunit.client.1.smithi149.stdout:3/241: mkdir d0/d2/d5a 0 2024-03-22T07:08:51.234 INFO:tasks.workunit.client.1.smithi149.stdout:5/280: rmdir d7/da/d20 39 2024-03-22T07:08:51.236 INFO:tasks.workunit.client.1.smithi149.stdout:3/242: truncate d0/d4/d8/f19 1103003 0 2024-03-22T07:08:51.236 INFO:tasks.workunit.client.1.smithi149.stdout:3/243: stat d0/d2/d36/f2e 0 2024-03-22T07:08:51.236 INFO:tasks.workunit.client.1.smithi149.stdout:8/343: dread d1/f24 [0,4194304] 0 2024-03-22T07:08:51.237 INFO:tasks.workunit.client.1.smithi149.stdout:3/244: chown d0/d4/d8/f23 1927005740 1 2024-03-22T07:08:51.238 INFO:tasks.workunit.client.1.smithi149.stdout:5/281: symlink d7/da/d20/l54 0 2024-03-22T07:08:51.238 INFO:tasks.workunit.client.1.smithi149.stdout:5/282: write d7/da/d20/d30/d3a/d3d/f34 [1202394,110571] 0 2024-03-22T07:08:51.238 INFO:tasks.workunit.client.1.smithi149.stdout:5/283: dread - d7/da/d20/d30/d3a/d3d/d2c/f4d zero size 2024-03-22T07:08:51.238 INFO:tasks.workunit.client.1.smithi149.stdout:8/344: chown d1/d2/d9/c2e 4712 1 2024-03-22T07:08:51.239 INFO:tasks.workunit.client.1.smithi149.stdout:5/284: read d7/da/fd [780124,98211] 0 2024-03-22T07:08:51.239 INFO:tasks.workunit.client.1.smithi149.stdout:1/320: dwrite d4/d10/d28/f2a [0,4194304] 0 2024-03-22T07:08:51.239 INFO:tasks.workunit.client.1.smithi149.stdout:3/245: truncate d0/d2/d30/d31/f33 1253047 0 2024-03-22T07:08:51.239 INFO:tasks.workunit.client.1.smithi149.stdout:3/246: readlink d0/l9 0 2024-03-22T07:08:51.239 INFO:tasks.workunit.client.1.smithi149.stdout:9/338: dwrite db/d42/f5a [0,4194304] 0 2024-03-22T07:08:51.240 INFO:tasks.workunit.client.1.smithi149.stdout:9/339: chown db/d22/c2a 5770 1 2024-03-22T07:08:51.241 INFO:tasks.workunit.client.1.smithi149.stdout:8/345: dread d1/d20/d41/f48 [0,4194304] 0 2024-03-22T07:08:51.241 INFO:tasks.workunit.client.1.smithi149.stdout:4/296: dwrite d0/d8/dc/f30 [4194304,4194304] 0 2024-03-22T07:08:51.242 INFO:tasks.workunit.client.1.smithi149.stdout:5/285: creat d7/d35/f55 x:0 0 0 2024-03-22T07:08:51.242 INFO:tasks.workunit.client.1.smithi149.stdout:5/286: write d7/da/fd [1636168,90288] 0 2024-03-22T07:08:51.243 INFO:tasks.workunit.client.1.smithi149.stdout:1/321: mknod d4/d1f/c6e 0 2024-03-22T07:08:51.244 INFO:tasks.workunit.client.1.smithi149.stdout:3/247: symlink d0/d2/d30/l5b 0 2024-03-22T07:08:51.249 INFO:tasks.workunit.client.1.smithi149.stdout:8/346: creat d1/d2/d18/f78 x:0 0 0 2024-03-22T07:08:51.251 INFO:tasks.workunit.client.1.smithi149.stdout:9/340: dread f3 [4194304,4194304] 0 2024-03-22T07:08:51.251 INFO:tasks.workunit.client.1.smithi149.stdout:9/341: creat db/d22/f69 x:0 0 0 2024-03-22T07:08:51.257 INFO:tasks.workunit.client.1.smithi149.stdout:5/287: creat d7/da/d20/d30/f56 x:0 0 0 2024-03-22T07:08:51.258 INFO:tasks.workunit.client.1.smithi149.stdout:1/322: creat d4/d20/d23/f6f x:0 0 0 2024-03-22T07:08:51.260 INFO:tasks.workunit.client.1.smithi149.stdout:9/342: symlink db/d19/d1c/l6a 0 2024-03-22T07:08:51.261 INFO:tasks.workunit.client.1.smithi149.stdout:5/288: link d7/db/c1f d7/da/d20/d30/d3a/c57 0 2024-03-22T07:08:51.262 INFO:tasks.workunit.client.1.smithi149.stdout:1/323: symlink d4/d10/l70 0 2024-03-22T07:08:51.263 INFO:tasks.workunit.client.1.smithi149.stdout:8/347: mknod d1/d2/d9/d11/d4d/c79 0 2024-03-22T07:08:51.265 INFO:tasks.workunit.client.1.smithi149.stdout:9/343: mkdir db/d21/d2e/d6b 0 2024-03-22T07:08:51.266 INFO:tasks.workunit.client.1.smithi149.stdout:8/348: write d1/d2/d9/f4a [636659,2920] 0 2024-03-22T07:08:51.266 INFO:tasks.workunit.client.1.smithi149.stdout:8/349: fsync d1/d2/d9/d11/f17 0 2024-03-22T07:08:51.266 INFO:tasks.workunit.client.1.smithi149.stdout:8/350: creat d1/d2/d9/d11/f7a x:0 0 0 2024-03-22T07:08:51.266 INFO:tasks.workunit.client.1.smithi149.stdout:4/297: dwrite d0/d29/f4a [0,4194304] 0 2024-03-22T07:08:51.266 INFO:tasks.workunit.client.1.smithi149.stdout:4/298: truncate d0/d8/d1d/f48 867679 0 2024-03-22T07:08:51.266 INFO:tasks.workunit.client.1.smithi149.stdout:4/299: chown d0/f3 12485 1 2024-03-22T07:08:51.267 INFO:tasks.workunit.client.1.smithi149.stdout:3/248: dwrite d0/d2/d36/f2e [0,4194304] 0 2024-03-22T07:08:51.268 INFO:tasks.workunit.client.1.smithi149.stdout:3/249: chown d0/d2/f2d 8438057 1 2024-03-22T07:08:51.268 INFO:tasks.workunit.client.1.smithi149.stdout:3/250: rename d0/d4/d8 to d0/d4/d8/d24/d40/d54/d5c 22 2024-03-22T07:08:51.268 INFO:tasks.workunit.client.1.smithi149.stdout:9/344: creat db/d21/d37/d3a/f6c x:0 0 0 2024-03-22T07:08:51.268 INFO:tasks.workunit.client.1.smithi149.stdout:9/345: creat db/d19/d39/d3b/f6d x:0 0 0 2024-03-22T07:08:51.268 INFO:tasks.workunit.client.1.smithi149.stdout:9/346: chown db/d42/d49/d4b 175544065 1 2024-03-22T07:08:51.268 INFO:tasks.workunit.client.1.smithi149.stdout:1/324: mknod d4/d1f/d6a/d6b/c71 0 2024-03-22T07:08:51.268 INFO:tasks.workunit.client.1.smithi149.stdout:1/325: read d4/d7/f33 [42114,111736] 0 2024-03-22T07:08:51.269 INFO:tasks.workunit.client.1.smithi149.stdout:1/326: fdatasync d4/d10/d28/f34 0 2024-03-22T07:08:51.269 INFO:tasks.workunit.client.1.smithi149.stdout:1/327: readlink d4/lf 0 2024-03-22T07:08:51.272 INFO:tasks.workunit.client.1.smithi149.stdout:8/351: dread d1/d20/f54 [0,4194304] 0 2024-03-22T07:08:51.278 INFO:tasks.workunit.client.1.smithi149.stdout:9/347: symlink db/d12/l6e 0 2024-03-22T07:08:51.278 INFO:tasks.workunit.client.1.smithi149.stdout:3/251: dread d0/f1a [0,4194304] 0 2024-03-22T07:08:51.279 INFO:tasks.workunit.client.1.smithi149.stdout:3/252: unlink d0/c51 0 2024-03-22T07:08:51.281 INFO:tasks.workunit.client.1.smithi149.stdout:9/348: mkdir db/d19/d39/d68/d6f 0 2024-03-22T07:08:51.282 INFO:tasks.workunit.client.1.smithi149.stdout:3/253: mkdir d0/d2/d30/d5d 0 2024-03-22T07:08:51.282 INFO:tasks.workunit.client.1.smithi149.stdout:3/254: stat d0/l9 0 2024-03-22T07:08:51.283 INFO:tasks.workunit.client.1.smithi149.stdout:3/255: creat d0/d2/d5a/f5e x:0 0 0 2024-03-22T07:08:51.284 INFO:tasks.workunit.client.1.smithi149.stdout:3/256: mkdir d0/d4/d8/d24/d40/d54/d5f 0 2024-03-22T07:08:51.285 INFO:tasks.workunit.client.1.smithi149.stdout:4/300: dwrite d0/f3 [0,4194304] 0 2024-03-22T07:08:51.285 INFO:tasks.workunit.client.1.smithi149.stdout:4/301: creat d0/d8/dc/d2e/f62 x:0 0 0 2024-03-22T07:08:51.288 INFO:tasks.workunit.client.1.smithi149.stdout:9/349: dread db/d33/f3d [0,4194304] 0 2024-03-22T07:08:51.288 INFO:tasks.workunit.client.1.smithi149.stdout:8/352: fsync d1/d2/d18/f78 0 2024-03-22T07:08:51.289 INFO:tasks.workunit.client.1.smithi149.stdout:6/306: sync 2024-03-22T07:08:51.290 INFO:tasks.workunit.client.1.smithi149.stdout:6/307: symlink d9/d1f/d24/d34/d37/d38/l61 0 2024-03-22T07:08:51.291 INFO:tasks.workunit.client.1.smithi149.stdout:6/308: creat d9/df/d22/f62 x:0 0 0 2024-03-22T07:08:51.294 INFO:tasks.workunit.client.1.smithi149.stdout:6/309: mkdir d9/df/d2e/d63 0 2024-03-22T07:08:51.297 INFO:tasks.workunit.client.1.smithi149.stdout:9/350: dread f3 [4194304,4194304] 0 2024-03-22T07:08:51.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/351: truncate f3 7496885 0 2024-03-22T07:08:51.299 INFO:tasks.workunit.client.1.smithi149.stdout:9/352: rename db/d19/d39/d3b to db/d19/d39/d3b/d70 22 2024-03-22T07:08:51.301 INFO:tasks.workunit.client.1.smithi149.stdout:9/353: mknod db/d21/c71 0 2024-03-22T07:08:51.313 INFO:tasks.workunit.client.1.smithi149.stdout:4/302: dwrite d0/d8/f1f [0,4194304] 0 2024-03-22T07:08:51.315 INFO:tasks.workunit.client.1.smithi149.stdout:4/303: rename d0/d8/d1d/d24/l27 to d0/d8/d5c/l63 0 2024-03-22T07:08:51.318 INFO:tasks.workunit.client.1.smithi149.stdout:8/353: dwrite d1/d20/d35/d22/f40 [0,4194304] 0 2024-03-22T07:08:51.320 INFO:tasks.workunit.client.1.smithi149.stdout:8/354: write d1/d2/d9/d11/d4d/f67 [349799,25600] 0 2024-03-22T07:08:51.320 INFO:tasks.workunit.client.1.smithi149.stdout:8/355: stat d1/d2/d18/f3c 0 2024-03-22T07:08:51.327 INFO:tasks.workunit.client.1.smithi149.stdout:6/310: dwrite d9/df/d22/f4b [0,4194304] 0 2024-03-22T07:08:51.330 INFO:tasks.workunit.client.1.smithi149.stdout:9/354: dwrite db/d42/f4c [0,4194304] 0 2024-03-22T07:08:51.330 INFO:tasks.workunit.client.1.smithi149.stdout:9/355: creat db/d19/f72 x:0 0 0 2024-03-22T07:08:51.333 INFO:tasks.workunit.client.1.smithi149.stdout:9/356: link db/d19/c2c db/d42/d49/d4b/d5f/c73 0 2024-03-22T07:08:51.334 INFO:tasks.workunit.client.1.smithi149.stdout:9/357: rename db/d21/d2e/d6b to db/d21/d74 0 2024-03-22T07:08:51.341 INFO:tasks.workunit.client.1.smithi149.stdout:8/356: dwrite d1/d2/d9/d11/f7a [0,4194304] 0 2024-03-22T07:08:51.341 INFO:tasks.workunit.client.1.smithi149.stdout:8/357: dread - d1/d20/d35/d22/d44/d4f/f56 zero size 2024-03-22T07:08:51.341 INFO:tasks.workunit.client.1.smithi149.stdout:4/304: dwrite d0/d8/dc/d14/d3e/f51 [0,4194304] 0 2024-03-22T07:08:51.345 INFO:tasks.workunit.client.1.smithi149.stdout:4/305: link d0/f32 d0/d8/d4c/f64 0 2024-03-22T07:08:51.345 INFO:tasks.workunit.client.1.smithi149.stdout:4/306: chown d0/d8/dc/d14/d3e/f61 432 1 2024-03-22T07:08:51.345 INFO:tasks.workunit.client.1.smithi149.stdout:4/307: getdents d0/d29/d55 0 2024-03-22T07:08:51.345 INFO:tasks.workunit.client.1.smithi149.stdout:4/308: fdatasync d0/d29/f49 0 2024-03-22T07:08:51.350 INFO:tasks.workunit.client.1.smithi149.stdout:9/358: dwrite db/d21/d37/f5c [0,4194304] 0 2024-03-22T07:08:51.353 INFO:tasks.workunit.client.1.smithi149.stdout:2/332: sync 2024-03-22T07:08:51.353 INFO:tasks.workunit.client.1.smithi149.stdout:2/333: write d1/d3/f54 [538684,82169] 0 2024-03-22T07:08:51.354 INFO:tasks.workunit.client.1.smithi149.stdout:9/359: truncate f3 4961800 0 2024-03-22T07:08:51.357 INFO:tasks.workunit.client.1.smithi149.stdout:2/334: mkdir d1/d3/dd/d71 0 2024-03-22T07:08:51.361 INFO:tasks.workunit.client.1.smithi149.stdout:9/360: dread db/d21/d37/f4e [0,4194304] 0 2024-03-22T07:08:51.362 INFO:tasks.workunit.client.1.smithi149.stdout:2/335: mkdir d1/d72 0 2024-03-22T07:08:51.362 INFO:tasks.workunit.client.1.smithi149.stdout:9/361: symlink db/d21/d37/l75 0 2024-03-22T07:08:51.362 INFO:tasks.workunit.client.1.smithi149.stdout:9/362: chown db/d21/d37/c3e 144723 1 2024-03-22T07:08:51.362 INFO:tasks.workunit.client.1.smithi149.stdout:9/363: creat db/d21/d74/f76 x:0 0 0 2024-03-22T07:08:51.364 INFO:tasks.workunit.client.1.smithi149.stdout:9/364: rename db/d21/d37/l75 to db/d21/d37/d3a/l77 0 2024-03-22T07:08:51.368 INFO:tasks.workunit.client.1.smithi149.stdout:8/358: dwrite d1/d2/d9/d11/f55 [0,4194304] 0 2024-03-22T07:08:51.368 INFO:tasks.workunit.client.1.smithi149.stdout:8/359: write d1/d2/d9/f4a [288993,16012] 0 2024-03-22T07:08:51.375 INFO:tasks.workunit.client.1.smithi149.stdout:9/365: dread db/d12/f35 [0,4194304] 0 2024-03-22T07:08:51.377 INFO:tasks.workunit.client.1.smithi149.stdout:8/360: write d1/d2/d9/f29 [324796,8672] 0 2024-03-22T07:08:51.377 INFO:tasks.workunit.client.1.smithi149.stdout:4/309: dwrite d0/d29/f3c [0,4194304] 0 2024-03-22T07:08:51.377 INFO:tasks.workunit.client.1.smithi149.stdout:4/310: write d0/f32 [60501,4337] 0 2024-03-22T07:08:51.377 INFO:tasks.workunit.client.1.smithi149.stdout:4/311: truncate d0/d8/f46 381823 0 2024-03-22T07:08:51.377 INFO:tasks.workunit.client.1.smithi149.stdout:4/312: stat d0/d29/f49 0 2024-03-22T07:08:51.378 INFO:tasks.workunit.client.1.smithi149.stdout:8/361: mknod d1/d2/d9/df/c7b 0 2024-03-22T07:08:51.378 INFO:tasks.workunit.client.1.smithi149.stdout:2/336: dwrite d1/d8/d34/f57 [0,4194304] 0 2024-03-22T07:08:51.379 INFO:tasks.workunit.client.1.smithi149.stdout:4/313: mknod d0/c65 0 2024-03-22T07:08:51.381 INFO:tasks.workunit.client.1.smithi149.stdout:8/362: rename d1/d20/d35/d22/d44/c5c to d1/d2/c7c 0 2024-03-22T07:08:51.386 INFO:tasks.workunit.client.1.smithi149.stdout:4/314: rename d0/d29/f49 to d0/d8/dc/d2e/f66 0 2024-03-22T07:08:51.389 INFO:tasks.workunit.client.1.smithi149.stdout:4/315: creat d0/f67 x:0 0 0 2024-03-22T07:08:51.394 INFO:tasks.workunit.client.1.smithi149.stdout:4/316: unlink d0/d8/dc/d2e/f66 0 2024-03-22T07:08:51.394 INFO:tasks.workunit.client.1.smithi149.stdout:4/317: fsync d0/d8/d1d/f48 0 2024-03-22T07:08:51.398 INFO:tasks.workunit.client.1.smithi149.stdout:9/366: dwrite db/d21/d74/f76 [0,4194304] 0 2024-03-22T07:08:51.398 INFO:tasks.workunit.client.1.smithi149.stdout:9/367: fsync db/d42/f4c 0 2024-03-22T07:08:51.399 INFO:tasks.workunit.client.1.smithi149.stdout:2/337: dwrite d1/d3/d5/f2b [0,4194304] 0 2024-03-22T07:08:51.401 INFO:tasks.workunit.client.1.smithi149.stdout:9/368: rename db/d21/d37/d3a/c40 to db/d19/d39/d3b/c78 0 2024-03-22T07:08:51.403 INFO:tasks.workunit.client.1.smithi149.stdout:4/318: dread d0/d29/f35 [0,4194304] 0 2024-03-22T07:08:51.404 INFO:tasks.workunit.client.1.smithi149.stdout:4/319: mkdir d0/d8/d1d/d68 0 2024-03-22T07:08:51.405 INFO:tasks.workunit.client.1.smithi149.stdout:4/320: rename d0/d8/dc/d4b/c50 to d0/d8/d1d/d24/c69 0 2024-03-22T07:08:51.405 INFO:tasks.workunit.client.1.smithi149.stdout:4/321: creat d0/d8/dc/d14/d3e/f6a x:0 0 0 2024-03-22T07:08:51.406 INFO:tasks.workunit.client.1.smithi149.stdout:4/322: write d0/d8/d4c/f64 [797370,110526] 0 2024-03-22T07:08:51.408 INFO:tasks.workunit.client.1.smithi149.stdout:4/323: truncate d0/d8/dc/fe 1655167 0 2024-03-22T07:08:51.409 INFO:tasks.workunit.client.1.smithi149.stdout:2/338: dread d1/d3/d5/f2b [0,4194304] 0 2024-03-22T07:08:51.411 INFO:tasks.workunit.client.1.smithi149.stdout:4/324: mkdir d0/d6b 0 2024-03-22T07:08:51.413 INFO:tasks.workunit.client.1.smithi149.stdout:9/369: read db/d42/f4c [1158327,22640] 0 2024-03-22T07:08:51.415 INFO:tasks.workunit.client.1.smithi149.stdout:9/370: mknod db/d21/d74/c79 0 2024-03-22T07:08:51.423 INFO:tasks.workunit.client.1.smithi149.stdout:4/325: dread d0/d8/dc/d4b/f4d [0,4194304] 0 2024-03-22T07:08:51.423 INFO:tasks.workunit.client.1.smithi149.stdout:4/326: write d0/d8/d1d/d24/f54 [914764,23941] 0 2024-03-22T07:08:51.426 INFO:tasks.workunit.client.1.smithi149.stdout:4/327: link d0/d8/dc/c25 d0/c6c 0 2024-03-22T07:08:51.427 INFO:tasks.workunit.client.1.smithi149.stdout:4/328: rename d0/d8/d1d/f48 to d0/d8/d4c/f6d 0 2024-03-22T07:08:51.427 INFO:tasks.workunit.client.1.smithi149.stdout:4/329: creat d0/d29/f6e x:0 0 0 2024-03-22T07:08:51.429 INFO:tasks.workunit.client.1.smithi149.stdout:4/330: mkdir d0/d8/dc/d2e/d6f 0 2024-03-22T07:08:51.429 INFO:tasks.workunit.client.1.smithi149.stdout:4/331: chown d0/c12 0 1 2024-03-22T07:08:51.429 INFO:tasks.workunit.client.1.smithi149.stdout:4/332: fdatasync d0/d29/f3c 0 2024-03-22T07:08:51.433 INFO:tasks.workunit.client.1.smithi149.stdout:2/339: dwrite d1/d3/f36 [0,4194304] 0 2024-03-22T07:08:51.434 INFO:tasks.workunit.client.1.smithi149.stdout:9/371: dwrite db/d12/f14 [4194304,4194304] 0 2024-03-22T07:08:51.434 INFO:tasks.workunit.client.1.smithi149.stdout:0/220: sync 2024-03-22T07:08:51.434 INFO:tasks.workunit.client.1.smithi149.stdout:0/221: fdatasync d1/d1a/f2d 0 2024-03-22T07:08:51.434 INFO:tasks.workunit.client.1.smithi149.stdout:0/222: fdatasync d1/d9/f13 0 2024-03-22T07:08:51.434 INFO:tasks.workunit.client.1.smithi149.stdout:0/223: truncate d1/d31/d36/f45 30714 0 2024-03-22T07:08:51.435 INFO:tasks.workunit.client.1.smithi149.stdout:2/340: dread d1/d3/d5/d41/f48 [0,4194304] 0 2024-03-22T07:08:51.440 INFO:tasks.workunit.client.1.smithi149.stdout:9/372: unlink db/d12/f35 0 2024-03-22T07:08:51.442 INFO:tasks.workunit.client.1.smithi149.stdout:0/224: symlink d1/d1a/l4c 0 2024-03-22T07:08:51.444 INFO:tasks.workunit.client.1.smithi149.stdout:2/341: mknod d1/d4d/d6a/c73 0 2024-03-22T07:08:51.445 INFO:tasks.workunit.client.1.smithi149.stdout:4/333: dwrite d0/d29/f4a [4194304,4194304] 0 2024-03-22T07:08:51.445 INFO:tasks.workunit.client.1.smithi149.stdout:4/334: creat d0/d8/dc/d4b/f70 x:0 0 0 2024-03-22T07:08:51.445 INFO:tasks.workunit.client.1.smithi149.stdout:4/335: fdatasync d0/f3 0 2024-03-22T07:08:51.445 INFO:tasks.workunit.client.1.smithi149.stdout:4/336: creat d0/d8/dc/d4b/f71 x:0 0 0 2024-03-22T07:08:51.446 INFO:tasks.workunit.client.1.smithi149.stdout:0/225: dread d1/d9/d14/f28 [0,4194304] 0 2024-03-22T07:08:51.446 INFO:tasks.workunit.client.1.smithi149.stdout:9/373: rename db/cf to db/d21/c7a 0 2024-03-22T07:08:51.446 INFO:tasks.workunit.client.1.smithi149.stdout:9/374: dread - db/d19/f72 zero size 2024-03-22T07:08:51.447 INFO:tasks.workunit.client.1.smithi149.stdout:9/375: read - db/d42/d49/d4b/d4d/f66 zero size 2024-03-22T07:08:51.447 INFO:tasks.workunit.client.1.smithi149.stdout:9/376: dread - db/d21/d37/d3a/f6c zero size 2024-03-22T07:08:51.447 INFO:tasks.workunit.client.1.smithi149.stdout:2/342: mkdir d1/d3/d6/d3f/d74 0 2024-03-22T07:08:51.452 INFO:tasks.workunit.client.1.smithi149.stdout:9/377: mknod db/d42/c7b 0 2024-03-22T07:08:51.453 INFO:tasks.workunit.client.1.smithi149.stdout:2/343: mkdir d1/d4d/d69/d75 0 2024-03-22T07:08:51.453 INFO:tasks.workunit.client.1.smithi149.stdout:2/344: write d1/d3/d5/f5a [1835345,117055] 0 2024-03-22T07:08:51.455 INFO:tasks.workunit.client.1.smithi149.stdout:2/345: creat d1/d3/dd/f76 x:0 0 0 2024-03-22T07:08:51.458 INFO:tasks.workunit.client.1.smithi149.stdout:2/346: dread d1/d3/f3e [0,4194304] 0 2024-03-22T07:08:51.467 INFO:tasks.workunit.client.1.smithi149.stdout:9/378: write db/d19/d1c/f4f [1835540,120231] 0 2024-03-22T07:08:51.468 INFO:tasks.workunit.client.1.smithi149.stdout:4/337: dwrite d0/d8/dc/f18 [0,4194304] 0 2024-03-22T07:08:51.468 INFO:tasks.workunit.client.1.smithi149.stdout:9/379: mkdir db/d21/d37/d7c 0 2024-03-22T07:08:51.469 INFO:tasks.workunit.client.1.smithi149.stdout:4/338: truncate d0/d8/d4c/f64 426576 0 2024-03-22T07:08:51.471 INFO:tasks.workunit.client.1.smithi149.stdout:4/339: symlink d0/d8/d5c/l72 0 2024-03-22T07:08:51.471 INFO:tasks.workunit.client.1.smithi149.stdout:4/340: write d0/d8/dc/d14/d3e/f61 [265398,21816] 0 2024-03-22T07:08:51.472 INFO:tasks.workunit.client.1.smithi149.stdout:9/380: rmdir db/d19/d39 39 2024-03-22T07:08:51.472 INFO:tasks.workunit.client.1.smithi149.stdout:9/381: creat db/d19/f7d x:0 0 0 2024-03-22T07:08:51.474 INFO:tasks.workunit.client.1.smithi149.stdout:4/341: rename d0/d8/dc/c33 to d0/d8/c73 0 2024-03-22T07:08:51.474 INFO:tasks.workunit.client.1.smithi149.stdout:4/342: write d0/f2d [879449,66252] 0 2024-03-22T07:08:51.474 INFO:tasks.workunit.client.1.smithi149.stdout:9/382: write db/d21/d37/f4e [1060499,57464] 0 2024-03-22T07:08:51.475 INFO:tasks.workunit.client.1.smithi149.stdout:4/343: mkdir d0/d8/dc/d74 0 2024-03-22T07:08:51.475 INFO:tasks.workunit.client.1.smithi149.stdout:9/383: symlink db/d42/d49/d4b/l7e 0 2024-03-22T07:08:51.475 INFO:tasks.workunit.client.1.smithi149.stdout:9/384: readlink db/d19/d1c/l56 0 2024-03-22T07:08:51.476 INFO:tasks.workunit.client.1.smithi149.stdout:9/385: fdatasync db/d22/f69 0 2024-03-22T07:08:51.476 INFO:tasks.workunit.client.1.smithi149.stdout:4/344: creat d0/d8/dc/d14/f75 x:0 0 0 2024-03-22T07:08:51.477 INFO:tasks.workunit.client.1.smithi149.stdout:4/345: mkdir d0/d8/dc/d74/d76 0 2024-03-22T07:08:51.477 INFO:tasks.workunit.client.1.smithi149.stdout:4/346: mknod d0/d8/dc/d2e/c77 0 2024-03-22T07:08:51.482 INFO:tasks.workunit.client.1.smithi149.stdout:0/226: dwrite d1/d9/f3a [0,4194304] 0 2024-03-22T07:08:51.484 INFO:tasks.workunit.client.1.smithi149.stdout:9/386: write db/d19/f3c [213905,36382] 0 2024-03-22T07:08:51.484 INFO:tasks.workunit.client.1.smithi149.stdout:9/387: dread - db/d33/f67 zero size 2024-03-22T07:08:51.485 INFO:tasks.workunit.client.1.smithi149.stdout:2/347: dwrite d1/d3/d5/d1f/d5e/f37 [0,4194304] 0 2024-03-22T07:08:51.485 INFO:tasks.workunit.client.1.smithi149.stdout:9/388: unlink db/d19/d28/c5d 0 2024-03-22T07:08:51.489 INFO:tasks.workunit.client.1.smithi149.stdout:2/348: symlink d1/d3/d5/d1f/d5e/d2f/l77 0 2024-03-22T07:08:51.489 INFO:tasks.workunit.client.1.smithi149.stdout:2/349: write d1/f27 [4777211,84593] 0 2024-03-22T07:08:51.490 INFO:tasks.workunit.client.1.smithi149.stdout:4/347: write d0/d8/dc/f30 [8203694,47216] 0 2024-03-22T07:08:51.491 INFO:tasks.workunit.client.1.smithi149.stdout:9/389: mkdir db/d42/d49/d4b/d5f/d7f 0 2024-03-22T07:08:51.494 INFO:tasks.workunit.client.1.smithi149.stdout:2/350: unlink d1/f27 0 2024-03-22T07:08:51.496 INFO:tasks.workunit.client.1.smithi149.stdout:4/348: mknod d0/d8/d1d/c78 0 2024-03-22T07:08:51.496 INFO:tasks.workunit.client.1.smithi149.stdout:4/349: write d0/d8/dc/d2e/f2f [4415561,98406] 0 2024-03-22T07:08:51.497 INFO:tasks.workunit.client.1.smithi149.stdout:2/351: stat d1/d8/l14 0 2024-03-22T07:08:51.500 INFO:tasks.workunit.client.1.smithi149.stdout:2/352: mknod d1/d3/c78 0 2024-03-22T07:08:51.501 INFO:tasks.workunit.client.1.smithi149.stdout:2/353: creat d1/d3/d5/d1f/d68/f79 x:0 0 0 2024-03-22T07:08:51.502 INFO:tasks.workunit.client.1.smithi149.stdout:9/390: dread f3 [0,4194304] 0 2024-03-22T07:08:51.503 INFO:tasks.workunit.client.1.smithi149.stdout:2/354: unlink d1/d3/d5/d1f/d5e/f37 0 2024-03-22T07:08:51.504 INFO:tasks.workunit.client.1.smithi149.stdout:9/391: unlink db/d21/d37/f4e 0 2024-03-22T07:08:51.504 INFO:tasks.workunit.client.1.smithi149.stdout:9/392: truncate db/d22/f69 277840 0 2024-03-22T07:08:51.505 INFO:tasks.workunit.client.1.smithi149.stdout:2/355: dread d1/d3/dd/f65 [0,4194304] 0 2024-03-22T07:08:51.505 INFO:tasks.workunit.client.1.smithi149.stdout:2/356: chown d1/d3/d5/d1f/f1b 0 1 2024-03-22T07:08:51.505 INFO:tasks.workunit.client.1.smithi149.stdout:9/393: rmdir db/d42/d49/d4b 39 2024-03-22T07:08:51.506 INFO:tasks.workunit.client.1.smithi149.stdout:4/350: dread d0/f3 [0,4194304] 0 2024-03-22T07:08:51.507 INFO:tasks.workunit.client.1.smithi149.stdout:0/227: dwrite d1/d31/f33 [0,4194304] 0 2024-03-22T07:08:51.508 INFO:tasks.workunit.client.1.smithi149.stdout:4/351: dread d0/d8/f46 [0,4194304] 0 2024-03-22T07:08:51.509 INFO:tasks.workunit.client.1.smithi149.stdout:9/394: dread db/d22/f36 [0,4194304] 0 2024-03-22T07:08:51.518 INFO:tasks.workunit.client.1.smithi149.stdout:4/352: mknod d0/d8/d1d/c79 0 2024-03-22T07:08:51.518 INFO:tasks.workunit.client.1.smithi149.stdout:4/353: chown d0/d8/dc/d14/f16 6 1 2024-03-22T07:08:51.518 INFO:tasks.workunit.client.1.smithi149.stdout:0/228: dread d1/d31/f34 [0,4194304] 0 2024-03-22T07:08:51.518 INFO:tasks.workunit.client.1.smithi149.stdout:0/229: dread - d1/d31/d36/f47 zero size 2024-03-22T07:08:51.518 INFO:tasks.workunit.client.1.smithi149.stdout:0/230: creat d1/d9/d14/d1f/f4d x:0 0 0 2024-03-22T07:08:51.518 INFO:tasks.workunit.client.1.smithi149.stdout:9/395: link db/d21/d37/f5c db/d19/d28/f80 0 2024-03-22T07:08:51.519 INFO:tasks.workunit.client.1.smithi149.stdout:4/354: mknod d0/d8/d4c/c7a 0 2024-03-22T07:08:51.521 INFO:tasks.workunit.client.1.smithi149.stdout:9/396: symlink db/d12/l81 0 2024-03-22T07:08:51.526 INFO:tasks.workunit.client.1.smithi149.stdout:1/328: sync 2024-03-22T07:08:51.526 INFO:tasks.workunit.client.1.smithi149.stdout:5/289: sync 2024-03-22T07:08:51.526 INFO:tasks.workunit.client.1.smithi149.stdout:3/257: sync 2024-03-22T07:08:51.526 INFO:tasks.workunit.client.1.smithi149.stdout:6/311: sync 2024-03-22T07:08:51.527 INFO:tasks.workunit.client.1.smithi149.stdout:7/385: sync 2024-03-22T07:08:51.527 INFO:tasks.workunit.client.1.smithi149.stdout:6/312: fdatasync d9/d1f/f21 0 2024-03-22T07:08:51.527 INFO:tasks.workunit.client.1.smithi149.stdout:7/386: fdatasync d1/d18/d1b/d51/d5a/d6a/f6e 0 2024-03-22T07:08:51.527 INFO:tasks.workunit.client.1.smithi149.stdout:5/290: truncate d7/da/d20/d30/d3a/d3d/d2c/d47/f52 253155 0 2024-03-22T07:08:51.527 INFO:tasks.workunit.client.1.smithi149.stdout:7/387: chown d1/d1c/d2f/l71 6 1 2024-03-22T07:08:51.527 INFO:tasks.workunit.client.1.smithi149.stdout:8/363: sync 2024-03-22T07:08:51.527 INFO:tasks.workunit.client.1.smithi149.stdout:8/364: write d1/d2/d3/f76 [728744,126075] 0 2024-03-22T07:08:51.527 INFO:tasks.workunit.client.1.smithi149.stdout:2/357: dwrite d1/f10 [8388608,4194304] 0 2024-03-22T07:08:51.527 INFO:tasks.workunit.client.1.smithi149.stdout:2/358: creat d1/d8/d34/f7a x:0 0 0 2024-03-22T07:08:51.528 INFO:tasks.workunit.client.1.smithi149.stdout:1/329: creat d4/d7/d5d/f72 x:0 0 0 2024-03-22T07:08:51.528 INFO:tasks.workunit.client.1.smithi149.stdout:1/330: chown d4/d1f/d6a/d6b 823 1 2024-03-22T07:08:51.528 INFO:tasks.workunit.client.1.smithi149.stdout:1/331: creat d4/d20/f73 x:0 0 0 2024-03-22T07:08:51.528 INFO:tasks.workunit.client.1.smithi149.stdout:1/332: readlink d4/d10/d28/l4e 0 2024-03-22T07:08:51.530 INFO:tasks.workunit.client.1.smithi149.stdout:6/313: symlink d9/d1f/d24/d42/l64 0 2024-03-22T07:08:51.530 INFO:tasks.workunit.client.1.smithi149.stdout:7/388: rmdir d1/d1c/d49 39 2024-03-22T07:08:51.530 INFO:tasks.workunit.client.1.smithi149.stdout:7/389: dread - d1/d18/d1b/d51/d5a/f78 zero size 2024-03-22T07:08:51.531 INFO:tasks.workunit.client.1.smithi149.stdout:4/355: dread d0/f3 [4194304,4194304] 0 2024-03-22T07:08:51.532 INFO:tasks.workunit.client.1.smithi149.stdout:9/397: read db/d21/d37/f5c [763753,16025] 0 2024-03-22T07:08:51.534 INFO:tasks.workunit.client.1.smithi149.stdout:5/291: link d7/d35/f40 d7/f58 0 2024-03-22T07:08:51.535 INFO:tasks.workunit.client.1.smithi149.stdout:7/390: getdents d1/d18/d1b 0 2024-03-22T07:08:51.536 INFO:tasks.workunit.client.1.smithi149.stdout:0/231: write d1/d31/d36/f43 [3499329,110777] 0 2024-03-22T07:08:51.536 INFO:tasks.workunit.client.1.smithi149.stdout:0/232: chown d1/d9 1617544318 1 2024-03-22T07:08:51.536 INFO:tasks.workunit.client.1.smithi149.stdout:0/233: read d1/f22 [3422174,90655] 0 2024-03-22T07:08:51.536 INFO:tasks.workunit.client.1.smithi149.stdout:6/314: dread d9/d1f/f21 [0,4194304] 0 2024-03-22T07:08:51.537 INFO:tasks.workunit.client.1.smithi149.stdout:1/333: link d4/d20/d23/d59/f5c d4/d20/d23/f74 0 2024-03-22T07:08:51.538 INFO:tasks.workunit.client.1.smithi149.stdout:5/292: mknod d7/da/d20/d30/d3a/d3d/d2c/d47/c59 0 2024-03-22T07:08:51.539 INFO:tasks.workunit.client.1.smithi149.stdout:7/391: creat d1/d1c/d4d/d56/f81 x:0 0 0 2024-03-22T07:08:51.541 INFO:tasks.workunit.client.1.smithi149.stdout:6/315: symlink d9/d1f/d24/d34/d37/d38/l65 0 2024-03-22T07:08:51.542 INFO:tasks.workunit.client.1.smithi149.stdout:1/334: mknod d4/d7/c75 0 2024-03-22T07:08:51.543 INFO:tasks.workunit.client.1.smithi149.stdout:5/293: symlink d7/da/d20/d36/d44/l5a 0 2024-03-22T07:08:51.543 INFO:tasks.workunit.client.1.smithi149.stdout:5/294: chown d7/f58 27414061 1 2024-03-22T07:08:51.545 INFO:tasks.workunit.client.1.smithi149.stdout:7/392: unlink d1/c24 0 2024-03-22T07:08:51.546 INFO:tasks.workunit.client.1.smithi149.stdout:0/234: mknod d1/c4e 0 2024-03-22T07:08:51.546 INFO:tasks.workunit.client.1.smithi149.stdout:0/235: fsync d1/d9/d14/d1f/f37 0 2024-03-22T07:08:51.546 INFO:tasks.workunit.client.1.smithi149.stdout:0/236: chown d1/d1a/l4c 55 1 2024-03-22T07:08:51.546 INFO:tasks.workunit.client.1.smithi149.stdout:0/237: dread - d1/d20/f3d zero size 2024-03-22T07:08:51.547 INFO:tasks.workunit.client.1.smithi149.stdout:5/295: dread d7/da/d20/f4c [0,4194304] 0 2024-03-22T07:08:51.548 INFO:tasks.workunit.client.1.smithi149.stdout:5/296: read d7/f9 [1343217,68565] 0 2024-03-22T07:08:51.548 INFO:tasks.workunit.client.1.smithi149.stdout:7/393: unlink d1/d18/d30/l6d 0 2024-03-22T07:08:51.548 INFO:tasks.workunit.client.1.smithi149.stdout:7/394: fdatasync d1/d1c/f7b 0 2024-03-22T07:08:51.549 INFO:tasks.workunit.client.1.smithi149.stdout:6/316: dread d9/df/f1e [0,4194304] 0 2024-03-22T07:08:51.550 INFO:tasks.workunit.client.1.smithi149.stdout:5/297: rename d7/da/d20/d30/d3a/d3d/d2c/d47/c53 to d7/d35/c5b 0 2024-03-22T07:08:51.550 INFO:tasks.workunit.client.1.smithi149.stdout:5/298: dread - d7/d35/f55 zero size 2024-03-22T07:08:51.551 INFO:tasks.workunit.client.1.smithi149.stdout:6/317: mknod d9/df/d2e/d4e/c66 0 2024-03-22T07:08:51.552 INFO:tasks.workunit.client.1.smithi149.stdout:5/299: mknod d7/db/d2b/c5c 0 2024-03-22T07:08:51.552 INFO:tasks.workunit.client.1.smithi149.stdout:4/356: read d0/d29/f3c [866545,22700] 0 2024-03-22T07:08:51.552 INFO:tasks.workunit.client.1.smithi149.stdout:4/357: chown d0/d8/d1d 2742545 1 2024-03-22T07:08:51.553 INFO:tasks.workunit.client.1.smithi149.stdout:4/358: truncate d0/d8/d5c/f5e 31231 0 2024-03-22T07:08:51.553 INFO:tasks.workunit.client.1.smithi149.stdout:3/258: dwrite d0/d2/d42/f4e [0,4194304] 0 2024-03-22T07:08:51.553 INFO:tasks.workunit.client.1.smithi149.stdout:3/259: stat d0/d2/d30/l3a 0 2024-03-22T07:08:51.553 INFO:tasks.workunit.client.1.smithi149.stdout:5/300: unlink d7/da/d20/d30/f56 0 2024-03-22T07:08:51.558 INFO:tasks.workunit.client.1.smithi149.stdout:5/301: creat d7/da/d20/d36/d44/f5d x:0 0 0 2024-03-22T07:08:51.558 INFO:tasks.workunit.client.1.smithi149.stdout:3/260: rmdir d0/d2/d5a 39 2024-03-22T07:08:51.558 INFO:tasks.workunit.client.1.smithi149.stdout:1/335: write d4/d10/d28/f65 [316996,119869] 0 2024-03-22T07:08:51.560 INFO:tasks.workunit.client.1.smithi149.stdout:5/302: creat d7/da/d20/f5e x:0 0 0 2024-03-22T07:08:51.561 INFO:tasks.workunit.client.1.smithi149.stdout:7/395: dread d1/d18/d1b/f42 [0,4194304] 0 2024-03-22T07:08:51.562 INFO:tasks.workunit.client.1.smithi149.stdout:3/261: mknod d0/d2/d30/c60 0 2024-03-22T07:08:51.562 INFO:tasks.workunit.client.1.smithi149.stdout:1/336: rmdir d4/d1f/d6a 39 2024-03-22T07:08:51.563 INFO:tasks.workunit.client.1.smithi149.stdout:1/337: chown d4/c6c 397 1 2024-03-22T07:08:51.566 INFO:tasks.workunit.client.1.smithi149.stdout:5/303: symlink d7/da/d20/d30/l5f 0 2024-03-22T07:08:51.566 INFO:tasks.workunit.client.1.smithi149.stdout:7/396: mknod d1/d18/d1b/d1d/c82 0 2024-03-22T07:08:51.568 INFO:tasks.workunit.client.1.smithi149.stdout:5/304: getdents d7/da/d20/d30/d3a/d3d 0 2024-03-22T07:08:51.576 INFO:tasks.workunit.client.1.smithi149.stdout:3/262: dread d0/d2/d42/f4e [0,4194304] 0 2024-03-22T07:08:51.584 INFO:tasks.workunit.client.1.smithi149.stdout:0/238: getdents d1 0 2024-03-22T07:08:51.586 INFO:tasks.workunit.client.1.smithi149.stdout:0/239: creat d1/d1a/f4f x:0 0 0 2024-03-22T07:08:51.586 INFO:tasks.workunit.client.1.smithi149.stdout:0/240: creat d1/d1a/f50 x:0 0 0 2024-03-22T07:08:51.587 INFO:tasks.workunit.client.1.smithi149.stdout:8/365: dwrite d1/d20/d35/d22/d44/d4f/f56 [0,4194304] 0 2024-03-22T07:08:51.591 INFO:tasks.workunit.client.1.smithi149.stdout:4/359: dwrite d0/d29/f4a [4194304,4194304] 0 2024-03-22T07:08:51.591 INFO:tasks.workunit.client.1.smithi149.stdout:8/366: getdents d1/d2/d9 0 2024-03-22T07:08:51.591 INFO:tasks.workunit.client.1.smithi149.stdout:8/367: fsync d1/d2/d9/f65 0 2024-03-22T07:08:51.592 INFO:tasks.workunit.client.1.smithi149.stdout:4/360: creat d0/d8/dc/d74/d76/f7b x:0 0 0 2024-03-22T07:08:51.593 INFO:tasks.workunit.client.1.smithi149.stdout:7/397: dwrite d1/d18/d1b/d1d/f16 [0,4194304] 0 2024-03-22T07:08:51.593 INFO:tasks.workunit.client.1.smithi149.stdout:6/318: dwrite d9/d1f/d24/d34/d37/f49 [0,4194304] 0 2024-03-22T07:08:51.594 INFO:tasks.workunit.client.1.smithi149.stdout:8/368: mknod d1/d20/c7d 0 2024-03-22T07:08:51.594 INFO:tasks.workunit.client.1.smithi149.stdout:5/305: fsync d7/da/d20/d36/d44/f5d 0 2024-03-22T07:08:51.594 INFO:tasks.workunit.client.1.smithi149.stdout:5/306: readlink d7/l21 0 2024-03-22T07:08:51.597 INFO:tasks.workunit.client.1.smithi149.stdout:6/319: unlink d9/f2c 0 2024-03-22T07:08:51.598 INFO:tasks.workunit.client.1.smithi149.stdout:6/320: creat d9/df/d22/f67 x:0 0 0 2024-03-22T07:08:51.605 INFO:tasks.workunit.client.1.smithi149.stdout:3/263: dwrite d0/d2/d30/d31/f3d [0,4194304] 0 2024-03-22T07:08:51.605 INFO:tasks.workunit.client.1.smithi149.stdout:0/241: write d1/d9/d14/f18 [1461465,103444] 0 2024-03-22T07:08:51.605 INFO:tasks.workunit.client.1.smithi149.stdout:0/242: stat d1/d1a/f4f 0 2024-03-22T07:08:51.605 INFO:tasks.workunit.client.1.smithi149.stdout:0/243: write d1/d27/f46 [380503,125170] 0 2024-03-22T07:08:51.607 INFO:tasks.workunit.client.1.smithi149.stdout:5/307: rename d7/da/d20/d30/d3a/d3d/d2c/d47/c59 to d7/da/d20/d36/c60 0 2024-03-22T07:08:51.609 INFO:tasks.workunit.client.1.smithi149.stdout:8/369: write d1/d20/d41/f63 [2765889,92098] 0 2024-03-22T07:08:51.609 INFO:tasks.workunit.client.1.smithi149.stdout:8/370: write d1/d2/d9/f3b [413277,23012] 0 2024-03-22T07:08:51.612 INFO:tasks.workunit.client.1.smithi149.stdout:3/264: symlink d0/d2/d30/d31/l61 0 2024-03-22T07:08:51.612 INFO:tasks.workunit.client.1.smithi149.stdout:5/308: mknod d7/db/d2b/c61 0 2024-03-22T07:08:51.613 INFO:tasks.workunit.client.1.smithi149.stdout:3/265: mkdir d0/d4/d8/d24/d40/d62 0 2024-03-22T07:08:51.614 INFO:tasks.workunit.client.1.smithi149.stdout:3/266: dread - d0/d4/d8/d24/f4f zero size 2024-03-22T07:08:51.616 INFO:tasks.workunit.client.1.smithi149.stdout:5/309: rename d7/l21 to d7/da/d20/d30/d3a/d3d/d2c/l62 0 2024-03-22T07:08:51.616 INFO:tasks.workunit.client.1.smithi149.stdout:5/310: write d7/da/d20/d30/d3a/d3d/d2c/f4d [225143,9203] 0 2024-03-22T07:08:51.620 INFO:tasks.workunit.client.1.smithi149.stdout:3/267: symlink d0/d4/d8/d24/l63 0 2024-03-22T07:08:51.621 INFO:tasks.workunit.client.1.smithi149.stdout:3/268: creat d0/d4/d8/d24/d40/d62/f64 x:0 0 0 2024-03-22T07:08:51.622 INFO:tasks.workunit.client.1.smithi149.stdout:3/269: stat d0/d4/d8/f21 0 2024-03-22T07:08:51.622 INFO:tasks.workunit.client.1.smithi149.stdout:3/270: write d0/d2/d36/f50 [824567,45245] 0 2024-03-22T07:08:51.623 INFO:tasks.workunit.client.1.smithi149.stdout:7/398: dwrite d1/d1c/f7b [0,4194304] 0 2024-03-22T07:08:51.624 INFO:tasks.workunit.client.1.smithi149.stdout:3/271: rmdir d0/d4/d8/d24/d40 39 2024-03-22T07:08:51.628 INFO:tasks.workunit.client.1.smithi149.stdout:3/272: rename d0/d2/d5a/f5e to d0/d2/d30/d5d/f65 0 2024-03-22T07:08:51.631 INFO:tasks.workunit.client.1.smithi149.stdout:4/361: dwrite d0/d8/dc/fe [0,4194304] 0 2024-03-22T07:08:51.631 INFO:tasks.workunit.client.1.smithi149.stdout:4/362: chown d0/c65 357487 1 2024-03-22T07:08:51.631 INFO:tasks.workunit.client.1.smithi149.stdout:3/273: truncate d0/f1a 2611401 0 2024-03-22T07:08:51.635 INFO:tasks.workunit.client.1.smithi149.stdout:3/274: rmdir d0/d2/d42 39 2024-03-22T07:08:51.636 INFO:tasks.workunit.client.1.smithi149.stdout:4/363: rmdir d0/d8/d5c 39 2024-03-22T07:08:51.636 INFO:tasks.workunit.client.1.smithi149.stdout:4/364: stat d0/d8/l1b 0 2024-03-22T07:08:51.636 INFO:tasks.workunit.client.1.smithi149.stdout:4/365: creat d0/d8/dc/f7c x:0 0 0 2024-03-22T07:08:51.637 INFO:tasks.workunit.client.1.smithi149.stdout:7/399: dread d1/d1c/d2f/f76 [0,4194304] 0 2024-03-22T07:08:51.637 INFO:tasks.workunit.client.1.smithi149.stdout:7/400: creat d1/d1c/d4d/f83 x:0 0 0 2024-03-22T07:08:51.637 INFO:tasks.workunit.client.1.smithi149.stdout:7/401: dread - d1/d18/d1b/d51/d5a/f78 zero size 2024-03-22T07:08:51.639 INFO:tasks.workunit.client.1.smithi149.stdout:4/366: creat d0/d6b/f7d x:0 0 0 2024-03-22T07:08:51.641 INFO:tasks.workunit.client.1.smithi149.stdout:6/321: dwrite d9/f19 [0,4194304] 0 2024-03-22T07:08:51.641 INFO:tasks.workunit.client.1.smithi149.stdout:6/322: dread - d9/d1f/f25 zero size 2024-03-22T07:08:51.642 INFO:tasks.workunit.client.1.smithi149.stdout:7/402: mknod d1/d18/d30/c84 0 2024-03-22T07:08:51.642 INFO:tasks.workunit.client.1.smithi149.stdout:7/403: chown d1/d1c/d2f/f43 0 1 2024-03-22T07:08:51.642 INFO:tasks.workunit.client.1.smithi149.stdout:7/404: getdents d1/d18/d1b/d1d/d72 0 2024-03-22T07:08:51.642 INFO:tasks.workunit.client.1.smithi149.stdout:7/405: write d1/d18/d1b/d51/d7a/d7d/d5d/f6b [6315,7518] 0 2024-03-22T07:08:51.643 INFO:tasks.workunit.client.1.smithi149.stdout:4/367: mknod d0/d8/dc/d14/c7e 0 2024-03-22T07:08:51.643 INFO:tasks.workunit.client.1.smithi149.stdout:4/368: dread - d0/d8/dc/f5b zero size 2024-03-22T07:08:51.643 INFO:tasks.workunit.client.1.smithi149.stdout:4/369: dread - d0/d8/dc/d14/d3e/f3d zero size 2024-03-22T07:08:51.644 INFO:tasks.workunit.client.1.smithi149.stdout:6/323: mkdir d9/d1f/d24/d68 0 2024-03-22T07:08:51.644 INFO:tasks.workunit.client.1.smithi149.stdout:6/324: stat d9/df/d2e/f20 0 2024-03-22T07:08:51.644 INFO:tasks.workunit.client.1.smithi149.stdout:6/325: write f8 [2198904,41222] 0 2024-03-22T07:08:51.646 INFO:tasks.workunit.client.1.smithi149.stdout:7/406: mkdir d1/d85 0 2024-03-22T07:08:51.646 INFO:tasks.workunit.client.1.smithi149.stdout:7/407: chown d1/d18/f1e 701 1 2024-03-22T07:08:51.646 INFO:tasks.workunit.client.1.smithi149.stdout:7/408: readlink d1/d18/d1b/d51/d7a/d7d/l61 0 2024-03-22T07:08:51.646 INFO:tasks.workunit.client.1.smithi149.stdout:7/409: fdatasync d1/d1c/d2f/f43 0 2024-03-22T07:08:51.646 INFO:tasks.workunit.client.1.smithi149.stdout:7/410: write d1/d18/d1b/d51/d5a/f78 [692793,32281] 0 2024-03-22T07:08:51.647 INFO:tasks.workunit.client.1.smithi149.stdout:4/370: creat d0/d8/dc/d2e/d6f/f7f x:0 0 0 2024-03-22T07:08:51.648 INFO:tasks.workunit.client.1.smithi149.stdout:0/244: dwrite d1/d1a/f4f [0,4194304] 0 2024-03-22T07:08:51.648 INFO:tasks.workunit.client.1.smithi149.stdout:5/311: dwrite d7/d35/f40 [4194304,4194304] 0 2024-03-22T07:08:51.649 INFO:tasks.workunit.client.1.smithi149.stdout:6/326: creat d9/df/d2e/d63/f69 x:0 0 0 2024-03-22T07:08:51.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/371: rename d0/f3 to d0/d8/dc/d4b/f80 0 2024-03-22T07:08:51.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/372: readlink d0/lf 0 2024-03-22T07:08:51.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/373: creat d0/d6b/f81 x:0 0 0 2024-03-22T07:08:51.656 INFO:tasks.workunit.client.1.smithi149.stdout:3/275: getdents d0/d2/d30/d5d 0 2024-03-22T07:08:51.659 INFO:tasks.workunit.client.1.smithi149.stdout:4/374: truncate d0/d29/f4a 2435467 0 2024-03-22T07:08:51.661 INFO:tasks.workunit.client.1.smithi149.stdout:8/371: dwrite d1/d20/d35/d22/f38 [0,4194304] 0 2024-03-22T07:08:51.661 INFO:tasks.workunit.client.1.smithi149.stdout:8/372: readlink d1/d20/d35/l14 0 2024-03-22T07:08:51.661 INFO:tasks.workunit.client.1.smithi149.stdout:8/373: fsync d1/d2/d9/df/f15 0 2024-03-22T07:08:51.661 INFO:tasks.workunit.client.1.smithi149.stdout:8/374: truncate d1/d2/d9/d11/d45/f71 821513 0 2024-03-22T07:08:51.661 INFO:tasks.workunit.client.1.smithi149.stdout:8/375: readlink d1/d20/d35/d22/d44/d4f/l64 0 2024-03-22T07:08:51.662 INFO:tasks.workunit.client.1.smithi149.stdout:8/376: creat d1/d20/d35/d22/d44/d4f/f7e x:0 0 0 2024-03-22T07:08:51.662 INFO:tasks.workunit.client.1.smithi149.stdout:3/276: mkdir d0/d2/d42/d66 0 2024-03-22T07:08:51.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/327: getdents d9/df/d22 0 2024-03-22T07:08:51.663 INFO:tasks.workunit.client.1.smithi149.stdout:4/375: mkdir d0/d8/d1d/d24/d82 0 2024-03-22T07:08:51.663 INFO:tasks.workunit.client.1.smithi149.stdout:4/376: creat d0/d8/dc/d2e/d6f/f83 x:0 0 0 2024-03-22T07:08:51.664 INFO:tasks.workunit.client.1.smithi149.stdout:0/245: rename d1/d31/d36 to d1/d31/d51 0 2024-03-22T07:08:51.667 INFO:tasks.workunit.client.1.smithi149.stdout:8/377: mknod d1/d2/c7f 0 2024-03-22T07:08:51.668 INFO:tasks.workunit.client.1.smithi149.stdout:8/378: truncate d1/d2/f43 685133 0 2024-03-22T07:08:51.668 INFO:tasks.workunit.client.1.smithi149.stdout:3/277: rmdir d0/d4/d8 39 2024-03-22T07:08:51.668 INFO:tasks.workunit.client.1.smithi149.stdout:3/278: readlink d0/l9 0 2024-03-22T07:08:51.670 INFO:tasks.workunit.client.1.smithi149.stdout:7/411: dwrite d1/d1c/d4d/f68 [0,4194304] 0 2024-03-22T07:08:51.672 INFO:tasks.workunit.client.1.smithi149.stdout:0/246: rename d1/d31/d44 to d1/d31/d52 0 2024-03-22T07:08:51.672 INFO:tasks.workunit.client.1.smithi149.stdout:0/247: read - d1/d9/d14/d1f/f4d zero size 2024-03-22T07:08:51.672 INFO:tasks.workunit.client.1.smithi149.stdout:0/248: fsync d1/d31/f34 0 2024-03-22T07:08:51.673 INFO:tasks.workunit.client.1.smithi149.stdout:8/379: rmdir d1/d20/d35 39 2024-03-22T07:08:51.673 INFO:tasks.workunit.client.1.smithi149.stdout:8/380: chown d1/d2/c7f 16230 1 2024-03-22T07:08:51.673 INFO:tasks.workunit.client.1.smithi149.stdout:6/328: dread d9/df/d22/f4b [0,4194304] 0 2024-03-22T07:08:51.673 INFO:tasks.workunit.client.1.smithi149.stdout:6/329: dread - d9/d1f/f41 zero size 2024-03-22T07:08:51.681 INFO:tasks.workunit.client.1.smithi149.stdout:3/279: write d0/d2/d30/d31/f3d [369136,41826] 0 2024-03-22T07:08:51.683 INFO:tasks.workunit.client.1.smithi149.stdout:7/412: write d1/d18/d1b/d1d/f1f [1451628,10432] 0 2024-03-22T07:08:51.683 INFO:tasks.workunit.client.1.smithi149.stdout:8/381: write d1/d20/d35/d22/f40 [1891693,47119] 0 2024-03-22T07:08:51.683 INFO:tasks.workunit.client.1.smithi149.stdout:8/382: write d1/d2/d3/f1a [1994461,95817] 0 2024-03-22T07:08:51.685 INFO:tasks.workunit.client.1.smithi149.stdout:0/249: rename d1/d27/f46 to d1/d31/f53 0 2024-03-22T07:08:51.686 INFO:tasks.workunit.client.1.smithi149.stdout:7/413: creat d1/d18/d1b/d51/d7a/d7d/f86 x:0 0 0 2024-03-22T07:08:51.686 INFO:tasks.workunit.client.1.smithi149.stdout:4/377: dwrite d0/d29/f31 [4194304,4194304] 0 2024-03-22T07:08:51.690 INFO:tasks.workunit.client.1.smithi149.stdout:0/250: write d1/d9/d14/f28 [447402,98258] 0 2024-03-22T07:08:51.691 INFO:tasks.workunit.client.1.smithi149.stdout:4/378: symlink d0/d8/d4c/l84 0 2024-03-22T07:08:51.691 INFO:tasks.workunit.client.1.smithi149.stdout:4/379: chown d0/d8/dc/d74/d76 2 1 2024-03-22T07:08:51.695 INFO:tasks.workunit.client.1.smithi149.stdout:0/251: link d1/d20/c21 d1/d1a/c54 0 2024-03-22T07:08:51.695 INFO:tasks.workunit.client.1.smithi149.stdout:6/330: write d9/d1f/d24/d34/d37/f49 [2927788,73743] 0 2024-03-22T07:08:51.699 INFO:tasks.workunit.client.1.smithi149.stdout:4/380: mkdir d0/d8/dc/d14/d85 0 2024-03-22T07:08:51.699 INFO:tasks.workunit.client.1.smithi149.stdout:4/381: write d0/d8/dc/d4b/f71 [260489,63612] 0 2024-03-22T07:08:51.699 INFO:tasks.workunit.client.1.smithi149.stdout:4/382: write d0/d6b/f7d [474045,24150] 0 2024-03-22T07:08:51.699 INFO:tasks.workunit.client.1.smithi149.stdout:4/383: creat d0/d8/dc/d4b/f86 x:0 0 0 2024-03-22T07:08:51.699 INFO:tasks.workunit.client.1.smithi149.stdout:4/384: fsync d0/d8/f46 0 2024-03-22T07:08:51.699 INFO:tasks.workunit.client.1.smithi149.stdout:4/385: fsync d0/d8/d1d/d24/f54 0 2024-03-22T07:08:51.699 INFO:tasks.workunit.client.1.smithi149.stdout:0/252: creat d1/d9/d14/f55 x:0 0 0 2024-03-22T07:08:51.701 INFO:tasks.workunit.client.1.smithi149.stdout:6/331: getdents d9/df/d2e/d4e 0 2024-03-22T07:08:51.703 INFO:tasks.workunit.client.1.smithi149.stdout:4/386: rmdir d0/d8/d1d/d24 39 2024-03-22T07:08:51.705 INFO:tasks.workunit.client.1.smithi149.stdout:7/414: dread d1/d1c/d2f/f43 [0,4194304] 0 2024-03-22T07:08:51.705 INFO:tasks.workunit.client.1.smithi149.stdout:4/387: link d0/d29/l2a d0/l87 0 2024-03-22T07:08:51.706 INFO:tasks.workunit.client.1.smithi149.stdout:4/388: creat d0/d8/d1d/d24/d82/f88 x:0 0 0 2024-03-22T07:08:51.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/332: read d9/d1f/d24/d34/d37/d38/f54 [1748268,75500] 0 2024-03-22T07:08:51.712 INFO:tasks.workunit.client.1.smithi149.stdout:8/383: dwrite d1/d2/d3/f1f [0,4194304] 0 2024-03-22T07:08:51.713 INFO:tasks.workunit.client.1.smithi149.stdout:6/333: symlink d9/d1f/d24/d34/l6a 0 2024-03-22T07:08:51.715 INFO:tasks.workunit.client.1.smithi149.stdout:8/384: dread d1/d2/d9/f3b [0,4194304] 0 2024-03-22T07:08:51.716 INFO:tasks.workunit.client.1.smithi149.stdout:6/334: rmdir d9/d1f/d5c 0 2024-03-22T07:08:51.716 INFO:tasks.workunit.client.1.smithi149.stdout:6/335: fsync d9/d1f/f48 0 2024-03-22T07:08:51.719 INFO:tasks.workunit.client.1.smithi149.stdout:8/385: rmdir d1/d2 39 2024-03-22T07:08:51.719 INFO:tasks.workunit.client.1.smithi149.stdout:8/386: write d1/d2/d9/d11/f58 [39724,34352] 0 2024-03-22T07:08:51.719 INFO:tasks.workunit.client.1.smithi149.stdout:3/280: dwrite d0/d4/d8/f21 [0,4194304] 0 2024-03-22T07:08:51.720 INFO:tasks.workunit.client.1.smithi149.stdout:8/387: symlink d1/d2/d9/d11/l80 0 2024-03-22T07:08:51.721 INFO:tasks.workunit.client.1.smithi149.stdout:3/281: symlink d0/d4/d8/d24/l67 0 2024-03-22T07:08:51.721 INFO:tasks.workunit.client.1.smithi149.stdout:3/282: read - d0/d4/d8/d24/f4f zero size 2024-03-22T07:08:51.726 INFO:tasks.workunit.client.1.smithi149.stdout:8/388: link d1/d2/d9/c30 d1/d20/d51/c81 0 2024-03-22T07:08:51.728 INFO:tasks.workunit.client.1.smithi149.stdout:8/389: mkdir d1/d20/d35/d22/d44/d82 0 2024-03-22T07:08:51.730 INFO:tasks.workunit.client.1.smithi149.stdout:8/390: symlink d1/d20/d35/d22/l83 0 2024-03-22T07:08:51.731 INFO:tasks.workunit.client.1.smithi149.stdout:8/391: truncate d1/d2/d9/d11/f7a 3151508 0 2024-03-22T07:08:51.734 INFO:tasks.workunit.client.1.smithi149.stdout:8/392: rename d1/d20/d41 to d1/d20/d35/d22/d44/d84 0 2024-03-22T07:08:51.734 INFO:tasks.workunit.client.1.smithi149.stdout:8/393: stat d1/d20/f54 0 2024-03-22T07:08:51.744 INFO:tasks.workunit.client.1.smithi149.stdout:2/359: sync 2024-03-22T07:08:51.744 INFO:tasks.workunit.client.1.smithi149.stdout:1/338: sync 2024-03-22T07:08:51.745 INFO:tasks.workunit.client.1.smithi149.stdout:2/360: mknod d1/d72/c7b 0 2024-03-22T07:08:51.745 INFO:tasks.workunit.client.1.smithi149.stdout:2/361: readlink d1/d8/l53 0 2024-03-22T07:08:51.745 INFO:tasks.workunit.client.1.smithi149.stdout:1/339: symlink d4/d7/l76 0 2024-03-22T07:08:51.745 INFO:tasks.workunit.client.1.smithi149.stdout:1/340: write d4/d10/f4d [4694688,87123] 0 2024-03-22T07:08:51.746 INFO:tasks.workunit.client.1.smithi149.stdout:2/362: symlink d1/d3/d6/d32/l7c 0 2024-03-22T07:08:51.746 INFO:tasks.workunit.client.1.smithi149.stdout:1/341: write d4/d20/d23/f44 [4468330,68144] 0 2024-03-22T07:08:51.746 INFO:tasks.workunit.client.1.smithi149.stdout:9/398: sync 2024-03-22T07:08:51.747 INFO:tasks.workunit.client.1.smithi149.stdout:1/342: symlink d4/d7/d5d/l77 0 2024-03-22T07:08:51.748 INFO:tasks.workunit.client.1.smithi149.stdout:9/399: mknod db/d42/d49/d4b/d5f/d7f/c82 0 2024-03-22T07:08:51.753 INFO:tasks.workunit.client.1.smithi149.stdout:9/400: link db/d21/d2e/c31 db/d33/c83 0 2024-03-22T07:08:51.753 INFO:tasks.workunit.client.1.smithi149.stdout:9/401: chown db/d19/d39/d3b/f48 359484 1 2024-03-22T07:08:51.753 INFO:tasks.workunit.client.1.smithi149.stdout:9/402: fdatasync db/d22/f45 0 2024-03-22T07:08:51.753 INFO:tasks.workunit.client.1.smithi149.stdout:0/253: dwrite d1/d9/d14/d1f/f4d [0,4194304] 0 2024-03-22T07:08:51.753 INFO:tasks.workunit.client.1.smithi149.stdout:4/389: dwrite d0/d8/dc/d4b/f86 [0,4194304] 0 2024-03-22T07:08:51.754 INFO:tasks.workunit.client.1.smithi149.stdout:3/283: dwrite d0/d2/d36/f50 [0,4194304] 0 2024-03-22T07:08:51.754 INFO:tasks.workunit.client.1.smithi149.stdout:3/284: chown d0/c4a 519185576 1 2024-03-22T07:08:51.756 INFO:tasks.workunit.client.1.smithi149.stdout:0/254: creat d1/f56 x:0 0 0 2024-03-22T07:08:51.756 INFO:tasks.workunit.client.1.smithi149.stdout:0/255: write d1/d9/d14/d1f/f24 [244299,72312] 0 2024-03-22T07:08:51.769 INFO:tasks.workunit.client.1.smithi149.stdout:4/390: symlink d0/d6b/l89 0 2024-03-22T07:08:51.770 INFO:tasks.workunit.client.1.smithi149.stdout:4/391: symlink d0/d8/dc/d74/l8a 0 2024-03-22T07:08:51.770 INFO:tasks.workunit.client.1.smithi149.stdout:4/392: write d0/f67 [797213,68009] 0 2024-03-22T07:08:51.771 INFO:tasks.workunit.client.1.smithi149.stdout:9/403: dread db/d42/f4c [0,4194304] 0 2024-03-22T07:08:51.771 INFO:tasks.workunit.client.1.smithi149.stdout:4/393: mknod d0/d29/c8b 0 2024-03-22T07:08:51.771 INFO:tasks.workunit.client.1.smithi149.stdout:4/394: fsync d0/d29/f35 0 2024-03-22T07:08:51.771 INFO:tasks.workunit.client.1.smithi149.stdout:6/336: dwrite d9/df/d2e/f23 [0,4194304] 0 2024-03-22T07:08:51.773 INFO:tasks.workunit.client.1.smithi149.stdout:4/395: mknod d0/d8/dc/d14/d3e/c8c 0 2024-03-22T07:08:51.776 INFO:tasks.workunit.client.1.smithi149.stdout:6/337: unlink d9/d1f/f25 0 2024-03-22T07:08:51.777 INFO:tasks.workunit.client.1.smithi149.stdout:9/404: dread db/d19/d39/d3b/f48 [0,4194304] 0 2024-03-22T07:08:51.777 INFO:tasks.workunit.client.1.smithi149.stdout:4/396: link d0/d8/dc/l17 d0/d29/l8d 0 2024-03-22T07:08:51.779 INFO:tasks.workunit.client.1.smithi149.stdout:6/338: rename d9/df/d2e/f4d to d9/d1f/f6b 0 2024-03-22T07:08:51.779 INFO:tasks.workunit.client.1.smithi149.stdout:6/339: readlink d9/d1f/d24/d34/l6a 0 2024-03-22T07:08:51.779 INFO:tasks.workunit.client.1.smithi149.stdout:6/340: fsync f8 0 2024-03-22T07:08:51.782 INFO:tasks.workunit.client.1.smithi149.stdout:9/405: creat db/d19/d28/f84 x:0 0 0 2024-03-22T07:08:51.783 INFO:tasks.workunit.client.1.smithi149.stdout:6/341: creat d9/df/f6c x:0 0 0 2024-03-22T07:08:51.785 INFO:tasks.workunit.client.1.smithi149.stdout:4/397: rmdir d0/d8/dc/d2e 39 2024-03-22T07:08:51.786 INFO:tasks.workunit.client.1.smithi149.stdout:4/398: write d0/d6b/f81 [446111,15007] 0 2024-03-22T07:08:51.786 INFO:tasks.workunit.client.1.smithi149.stdout:4/399: dread - d0/d8/dc/f5b zero size 2024-03-22T07:08:51.786 INFO:tasks.workunit.client.1.smithi149.stdout:4/400: write d0/d8/dc/d4b/f71 [1065965,62866] 0 2024-03-22T07:08:51.786 INFO:tasks.workunit.client.1.smithi149.stdout:4/401: truncate d0/f2d 1231420 0 2024-03-22T07:08:51.786 INFO:tasks.workunit.client.1.smithi149.stdout:4/402: write d0/d8/dc/d2e/d6f/f83 [125055,22911] 0 2024-03-22T07:08:51.786 INFO:tasks.workunit.client.1.smithi149.stdout:2/363: dwrite d1/d3/d5/d1f/f1b [0,4194304] 0 2024-03-22T07:08:51.787 INFO:tasks.workunit.client.1.smithi149.stdout:2/364: fdatasync d1/d8/d34/f63 0 2024-03-22T07:08:51.787 INFO:tasks.workunit.client.1.smithi149.stdout:1/343: dwrite d4/d10/d28/f53 [0,4194304] 0 2024-03-22T07:08:51.787 INFO:tasks.workunit.client.1.smithi149.stdout:3/285: dwrite d0/d4/f2f [0,4194304] 0 2024-03-22T07:08:51.789 INFO:tasks.workunit.client.1.smithi149.stdout:6/342: unlink d9/df/d22/f4b 0 2024-03-22T07:08:51.789 INFO:tasks.workunit.client.1.smithi149.stdout:6/343: stat d9/d47 0 2024-03-22T07:08:51.790 INFO:tasks.workunit.client.1.smithi149.stdout:0/256: dwrite d1/d31/f34 [0,4194304] 0 2024-03-22T07:08:51.792 INFO:tasks.workunit.client.1.smithi149.stdout:3/286: dread d0/d2/d30/f3f [0,4194304] 0 2024-03-22T07:08:51.794 INFO:tasks.workunit.client.1.smithi149.stdout:2/365: mkdir d1/d3/d7d 0 2024-03-22T07:08:51.795 INFO:tasks.workunit.client.1.smithi149.stdout:1/344: mknod d4/d1f/d6a/c78 0 2024-03-22T07:08:51.795 INFO:tasks.workunit.client.1.smithi149.stdout:1/345: creat d4/d7/f79 x:0 0 0 2024-03-22T07:08:51.795 INFO:tasks.workunit.client.1.smithi149.stdout:1/346: read d4/d7/fe [712556,8669] 0 2024-03-22T07:08:51.796 INFO:tasks.workunit.client.1.smithi149.stdout:6/344: dread d9/d47/f51 [0,4194304] 0 2024-03-22T07:08:51.797 INFO:tasks.workunit.client.1.smithi149.stdout:4/403: rmdir d0/d8/dc/d4b 39 2024-03-22T07:08:51.799 INFO:tasks.workunit.client.1.smithi149.stdout:0/257: creat d1/d9/f57 x:0 0 0 2024-03-22T07:08:51.800 INFO:tasks.workunit.client.1.smithi149.stdout:3/287: symlink d0/d2/d30/d31/l68 0 2024-03-22T07:08:51.801 INFO:tasks.workunit.client.1.smithi149.stdout:3/288: write d0/d4/d8/f25 [485004,84681] 0 2024-03-22T07:08:51.801 INFO:tasks.workunit.client.1.smithi149.stdout:2/366: mkdir d1/d4d/d69/d75/d7e 0 2024-03-22T07:08:51.804 INFO:tasks.workunit.client.1.smithi149.stdout:6/345: mkdir d9/df/d22/d6d 0 2024-03-22T07:08:51.804 INFO:tasks.workunit.client.1.smithi149.stdout:6/346: fdatasync d9/df/d22/f67 0 2024-03-22T07:08:51.804 INFO:tasks.workunit.client.1.smithi149.stdout:6/347: write d9/d1f/f48 [36689,33630] 0 2024-03-22T07:08:51.804 INFO:tasks.workunit.client.1.smithi149.stdout:9/406: rmdir db/d19/d28 39 2024-03-22T07:08:51.806 INFO:tasks.workunit.client.1.smithi149.stdout:4/404: unlink d0/l87 0 2024-03-22T07:08:51.808 INFO:tasks.workunit.client.1.smithi149.stdout:0/258: mknod d1/d9/c58 0 2024-03-22T07:08:51.810 INFO:tasks.workunit.client.1.smithi149.stdout:3/289: creat d0/d4/d8/d24/d40/d54/d5f/f69 x:0 0 0 2024-03-22T07:08:51.813 INFO:tasks.workunit.client.1.smithi149.stdout:2/367: creat d1/d3/d6/d3f/f7f x:0 0 0 2024-03-22T07:08:51.813 INFO:tasks.workunit.client.1.smithi149.stdout:2/368: chown d1/d3/l1d 7268848 1 2024-03-22T07:08:51.813 INFO:tasks.workunit.client.1.smithi149.stdout:1/347: symlink d4/d20/d23/l7a 0 2024-03-22T07:08:51.816 INFO:tasks.workunit.client.1.smithi149.stdout:3/290: rename d0/d4/d8/d24/d40/c4c to d0/d2/d5a/c6a 0 2024-03-22T07:08:51.818 INFO:tasks.workunit.client.1.smithi149.stdout:1/348: mknod d4/d20/d23/d59/c7b 0 2024-03-22T07:08:51.818 INFO:tasks.workunit.client.1.smithi149.stdout:1/349: write d4/d7/f57 [4347179,101485] 0 2024-03-22T07:08:51.818 INFO:tasks.workunit.client.1.smithi149.stdout:1/350: readlink d4/d1f/l51 0 2024-03-22T07:08:51.818 INFO:tasks.workunit.client.1.smithi149.stdout:1/351: chown d4/d7/d5d 1760062 1 2024-03-22T07:08:51.821 INFO:tasks.workunit.client.1.smithi149.stdout:6/348: write d9/d1f/f2a [437176,72497] 0 2024-03-22T07:08:51.821 INFO:tasks.workunit.client.1.smithi149.stdout:6/349: chown d9/df/d2e/f20 944097 1 2024-03-22T07:08:51.821 INFO:tasks.workunit.client.1.smithi149.stdout:3/291: mknod d0/d2/d36/c6b 0 2024-03-22T07:08:51.825 INFO:tasks.workunit.client.1.smithi149.stdout:3/292: write d0/d2/d30/f3f [171050,14201] 0 2024-03-22T07:08:51.828 INFO:tasks.workunit.client.1.smithi149.stdout:3/293: mkdir d0/d4/d6c 0 2024-03-22T07:08:51.828 INFO:tasks.workunit.client.1.smithi149.stdout:3/294: chown d0/d2/d30/d5d/f65 5978913 1 2024-03-22T07:08:51.829 INFO:tasks.workunit.client.1.smithi149.stdout:3/295: mknod d0/d4/d8/d24/d40/d54/c6d 0 2024-03-22T07:08:51.830 INFO:tasks.workunit.client.1.smithi149.stdout:1/352: dread d4/d10/d28/f37 [0,4194304] 0 2024-03-22T07:08:51.838 INFO:tasks.workunit.client.1.smithi149.stdout:3/296: dread d0/d4/d8/f19 [0,4194304] 0 2024-03-22T07:08:51.838 INFO:tasks.workunit.client.1.smithi149.stdout:3/297: creat d0/d4/d8/d24/f6e x:0 0 0 2024-03-22T07:08:51.838 INFO:tasks.workunit.client.1.smithi149.stdout:3/298: fdatasync d0/d4/fe 0 2024-03-22T07:08:51.840 INFO:tasks.workunit.client.1.smithi149.stdout:3/299: getdents d0/d4/d8 0 2024-03-22T07:08:51.842 INFO:tasks.workunit.client.1.smithi149.stdout:3/300: creat d0/d4/d8/d24/d40/d54/f6f x:0 0 0 2024-03-22T07:08:51.844 INFO:tasks.workunit.client.1.smithi149.stdout:3/301: unlink d0/d2/d30/d31/l61 0 2024-03-22T07:08:51.848 INFO:tasks.workunit.client.1.smithi149.stdout:0/259: dwrite d1/d9/d14/f28 [0,4194304] 0 2024-03-22T07:08:51.848 INFO:tasks.workunit.client.1.smithi149.stdout:0/260: readlink d1/d1a/l4c 0 2024-03-22T07:08:51.850 INFO:tasks.workunit.client.1.smithi149.stdout:0/261: mkdir d1/d9/d59 0 2024-03-22T07:08:51.853 INFO:tasks.workunit.client.1.smithi149.stdout:9/407: dwrite db/d19/d39/d3b/f53 [0,4194304] 0 2024-03-22T07:08:51.856 INFO:tasks.workunit.client.1.smithi149.stdout:9/408: dread db/d22/f36 [0,4194304] 0 2024-03-22T07:08:51.858 INFO:tasks.workunit.client.1.smithi149.stdout:2/369: dwrite d1/d3/d5/d41/f48 [0,4194304] 0 2024-03-22T07:08:51.858 INFO:tasks.workunit.client.1.smithi149.stdout:2/370: chown d1/d3/d5/d1f/d68/c70 5770 1 2024-03-22T07:08:51.859 INFO:tasks.workunit.client.1.smithi149.stdout:2/371: chown d1/d8/d34/c67 19072 1 2024-03-22T07:08:51.862 INFO:tasks.workunit.client.1.smithi149.stdout:6/350: dwrite d9/d1f/d24/f2d [0,4194304] 0 2024-03-22T07:08:51.862 INFO:tasks.workunit.client.1.smithi149.stdout:3/302: dread d0/d2/d36/f20 [0,4194304] 0 2024-03-22T07:08:51.862 INFO:tasks.workunit.client.1.smithi149.stdout:3/303: creat d0/d2/d30/f70 x:0 0 0 2024-03-22T07:08:51.863 INFO:tasks.workunit.client.1.smithi149.stdout:1/353: dwrite d4/d7/d5a/f5e [0,4194304] 0 2024-03-22T07:08:51.863 INFO:tasks.workunit.client.1.smithi149.stdout:6/351: unlink f8 0 2024-03-22T07:08:51.866 INFO:tasks.workunit.client.1.smithi149.stdout:3/304: creat d0/d4/d8/d24/d39/f71 x:0 0 0 2024-03-22T07:08:51.866 INFO:tasks.workunit.client.1.smithi149.stdout:3/305: creat d0/f72 x:0 0 0 2024-03-22T07:08:51.866 INFO:tasks.workunit.client.1.smithi149.stdout:3/306: write d0/d4/d8/d24/f47 [706200,23397] 0 2024-03-22T07:08:51.866 INFO:tasks.workunit.client.1.smithi149.stdout:3/307: creat d0/d2/d30/f73 x:0 0 0 2024-03-22T07:08:51.869 INFO:tasks.workunit.client.1.smithi149.stdout:6/352: symlink d9/d1f/d24/d34/d4f/d55/l6e 0 2024-03-22T07:08:51.869 INFO:tasks.workunit.client.1.smithi149.stdout:6/353: readlink d9/d1f/d24/d34/d4f/d55/l6e 0 2024-03-22T07:08:51.870 INFO:tasks.workunit.client.1.smithi149.stdout:0/262: dwrite d1/d9/d14/f18 [0,4194304] 0 2024-03-22T07:08:51.872 INFO:tasks.workunit.client.1.smithi149.stdout:3/308: unlink d0/d4/d8/d24/l45 0 2024-03-22T07:08:51.875 INFO:tasks.workunit.client.1.smithi149.stdout:6/354: truncate d9/d1f/d24/d34/d37/d38/f53 4099348 0 2024-03-22T07:08:51.875 INFO:tasks.workunit.client.1.smithi149.stdout:6/355: write d9/d1f/f6b [1006090,43804] 0 2024-03-22T07:08:51.875 INFO:tasks.workunit.client.1.smithi149.stdout:6/356: readlink d9/d1f/d24/d34/d4f/d55/l6e 0 2024-03-22T07:08:51.878 INFO:tasks.workunit.client.1.smithi149.stdout:6/357: dread d9/f17 [4194304,4194304] 0 2024-03-22T07:08:51.878 INFO:tasks.workunit.client.1.smithi149.stdout:6/358: write d9/df/d2e/f1b [4735634,30093] 0 2024-03-22T07:08:51.878 INFO:tasks.workunit.client.1.smithi149.stdout:1/354: rename d4/d1f to d4/d10/d7c 0 2024-03-22T07:08:51.880 INFO:tasks.workunit.client.1.smithi149.stdout:6/359: link d9/d1f/l33 d9/d1f/d24/d34/l6f 0 2024-03-22T07:08:51.880 INFO:tasks.workunit.client.1.smithi149.stdout:6/360: chown d9/df/d2e/d4e 633552 1 2024-03-22T07:08:51.881 INFO:tasks.workunit.client.1.smithi149.stdout:2/372: dwrite d1/d3/f36 [0,4194304] 0 2024-03-22T07:08:51.882 INFO:tasks.workunit.client.1.smithi149.stdout:1/355: creat d4/d10/d7c/f7d x:0 0 0 2024-03-22T07:08:51.884 INFO:tasks.workunit.client.1.smithi149.stdout:0/263: dread d1/d31/d51/f43 [0,4194304] 0 2024-03-22T07:08:51.884 INFO:tasks.workunit.client.1.smithi149.stdout:2/373: symlink d1/d3/l80 0 2024-03-22T07:08:51.884 INFO:tasks.workunit.client.1.smithi149.stdout:2/374: chown d1/d8/d34/l49 2606339 1 2024-03-22T07:08:51.885 INFO:tasks.workunit.client.1.smithi149.stdout:1/356: symlink d4/d20/d2d/d64/l7e 0 2024-03-22T07:08:51.885 INFO:tasks.workunit.client.1.smithi149.stdout:1/357: chown d4/d10/d7c/d6a/c78 121 1 2024-03-22T07:08:51.885 INFO:tasks.workunit.client.1.smithi149.stdout:1/358: dread - d4/d7/f50 zero size 2024-03-22T07:08:51.886 INFO:tasks.workunit.client.1.smithi149.stdout:1/359: chown d4/d7/c18 175 1 2024-03-22T07:08:51.886 INFO:tasks.workunit.client.1.smithi149.stdout:1/360: fdatasync d4/d10/f4c 0 2024-03-22T07:08:51.886 INFO:tasks.workunit.client.1.smithi149.stdout:1/361: chown d4/d7/l2b 401720568 1 2024-03-22T07:08:51.887 INFO:tasks.workunit.client.1.smithi149.stdout:2/375: unlink d1/d3/d6/l38 0 2024-03-22T07:08:51.897 INFO:tasks.workunit.client.1.smithi149.stdout:3/309: dwrite d0/d2/d36/f59 [0,4194304] 0 2024-03-22T07:08:51.898 INFO:tasks.workunit.client.1.smithi149.stdout:6/361: dread d9/df/f15 [0,4194304] 0 2024-03-22T07:08:51.898 INFO:tasks.workunit.client.1.smithi149.stdout:6/362: read - d9/df/d22/f67 zero size 2024-03-22T07:08:51.901 INFO:tasks.workunit.client.1.smithi149.stdout:3/310: mkdir d0/d2/d36/d74 0 2024-03-22T07:08:51.901 INFO:tasks.workunit.client.1.smithi149.stdout:6/363: symlink d9/d1f/d24/l70 0 2024-03-22T07:08:51.903 INFO:tasks.workunit.client.1.smithi149.stdout:3/311: creat d0/d2/d36/d74/f75 x:0 0 0 2024-03-22T07:08:51.905 INFO:tasks.workunit.client.1.smithi149.stdout:6/364: rename d9/d1f/d24/f2d to d9/d1f/d24/d34/d37/d38/f71 0 2024-03-22T07:08:51.912 INFO:tasks.workunit.client.1.smithi149.stdout:0/264: dwrite d1/d1a/f25 [0,4194304] 0 2024-03-22T07:08:51.914 INFO:tasks.workunit.client.1.smithi149.stdout:0/265: creat d1/d31/d52/f5a x:0 0 0 2024-03-22T07:08:51.914 INFO:tasks.workunit.client.1.smithi149.stdout:0/266: chown d1/c4e 127973 1 2024-03-22T07:08:51.915 INFO:tasks.workunit.client.1.smithi149.stdout:0/267: creat d1/d31/d51/f5b x:0 0 0 2024-03-22T07:08:51.917 INFO:tasks.workunit.client.1.smithi149.stdout:1/362: dwrite d4/d7/d5d/f72 [0,4194304] 0 2024-03-22T07:08:51.917 INFO:tasks.workunit.client.1.smithi149.stdout:0/268: link d1/fb d1/d31/f5c 0 2024-03-22T07:08:51.918 INFO:tasks.workunit.client.1.smithi149.stdout:7/415: sync 2024-03-22T07:08:51.918 INFO:tasks.workunit.client.1.smithi149.stdout:5/312: sync 2024-03-22T07:08:51.919 INFO:tasks.workunit.client.1.smithi149.stdout:2/376: dwrite d1/d3/f54 [0,4194304] 0 2024-03-22T07:08:51.919 INFO:tasks.workunit.client.1.smithi149.stdout:6/365: dread d9/df/d22/f58 [4194304,4194304] 0 2024-03-22T07:08:51.926 INFO:tasks.workunit.client.1.smithi149.stdout:7/416: link d1/d18/d1b/d51/d7a/d7d/f3b d1/d18/d1b/d1d/d28/f87 0 2024-03-22T07:08:51.930 INFO:tasks.workunit.client.1.smithi149.stdout:6/366: rmdir d9/d1f/d24/d5f 0 2024-03-22T07:08:51.930 INFO:tasks.workunit.client.1.smithi149.stdout:6/367: rename d9/df to d9/df/d22/d72 22 2024-03-22T07:08:51.930 INFO:tasks.workunit.client.1.smithi149.stdout:6/368: write d9/df/d2e/f1c [1665134,92110] 0 2024-03-22T07:08:51.930 INFO:tasks.workunit.client.1.smithi149.stdout:6/369: chown d9/df/d22/c3d 418 1 2024-03-22T07:08:51.930 INFO:tasks.workunit.client.1.smithi149.stdout:6/370: read d9/df/d2e/f12 [1390014,98878] 0 2024-03-22T07:08:51.931 INFO:tasks.workunit.client.1.smithi149.stdout:7/417: creat d1/d18/d1b/d1d/d72/f88 x:0 0 0 2024-03-22T07:08:51.932 INFO:tasks.workunit.client.1.smithi149.stdout:5/313: dread d7/d35/f40 [4194304,4194304] 0 2024-03-22T07:08:51.933 INFO:tasks.workunit.client.1.smithi149.stdout:5/314: write d7/f58 [1547240,45148] 0 2024-03-22T07:08:51.933 INFO:tasks.workunit.client.1.smithi149.stdout:6/371: unlink d9/df/d22/f62 0 2024-03-22T07:08:51.936 INFO:tasks.workunit.client.1.smithi149.stdout:5/315: mkdir d7/da/d20/d30/d3a/d3d/d2c/d63 0 2024-03-22T07:08:51.936 INFO:tasks.workunit.client.1.smithi149.stdout:5/316: chown d7/da/d20/d30/d3a/d3d/d2c/d47/f52 27689152 1 2024-03-22T07:08:51.938 INFO:tasks.workunit.client.1.smithi149.stdout:6/372: dread d9/df/d22/f31 [0,4194304] 0 2024-03-22T07:08:51.938 INFO:tasks.workunit.client.1.smithi149.stdout:5/317: symlink d7/da/l64 0 2024-03-22T07:08:51.939 INFO:tasks.workunit.client.1.smithi149.stdout:6/373: mkdir d9/d1f/d24/d68/d73 0 2024-03-22T07:08:51.940 INFO:tasks.workunit.client.1.smithi149.stdout:7/418: dread d1/f5 [0,4194304] 0 2024-03-22T07:08:51.940 INFO:tasks.workunit.client.1.smithi149.stdout:7/419: write d1/d18/d1b/d1d/f13 [574360,32803] 0 2024-03-22T07:08:51.940 INFO:tasks.workunit.client.1.smithi149.stdout:7/420: fdatasync d1/d18/d1b/d1d/d28/f87 0 2024-03-22T07:08:51.940 INFO:tasks.workunit.client.1.smithi149.stdout:7/421: fdatasync d1/d18/d1b/d51/f52 0 2024-03-22T07:08:51.941 INFO:tasks.workunit.client.1.smithi149.stdout:5/318: rename d7/da/d20/l54 to d7/da/d20/d36/d44/l65 0 2024-03-22T07:08:51.941 INFO:tasks.workunit.client.1.smithi149.stdout:2/377: dwrite d1/d3/dd/f65 [0,4194304] 0 2024-03-22T07:08:51.942 INFO:tasks.workunit.client.1.smithi149.stdout:6/374: creat d9/df/d22/f74 x:0 0 0 2024-03-22T07:08:51.942 INFO:tasks.workunit.client.1.smithi149.stdout:0/269: dwrite d1/d9/f57 [0,4194304] 0 2024-03-22T07:08:51.946 INFO:tasks.workunit.client.1.smithi149.stdout:2/378: mkdir d1/d3/dd/d71/d81 0 2024-03-22T07:08:51.948 INFO:tasks.workunit.client.1.smithi149.stdout:6/375: truncate d9/d1f/d24/d34/d37/d38/f53 2504365 0 2024-03-22T07:08:51.948 INFO:tasks.workunit.client.1.smithi149.stdout:6/376: stat d9/d1f/d24/d34/d4f 0 2024-03-22T07:08:51.948 INFO:tasks.workunit.client.1.smithi149.stdout:6/377: fdatasync d9/df/d22/f67 0 2024-03-22T07:08:51.954 INFO:tasks.workunit.client.1.smithi149.stdout:0/270: creat d1/d9/d59/f5d x:0 0 0 2024-03-22T07:08:51.956 INFO:tasks.workunit.client.1.smithi149.stdout:7/422: rmdir d1/d1c/d4d 39 2024-03-22T07:08:51.956 INFO:tasks.workunit.client.1.smithi149.stdout:7/423: stat d1/d18/d30/f45 0 2024-03-22T07:08:51.956 INFO:tasks.workunit.client.1.smithi149.stdout:5/319: rmdir d7/da/d20/d36/d44 39 2024-03-22T07:08:51.957 INFO:tasks.workunit.client.1.smithi149.stdout:5/320: write d7/da/f37 [5075401,3172] 0 2024-03-22T07:08:51.957 INFO:tasks.workunit.client.1.smithi149.stdout:6/378: creat d9/d1f/d24/d42/f75 x:0 0 0 2024-03-22T07:08:51.958 INFO:tasks.workunit.client.1.smithi149.stdout:0/271: mknod d1/d9/d14/c5e 0 2024-03-22T07:08:51.961 INFO:tasks.workunit.client.1.smithi149.stdout:7/424: truncate d1/d18/d1b/d1d/d28/f50 3289764 0 2024-03-22T07:08:51.962 INFO:tasks.workunit.client.1.smithi149.stdout:5/321: creat d7/da/d20/d36/f66 x:0 0 0 2024-03-22T07:08:51.963 INFO:tasks.workunit.client.1.smithi149.stdout:6/379: getdents d9/d47 0 2024-03-22T07:08:51.965 INFO:tasks.workunit.client.1.smithi149.stdout:0/272: rename d1/d31/f34 to d1/d9/d14/d1f/f5f 0 2024-03-22T07:08:51.966 INFO:tasks.workunit.client.1.smithi149.stdout:7/425: symlink d1/d18/d30/l89 0 2024-03-22T07:08:51.968 INFO:tasks.workunit.client.1.smithi149.stdout:5/322: rename d7/da/d20/d30/l5f to d7/da/d20/d30/l67 0 2024-03-22T07:08:51.968 INFO:tasks.workunit.client.1.smithi149.stdout:2/379: dwrite d1/d3/d5/d1f/f1b [4194304,4194304] 0 2024-03-22T07:08:51.970 INFO:tasks.workunit.client.1.smithi149.stdout:2/380: symlink d1/d4d/d69/l82 0 2024-03-22T07:08:51.972 INFO:tasks.workunit.client.1.smithi149.stdout:6/380: dread d9/df/d2e/f12 [0,4194304] 0 2024-03-22T07:08:51.972 INFO:tasks.workunit.client.1.smithi149.stdout:6/381: chown d9/df/d2e 25373011 1 2024-03-22T07:08:51.972 INFO:tasks.workunit.client.1.smithi149.stdout:2/381: chown d1/d3/d6/d3f 6347 1 2024-03-22T07:08:51.973 INFO:tasks.workunit.client.1.smithi149.stdout:5/323: dread d7/da/d20/d30/d3a/d3d/f34 [0,4194304] 0 2024-03-22T07:08:51.973 INFO:tasks.workunit.client.1.smithi149.stdout:6/382: truncate f5 2064230 0 2024-03-22T07:08:51.975 INFO:tasks.workunit.client.1.smithi149.stdout:6/383: write d9/df/d2e/f27 [360294,7130] 0 2024-03-22T07:08:51.975 INFO:tasks.workunit.client.1.smithi149.stdout:6/384: write d9/d1f/f2a [1042008,86961] 0 2024-03-22T07:08:51.977 INFO:tasks.workunit.client.1.smithi149.stdout:6/385: mknod d9/d1f/d24/d34/c76 0 2024-03-22T07:08:51.977 INFO:tasks.workunit.client.1.smithi149.stdout:6/386: truncate d9/df/d2e/d4e/f5e 246171 0 2024-03-22T07:08:51.977 INFO:tasks.workunit.client.1.smithi149.stdout:0/273: dread d1/d9/f57 [0,4194304] 0 2024-03-22T07:08:51.977 INFO:tasks.workunit.client.1.smithi149.stdout:0/274: truncate d1/d9/d14/d1f/f24 1123241 0 2024-03-22T07:08:51.978 INFO:tasks.workunit.client.1.smithi149.stdout:7/426: read d1/d18/d1b/d51/d5a/d6a/f6e [1899977,41954] 0 2024-03-22T07:08:51.979 INFO:tasks.workunit.client.1.smithi149.stdout:6/387: symlink d9/d1f/d24/d68/l77 0 2024-03-22T07:08:51.979 INFO:tasks.workunit.client.1.smithi149.stdout:6/388: write d9/df/d22/f74 [906454,69803] 0 2024-03-22T07:08:51.983 INFO:tasks.workunit.client.1.smithi149.stdout:0/275: link d1/d27/l40 d1/d9/d14/d1f/l60 0 2024-03-22T07:08:51.985 INFO:tasks.workunit.client.1.smithi149.stdout:7/427: creat d1/d1c/d2f/d74/f8a x:0 0 0 2024-03-22T07:08:51.985 INFO:tasks.workunit.client.1.smithi149.stdout:2/382: dread d1/d3/d6/f7 [0,4194304] 0 2024-03-22T07:08:51.986 INFO:tasks.workunit.client.1.smithi149.stdout:4/405: sync 2024-03-22T07:08:51.986 INFO:tasks.workunit.client.1.smithi149.stdout:8/394: sync 2024-03-22T07:08:51.987 INFO:tasks.workunit.client.1.smithi149.stdout:6/389: dread d9/d1f/d24/d34/d37/d38/f54 [0,4194304] 0 2024-03-22T07:08:51.987 INFO:tasks.workunit.client.1.smithi149.stdout:6/390: write d9/df/d22/f67 [258530,97116] 0 2024-03-22T07:08:51.988 INFO:tasks.workunit.client.1.smithi149.stdout:0/276: unlink d1/d31/d51/f43 0 2024-03-22T07:08:51.990 INFO:tasks.workunit.client.1.smithi149.stdout:2/383: symlink d1/d3/d5/d41/l83 0 2024-03-22T07:08:51.992 INFO:tasks.workunit.client.1.smithi149.stdout:2/384: dread d1/d3/d5/d1f/d5e/d2f/f33 [0,4194304] 0 2024-03-22T07:08:51.992 INFO:tasks.workunit.client.1.smithi149.stdout:2/385: chown d1/d8/c43 599715951 1 2024-03-22T07:08:51.992 INFO:tasks.workunit.client.1.smithi149.stdout:2/386: dread - d1/d3/dd/f6e zero size 2024-03-22T07:08:51.992 INFO:tasks.workunit.client.1.smithi149.stdout:2/387: creat d1/d3/d5/d1f/f84 x:0 0 0 2024-03-22T07:08:51.992 INFO:tasks.workunit.client.1.smithi149.stdout:4/406: creat d0/d8/dc/d2e/d6f/f8e x:0 0 0 2024-03-22T07:08:51.993 INFO:tasks.workunit.client.1.smithi149.stdout:8/395: symlink d1/d2/d18/l85 0 2024-03-22T07:08:51.995 INFO:tasks.workunit.client.1.smithi149.stdout:6/391: creat d9/df/d22/d6d/f78 x:0 0 0 2024-03-22T07:08:51.997 INFO:tasks.workunit.client.1.smithi149.stdout:0/277: link d1/d9/d14/d1f/f24 d1/d9/d59/f61 0 2024-03-22T07:08:51.999 INFO:tasks.workunit.client.1.smithi149.stdout:5/324: dwrite d7/da/f1b [0,4194304] 0 2024-03-22T07:08:51.999 INFO:tasks.workunit.client.1.smithi149.stdout:5/325: dread - d7/da/d20/d36/d44/f5d zero size 2024-03-22T07:08:52.001 INFO:tasks.workunit.client.1.smithi149.stdout:4/407: link d0/d8/dc/d14/f16 d0/d8/d1d/f8f 0 2024-03-22T07:08:52.001 INFO:tasks.workunit.client.1.smithi149.stdout:4/408: fdatasync d0/d8/dc/f13 0 2024-03-22T07:08:52.001 INFO:tasks.workunit.client.1.smithi149.stdout:8/396: stat d1/d2/l36 0 2024-03-22T07:08:52.002 INFO:tasks.workunit.client.1.smithi149.stdout:8/397: stat d1/d20/d35/d22/d44/f52 0 2024-03-22T07:08:52.002 INFO:tasks.workunit.client.1.smithi149.stdout:6/392: mknod d9/df/d2e/d63/c79 0 2024-03-22T07:08:52.004 INFO:tasks.workunit.client.1.smithi149.stdout:2/388: write d1/f10 [1416724,109794] 0 2024-03-22T07:08:52.004 INFO:tasks.workunit.client.1.smithi149.stdout:2/389: dread - d1/d3/f29 zero size 2024-03-22T07:08:52.004 INFO:tasks.workunit.client.1.smithi149.stdout:2/390: chown d1/d8/l53 549977 1 2024-03-22T07:08:52.004 INFO:tasks.workunit.client.1.smithi149.stdout:2/391: readlink d1/d3/d5/d1f/d5e/l2c 0 2024-03-22T07:08:52.004 INFO:tasks.workunit.client.1.smithi149.stdout:0/278: link d1/f56 d1/d27/f62 0 2024-03-22T07:08:52.004 INFO:tasks.workunit.client.1.smithi149.stdout:0/279: creat d1/f63 x:0 0 0 2024-03-22T07:08:52.006 INFO:tasks.workunit.client.1.smithi149.stdout:7/428: getdents d1/d1c/d2f/d74 0 2024-03-22T07:08:52.010 INFO:tasks.workunit.client.1.smithi149.stdout:5/326: dread d7/da/f37 [0,4194304] 0 2024-03-22T07:08:52.011 INFO:tasks.workunit.client.1.smithi149.stdout:4/409: mkdir d0/d8/dc/d74/d90 0 2024-03-22T07:08:52.012 INFO:tasks.workunit.client.1.smithi149.stdout:8/398: truncate d1/d20/d35/d22/d44/d84/f48 723496 0 2024-03-22T07:08:52.012 INFO:tasks.workunit.client.1.smithi149.stdout:8/399: truncate d1/d2/d18/f78 398546 0 2024-03-22T07:08:52.012 INFO:tasks.workunit.client.1.smithi149.stdout:8/400: write d1/d20/d35/d22/f40 [3508543,128081] 0 2024-03-22T07:08:52.012 INFO:tasks.workunit.client.1.smithi149.stdout:8/401: creat d1/d2/d3/f86 x:0 0 0 2024-03-22T07:08:52.013 INFO:tasks.workunit.client.1.smithi149.stdout:2/392: write d1/d3/d5/f2b [666411,13265] 0 2024-03-22T07:08:52.013 INFO:tasks.workunit.client.1.smithi149.stdout:6/393: creat d9/d1f/d24/d34/d37/f7a x:0 0 0 2024-03-22T07:08:52.016 INFO:tasks.workunit.client.1.smithi149.stdout:0/280: mkdir d1/d31/d51/d64 0 2024-03-22T07:08:52.019 INFO:tasks.workunit.client.1.smithi149.stdout:7/429: rename d1/d1c/d49/f5e to d1/d18/d30/f8b 0 2024-03-22T07:08:52.020 INFO:tasks.workunit.client.1.smithi149.stdout:4/410: truncate d0/d8/dc/d2e/f2f 1230743 0 2024-03-22T07:08:52.020 INFO:tasks.workunit.client.1.smithi149.stdout:4/411: creat d0/d8/d1d/d24/d82/f91 x:0 0 0 2024-03-22T07:08:52.020 INFO:tasks.workunit.client.1.smithi149.stdout:4/412: chown d0/d8/dc 5229875 1 2024-03-22T07:08:52.020 INFO:tasks.workunit.client.1.smithi149.stdout:4/413: fdatasync d0/d8/d1d/f28 0 2024-03-22T07:08:52.020 INFO:tasks.workunit.client.1.smithi149.stdout:4/414: dread - d0/d8/dc/d2e/d6f/f8e zero size 2024-03-22T07:08:52.021 INFO:tasks.workunit.client.1.smithi149.stdout:4/415: dread d0/f32 [0,4194304] 0 2024-03-22T07:08:52.024 INFO:tasks.workunit.client.1.smithi149.stdout:8/402: mknod d1/d2/d9/c87 0 2024-03-22T07:08:52.024 INFO:tasks.workunit.client.1.smithi149.stdout:8/403: fsync d1/d20/f69 0 2024-03-22T07:08:52.024 INFO:tasks.workunit.client.1.smithi149.stdout:8/404: write d1/d2/d9/f29 [2528037,11565] 0 2024-03-22T07:08:52.024 INFO:tasks.workunit.client.1.smithi149.stdout:8/405: chown d1/d20/d35/c4b 436115 1 2024-03-22T07:08:52.026 INFO:tasks.workunit.client.1.smithi149.stdout:2/393: symlink d1/d3/d6/d3f/l85 0 2024-03-22T07:08:52.026 INFO:tasks.workunit.client.1.smithi149.stdout:6/394: symlink d9/d1f/d24/l7b 0 2024-03-22T07:08:52.027 INFO:tasks.workunit.client.1.smithi149.stdout:0/281: rename d1/c4e to d1/d9/d14/c65 0 2024-03-22T07:08:52.029 INFO:tasks.workunit.client.1.smithi149.stdout:7/430: link d1/d1c/d4d/f75 d1/d18/f8c 0 2024-03-22T07:08:52.029 INFO:tasks.workunit.client.1.smithi149.stdout:7/431: dread - d1/d18/d1b/d51/d7a/f7f zero size 2024-03-22T07:08:52.029 INFO:tasks.workunit.client.1.smithi149.stdout:7/432: chown d1/d18/d1b/d51/d7a/d7d 547 1 2024-03-22T07:08:52.029 INFO:tasks.workunit.client.1.smithi149.stdout:7/433: readlink d1/d18/d1b/l4b 0 2024-03-22T07:08:52.029 INFO:tasks.workunit.client.1.smithi149.stdout:7/434: chown d1/d18/d1b/d51/d7a/d7d/d5d/c7e 218256 1 2024-03-22T07:08:52.030 INFO:tasks.workunit.client.1.smithi149.stdout:4/416: write d0/d8/dc/f18 [3536019,115082] 0 2024-03-22T07:08:52.030 INFO:tasks.workunit.client.1.smithi149.stdout:4/417: read d0/d8/d5c/f5e [23680,109343] 0 2024-03-22T07:08:52.031 INFO:tasks.workunit.client.1.smithi149.stdout:8/406: mknod d1/d20/d35/d22/c88 0 2024-03-22T07:08:52.031 INFO:tasks.workunit.client.1.smithi149.stdout:8/407: write d1/d2/d3/f86 [155415,14828] 0 2024-03-22T07:08:52.033 INFO:tasks.workunit.client.1.smithi149.stdout:6/395: mknod d9/d1f/c7c 0 2024-03-22T07:08:52.033 INFO:tasks.workunit.client.1.smithi149.stdout:9/409: sync 2024-03-22T07:08:52.034 INFO:tasks.workunit.client.1.smithi149.stdout:7/435: symlink d1/d1c/d2f/d74/l8d 0 2024-03-22T07:08:52.035 INFO:tasks.workunit.client.1.smithi149.stdout:6/396: mkdir d9/df/d22/d6d/d7d 0 2024-03-22T07:08:52.038 INFO:tasks.workunit.client.1.smithi149.stdout:2/394: read d1/d8/d34/f57 [1601621,116920] 0 2024-03-22T07:08:52.038 INFO:tasks.workunit.client.1.smithi149.stdout:2/395: stat d1/d8/d59 0 2024-03-22T07:08:52.039 INFO:tasks.workunit.client.1.smithi149.stdout:2/396: fdatasync d1/d3/d5/f5a 0 2024-03-22T07:08:52.039 INFO:tasks.workunit.client.1.smithi149.stdout:2/397: fdatasync d1/d8/d34/f63 0 2024-03-22T07:08:52.039 INFO:tasks.workunit.client.1.smithi149.stdout:2/398: stat d1/d3/d5/d1f/f55 0 2024-03-22T07:08:52.042 INFO:tasks.workunit.client.1.smithi149.stdout:9/410: mkdir db/d19/d1c/d85 0 2024-03-22T07:08:52.042 INFO:tasks.workunit.client.1.smithi149.stdout:6/397: rmdir d9/d1f/d24/d34/d37/d38 39 2024-03-22T07:08:52.042 INFO:tasks.workunit.client.1.smithi149.stdout:7/436: fsync d1/d18/d1b/d51/d7a/d7d/d5b/f79 0 2024-03-22T07:08:52.042 INFO:tasks.workunit.client.1.smithi149.stdout:7/437: fdatasync d1/d18/d1b/d1d/f62 0 2024-03-22T07:08:52.042 INFO:tasks.workunit.client.1.smithi149.stdout:7/438: truncate d1/d18/d1b/d1d/d28/d53/f67 72675 0 2024-03-22T07:08:52.042 INFO:tasks.workunit.client.1.smithi149.stdout:8/408: dread d1/d2/d3/f1f [0,4194304] 0 2024-03-22T07:08:52.043 INFO:tasks.workunit.client.1.smithi149.stdout:5/327: dwrite d7/da/f1b [0,4194304] 0 2024-03-22T07:08:52.046 INFO:tasks.workunit.client.1.smithi149.stdout:2/399: mkdir d1/d3/d6/d3f/d86 0 2024-03-22T07:08:52.046 INFO:tasks.workunit.client.1.smithi149.stdout:9/411: symlink db/d19/l86 0 2024-03-22T07:08:52.046 INFO:tasks.workunit.client.1.smithi149.stdout:7/439: rename d1/d18/d1b/d1d/d28 to d1/d1c/d49/d8e 0 2024-03-22T07:08:52.046 INFO:tasks.workunit.client.1.smithi149.stdout:7/440: readlink d1/d1c/d2f/d74/l8d 0 2024-03-22T07:08:52.047 INFO:tasks.workunit.client.1.smithi149.stdout:7/441: chown d1/d1c/d49/d8e/l66 31055620 1 2024-03-22T07:08:52.050 INFO:tasks.workunit.client.1.smithi149.stdout:5/328: creat d7/db/d2b/f68 x:0 0 0 2024-03-22T07:08:52.050 INFO:tasks.workunit.client.1.smithi149.stdout:5/329: creat d7/da/d20/d45/f69 x:0 0 0 2024-03-22T07:08:52.054 INFO:tasks.workunit.client.1.smithi149.stdout:5/330: link d7/da/d20/d45/f51 d7/da/d20/d30/d3a/d3d/d2c/d4b/f6a 0 2024-03-22T07:08:52.054 INFO:tasks.workunit.client.1.smithi149.stdout:2/400: mkdir d1/d3/d6/d87 0 2024-03-22T07:08:52.054 INFO:tasks.workunit.client.1.smithi149.stdout:9/412: symlink db/d21/d37/l87 0 2024-03-22T07:08:52.054 INFO:tasks.workunit.client.1.smithi149.stdout:5/331: truncate d7/d35/f55 965532 0 2024-03-22T07:08:52.054 INFO:tasks.workunit.client.1.smithi149.stdout:2/401: chown d1/d8/l53 244392623 1 2024-03-22T07:08:52.055 INFO:tasks.workunit.client.1.smithi149.stdout:5/332: write d7/da/f37 [2163427,24366] 0 2024-03-22T07:08:52.055 INFO:tasks.workunit.client.1.smithi149.stdout:5/333: stat d7/db/l42 0 2024-03-22T07:08:52.055 INFO:tasks.workunit.client.1.smithi149.stdout:6/398: write d9/d1f/d24/d34/d37/d38/f53 [2087320,60869] 0 2024-03-22T07:08:52.057 INFO:tasks.workunit.client.1.smithi149.stdout:5/334: write d7/da/d20/f4c [1334421,106494] 0 2024-03-22T07:08:52.057 INFO:tasks.workunit.client.1.smithi149.stdout:9/413: link db/d21/c71 db/d42/d49/d4b/c88 0 2024-03-22T07:08:52.057 INFO:tasks.workunit.client.1.smithi149.stdout:2/402: mknod d1/d4d/d69/d75/c88 0 2024-03-22T07:08:52.058 INFO:tasks.workunit.client.1.smithi149.stdout:6/399: link d9/df/d2e/f12 d9/d1f/d24/d34/f7e 0 2024-03-22T07:08:52.058 INFO:tasks.workunit.client.1.smithi149.stdout:2/403: read - d1/d3/dd/f4c zero size 2024-03-22T07:08:52.058 INFO:tasks.workunit.client.1.smithi149.stdout:2/404: chown d1/d3/d6/d87 16847 1 2024-03-22T07:08:52.059 INFO:tasks.workunit.client.1.smithi149.stdout:9/414: rename db/d19/f72 to db/d42/d49/d4b/f89 0 2024-03-22T07:08:52.060 INFO:tasks.workunit.client.1.smithi149.stdout:6/400: creat d9/d1f/d24/d68/d73/f7f x:0 0 0 2024-03-22T07:08:52.061 INFO:tasks.workunit.client.1.smithi149.stdout:2/405: dread d1/d8/f24 [4194304,4194304] 0 2024-03-22T07:08:52.062 INFO:tasks.workunit.client.1.smithi149.stdout:2/406: fsync d1/d3/d5/d1f/f55 0 2024-03-22T07:08:52.062 INFO:tasks.workunit.client.1.smithi149.stdout:2/407: chown d1/d3/d5/d1f/d5e/c2d 40120 1 2024-03-22T07:08:52.062 INFO:tasks.workunit.client.1.smithi149.stdout:9/415: rename db/d21/l60 to db/d33/l8a 0 2024-03-22T07:08:52.062 INFO:tasks.workunit.client.1.smithi149.stdout:9/416: chown db/d21/d74 411344 1 2024-03-22T07:08:52.062 INFO:tasks.workunit.client.1.smithi149.stdout:6/401: creat d9/d1f/d24/d34/d4f/f80 x:0 0 0 2024-03-22T07:08:52.062 INFO:tasks.workunit.client.1.smithi149.stdout:6/402: fsync d9/df/d22/f31 0 2024-03-22T07:08:52.063 INFO:tasks.workunit.client.1.smithi149.stdout:2/408: chown d1/d3/dd/l5c 1596 1 2024-03-22T07:08:52.063 INFO:tasks.workunit.client.1.smithi149.stdout:6/403: symlink d9/df/d22/d6d/d7d/l81 0 2024-03-22T07:08:52.064 INFO:tasks.workunit.client.1.smithi149.stdout:2/409: mkdir d1/d4d/d69/d75/d89 0 2024-03-22T07:08:52.064 INFO:tasks.workunit.client.1.smithi149.stdout:2/410: truncate d1/f60 146637 0 2024-03-22T07:08:52.064 INFO:tasks.workunit.client.1.smithi149.stdout:9/417: mknod db/c8b 0 2024-03-22T07:08:52.065 INFO:tasks.workunit.client.1.smithi149.stdout:7/442: dread d1/d1c/f7b [0,4194304] 0 2024-03-22T07:08:52.065 INFO:tasks.workunit.client.1.smithi149.stdout:6/404: symlink d9/df/d2e/l82 0 2024-03-22T07:08:52.067 INFO:tasks.workunit.client.1.smithi149.stdout:9/418: mkdir db/d21/d8c 0 2024-03-22T07:08:52.067 INFO:tasks.workunit.client.1.smithi149.stdout:9/419: chown db/d21/d74/f76 13 1 2024-03-22T07:08:52.067 INFO:tasks.workunit.client.1.smithi149.stdout:9/420: chown db 61277908 1 2024-03-22T07:08:52.067 INFO:tasks.workunit.client.1.smithi149.stdout:9/421: write db/d19/f7d [959215,76217] 0 2024-03-22T07:08:52.068 INFO:tasks.workunit.client.1.smithi149.stdout:6/405: mknod d9/d1f/d24/d34/d37/c83 0 2024-03-22T07:08:52.068 INFO:tasks.workunit.client.1.smithi149.stdout:2/411: rmdir d1/d3/dd 39 2024-03-22T07:08:52.069 INFO:tasks.workunit.client.1.smithi149.stdout:6/406: symlink d9/df/d22/d6d/l84 0 2024-03-22T07:08:52.069 INFO:tasks.workunit.client.1.smithi149.stdout:2/412: creat d1/d4d/f8a x:0 0 0 2024-03-22T07:08:52.071 INFO:tasks.workunit.client.1.smithi149.stdout:6/407: creat d9/d1f/d24/d34/d37/d38/f85 x:0 0 0 2024-03-22T07:08:52.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/409: dwrite d1/f24 [0,4194304] 0 2024-03-22T07:08:52.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/408: mkdir d9/d1f/d24/d34/d4f/d86 0 2024-03-22T07:08:52.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/409: write d9/d47/d52/f60 [561026,85494] 0 2024-03-22T07:08:52.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/410: write d9/df/f6c [550948,92469] 0 2024-03-22T07:08:52.075 INFO:tasks.workunit.client.1.smithi149.stdout:5/335: dread d7/da/d20/d30/d3a/d3d/d2c/d4b/f6a [0,4194304] 0 2024-03-22T07:08:52.076 INFO:tasks.workunit.client.1.smithi149.stdout:4/418: dwrite d0/d8/dc/d14/f75 [0,4194304] 0 2024-03-22T07:08:52.076 INFO:tasks.workunit.client.1.smithi149.stdout:8/410: rename d1/d2/d9/c30 to d1/d2/d9/d11/d45/c89 0 2024-03-22T07:08:52.076 INFO:tasks.workunit.client.1.smithi149.stdout:0/282: dwrite d1/d31/d51/f47 [0,4194304] 0 2024-03-22T07:08:52.084 INFO:tasks.workunit.client.1.smithi149.stdout:4/419: mknod d0/d8/d1d/d24/c92 0 2024-03-22T07:08:52.089 INFO:tasks.workunit.client.1.smithi149.stdout:4/420: mkdir d0/d6b/d93 0 2024-03-22T07:08:52.090 INFO:tasks.workunit.client.1.smithi149.stdout:4/421: dread - d0/d8/dc/d14/d3e/f3d zero size 2024-03-22T07:08:52.093 INFO:tasks.workunit.client.1.smithi149.stdout:7/443: dwrite d1/f41 [0,4194304] 0 2024-03-22T07:08:52.093 INFO:tasks.workunit.client.1.smithi149.stdout:7/444: write d1/d18/d1b/d1d/fd [581295,118468] 0 2024-03-22T07:08:52.093 INFO:tasks.workunit.client.1.smithi149.stdout:0/283: write d1/d31/d51/f47 [3414814,36892] 0 2024-03-22T07:08:52.098 INFO:tasks.workunit.client.1.smithi149.stdout:7/445: symlink d1/d18/d1b/d1d/d72/l8f 0 2024-03-22T07:08:52.098 INFO:tasks.workunit.client.1.smithi149.stdout:4/422: dread d0/d8/d1d/f8f [0,4194304] 0 2024-03-22T07:08:52.101 INFO:tasks.workunit.client.1.smithi149.stdout:7/446: mknod d1/d18/d1b/d1d/c90 0 2024-03-22T07:08:52.102 INFO:tasks.workunit.client.1.smithi149.stdout:1/363: sync 2024-03-22T07:08:52.102 INFO:tasks.workunit.client.1.smithi149.stdout:1/364: dread - d4/d10/f4c zero size 2024-03-22T07:08:52.104 INFO:tasks.workunit.client.1.smithi149.stdout:5/336: getdents d7/db/d2b 0 2024-03-22T07:08:52.104 INFO:tasks.workunit.client.1.smithi149.stdout:5/337: chown d7/c27 747015 1 2024-03-22T07:08:52.105 INFO:tasks.workunit.client.1.smithi149.stdout:3/312: sync 2024-03-22T07:08:52.106 INFO:tasks.workunit.client.1.smithi149.stdout:1/365: symlink d4/d20/d23/d2f/l7f 0 2024-03-22T07:08:52.106 INFO:tasks.workunit.client.1.smithi149.stdout:1/366: dread - d4/d10/d28/f2e zero size 2024-03-22T07:08:52.107 INFO:tasks.workunit.client.1.smithi149.stdout:5/338: symlink d7/da/d20/l6b 0 2024-03-22T07:08:52.109 INFO:tasks.workunit.client.1.smithi149.stdout:7/447: mknod d1/d1c/c91 0 2024-03-22T07:08:52.111 INFO:tasks.workunit.client.1.smithi149.stdout:1/367: mkdir d4/d80 0 2024-03-22T07:08:52.111 INFO:tasks.workunit.client.1.smithi149.stdout:1/368: dread - d4/d10/d28/f60 zero size 2024-03-22T07:08:52.111 INFO:tasks.workunit.client.1.smithi149.stdout:3/313: dread d0/d2/d36/f20 [0,4194304] 0 2024-03-22T07:08:52.111 INFO:tasks.workunit.client.1.smithi149.stdout:3/314: readlink d0/d4/d8/l26 0 2024-03-22T07:08:52.113 INFO:tasks.workunit.client.1.smithi149.stdout:5/339: rename d7/da/d20/d30/d3a/d3d/d2c/d4b to d7/db/d6c 0 2024-03-22T07:08:52.114 INFO:tasks.workunit.client.1.smithi149.stdout:5/340: symlink d7/da/d20/d36/d44/d4a/l6d 0 2024-03-22T07:08:52.117 INFO:tasks.workunit.client.1.smithi149.stdout:5/341: getdents d7/da/d20/d30/d3a/d3d/d2c/d47 0 2024-03-22T07:08:52.119 INFO:tasks.workunit.client.1.smithi149.stdout:5/342: rename d7/da/fe to d7/da/d20/d30/d3a/d3d/d2c/d47/f6e 0 2024-03-22T07:08:52.123 INFO:tasks.workunit.client.1.smithi149.stdout:1/369: dread d4/d7/f39 [0,4194304] 0 2024-03-22T07:08:52.136 INFO:tasks.workunit.client.1.smithi149.stdout:6/411: dwrite d9/d47/f5a [0,4194304] 0 2024-03-22T07:08:52.136 INFO:tasks.workunit.client.1.smithi149.stdout:1/370: write d4/d10/f63 [3017512,97119] 0 2024-03-22T07:08:52.137 INFO:tasks.workunit.client.1.smithi149.stdout:9/422: dwrite db/d22/f2b [0,4194304] 0 2024-03-22T07:08:52.138 INFO:tasks.workunit.client.1.smithi149.stdout:2/413: dwrite d1/d8/f24 [0,4194304] 0 2024-03-22T07:08:52.138 INFO:tasks.workunit.client.1.smithi149.stdout:3/315: dwrite d0/d4/d8/f25 [0,4194304] 0 2024-03-22T07:08:52.140 INFO:tasks.workunit.client.1.smithi149.stdout:4/423: dwrite d0/d8/d1d/f28 [0,4194304] 0 2024-03-22T07:08:52.140 INFO:tasks.workunit.client.1.smithi149.stdout:4/424: dread - d0/d8/d1d/d24/d82/f88 zero size 2024-03-22T07:08:52.140 INFO:tasks.workunit.client.1.smithi149.stdout:7/448: dwrite d1/d18/d1b/d1d/f13 [0,4194304] 0 2024-03-22T07:08:52.140 INFO:tasks.workunit.client.1.smithi149.stdout:1/371: creat d4/d10/d7c/d6a/d6b/f81 x:0 0 0 2024-03-22T07:08:52.141 INFO:tasks.workunit.client.1.smithi149.stdout:9/423: mknod db/d19/d1c/d85/c8d 0 2024-03-22T07:08:52.142 INFO:tasks.workunit.client.1.smithi149.stdout:9/424: dread - db/d19/d39/f5e zero size 2024-03-22T07:08:52.142 INFO:tasks.workunit.client.1.smithi149.stdout:9/425: fsync db/d22/f36 0 2024-03-22T07:08:52.142 INFO:tasks.workunit.client.1.smithi149.stdout:3/316: dread d0/d4/d8/d24/f47 [0,4194304] 0 2024-03-22T07:08:52.146 INFO:tasks.workunit.client.1.smithi149.stdout:2/414: mkdir d1/d4d/d8b 0 2024-03-22T07:08:52.147 INFO:tasks.workunit.client.1.smithi149.stdout:2/415: write d1/d3/f13 [4666058,27565] 0 2024-03-22T07:08:52.147 INFO:tasks.workunit.client.1.smithi149.stdout:2/416: write d1/d3/d5/d1f/f84 [877768,105895] 0 2024-03-22T07:08:52.147 INFO:tasks.workunit.client.1.smithi149.stdout:6/412: rmdir d9/d1f/d24/d34/d37 39 2024-03-22T07:08:52.150 INFO:tasks.workunit.client.1.smithi149.stdout:7/449: unlink d1/d18/d30/l54 0 2024-03-22T07:08:52.150 INFO:tasks.workunit.client.1.smithi149.stdout:7/450: fdatasync d1/d18/f34 0 2024-03-22T07:08:52.150 INFO:tasks.workunit.client.1.smithi149.stdout:7/451: chown d1/d1c/d2f/f76 456646498 1 2024-03-22T07:08:52.150 INFO:tasks.workunit.client.1.smithi149.stdout:7/452: write d1/d18/d1b/d1d/f8 [3481690,51350] 0 2024-03-22T07:08:52.151 INFO:tasks.workunit.client.1.smithi149.stdout:1/372: rmdir d4/d10/d7c 39 2024-03-22T07:08:52.151 INFO:tasks.workunit.client.1.smithi149.stdout:1/373: chown d4/d7/f33 3 1 2024-03-22T07:08:52.151 INFO:tasks.workunit.client.1.smithi149.stdout:1/374: truncate d4/d7/f79 506421 0 2024-03-22T07:08:52.151 INFO:tasks.workunit.client.1.smithi149.stdout:1/375: truncate d4/d20/f73 78562 0 2024-03-22T07:08:52.152 INFO:tasks.workunit.client.1.smithi149.stdout:3/317: getdents d0/d4/d8/d24/d40 0 2024-03-22T07:08:52.153 INFO:tasks.workunit.client.1.smithi149.stdout:3/318: creat d0/d2/f76 x:0 0 0 2024-03-22T07:08:52.153 INFO:tasks.workunit.client.1.smithi149.stdout:7/453: mkdir d1/d1c/d4d/d56/d92 0 2024-03-22T07:08:52.153 INFO:tasks.workunit.client.1.smithi149.stdout:1/376: truncate d4/d20/d23/d2f/f35 87640 0 2024-03-22T07:08:52.154 INFO:tasks.workunit.client.1.smithi149.stdout:7/454: mkdir d1/d1c/d49/d8e/d53/d5c/d93 0 2024-03-22T07:08:52.156 INFO:tasks.workunit.client.1.smithi149.stdout:1/377: rmdir d4/d80 0 2024-03-22T07:08:52.158 INFO:tasks.workunit.client.1.smithi149.stdout:1/378: mkdir d4/d7/d5d/d82 0 2024-03-22T07:08:52.158 INFO:tasks.workunit.client.1.smithi149.stdout:1/379: dread - d4/d20/d23/d2f/f62 zero size 2024-03-22T07:08:52.159 INFO:tasks.workunit.client.1.smithi149.stdout:2/417: fdatasync d1/d3/d5/d1f/f84 0 2024-03-22T07:08:52.161 INFO:tasks.workunit.client.1.smithi149.stdout:1/380: rename d4/d20/d23/d2f/f3b to d4/d10/d28/f83 0 2024-03-22T07:08:52.161 INFO:tasks.workunit.client.1.smithi149.stdout:1/381: creat d4/d10/d7c/d6a/d6b/f84 x:0 0 0 2024-03-22T07:08:52.162 INFO:tasks.workunit.client.1.smithi149.stdout:2/418: mkdir d1/d4d/d69/d75/d7e/d8c 0 2024-03-22T07:08:52.168 INFO:tasks.workunit.client.1.smithi149.stdout:7/455: dread d1/d1c/d49/d8e/f50 [0,4194304] 0 2024-03-22T07:08:52.171 INFO:tasks.workunit.client.1.smithi149.stdout:9/426: dwrite db/f2f [4194304,4194304] 0 2024-03-22T07:08:52.171 INFO:tasks.workunit.client.1.smithi149.stdout:9/427: dread - db/d19/d39/d3b/f6d zero size 2024-03-22T07:08:52.173 INFO:tasks.workunit.client.1.smithi149.stdout:2/419: dread d1/f10 [8388608,4194304] 0 2024-03-22T07:08:52.173 INFO:tasks.workunit.client.1.smithi149.stdout:2/420: chown d1/f60 180 1 2024-03-22T07:08:52.173 INFO:tasks.workunit.client.1.smithi149.stdout:9/428: stat db/d21/d37/l54 0 2024-03-22T07:08:52.174 INFO:tasks.workunit.client.1.smithi149.stdout:2/421: symlink d1/d3/d5/d1f/d5e/l8d 0 2024-03-22T07:08:52.175 INFO:tasks.workunit.client.1.smithi149.stdout:9/429: creat db/d42/d49/f8e x:0 0 0 2024-03-22T07:08:52.177 INFO:tasks.workunit.client.1.smithi149.stdout:2/422: symlink d1/d3/dd/d71/d81/l8e 0 2024-03-22T07:08:52.178 INFO:tasks.workunit.client.1.smithi149.stdout:9/430: getdents db/d19/d39/d3b 0 2024-03-22T07:08:52.179 INFO:tasks.workunit.client.1.smithi149.stdout:7/456: write d1/d1c/f7b [1842078,62810] 0 2024-03-22T07:08:52.179 INFO:tasks.workunit.client.1.smithi149.stdout:7/457: fdatasync d1/d18/f34 0 2024-03-22T07:08:52.181 INFO:tasks.workunit.client.1.smithi149.stdout:9/431: rename db/d21/c7a to db/d12/c8f 0 2024-03-22T07:08:52.181 INFO:tasks.workunit.client.1.smithi149.stdout:9/432: readlink db/d12/l6e 0 2024-03-22T07:08:52.182 INFO:tasks.workunit.client.1.smithi149.stdout:9/433: symlink db/d21/d2e/l90 0 2024-03-22T07:08:52.182 INFO:tasks.workunit.client.1.smithi149.stdout:9/434: write db/d42/f4c [1697032,108134] 0 2024-03-22T07:08:52.182 INFO:tasks.workunit.client.1.smithi149.stdout:9/435: dread db/d42/d49/d4b/d4d/f55 [0,4194304] 0 2024-03-22T07:08:52.182 INFO:tasks.workunit.client.1.smithi149.stdout:9/436: fsync db/d21/d37/f64 0 2024-03-22T07:08:52.182 INFO:tasks.workunit.client.1.smithi149.stdout:9/437: write db/d19/d28/f80 [4755674,68192] 0 2024-03-22T07:08:52.182 INFO:tasks.workunit.client.1.smithi149.stdout:9/438: stat db/d42/f5a 0 2024-03-22T07:08:52.183 INFO:tasks.workunit.client.1.smithi149.stdout:9/439: stat db/d42/d49/d4b 0 2024-03-22T07:08:52.183 INFO:tasks.workunit.client.1.smithi149.stdout:9/440: symlink db/d21/l91 0 2024-03-22T07:08:52.183 INFO:tasks.workunit.client.1.smithi149.stdout:9/441: creat db/d33/f92 x:0 0 0 2024-03-22T07:08:52.184 INFO:tasks.workunit.client.1.smithi149.stdout:6/413: dwrite d9/df/f56 [0,4194304] 0 2024-03-22T07:08:52.184 INFO:tasks.workunit.client.1.smithi149.stdout:6/414: creat d9/df/d22/f87 x:0 0 0 2024-03-22T07:08:52.189 INFO:tasks.workunit.client.1.smithi149.stdout:7/458: dread d1/d18/d1b/d51/d7a/d7d/f3b [0,4194304] 0 2024-03-22T07:08:52.189 INFO:tasks.workunit.client.1.smithi149.stdout:7/459: write d1/d18/d1b/d1d/f13 [4276810,12740] 0 2024-03-22T07:08:52.189 INFO:tasks.workunit.client.1.smithi149.stdout:3/319: dwrite d0/d2/d30/f3f [0,4194304] 0 2024-03-22T07:08:52.189 INFO:tasks.workunit.client.1.smithi149.stdout:3/320: write d0/d2/d30/d5d/f65 [19147,127681] 0 2024-03-22T07:08:52.189 INFO:tasks.workunit.client.1.smithi149.stdout:2/423: write d1/d3/d6/f7 [2093051,115461] 0 2024-03-22T07:08:52.189 INFO:tasks.workunit.client.1.smithi149.stdout:2/424: chown d1/d72 101141 1 2024-03-22T07:08:52.196 INFO:tasks.workunit.client.1.smithi149.stdout:3/321: mknod d0/d4/d8/c77 0 2024-03-22T07:08:52.197 INFO:tasks.workunit.client.1.smithi149.stdout:6/415: write d9/df/d2e/f27 [2887454,33767] 0 2024-03-22T07:08:52.197 INFO:tasks.workunit.client.1.smithi149.stdout:2/425: creat d1/d3/d6/f8f x:0 0 0 2024-03-22T07:08:52.197 INFO:tasks.workunit.client.1.smithi149.stdout:2/426: fsync d1/d3/d5/d41/f48 0 2024-03-22T07:08:52.197 INFO:tasks.workunit.client.1.smithi149.stdout:2/427: write d1/d3/d5/d1f/d5e/f4a [867073,110219] 0 2024-03-22T07:08:52.200 INFO:tasks.workunit.client.1.smithi149.stdout:2/428: mkdir d1/d4d/d90 0 2024-03-22T07:08:52.201 INFO:tasks.workunit.client.1.smithi149.stdout:3/322: getdents d0/d4/d8/d24/d40/d54/d5f 0 2024-03-22T07:08:52.203 INFO:tasks.workunit.client.1.smithi149.stdout:2/429: truncate d1/d3/d5/f2b 1390207 0 2024-03-22T07:08:52.204 INFO:tasks.workunit.client.1.smithi149.stdout:3/323: truncate d0/d2/f2d 391796 0 2024-03-22T07:08:52.204 INFO:tasks.workunit.client.1.smithi149.stdout:2/430: fsync d1/d3/dd/f5f 0 2024-03-22T07:08:52.205 INFO:tasks.workunit.client.1.smithi149.stdout:3/324: symlink d0/d4/d8/d24/d40/l78 0 2024-03-22T07:08:52.206 INFO:tasks.workunit.client.1.smithi149.stdout:2/431: symlink d1/d4d/d90/l91 0 2024-03-22T07:08:52.207 INFO:tasks.workunit.client.1.smithi149.stdout:2/432: creat d1/d3/d6/f92 x:0 0 0 2024-03-22T07:08:52.208 INFO:tasks.workunit.client.1.smithi149.stdout:2/433: rmdir d1/d3/d5/d41 39 2024-03-22T07:08:52.208 INFO:tasks.workunit.client.1.smithi149.stdout:2/434: chown d1/d3/d6/f8f 36049 1 2024-03-22T07:08:52.210 INFO:tasks.workunit.client.1.smithi149.stdout:2/435: link d1/d3/f25 d1/d3/d5/d1f/f93 0 2024-03-22T07:08:52.210 INFO:tasks.workunit.client.1.smithi149.stdout:2/436: read d1/d3/d6/f7 [2594990,113784] 0 2024-03-22T07:08:52.212 INFO:tasks.workunit.client.1.smithi149.stdout:2/437: mkdir d1/d3/d5/d1f/d68/d94 0 2024-03-22T07:08:52.212 INFO:tasks.workunit.client.1.smithi149.stdout:2/438: fdatasync d1/d3/d5/d41/f48 0 2024-03-22T07:08:52.212 INFO:tasks.workunit.client.1.smithi149.stdout:2/439: write d1/d3/d5/d1f/d5e/d2f/f33 [616879,8774] 0 2024-03-22T07:08:52.214 INFO:tasks.workunit.client.1.smithi149.stdout:7/460: dwrite d1/d18/d1b/d1d/f62 [0,4194304] 0 2024-03-22T07:08:52.218 INFO:tasks.workunit.client.1.smithi149.stdout:2/440: dread d1/d3/f25 [0,4194304] 0 2024-03-22T07:08:52.219 INFO:tasks.workunit.client.1.smithi149.stdout:7/461: creat d1/d1c/f94 x:0 0 0 2024-03-22T07:08:52.219 INFO:tasks.workunit.client.1.smithi149.stdout:7/462: write d1/f5 [1880134,57510] 0 2024-03-22T07:08:52.219 INFO:tasks.workunit.client.1.smithi149.stdout:7/463: dread - d1/d1c/d4d/d56/f81 zero size 2024-03-22T07:08:52.219 INFO:tasks.workunit.client.1.smithi149.stdout:7/464: write d1/d18/d1b/d51/d7a/d7d/f3b [4893715,63051] 0 2024-03-22T07:08:52.221 INFO:tasks.workunit.client.1.smithi149.stdout:2/441: unlink d1/d4d/d69/l82 0 2024-03-22T07:08:52.221 INFO:tasks.workunit.client.1.smithi149.stdout:9/442: dwrite db/d33/f3d [0,4194304] 0 2024-03-22T07:08:52.221 INFO:tasks.workunit.client.1.smithi149.stdout:9/443: write db/d19/d39/f5e [1040053,48950] 0 2024-03-22T07:08:52.225 INFO:tasks.workunit.client.1.smithi149.stdout:2/442: rmdir d1/d3/dd/d71/d81 39 2024-03-22T07:08:52.227 INFO:tasks.workunit.client.1.smithi149.stdout:9/444: getdents db/d33 0 2024-03-22T07:08:52.228 INFO:tasks.workunit.client.1.smithi149.stdout:2/443: mknod d1/d3/d5/d1f/d68/d94/c95 0 2024-03-22T07:08:52.228 INFO:tasks.workunit.client.1.smithi149.stdout:2/444: chown d1/d3/d5/d1f/d5e/d2f/l44 898471870 1 2024-03-22T07:08:52.230 INFO:tasks.workunit.client.1.smithi149.stdout:9/445: getdents db/d19/d39 0 2024-03-22T07:08:52.231 INFO:tasks.workunit.client.1.smithi149.stdout:2/445: symlink d1/d3/d6/d3f/d74/l96 0 2024-03-22T07:08:52.231 INFO:tasks.workunit.client.1.smithi149.stdout:2/446: readlink d1/l20 0 2024-03-22T07:08:52.232 INFO:tasks.workunit.client.1.smithi149.stdout:9/446: mkdir db/d19/d93 0 2024-03-22T07:08:52.232 INFO:tasks.workunit.client.1.smithi149.stdout:9/447: creat db/d19/f94 x:0 0 0 2024-03-22T07:08:52.238 INFO:tasks.workunit.client.1.smithi149.stdout:3/325: dwrite d0/d4/d8/d24/d40/d54/d5f/f69 [0,4194304] 0 2024-03-22T07:08:52.238 INFO:tasks.workunit.client.1.smithi149.stdout:3/326: write d0/d2/d42/f4e [128820,122420] 0 2024-03-22T07:08:52.238 INFO:tasks.workunit.client.1.smithi149.stdout:3/327: chown d0/d4/l6 0 1 2024-03-22T07:08:52.241 INFO:tasks.workunit.client.1.smithi149.stdout:3/328: symlink d0/d2/d30/l79 0 2024-03-22T07:08:52.243 INFO:tasks.workunit.client.1.smithi149.stdout:3/329: truncate d0/d2/d36/f59 3563130 0 2024-03-22T07:08:52.243 INFO:tasks.workunit.client.1.smithi149.stdout:3/330: chown d0 236950 1 2024-03-22T07:08:52.243 INFO:tasks.workunit.client.1.smithi149.stdout:3/331: fsync d0/d4/d8/d24/d40/d54/d5f/f69 0 2024-03-22T07:08:52.244 INFO:tasks.workunit.client.1.smithi149.stdout:3/332: link d0/d2/f76 d0/d2/d42/d66/f7a 0 2024-03-22T07:08:52.246 INFO:tasks.workunit.client.1.smithi149.stdout:3/333: mknod d0/d4/d8/d24/d40/d62/c7b 0 2024-03-22T07:08:52.247 INFO:tasks.workunit.client.1.smithi149.stdout:3/334: creat d0/d2/d42/f7c x:0 0 0 2024-03-22T07:08:52.252 INFO:tasks.workunit.client.1.smithi149.stdout:3/335: dread d0/d2/f2d [0,4194304] 0 2024-03-22T07:08:52.252 INFO:tasks.workunit.client.1.smithi149.stdout:3/336: chown d0/d4/d8/d24/d40/d54/d5f/f69 52335 1 2024-03-22T07:08:52.252 INFO:tasks.workunit.client.1.smithi149.stdout:3/337: chown d0/d2/d30/d31/f33 55129 1 2024-03-22T07:08:52.257 INFO:tasks.workunit.client.1.smithi149.stdout:3/338: write d0/d4/fe [6355519,8446] 0 2024-03-22T07:08:52.257 INFO:tasks.workunit.client.1.smithi149.stdout:3/339: fdatasync d0/d4/d8/f21 0 2024-03-22T07:08:52.257 INFO:tasks.workunit.client.1.smithi149.stdout:3/340: write d0/d2/d42/f4e [1358173,116882] 0 2024-03-22T07:08:52.258 INFO:tasks.workunit.client.1.smithi149.stdout:6/416: dwrite d9/df/f56 [0,4194304] 0 2024-03-22T07:08:52.259 INFO:tasks.workunit.client.1.smithi149.stdout:2/447: dwrite d1/d8/d34/f63 [0,4194304] 0 2024-03-22T07:08:52.259 INFO:tasks.workunit.client.1.smithi149.stdout:2/448: truncate d1/d8/d34/f7a 626234 0 2024-03-22T07:08:52.259 INFO:tasks.workunit.client.1.smithi149.stdout:2/449: dread - d1/d3/d6/d3f/f7f zero size 2024-03-22T07:08:52.259 INFO:tasks.workunit.client.1.smithi149.stdout:3/341: rename d0/d4/f1e to d0/d2/d42/d66/f7d 0 2024-03-22T07:08:52.259 INFO:tasks.workunit.client.1.smithi149.stdout:9/448: dwrite db/d22/f69 [0,4194304] 0 2024-03-22T07:08:52.260 INFO:tasks.workunit.client.1.smithi149.stdout:6/417: mknod d9/d1f/d24/d34/d4f/d55/c88 0 2024-03-22T07:08:52.263 INFO:tasks.workunit.client.1.smithi149.stdout:3/342: rename d0/d4/d8/d24/d40/d54/d5f to d0/d2/d30/d7e 0 2024-03-22T07:08:52.266 INFO:tasks.workunit.client.1.smithi149.stdout:9/449: unlink db/d19/d28/f80 0 2024-03-22T07:08:52.266 INFO:tasks.workunit.client.1.smithi149.stdout:3/343: mknod d0/d4/d8/d24/d40/d62/c7f 0 2024-03-22T07:08:52.266 INFO:tasks.workunit.client.1.smithi149.stdout:3/344: creat d0/d2/d36/d74/f80 x:0 0 0 2024-03-22T07:08:52.267 INFO:tasks.workunit.client.1.smithi149.stdout:9/450: mknod db/d19/d39/d3b/c95 0 2024-03-22T07:08:52.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/418: dread d9/d1f/d24/d34/d37/d38/f54 [0,4194304] 0 2024-03-22T07:08:52.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/419: creat d9/df/d22/f89 x:0 0 0 2024-03-22T07:08:52.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/420: readlink d9/d1f/d24/d34/d37/d38/l65 0 2024-03-22T07:08:52.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/421: chown d9/d1f/d24/d68/d73 85 1 2024-03-22T07:08:52.269 INFO:tasks.workunit.client.1.smithi149.stdout:3/345: mknod d0/d2/d30/c81 0 2024-03-22T07:08:52.270 INFO:tasks.workunit.client.1.smithi149.stdout:3/346: chown d0/d4/d8/l44 1219 1 2024-03-22T07:08:52.270 INFO:tasks.workunit.client.1.smithi149.stdout:3/347: truncate d0/d4/d8/f25 4795643 0 2024-03-22T07:08:52.272 INFO:tasks.workunit.client.1.smithi149.stdout:9/451: creat db/d21/d8c/f96 x:0 0 0 2024-03-22T07:08:52.274 INFO:tasks.workunit.client.1.smithi149.stdout:6/422: rmdir d9/df/d2e/d4e 39 2024-03-22T07:08:52.276 INFO:tasks.workunit.client.1.smithi149.stdout:2/450: dwrite d1/d3/d5/ff [0,4194304] 0 2024-03-22T07:08:52.276 INFO:tasks.workunit.client.1.smithi149.stdout:2/451: write d1/d3/dd/f40 [2435380,29955] 0 2024-03-22T07:08:52.276 INFO:tasks.workunit.client.1.smithi149.stdout:9/452: getdents db/d19/d1c/d85 0 2024-03-22T07:08:52.276 INFO:tasks.workunit.client.1.smithi149.stdout:9/453: readlink db/d12/l6e 0 2024-03-22T07:08:52.279 INFO:tasks.workunit.client.1.smithi149.stdout:9/454: mknod db/d19/d39/d3b/c97 0 2024-03-22T07:08:52.294 INFO:tasks.workunit.client.1.smithi149.stdout:9/455: dread db/d19/d39/d3b/f53 [0,4194304] 0 2024-03-22T07:08:52.294 INFO:tasks.workunit.client.1.smithi149.stdout:9/456: creat db/d21/d8c/f98 x:0 0 0 2024-03-22T07:08:52.295 INFO:tasks.workunit.client.1.smithi149.stdout:9/457: rmdir db/d33 39 2024-03-22T07:08:52.295 INFO:tasks.workunit.client.1.smithi149.stdout:9/458: stat db/l27 0 2024-03-22T07:08:52.296 INFO:tasks.workunit.client.1.smithi149.stdout:9/459: creat db/d21/d2e/f99 x:0 0 0 2024-03-22T07:08:52.297 INFO:tasks.workunit.client.1.smithi149.stdout:2/452: dwrite d1/d3/f4b [0,4194304] 0 2024-03-22T07:08:52.297 INFO:tasks.workunit.client.1.smithi149.stdout:2/453: truncate d1/d3/dd/f4c 173600 0 2024-03-22T07:08:52.298 INFO:tasks.workunit.client.1.smithi149.stdout:6/423: dwrite d9/df/d22/f89 [0,4194304] 0 2024-03-22T07:08:52.298 INFO:tasks.workunit.client.1.smithi149.stdout:6/424: chown d9/d1f/d24/l70 27 1 2024-03-22T07:08:52.298 INFO:tasks.workunit.client.1.smithi149.stdout:6/425: creat d9/d47/d52/f8a x:0 0 0 2024-03-22T07:08:52.301 INFO:tasks.workunit.client.1.smithi149.stdout:2/454: unlink d1/d3/d5/f28 0 2024-03-22T07:08:52.316 INFO:tasks.workunit.client.1.smithi149.stdout:9/460: dwrite db/d42/d49/f8e [0,4194304] 0 2024-03-22T07:08:52.316 INFO:tasks.workunit.client.1.smithi149.stdout:9/461: truncate db/d19/d39/d3b/f53 4374486 0 2024-03-22T07:08:52.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:52 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:52.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:52 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:52.319 INFO:tasks.workunit.client.1.smithi149.stdout:9/462: creat db/d19/d28/f9a x:0 0 0 2024-03-22T07:08:52.320 INFO:tasks.workunit.client.1.smithi149.stdout:9/463: getdents db/d42/d49/d4b/d5f/d7f 0 2024-03-22T07:08:52.333 INFO:tasks.workunit.client.1.smithi149.stdout:6/426: dwrite d9/d1f/d24/d34/d37/d38/f53 [0,4194304] 0 2024-03-22T07:08:52.333 INFO:tasks.workunit.client.1.smithi149.stdout:6/427: fsync d9/df/d2e/d63/f69 0 2024-03-22T07:08:52.333 INFO:tasks.workunit.client.1.smithi149.stdout:6/428: truncate d9/d1f/d24/d34/d37/d38/f85 513937 0 2024-03-22T07:08:52.333 INFO:tasks.workunit.client.1.smithi149.stdout:6/429: creat d9/d1f/d24/d68/d73/f8b x:0 0 0 2024-03-22T07:08:52.334 INFO:tasks.workunit.client.1.smithi149.stdout:6/430: chown d9/df/d2e/c10 22 1 2024-03-22T07:08:52.334 INFO:tasks.workunit.client.1.smithi149.stdout:6/431: dread - d9/d1f/d24/d68/d73/f8b zero size 2024-03-22T07:08:52.334 INFO:tasks.workunit.client.1.smithi149.stdout:6/432: chown d9/df/d22/c50 7582493 1 2024-03-22T07:08:52.334 INFO:tasks.workunit.client.1.smithi149.stdout:2/455: dwrite d1/f42 [0,4194304] 0 2024-03-22T07:08:52.336 INFO:tasks.workunit.client.1.smithi149.stdout:6/433: getdents d9/d1f/d24/d34/d4f/d55 0 2024-03-22T07:08:52.337 INFO:tasks.workunit.client.1.smithi149.stdout:6/434: creat d9/d1f/d24/d68/f8c x:0 0 0 2024-03-22T07:08:52.340 INFO:tasks.workunit.client.1.smithi149.stdout:6/435: creat d9/df/d2e/f8d x:0 0 0 2024-03-22T07:08:52.341 INFO:tasks.workunit.client.1.smithi149.stdout:6/436: dread d9/d1f/f6b [0,4194304] 0 2024-03-22T07:08:52.343 INFO:tasks.workunit.client.1.smithi149.stdout:6/437: creat d9/d1f/d24/d34/d4f/d55/f8e x:0 0 0 2024-03-22T07:08:52.343 INFO:tasks.workunit.client.1.smithi149.stdout:6/438: getdents d9/d1f/d24/d34/d4f/d86 0 2024-03-22T07:08:52.343 INFO:tasks.workunit.client.1.smithi149.stdout:6/439: write d9/d1f/d24/d34/d37/d38/f43 [1165040,60156] 0 2024-03-22T07:08:52.350 INFO:tasks.workunit.client.1.smithi149.stdout:2/456: dwrite d1/f4 [4194304,4194304] 0 2024-03-22T07:08:52.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:52 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:52.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:52 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:52.355 INFO:tasks.workunit.client.1.smithi149.stdout:2/457: write d1/d3/d5/d1f/f39 [495979,47815] 0 2024-03-22T07:08:52.357 INFO:tasks.workunit.client.1.smithi149.stdout:2/458: mkdir d1/d3/d5/d1f/d68/d97 0 2024-03-22T07:08:52.358 INFO:tasks.workunit.client.1.smithi149.stdout:2/459: mkdir d1/d4d/d69/d75/d7e/d8c/d98 0 2024-03-22T07:08:52.365 INFO:tasks.workunit.client.1.smithi149.stdout:6/440: dwrite d9/d1f/d24/d34/d37/f3e [0,4194304] 0 2024-03-22T07:08:52.366 INFO:tasks.workunit.client.1.smithi149.stdout:6/441: symlink d9/d1f/d24/d34/l8f 0 2024-03-22T07:08:52.367 INFO:tasks.workunit.client.1.smithi149.stdout:6/442: creat d9/d1f/f90 x:0 0 0 2024-03-22T07:08:52.367 INFO:tasks.workunit.client.1.smithi149.stdout:6/443: write f5 [2491913,110524] 0 2024-03-22T07:08:52.368 INFO:tasks.workunit.client.1.smithi149.stdout:6/444: chown d9/d1f/d24/d34/d4f/d55/c88 194603 1 2024-03-22T07:08:52.368 INFO:tasks.workunit.client.1.smithi149.stdout:6/445: fdatasync d9/df/d2e/f8d 0 2024-03-22T07:08:52.368 INFO:tasks.workunit.client.1.smithi149.stdout:6/446: creat d9/df/f91 x:0 0 0 2024-03-22T07:08:52.371 INFO:tasks.workunit.client.1.smithi149.stdout:6/447: getdents d9/d1f/d24/d42 0 2024-03-22T07:08:52.372 INFO:tasks.workunit.client.1.smithi149.stdout:6/448: mknod d9/df/d22/d6d/c92 0 2024-03-22T07:08:52.373 INFO:tasks.workunit.client.1.smithi149.stdout:6/449: truncate d9/d1f/d24/d34/d37/d38/f53 3544427 0 2024-03-22T07:08:52.373 INFO:tasks.workunit.client.1.smithi149.stdout:6/450: creat d9/d47/d52/f93 x:0 0 0 2024-03-22T07:08:52.373 INFO:tasks.workunit.client.1.smithi149.stdout:2/460: write d1/d3/d5/d41/f48 [1539395,108745] 0 2024-03-22T07:08:52.376 INFO:tasks.workunit.client.1.smithi149.stdout:2/461: link d1/d3/d5/d1f/d68/f79 d1/d4d/d90/f99 0 2024-03-22T07:08:52.377 INFO:tasks.workunit.client.1.smithi149.stdout:5/343: sync 2024-03-22T07:08:52.377 INFO:tasks.workunit.client.1.smithi149.stdout:0/284: sync 2024-03-22T07:08:52.377 INFO:tasks.workunit.client.1.smithi149.stdout:8/411: sync 2024-03-22T07:08:52.377 INFO:tasks.workunit.client.1.smithi149.stdout:0/285: stat d1/d1a/l4c 0 2024-03-22T07:08:52.377 INFO:tasks.workunit.client.1.smithi149.stdout:8/412: chown d1/d20/c61 1458852674 1 2024-03-22T07:08:52.379 INFO:tasks.workunit.client.1.smithi149.stdout:2/462: rename d1/d4d/d69/d75/d7e/d8c/d98 to d1/d3/d9a 0 2024-03-22T07:08:52.381 INFO:tasks.workunit.client.1.smithi149.stdout:5/344: fdatasync d7/da/d20/d30/d3a/d3d/d2c/d47/f52 0 2024-03-22T07:08:52.383 INFO:tasks.workunit.client.1.smithi149.stdout:8/413: symlink d1/d20/l8a 0 2024-03-22T07:08:52.385 INFO:tasks.workunit.client.1.smithi149.stdout:0/286: rmdir d1/d31 39 2024-03-22T07:08:52.386 INFO:tasks.workunit.client.1.smithi149.stdout:5/345: truncate d7/da/f1b 340462 0 2024-03-22T07:08:52.387 INFO:tasks.workunit.client.1.smithi149.stdout:0/287: unlink d1/d20/c21 0 2024-03-22T07:08:52.388 INFO:tasks.workunit.client.1.smithi149.stdout:5/346: link d7/da/d20/d30/d3a/d3d/d2c/d47/f52 d7/da/d20/d36/d44/f6f 0 2024-03-22T07:08:52.388 INFO:tasks.workunit.client.1.smithi149.stdout:5/347: write d7/da/d20/f5e [835163,92859] 0 2024-03-22T07:08:52.388 INFO:tasks.workunit.client.1.smithi149.stdout:5/348: dread d7/da/d20/d30/d3a/d3d/d2c/d47/f52 [0,4194304] 0 2024-03-22T07:08:52.388 INFO:tasks.workunit.client.1.smithi149.stdout:5/349: write d7/da/d20/f49 [742898,99631] 0 2024-03-22T07:08:52.389 INFO:tasks.workunit.client.1.smithi149.stdout:0/288: mkdir d1/d27/d66 0 2024-03-22T07:08:52.389 INFO:tasks.workunit.client.1.smithi149.stdout:0/289: fsync d1/d9/d14/d1f/f2a 0 2024-03-22T07:08:52.392 INFO:tasks.workunit.client.1.smithi149.stdout:0/290: mknod d1/d9/c67 0 2024-03-22T07:08:52.392 INFO:tasks.workunit.client.1.smithi149.stdout:0/291: chown d1/d1a/c54 968348 1 2024-03-22T07:08:52.393 INFO:tasks.workunit.client.1.smithi149.stdout:0/292: mknod d1/d31/d51/c68 0 2024-03-22T07:08:52.393 INFO:tasks.workunit.client.1.smithi149.stdout:0/293: dread - d1/d1a/f50 zero size 2024-03-22T07:08:52.395 INFO:tasks.workunit.client.1.smithi149.stdout:0/294: truncate d1/d9/d14/f28 3442991 0 2024-03-22T07:08:52.395 INFO:tasks.workunit.client.1.smithi149.stdout:2/463: dread d1/d8/d34/f57 [0,4194304] 0 2024-03-22T07:08:52.396 INFO:tasks.workunit.client.1.smithi149.stdout:8/414: dread d1/d2/d9/d11/f55 [0,4194304] 0 2024-03-22T07:08:52.396 INFO:tasks.workunit.client.1.smithi149.stdout:2/464: link d1/d3/d5/d1f/d5e/c2d d1/d3/d6/d3f/d86/c9b 0 2024-03-22T07:08:52.397 INFO:tasks.workunit.client.1.smithi149.stdout:8/415: rmdir d1/d20/d35/d22 39 2024-03-22T07:08:52.397 INFO:tasks.workunit.client.1.smithi149.stdout:2/465: creat d1/d3/d5/d1f/d68/d94/f9c x:0 0 0 2024-03-22T07:08:52.398 INFO:tasks.workunit.client.1.smithi149.stdout:5/350: dread d7/d35/f40 [0,4194304] 0 2024-03-22T07:08:52.398 INFO:tasks.workunit.client.1.smithi149.stdout:8/416: mkdir d1/d2/d9/d11/d8b 0 2024-03-22T07:08:52.399 INFO:tasks.workunit.client.1.smithi149.stdout:8/417: fsync d1/d2/d18/f3c 0 2024-03-22T07:08:52.401 INFO:tasks.workunit.client.1.smithi149.stdout:2/466: unlink d1/d3/dd/f6e 0 2024-03-22T07:08:52.402 INFO:tasks.workunit.client.1.smithi149.stdout:5/351: mknod d7/da/c70 0 2024-03-22T07:08:52.408 INFO:tasks.workunit.client.1.smithi149.stdout:2/467: symlink d1/d3/d6/d32/l9d 0 2024-03-22T07:08:52.414 INFO:tasks.workunit.client.1.smithi149.stdout:5/352: rmdir d7/da 39 2024-03-22T07:08:52.415 INFO:tasks.workunit.client.1.smithi149.stdout:2/468: creat d1/d72/f9e x:0 0 0 2024-03-22T07:08:52.416 INFO:tasks.workunit.client.1.smithi149.stdout:2/469: dread d1/f60 [0,4194304] 0 2024-03-22T07:08:52.416 INFO:tasks.workunit.client.1.smithi149.stdout:5/353: creat d7/da/d20/d30/d3a/d3d/d2c/d63/f71 x:0 0 0 2024-03-22T07:08:52.417 INFO:tasks.workunit.client.1.smithi149.stdout:2/470: mkdir d1/d3/d5/d41/d9f 0 2024-03-22T07:08:52.417 INFO:tasks.workunit.client.1.smithi149.stdout:2/471: fsync d1/f4 0 2024-03-22T07:08:52.424 INFO:tasks.workunit.client.1.smithi149.stdout:0/295: dwrite d1/d9/d14/f28 [0,4194304] 0 2024-03-22T07:08:52.426 INFO:tasks.workunit.client.1.smithi149.stdout:0/296: rename d1/c49 to d1/d31/d51/d64/c69 0 2024-03-22T07:08:52.427 INFO:tasks.workunit.client.1.smithi149.stdout:0/297: rename d1/f56 to d1/f6a 0 2024-03-22T07:08:52.429 INFO:tasks.workunit.client.1.smithi149.stdout:2/472: write d1/d3/d5/d1f/f1b [1812358,102614] 0 2024-03-22T07:08:52.436 INFO:tasks.workunit.client.1.smithi149.stdout:0/298: dread d1/d9/d14/d1f/f5f [0,4194304] 0 2024-03-22T07:08:52.437 INFO:tasks.workunit.client.1.smithi149.stdout:0/299: mkdir d1/d6b 0 2024-03-22T07:08:52.438 INFO:tasks.workunit.client.1.smithi149.stdout:0/300: creat d1/d9/d59/f6c x:0 0 0 2024-03-22T07:08:52.439 INFO:tasks.workunit.client.1.smithi149.stdout:5/354: dwrite d7/da/f37 [4194304,4194304] 0 2024-03-22T07:08:52.440 INFO:tasks.workunit.client.1.smithi149.stdout:0/301: unlink d1/c5 0 2024-03-22T07:08:52.442 INFO:tasks.workunit.client.1.smithi149.stdout:0/302: symlink d1/l6d 0 2024-03-22T07:08:52.443 INFO:tasks.workunit.client.1.smithi149.stdout:0/303: rmdir d1/d1a 39 2024-03-22T07:08:52.443 INFO:tasks.workunit.client.1.smithi149.stdout:0/304: creat d1/d27/f6e x:0 0 0 2024-03-22T07:08:52.446 INFO:tasks.workunit.client.1.smithi149.stdout:0/305: symlink d1/l6f 0 2024-03-22T07:08:52.447 INFO:tasks.workunit.client.1.smithi149.stdout:0/306: write d1/d9/d14/d1f/f2a [3236970,4203] 0 2024-03-22T07:08:52.447 INFO:tasks.workunit.client.1.smithi149.stdout:0/307: fsync d1/d9/d14/f55 0 2024-03-22T07:08:52.447 INFO:tasks.workunit.client.1.smithi149.stdout:0/308: creat d1/d9/d14/f70 x:0 0 0 2024-03-22T07:08:52.447 INFO:tasks.workunit.client.1.smithi149.stdout:0/309: read - d1/d9/d14/d1f/f37 zero size 2024-03-22T07:08:52.449 INFO:tasks.workunit.client.1.smithi149.stdout:0/310: rename d1/f22 to d1/d9/f71 0 2024-03-22T07:08:52.450 INFO:tasks.workunit.client.1.smithi149.stdout:2/473: dwrite d1/f42 [0,4194304] 0 2024-03-22T07:08:52.451 INFO:tasks.workunit.client.1.smithi149.stdout:0/311: rename d1/d27/l40 to d1/d9/d59/l72 0 2024-03-22T07:08:52.451 INFO:tasks.workunit.client.1.smithi149.stdout:0/312: stat d1/d9/d14/c5e 0 2024-03-22T07:08:52.451 INFO:tasks.workunit.client.1.smithi149.stdout:0/313: chown d1/d1a 56 1 2024-03-22T07:08:52.455 INFO:tasks.workunit.client.1.smithi149.stdout:2/474: symlink d1/d3/dd/d71/d81/la0 0 2024-03-22T07:08:52.455 INFO:tasks.workunit.client.1.smithi149.stdout:2/475: truncate d1/d3/d5/d1f/d68/d94/f9c 772990 0 2024-03-22T07:08:52.455 INFO:tasks.workunit.client.1.smithi149.stdout:2/476: chown d1 1343551850 1 2024-03-22T07:08:52.455 INFO:tasks.workunit.client.1.smithi149.stdout:2/477: write d1/d3/f29 [737819,35443] 0 2024-03-22T07:08:52.456 INFO:tasks.workunit.client.1.smithi149.stdout:2/478: creat d1/d4d/d69/d75/fa1 x:0 0 0 2024-03-22T07:08:52.456 INFO:tasks.workunit.client.1.smithi149.stdout:5/355: dwrite d7/da/d20/d36/d44/f5d [0,4194304] 0 2024-03-22T07:08:52.456 INFO:tasks.workunit.client.1.smithi149.stdout:5/356: creat d7/da/d20/d30/d3a/d3d/d2c/d63/f72 x:0 0 0 2024-03-22T07:08:52.457 INFO:tasks.workunit.client.1.smithi149.stdout:0/314: rename d1/d1a/l1b to d1/d31/d51/l73 0 2024-03-22T07:08:52.460 INFO:tasks.workunit.client.1.smithi149.stdout:0/315: mkdir d1/d9/d14/d74 0 2024-03-22T07:08:52.463 INFO:tasks.workunit.client.1.smithi149.stdout:0/316: symlink d1/d20/l75 0 2024-03-22T07:08:52.463 INFO:tasks.workunit.client.1.smithi149.stdout:0/317: fsync d1/d9/f3a 0 2024-03-22T07:08:52.463 INFO:tasks.workunit.client.1.smithi149.stdout:0/318: readlink d1/d31/d51/l73 0 2024-03-22T07:08:52.466 INFO:tasks.workunit.client.1.smithi149.stdout:2/479: write d1/d3/f54 [1211293,13484] 0 2024-03-22T07:08:52.468 INFO:tasks.workunit.client.1.smithi149.stdout:2/480: symlink d1/d3/dd/d71/la2 0 2024-03-22T07:08:52.468 INFO:tasks.workunit.client.1.smithi149.stdout:2/481: chown d1/d4d/d69/d75/fa1 1884 1 2024-03-22T07:08:52.470 INFO:tasks.workunit.client.1.smithi149.stdout:2/482: symlink d1/d3/d5/la3 0 2024-03-22T07:08:52.470 INFO:tasks.workunit.client.1.smithi149.stdout:5/357: write d7/db/f22 [3122642,44610] 0 2024-03-22T07:08:52.472 INFO:tasks.workunit.client.1.smithi149.stdout:0/319: dread d1/d9/f57 [0,4194304] 0 2024-03-22T07:08:52.472 INFO:tasks.workunit.client.1.smithi149.stdout:5/358: creat d7/db/d2b/f73 x:0 0 0 2024-03-22T07:08:52.473 INFO:tasks.workunit.client.1.smithi149.stdout:2/483: rmdir d1/d3/d6 39 2024-03-22T07:08:52.474 INFO:tasks.workunit.client.1.smithi149.stdout:2/484: mknod d1/d3/dd/ca4 0 2024-03-22T07:08:52.474 INFO:tasks.workunit.client.1.smithi149.stdout:2/485: stat d1/d3/d5/d41/f48 0 2024-03-22T07:08:52.475 INFO:tasks.workunit.client.1.smithi149.stdout:2/486: mkdir d1/d3/d5/d41/da5 0 2024-03-22T07:08:52.476 INFO:tasks.workunit.client.1.smithi149.stdout:4/425: sync 2024-03-22T07:08:52.476 INFO:tasks.workunit.client.1.smithi149.stdout:4/426: fdatasync d0/d8/f56 0 2024-03-22T07:08:52.476 INFO:tasks.workunit.client.1.smithi149.stdout:4/427: creat d0/d6b/f94 x:0 0 0 2024-03-22T07:08:52.477 INFO:tasks.workunit.client.1.smithi149.stdout:2/487: link d1/d3/d5/la3 d1/d3/d6/d3f/d74/la6 0 2024-03-22T07:08:52.479 INFO:tasks.workunit.client.1.smithi149.stdout:4/428: creat d0/d8/dc/d74/f95 x:0 0 0 2024-03-22T07:08:52.480 INFO:tasks.workunit.client.1.smithi149.stdout:0/320: dread d1/d9/f57 [0,4194304] 0 2024-03-22T07:08:52.481 INFO:tasks.workunit.client.1.smithi149.stdout:2/488: rename d1/d3/dd/d71/d81 to d1/d3/d5/d41/d9f/da7 0 2024-03-22T07:08:52.483 INFO:tasks.workunit.client.1.smithi149.stdout:4/429: mkdir d0/d8/dc/d2e/d6f/d96 0 2024-03-22T07:08:52.483 INFO:tasks.workunit.client.1.smithi149.stdout:4/430: read d0/d8/f56 [1541287,11696] 0 2024-03-22T07:08:52.483 INFO:tasks.workunit.client.1.smithi149.stdout:4/431: fdatasync d0/d8/d5c/f5e 0 2024-03-22T07:08:52.484 INFO:tasks.workunit.client.1.smithi149.stdout:0/321: unlink d1/c6 0 2024-03-22T07:08:52.485 INFO:tasks.workunit.client.1.smithi149.stdout:2/489: link d1/d3/dd/f4c d1/d4d/d69/d75/d89/fa8 0 2024-03-22T07:08:52.485 INFO:tasks.workunit.client.1.smithi149.stdout:5/359: dwrite d7/da/d20/d30/d3a/d3d/d2c/f4d [0,4194304] 0 2024-03-22T07:08:52.486 INFO:tasks.workunit.client.1.smithi149.stdout:4/432: rename d0/d8/d1d/d68 to d0/d8/dc/d14/d85/d97 0 2024-03-22T07:08:52.486 INFO:tasks.workunit.client.1.smithi149.stdout:4/433: write d0/d8/d4c/f57 [759262,34213] 0 2024-03-22T07:08:52.487 INFO:tasks.workunit.client.1.smithi149.stdout:4/434: readlink d0/d8/dc/d4b/l59 0 2024-03-22T07:08:52.490 INFO:tasks.workunit.client.1.smithi149.stdout:5/360: mknod d7/da/d20/d36/c74 0 2024-03-22T07:08:52.495 INFO:tasks.workunit.client.1.smithi149.stdout:5/361: mknod d7/da/d20/d30/d3a/d3d/c75 0 2024-03-22T07:08:52.496 INFO:tasks.workunit.client.1.smithi149.stdout:5/362: read - d7/da/d20/d45/f4f zero size 2024-03-22T07:08:52.496 INFO:tasks.workunit.client.1.smithi149.stdout:2/490: write d1/d3/dd/f21 [312485,13097] 0 2024-03-22T07:08:52.497 INFO:tasks.workunit.client.1.smithi149.stdout:4/435: read d0/d8/f1f [1049242,68289] 0 2024-03-22T07:08:52.497 INFO:tasks.workunit.client.1.smithi149.stdout:5/363: mknod d7/da/c76 0 2024-03-22T07:08:52.497 INFO:tasks.workunit.client.1.smithi149.stdout:5/364: fdatasync d7/da/d20/d36/d44/f6f 0 2024-03-22T07:08:52.499 INFO:tasks.workunit.client.1.smithi149.stdout:2/491: symlink d1/la9 0 2024-03-22T07:08:52.499 INFO:tasks.workunit.client.1.smithi149.stdout:5/365: write d7/da/f1b [163593,19814] 0 2024-03-22T07:08:52.502 INFO:tasks.workunit.client.1.smithi149.stdout:0/322: rmdir d1 39 2024-03-22T07:08:52.504 INFO:tasks.workunit.client.1.smithi149.stdout:2/492: unlink d1/d3/d5/d41/f48 0 2024-03-22T07:08:52.505 INFO:tasks.workunit.client.1.smithi149.stdout:2/493: chown d1/d4d/d69/d75/d89/fa8 41552564 1 2024-03-22T07:08:52.505 INFO:tasks.workunit.client.1.smithi149.stdout:5/366: dread d7/da/d20/f5e [0,4194304] 0 2024-03-22T07:08:52.514 INFO:tasks.workunit.client.1.smithi149.stdout:4/436: dwrite d0/f32 [0,4194304] 0 2024-03-22T07:08:52.524 INFO:tasks.workunit.client.1.smithi149.stdout:0/323: rename d1/d9/f13 to d1/d9/d14/f76 0 2024-03-22T07:08:52.525 INFO:tasks.workunit.client.1.smithi149.stdout:5/367: creat d7/da/d20/d36/d44/f77 x:0 0 0 2024-03-22T07:08:52.531 INFO:tasks.workunit.client.1.smithi149.stdout:2/494: dwrite d1/d4d/d90/f99 [0,4194304] 0 2024-03-22T07:08:52.531 INFO:tasks.workunit.client.1.smithi149.stdout:2/495: write d1/f60 [84206,39317] 0 2024-03-22T07:08:52.533 INFO:tasks.workunit.client.1.smithi149.stdout:4/437: dwrite d0/d8/dc/d14/d3e/f3d [0,4194304] 0 2024-03-22T07:08:52.549 INFO:tasks.workunit.client.1.smithi149.stdout:0/324: mkdir d1/d9/d14/d74/d77 0 2024-03-22T07:08:52.556 INFO:tasks.workunit.client.1.smithi149.stdout:2/496: mknod d1/d72/caa 0 2024-03-22T07:08:52.559 INFO:tasks.workunit.client.1.smithi149.stdout:4/438: truncate d0/d8/dc/fe 1154477 0 2024-03-22T07:08:52.560 INFO:tasks.workunit.client.1.smithi149.stdout:0/325: creat d1/d6b/f78 x:0 0 0 2024-03-22T07:08:52.562 INFO:tasks.workunit.client.1.smithi149.stdout:4/439: getdents d0/d8 0 2024-03-22T07:08:52.562 INFO:tasks.workunit.client.1.smithi149.stdout:4/440: chown d0/d8/dc/d74/d90 6767605 1 2024-03-22T07:08:52.563 INFO:tasks.workunit.client.1.smithi149.stdout:2/497: dread d1/d3/dd/f40 [0,4194304] 0 2024-03-22T07:08:52.563 INFO:tasks.workunit.client.1.smithi149.stdout:0/326: mknod d1/d6b/c79 0 2024-03-22T07:08:52.565 INFO:tasks.workunit.client.1.smithi149.stdout:2/498: rename d1/d3/dd/f4c to d1/d3/d5/d1f/d5e/d2f/fab 0 2024-03-22T07:08:52.565 INFO:tasks.workunit.client.1.smithi149.stdout:2/499: creat d1/d3/d5/d1f/d68/d94/fac x:0 0 0 2024-03-22T07:08:52.565 INFO:tasks.workunit.client.1.smithi149.stdout:2/500: truncate d1/d3/d6/f8f 662980 0 2024-03-22T07:08:52.565 INFO:tasks.workunit.client.1.smithi149.stdout:2/501: chown d1/la9 69840826 1 2024-03-22T07:08:52.568 INFO:tasks.workunit.client.1.smithi149.stdout:5/368: mknod d7/c78 0 2024-03-22T07:08:52.568 INFO:tasks.workunit.client.1.smithi149.stdout:2/502: dread d1/d3/d6/f8f [0,4194304] 0 2024-03-22T07:08:52.569 INFO:tasks.workunit.client.1.smithi149.stdout:5/369: creat d7/da/d20/d30/d3a/f79 x:0 0 0 2024-03-22T07:08:52.572 INFO:tasks.workunit.client.1.smithi149.stdout:5/370: getdents d7/db/d2b 0 2024-03-22T07:08:52.574 INFO:tasks.workunit.client.1.smithi149.stdout:5/371: symlink d7/d35/l7a 0 2024-03-22T07:08:52.582 INFO:tasks.workunit.client.1.smithi149.stdout:2/503: read d1/d3/dd/f65 [1808348,115143] 0 2024-03-22T07:08:52.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/441: dwrite d0/d8/f34 [0,4194304] 0 2024-03-22T07:08:52.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/442: truncate d0/d8/dc/f7c 857586 0 2024-03-22T07:08:52.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/443: chown d0/d8/d5c/f5e 1047536591 1 2024-03-22T07:08:52.590 INFO:tasks.workunit.client.1.smithi149.stdout:0/327: dwrite d1/d9/f3a [0,4194304] 0 2024-03-22T07:08:52.590 INFO:tasks.workunit.client.1.smithi149.stdout:0/328: creat d1/d6b/f7a x:0 0 0 2024-03-22T07:08:52.591 INFO:tasks.workunit.client.1.smithi149.stdout:7/465: sync 2024-03-22T07:08:52.591 INFO:tasks.workunit.client.1.smithi149.stdout:9/464: sync 2024-03-22T07:08:52.591 INFO:tasks.workunit.client.1.smithi149.stdout:1/382: sync 2024-03-22T07:08:52.591 INFO:tasks.workunit.client.1.smithi149.stdout:7/466: write d1/d18/d1b/d1d/f8 [992939,68221] 0 2024-03-22T07:08:52.591 INFO:tasks.workunit.client.1.smithi149.stdout:7/467: write d1/f5 [1267886,18324] 0 2024-03-22T07:08:52.592 INFO:tasks.workunit.client.1.smithi149.stdout:8/418: sync 2024-03-22T07:08:52.592 INFO:tasks.workunit.client.1.smithi149.stdout:8/419: write d1/d2/d9/f65 [143192,61450] 0 2024-03-22T07:08:52.592 INFO:tasks.workunit.client.1.smithi149.stdout:8/420: chown d1/f24 0 1 2024-03-22T07:08:52.592 INFO:tasks.workunit.client.1.smithi149.stdout:8/421: chown d1/d20/d51/l59 298618 1 2024-03-22T07:08:52.593 INFO:tasks.workunit.client.1.smithi149.stdout:8/422: fsync d1/d2/d18/f78 0 2024-03-22T07:08:52.593 INFO:tasks.workunit.client.1.smithi149.stdout:6/451: sync 2024-03-22T07:08:52.593 INFO:tasks.workunit.client.1.smithi149.stdout:3/348: sync 2024-03-22T07:08:52.593 INFO:tasks.workunit.client.1.smithi149.stdout:3/349: dread - d0/f38 zero size 2024-03-22T07:08:52.593 INFO:tasks.workunit.client.1.smithi149.stdout:0/329: symlink d1/d9/d14/d1f/l7b 0 2024-03-22T07:08:52.594 INFO:tasks.workunit.client.1.smithi149.stdout:8/423: dread d1/d2/d18/f78 [0,4194304] 0 2024-03-22T07:08:52.594 INFO:tasks.workunit.client.1.smithi149.stdout:1/383: mkdir d4/d20/d23/d3c/d85 0 2024-03-22T07:08:52.594 INFO:tasks.workunit.client.1.smithi149.stdout:1/384: chown d4/c54 1 1 2024-03-22T07:08:52.594 INFO:tasks.workunit.client.1.smithi149.stdout:1/385: chown d4/d10/d7c/d6a/c78 60506785 1 2024-03-22T07:08:52.596 INFO:tasks.workunit.client.1.smithi149.stdout:7/468: link d1/d1c/d49/d8e/l39 d1/d1c/d4d/l95 0 2024-03-22T07:08:52.596 INFO:tasks.workunit.client.1.smithi149.stdout:7/469: truncate d1/d1c/d2f/d74/f8a 666843 0 2024-03-22T07:08:52.596 INFO:tasks.workunit.client.1.smithi149.stdout:7/470: read - d1/d1c/d4d/f83 zero size 2024-03-22T07:08:52.596 INFO:tasks.workunit.client.1.smithi149.stdout:7/471: fsync d1/d1c/d4d/f75 0 2024-03-22T07:08:52.598 INFO:tasks.workunit.client.1.smithi149.stdout:6/452: rename d9/d1f/d24/d34/d37/d38/c57 to d9/c94 0 2024-03-22T07:08:52.599 INFO:tasks.workunit.client.1.smithi149.stdout:3/350: rmdir d0/d2/d30/d7e 39 2024-03-22T07:08:52.599 INFO:tasks.workunit.client.1.smithi149.stdout:6/453: dread - d9/d1f/d24/d68/d73/f8b zero size 2024-03-22T07:08:52.599 INFO:tasks.workunit.client.1.smithi149.stdout:6/454: write d9/df/f91 [98293,98209] 0 2024-03-22T07:08:52.599 INFO:tasks.workunit.client.1.smithi149.stdout:6/455: write d9/d1f/d24/d68/d73/f8b [729332,21877] 0 2024-03-22T07:08:52.600 INFO:tasks.workunit.client.1.smithi149.stdout:1/386: rename d4/d20/d23/d3c/f41 to d4/d20/d2d/d64/f86 0 2024-03-22T07:08:52.600 INFO:tasks.workunit.client.1.smithi149.stdout:0/330: readlink d1/d1a/l1d 0 2024-03-22T07:08:52.600 INFO:tasks.workunit.client.1.smithi149.stdout:0/331: truncate d1/d20/f3d 603639 0 2024-03-22T07:08:52.602 INFO:tasks.workunit.client.1.smithi149.stdout:2/504: dread d1/d3/d5/d1f/d68/f79 [0,4194304] 0 2024-03-22T07:08:52.602 INFO:tasks.workunit.client.1.smithi149.stdout:2/505: fsync d1/d3/d5/d1f/f55 0 2024-03-22T07:08:52.607 INFO:tasks.workunit.client.1.smithi149.stdout:1/387: link d4/d10/d7c/f4f d4/d7/d5a/f87 0 2024-03-22T07:08:52.607 INFO:tasks.workunit.client.1.smithi149.stdout:1/388: truncate d4/d7/f57 4733276 0 2024-03-22T07:08:52.607 INFO:tasks.workunit.client.1.smithi149.stdout:0/332: mknod d1/d9/d59/c7c 0 2024-03-22T07:08:52.608 INFO:tasks.workunit.client.1.smithi149.stdout:2/506: creat d1/d3/d9a/fad x:0 0 0 2024-03-22T07:08:52.609 INFO:tasks.workunit.client.1.smithi149.stdout:9/465: dread db/d42/f4c [0,4194304] 0 2024-03-22T07:08:52.609 INFO:tasks.workunit.client.1.smithi149.stdout:1/389: mkdir d4/d10/d28/d88 0 2024-03-22T07:08:52.609 INFO:tasks.workunit.client.1.smithi149.stdout:0/333: rename d1/d1a/l1d to d1/d20/l7d 0 2024-03-22T07:08:52.610 INFO:tasks.workunit.client.1.smithi149.stdout:9/466: truncate db/d22/f45 3982155 0 2024-03-22T07:08:52.610 INFO:tasks.workunit.client.1.smithi149.stdout:0/334: rmdir d1/d9/d59 39 2024-03-22T07:08:52.612 INFO:tasks.workunit.client.1.smithi149.stdout:2/507: rename d1/d3/d5/d1f/d5e/l8d to d1/d3/d6/d3f/lae 0 2024-03-22T07:08:52.615 INFO:tasks.workunit.client.1.smithi149.stdout:2/508: creat d1/d4d/d69/d75/d89/faf x:0 0 0 2024-03-22T07:08:52.615 INFO:tasks.workunit.client.1.smithi149.stdout:2/509: fdatasync d1/d3/d5/d1f/f93 0 2024-03-22T07:08:52.617 INFO:tasks.workunit.client.1.smithi149.stdout:2/510: creat d1/d8/fb0 x:0 0 0 2024-03-22T07:08:52.617 INFO:tasks.workunit.client.1.smithi149.stdout:2/511: write d1/f60 [518958,28248] 0 2024-03-22T07:08:52.617 INFO:tasks.workunit.client.1.smithi149.stdout:2/512: stat d1/d3/f3e 0 2024-03-22T07:08:52.619 INFO:tasks.workunit.client.1.smithi149.stdout:2/513: mknod d1/d4d/d8b/cb1 0 2024-03-22T07:08:52.622 INFO:tasks.workunit.client.1.smithi149.stdout:2/514: rename d1/d3/dd to d1/d3/d6/db2 0 2024-03-22T07:08:52.630 INFO:tasks.workunit.client.1.smithi149.stdout:5/372: dwrite d7/da/d20/d30/d3a/d3d/d2c/d63/f72 [0,4194304] 0 2024-03-22T07:08:52.630 INFO:tasks.workunit.client.1.smithi149.stdout:5/373: dread - d7/da/d20/d45/f4f zero size 2024-03-22T07:08:52.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/515: read d1/d3/f29 [586440,51756] 0 2024-03-22T07:08:52.635 INFO:tasks.workunit.client.1.smithi149.stdout:2/516: symlink d1/d3/d6/db2/d71/lb3 0 2024-03-22T07:08:52.638 INFO:tasks.workunit.client.1.smithi149.stdout:2/517: unlink d1/d4d/d69/d75/c88 0 2024-03-22T07:08:52.638 INFO:tasks.workunit.client.1.smithi149.stdout:2/518: fsync d1/d3/d5/d1f/d5e/d2f/fab 0 2024-03-22T07:08:52.639 INFO:tasks.workunit.client.1.smithi149.stdout:3/351: dwrite d0/d2/d36/f2e [0,4194304] 0 2024-03-22T07:08:52.640 INFO:tasks.workunit.client.1.smithi149.stdout:5/374: rename d7/db to d7/da/d20/d30/d3a/d3d/d2c/d47/d7b 0 2024-03-22T07:08:52.640 INFO:tasks.workunit.client.1.smithi149.stdout:5/375: fdatasync d7/da/f37 0 2024-03-22T07:08:52.641 INFO:tasks.workunit.client.1.smithi149.stdout:3/352: mkdir d0/d2/d42/d66/d82 0 2024-03-22T07:08:52.641 INFO:tasks.workunit.client.1.smithi149.stdout:3/353: creat d0/d2/f83 x:0 0 0 2024-03-22T07:08:52.642 INFO:tasks.workunit.client.1.smithi149.stdout:2/519: rename d1/d3/f54 to d1/d3/d5/d1f/d68/fb4 0 2024-03-22T07:08:52.643 INFO:tasks.workunit.client.1.smithi149.stdout:2/520: chown d1/d3/l6c 508657 1 2024-03-22T07:08:52.648 INFO:tasks.workunit.client.1.smithi149.stdout:7/472: dwrite d1/d18/d1b/f2c [0,4194304] 0 2024-03-22T07:08:52.651 INFO:tasks.workunit.client.1.smithi149.stdout:8/424: dwrite d1/d20/d35/fb [0,4194304] 0 2024-03-22T07:08:52.652 INFO:tasks.workunit.client.1.smithi149.stdout:8/425: write d1/d2/d18/f3c [156273,111730] 0 2024-03-22T07:08:52.653 INFO:tasks.workunit.client.1.smithi149.stdout:0/335: dwrite d1/d9/d14/f70 [0,4194304] 0 2024-03-22T07:08:52.653 INFO:tasks.workunit.client.1.smithi149.stdout:9/467: dwrite db/d33/f67 [0,4194304] 0 2024-03-22T07:08:52.653 INFO:tasks.workunit.client.1.smithi149.stdout:9/468: chown db/d21/d37/d7c 63506356 1 2024-03-22T07:08:52.655 INFO:tasks.workunit.client.1.smithi149.stdout:8/426: rmdir d1/d20/d35/d22/d44/d4f 39 2024-03-22T07:08:52.656 INFO:tasks.workunit.client.1.smithi149.stdout:2/521: write d1/f42 [3758842,60245] 0 2024-03-22T07:08:52.658 INFO:tasks.workunit.client.1.smithi149.stdout:6/456: dwrite d9/d1f/d24/d34/d37/f7a [0,4194304] 0 2024-03-22T07:08:52.659 INFO:tasks.workunit.client.1.smithi149.stdout:0/336: mkdir d1/d31/d51/d64/d7e 0 2024-03-22T07:08:52.661 INFO:tasks.workunit.client.1.smithi149.stdout:5/376: dwrite d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/fc [0,4194304] 0 2024-03-22T07:08:52.664 INFO:tasks.workunit.client.1.smithi149.stdout:8/427: unlink d1/d2/d18/l85 0 2024-03-22T07:08:52.665 INFO:tasks.workunit.client.1.smithi149.stdout:2/522: creat d1/d8/fb5 x:0 0 0 2024-03-22T07:08:52.667 INFO:tasks.workunit.client.1.smithi149.stdout:6/457: rmdir d9/d1f/d24/d42 39 2024-03-22T07:08:52.668 INFO:tasks.workunit.client.1.smithi149.stdout:0/337: mknod d1/d9/d14/d74/d77/c7f 0 2024-03-22T07:08:52.669 INFO:tasks.workunit.client.1.smithi149.stdout:0/338: dread d1/d20/f3d [0,4194304] 0 2024-03-22T07:08:52.669 INFO:tasks.workunit.client.1.smithi149.stdout:5/377: creat d7/da/d20/d45/f7c x:0 0 0 2024-03-22T07:08:52.670 INFO:tasks.workunit.client.1.smithi149.stdout:9/469: dwrite db/d19/f7d [0,4194304] 0 2024-03-22T07:08:52.670 INFO:tasks.workunit.client.1.smithi149.stdout:9/470: truncate db/d21/d37/d3a/f6c 3856 0 2024-03-22T07:08:52.670 INFO:tasks.workunit.client.1.smithi149.stdout:8/428: stat d1/d2/d3/c60 0 2024-03-22T07:08:52.672 INFO:tasks.workunit.client.1.smithi149.stdout:2/523: mkdir d1/d4d/d69/d75/db6 0 2024-03-22T07:08:52.673 INFO:tasks.workunit.client.1.smithi149.stdout:6/458: rename d9/d1f/d24/d68/f8c to d9/df/d22/f95 0 2024-03-22T07:08:52.673 INFO:tasks.workunit.client.1.smithi149.stdout:6/459: write d9/df/d22/f87 [755406,76377] 0 2024-03-22T07:08:52.673 INFO:tasks.workunit.client.1.smithi149.stdout:6/460: stat d9/df/f6c 0 2024-03-22T07:08:52.673 INFO:tasks.workunit.client.1.smithi149.stdout:6/461: write d9/d1f/d24/d34/f7e [1848146,120458] 0 2024-03-22T07:08:52.674 INFO:tasks.workunit.client.1.smithi149.stdout:6/462: readlink d9/d1f/d24/d42/l59 0 2024-03-22T07:08:52.675 INFO:tasks.workunit.client.1.smithi149.stdout:0/339: unlink d1/d6b/f7a 0 2024-03-22T07:08:52.675 INFO:tasks.workunit.client.1.smithi149.stdout:5/378: getdents d7/da/d20/d36/d44/d4a 0 2024-03-22T07:08:52.679 INFO:tasks.workunit.client.1.smithi149.stdout:9/471: creat db/d21/d37/d7c/f9b x:0 0 0 2024-03-22T07:08:52.679 INFO:tasks.workunit.client.1.smithi149.stdout:8/429: creat d1/d20/d35/d22/d44/d84/f8c x:0 0 0 2024-03-22T07:08:52.681 INFO:tasks.workunit.client.1.smithi149.stdout:2/524: getdents d1/d3/d6/d3f 0 2024-03-22T07:08:52.683 INFO:tasks.workunit.client.1.smithi149.stdout:0/340: symlink d1/d27/l80 0 2024-03-22T07:08:52.683 INFO:tasks.workunit.client.1.smithi149.stdout:0/341: write d1/d9/f71 [477878,45525] 0 2024-03-22T07:08:52.684 INFO:tasks.workunit.client.1.smithi149.stdout:5/379: rmdir d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d2b 39 2024-03-22T07:08:52.685 INFO:tasks.workunit.client.1.smithi149.stdout:9/472: truncate f3 3075122 0 2024-03-22T07:08:52.685 INFO:tasks.workunit.client.1.smithi149.stdout:9/473: chown db/d21/d74 468262455 1 2024-03-22T07:08:52.686 INFO:tasks.workunit.client.1.smithi149.stdout:8/430: truncate d1/d20/d35/f6f 345138 0 2024-03-22T07:08:52.688 INFO:tasks.workunit.client.1.smithi149.stdout:0/342: truncate d1/f39 1154924 0 2024-03-22T07:08:52.688 INFO:tasks.workunit.client.1.smithi149.stdout:0/343: truncate d1/d31/d51/f5b 946590 0 2024-03-22T07:08:52.691 INFO:tasks.workunit.client.1.smithi149.stdout:5/380: rmdir d7/da/d20/d30/d3a 39 2024-03-22T07:08:52.691 INFO:tasks.workunit.client.1.smithi149.stdout:5/381: dread - d7/da/d20/d45/f4f zero size 2024-03-22T07:08:52.692 INFO:tasks.workunit.client.1.smithi149.stdout:5/382: stat d7/f8 0 2024-03-22T07:08:52.693 INFO:tasks.workunit.client.1.smithi149.stdout:6/463: dwrite d9/d1f/f41 [0,4194304] 0 2024-03-22T07:08:52.694 INFO:tasks.workunit.client.1.smithi149.stdout:6/464: stat d9/df/d2e/f1b 0 2024-03-22T07:08:52.701 INFO:tasks.workunit.client.1.smithi149.stdout:9/474: write db/d21/d74/f76 [1987983,18986] 0 2024-03-22T07:08:52.701 INFO:tasks.workunit.client.1.smithi149.stdout:9/475: truncate db/d19/d39/d3b/f6d 423197 0 2024-03-22T07:08:52.702 INFO:tasks.workunit.client.1.smithi149.stdout:9/476: fdatasync f3 0 2024-03-22T07:08:52.702 INFO:tasks.workunit.client.1.smithi149.stdout:0/344: write d1/d9/d14/f18 [1365191,109940] 0 2024-03-22T07:08:52.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/465: rename d9/d1f to d9/df/d22/d6d/d7d/d96 0 2024-03-22T07:08:52.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/466: write d9/df/f91 [1234647,120243] 0 2024-03-22T07:08:52.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/467: write d9/df/d22/d6d/d7d/d96/d24/d34/d4f/f80 [189506,40307] 0 2024-03-22T07:08:52.709 INFO:tasks.workunit.client.1.smithi149.stdout:9/477: rename db/l4a to db/d21/d37/d3a/l9c 0 2024-03-22T07:08:52.710 INFO:tasks.workunit.client.1.smithi149.stdout:6/468: creat d9/df/d22/d6d/d7d/d96/d24/d34/d4f/d86/f97 x:0 0 0 2024-03-22T07:08:52.711 INFO:tasks.workunit.client.1.smithi149.stdout:0/345: rename d1/d9/d14/f70 to d1/d9/d14/d74/f81 0 2024-03-22T07:08:52.714 INFO:tasks.workunit.client.1.smithi149.stdout:5/383: dwrite d7/da/d20/d36/d44/f5d [0,4194304] 0 2024-03-22T07:08:52.714 INFO:tasks.workunit.client.1.smithi149.stdout:5/384: fsync d7/da/d20/d36/d44/f6f 0 2024-03-22T07:08:52.716 INFO:tasks.workunit.client.1.smithi149.stdout:0/346: mknod d1/d31/d52/c82 0 2024-03-22T07:08:52.716 INFO:tasks.workunit.client.1.smithi149.stdout:0/347: fsync d1/f4a 0 2024-03-22T07:08:52.717 INFO:tasks.workunit.client.1.smithi149.stdout:8/431: dwrite d1/d2/d9/d11/f55 [0,4194304] 0 2024-03-22T07:08:52.719 INFO:tasks.workunit.client.1.smithi149.stdout:0/348: unlink d1/l6d 0 2024-03-22T07:08:52.721 INFO:tasks.workunit.client.1.smithi149.stdout:9/478: truncate db/d21/d74/f76 2945893 0 2024-03-22T07:08:52.721 INFO:tasks.workunit.client.1.smithi149.stdout:9/479: write db/d19/f94 [680878,75892] 0 2024-03-22T07:08:52.723 INFO:tasks.workunit.client.1.smithi149.stdout:9/480: write db/d12/f17 [2143211,82812] 0 2024-03-22T07:08:52.723 INFO:tasks.workunit.client.1.smithi149.stdout:9/481: read - db/d21/d8c/f96 zero size 2024-03-22T07:08:52.725 INFO:tasks.workunit.client.1.smithi149.stdout:6/469: rename d9/df/d22/d6d/d7d/d96/d24/d34/d4f/d86 to d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98 0 2024-03-22T07:08:52.726 INFO:tasks.workunit.client.1.smithi149.stdout:9/482: truncate db/d42/f5a 2366530 0 2024-03-22T07:08:52.726 INFO:tasks.workunit.client.1.smithi149.stdout:9/483: dread - db/d19/d28/f84 zero size 2024-03-22T07:08:52.729 INFO:tasks.workunit.client.1.smithi149.stdout:9/484: mknod db/d42/d49/c9d 0 2024-03-22T07:08:52.731 INFO:tasks.workunit.client.1.smithi149.stdout:8/432: dread d1/d20/d35/d22/d44/d4f/f56 [0,4194304] 0 2024-03-22T07:08:52.731 INFO:tasks.workunit.client.1.smithi149.stdout:8/433: chown d1/d20/c61 1046911 1 2024-03-22T07:08:52.732 INFO:tasks.workunit.client.1.smithi149.stdout:8/434: getdents d1/d20 0 2024-03-22T07:08:52.733 INFO:tasks.workunit.client.1.smithi149.stdout:8/435: mkdir d1/d2/d9/d11/d8b/d8d 0 2024-03-22T07:08:52.734 INFO:tasks.workunit.client.1.smithi149.stdout:8/436: truncate d1/d2/d9/f37 1024707 0 2024-03-22T07:08:52.735 INFO:tasks.workunit.client.1.smithi149.stdout:8/437: mknod d1/d20/d35/d22/d44/d4f/c8e 0 2024-03-22T07:08:52.735 INFO:tasks.workunit.client.1.smithi149.stdout:8/438: chown d1/d2/d3/c39 321617 1 2024-03-22T07:08:52.735 INFO:tasks.workunit.client.1.smithi149.stdout:8/439: readlink d1/d2/d3/l75 0 2024-03-22T07:08:52.735 INFO:tasks.workunit.client.1.smithi149.stdout:8/440: chown d1/d2/d9/df/f15 478161179 1 2024-03-22T07:08:52.738 INFO:tasks.workunit.client.1.smithi149.stdout:5/385: dwrite d7/da/d20/d30/d3a/d3d/d2c/f4d [0,4194304] 0 2024-03-22T07:08:52.738 INFO:tasks.workunit.client.1.smithi149.stdout:5/386: chown d7/da/c70 30138 1 2024-03-22T07:08:52.739 INFO:tasks.workunit.client.1.smithi149.stdout:8/441: dread d1/d2/d3/f86 [0,4194304] 0 2024-03-22T07:08:52.739 INFO:tasks.workunit.client.1.smithi149.stdout:5/387: truncate d7/da/d20/d30/d3a/d3d/d2c/d47/f6e 5997314 0 2024-03-22T07:08:52.741 INFO:tasks.workunit.client.1.smithi149.stdout:5/388: mkdir d7/da/d7d 0 2024-03-22T07:08:52.742 INFO:tasks.workunit.client.1.smithi149.stdout:0/349: dwrite d1/d9/d14/f76 [0,4194304] 0 2024-03-22T07:08:52.742 INFO:tasks.workunit.client.1.smithi149.stdout:0/350: readlink d1/d9/d14/d1f/l60 0 2024-03-22T07:08:52.743 INFO:tasks.workunit.client.1.smithi149.stdout:5/389: truncate d7/da/d20/f4c 1511673 0 2024-03-22T07:08:52.743 INFO:tasks.workunit.client.1.smithi149.stdout:5/390: fsync d7/da/d20/d45/f4f 0 2024-03-22T07:08:52.743 INFO:tasks.workunit.client.1.smithi149.stdout:5/391: fdatasync d7/da/d20/d45/f51 0 2024-03-22T07:08:52.743 INFO:tasks.workunit.client.1.smithi149.stdout:5/392: read d7/f9 [4610951,128703] 0 2024-03-22T07:08:52.743 INFO:tasks.workunit.client.1.smithi149.stdout:5/393: write d7/da/d20/f5e [1051719,91376] 0 2024-03-22T07:08:52.745 INFO:tasks.workunit.client.1.smithi149.stdout:6/470: dread d9/df/d22/d6d/d7d/d96/f2a [0,4194304] 0 2024-03-22T07:08:52.745 INFO:tasks.workunit.client.1.smithi149.stdout:0/351: symlink d1/d9/d14/d74/d77/l83 0 2024-03-22T07:08:52.747 INFO:tasks.workunit.client.1.smithi149.stdout:5/394: creat d7/da/d20/f7e x:0 0 0 2024-03-22T07:08:52.747 INFO:tasks.workunit.client.1.smithi149.stdout:5/395: truncate d7/da/d20/f49 1835641 0 2024-03-22T07:08:52.749 INFO:tasks.workunit.client.1.smithi149.stdout:6/471: unlink d9/df/d22/d6d/d7d/d96/d24/d68/d73/f7f 0 2024-03-22T07:08:52.750 INFO:tasks.workunit.client.1.smithi149.stdout:0/352: rmdir d1/d1a 39 2024-03-22T07:08:52.750 INFO:tasks.workunit.client.1.smithi149.stdout:0/353: chown d1/d31/d52/f5a 982461678 1 2024-03-22T07:08:52.750 INFO:tasks.workunit.client.1.smithi149.stdout:0/354: fdatasync d1/d9/d59/f6c 0 2024-03-22T07:08:52.750 INFO:tasks.workunit.client.1.smithi149.stdout:0/355: readlink d1/d9/d14/d1f/l7b 0 2024-03-22T07:08:52.751 INFO:tasks.workunit.client.1.smithi149.stdout:5/396: creat d7/da/d20/d30/d3a/d3d/d2c/d63/f7f x:0 0 0 2024-03-22T07:08:52.753 INFO:tasks.workunit.client.1.smithi149.stdout:5/397: mknod d7/da/d20/d30/d3a/c80 0 2024-03-22T07:08:52.756 INFO:tasks.workunit.client.1.smithi149.stdout:5/398: rename d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d6c to d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d81 0 2024-03-22T07:08:52.758 INFO:tasks.workunit.client.1.smithi149.stdout:5/399: link d7/c78 d7/da/d20/d36/d44/c82 0 2024-03-22T07:08:52.758 INFO:tasks.workunit.client.1.smithi149.stdout:5/400: stat d7/d35/f55 0 2024-03-22T07:08:52.758 INFO:tasks.workunit.client.1.smithi149.stdout:5/401: chown d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/c2d 617604 1 2024-03-22T07:08:52.763 INFO:tasks.workunit.client.1.smithi149.stdout:5/402: mknod d7/da/d20/d30/d3a/d3d/c83 0 2024-03-22T07:08:52.765 INFO:tasks.workunit.client.1.smithi149.stdout:5/403: rmdir d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d2b 39 2024-03-22T07:08:52.767 INFO:tasks.workunit.client.1.smithi149.stdout:5/404: mkdir d7/da/d20/d36/d84 0 2024-03-22T07:08:52.768 INFO:tasks.workunit.client.1.smithi149.stdout:5/405: symlink d7/da/d20/l85 0 2024-03-22T07:08:52.768 INFO:tasks.workunit.client.1.smithi149.stdout:5/406: truncate d7/da/d20/d30/d3a/d3d/d2c/d47/f52 1010840 0 2024-03-22T07:08:52.769 INFO:tasks.workunit.client.1.smithi149.stdout:5/407: symlink d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d81/l86 0 2024-03-22T07:08:52.770 INFO:tasks.workunit.client.1.smithi149.stdout:5/408: link d7/da/c76 d7/da/d20/d30/d3a/d3d/d2c/d63/c87 0 2024-03-22T07:08:52.771 INFO:tasks.workunit.client.1.smithi149.stdout:5/409: creat d7/da/d20/d30/d3a/d3d/d2c/f88 x:0 0 0 2024-03-22T07:08:52.773 INFO:tasks.workunit.client.1.smithi149.stdout:5/410: mkdir d7/da/d20/d89 0 2024-03-22T07:08:52.773 INFO:tasks.workunit.client.1.smithi149.stdout:5/411: fsync d7/f33 0 2024-03-22T07:08:52.773 INFO:tasks.workunit.client.1.smithi149.stdout:5/412: chown d7/da/d20/d30/d3a/d3d/d2c/d63/f7f 79538713 1 2024-03-22T07:08:52.773 INFO:tasks.workunit.client.1.smithi149.stdout:5/413: mknod d7/da/d20/d30/d3a/d3d/d2c/d63/c8a 0 2024-03-22T07:08:52.774 INFO:tasks.workunit.client.1.smithi149.stdout:5/414: creat d7/da/d20/d30/d3a/d3d/d2c/f8b x:0 0 0 2024-03-22T07:08:52.777 INFO:tasks.workunit.client.1.smithi149.stdout:9/485: dwrite db/d21/d74/f76 [0,4194304] 0 2024-03-22T07:08:52.779 INFO:tasks.workunit.client.1.smithi149.stdout:9/486: mkdir db/d21/d9e 0 2024-03-22T07:08:52.779 INFO:tasks.workunit.client.1.smithi149.stdout:8/442: dwrite d1/d20/d35/d22/d44/d84/f8c [0,4194304] 0 2024-03-22T07:08:52.779 INFO:tasks.workunit.client.1.smithi149.stdout:8/443: write d1/d2/d9/d11/d45/f70 [708271,82755] 0 2024-03-22T07:08:52.781 INFO:tasks.workunit.client.1.smithi149.stdout:0/356: dwrite d1/d9/d14/d1f/f4d [0,4194304] 0 2024-03-22T07:08:52.782 INFO:tasks.workunit.client.1.smithi149.stdout:0/357: truncate d1/d31/d52/f5a 892863 0 2024-03-22T07:08:52.782 INFO:tasks.workunit.client.1.smithi149.stdout:6/472: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f43 [0,4194304] 0 2024-03-22T07:08:52.782 INFO:tasks.workunit.client.1.smithi149.stdout:6/473: chown d9/f19 685 1 2024-03-22T07:08:52.782 INFO:tasks.workunit.client.1.smithi149.stdout:6/474: write d9/df/d2e/f8d [796709,95703] 0 2024-03-22T07:08:52.782 INFO:tasks.workunit.client.1.smithi149.stdout:6/475: readlink d9/df/d22/d6d/d7d/d96/d24/d34/l6a 0 2024-03-22T07:08:52.782 INFO:tasks.workunit.client.1.smithi149.stdout:6/476: dread - d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/f97 zero size 2024-03-22T07:08:52.782 INFO:tasks.workunit.client.1.smithi149.stdout:6/477: stat d9/df/d22/d6d/d7d/d96/d24/d68 0 2024-03-22T07:08:52.782 INFO:tasks.workunit.client.1.smithi149.stdout:8/444: creat d1/d2/f8f x:0 0 0 2024-03-22T07:08:52.785 INFO:tasks.workunit.client.1.smithi149.stdout:0/358: mknod d1/d31/c84 0 2024-03-22T07:08:52.787 INFO:tasks.workunit.client.1.smithi149.stdout:6/478: creat d9/df/d22/d6d/d7d/d96/d24/f99 x:0 0 0 2024-03-22T07:08:52.791 INFO:tasks.workunit.client.1.smithi149.stdout:5/415: dwrite d7/da/d20/f49 [0,4194304] 0 2024-03-22T07:08:52.795 INFO:tasks.workunit.client.1.smithi149.stdout:5/416: mknod d7/da/d20/c8c 0 2024-03-22T07:08:52.795 INFO:tasks.workunit.client.1.smithi149.stdout:5/417: chown d7/da/d20/d30/d3a/d3d/d2c/d47/f6e 2622186 1 2024-03-22T07:08:52.795 INFO:tasks.workunit.client.1.smithi149.stdout:5/418: write d7/da/d20/d36/d44/f6f [1156864,23631] 0 2024-03-22T07:08:52.795 INFO:tasks.workunit.client.1.smithi149.stdout:5/419: chown d7/da/d20/d36/d44/l65 0 1 2024-03-22T07:08:52.797 INFO:tasks.workunit.client.1.smithi149.stdout:5/420: mknod d7/da/d20/d30/d3a/d3d/d2c/c8d 0 2024-03-22T07:08:52.798 INFO:tasks.workunit.client.1.smithi149.stdout:5/421: mkdir d7/d35/d8e 0 2024-03-22T07:08:52.800 INFO:tasks.workunit.client.1.smithi149.stdout:5/422: mknod d7/d35/c8f 0 2024-03-22T07:08:52.800 INFO:tasks.workunit.client.1.smithi149.stdout:6/479: dread d9/df/d2e/f20 [0,4194304] 0 2024-03-22T07:08:52.801 INFO:tasks.workunit.client.1.smithi149.stdout:5/423: creat d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d81/f90 x:0 0 0 2024-03-22T07:08:52.801 INFO:tasks.workunit.client.1.smithi149.stdout:5/424: chown d7/da/d20/d30/d3a/d3d/d2c/d63/f72 204 1 2024-03-22T07:08:52.801 INFO:tasks.workunit.client.1.smithi149.stdout:5/425: stat d7/da/d20/d30/d3a/d3d/d2c/d63 0 2024-03-22T07:08:52.802 INFO:tasks.workunit.client.1.smithi149.stdout:6/480: rmdir d9/df/d22/d6d/d7d 39 2024-03-22T07:08:52.804 INFO:tasks.workunit.client.1.smithi149.stdout:6/481: truncate d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f71 87886 0 2024-03-22T07:08:52.807 INFO:tasks.workunit.client.1.smithi149.stdout:6/482: dread d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 [0,4194304] 0 2024-03-22T07:08:52.807 INFO:tasks.workunit.client.1.smithi149.stdout:6/483: dread - d9/df/d22/d6d/d7d/d96/f90 zero size 2024-03-22T07:08:52.811 INFO:tasks.workunit.client.1.smithi149.stdout:6/484: rename d9/df/d22/d6d/d7d/d96/d24/d34/d37/f49 to d9/df/f9a 0 2024-03-22T07:08:52.811 INFO:tasks.workunit.client.1.smithi149.stdout:6/485: truncate d9/df/d22/d6d/f78 147178 0 2024-03-22T07:08:52.813 INFO:tasks.workunit.client.1.smithi149.stdout:6/486: unlink d9/df/d22/d6d/d7d/d96/f90 0 2024-03-22T07:08:52.823 INFO:tasks.workunit.client.1.smithi149.stdout:9/487: dwrite db/d21/d37/f64 [0,4194304] 0 2024-03-22T07:08:52.823 INFO:tasks.workunit.client.1.smithi149.stdout:9/488: dread - db/d21/d37/d7c/f9b zero size 2024-03-22T07:08:52.823 INFO:tasks.workunit.client.1.smithi149.stdout:9/489: chown db/d42/d49/d4b 57 1 2024-03-22T07:08:52.824 INFO:tasks.workunit.client.1.smithi149.stdout:5/426: dwrite d7/da/f37 [4194304,4194304] 0 2024-03-22T07:08:52.824 INFO:tasks.workunit.client.1.smithi149.stdout:0/359: dwrite d1/f4a [0,4194304] 0 2024-03-22T07:08:52.824 INFO:tasks.workunit.client.1.smithi149.stdout:8/445: dwrite d1/d2/d9/d11/f58 [0,4194304] 0 2024-03-22T07:08:52.824 INFO:tasks.workunit.client.1.smithi149.stdout:8/446: chown d1/d2/d9/df/l6b 3410427 1 2024-03-22T07:08:52.824 INFO:tasks.workunit.client.1.smithi149.stdout:8/447: truncate d1/d2/d9/df/f15 4504745 0 2024-03-22T07:08:52.824 INFO:tasks.workunit.client.1.smithi149.stdout:8/448: chown d1/d20/d51 951081181 1 2024-03-22T07:08:52.824 INFO:tasks.workunit.client.1.smithi149.stdout:8/449: readlink d1/d2/d9/d11/l80 0 2024-03-22T07:08:52.832 INFO:tasks.workunit.client.1.smithi149.stdout:5/427: link d7/da/d20/d45/f7c d7/da/d20/d36/f91 0 2024-03-22T07:08:52.835 INFO:tasks.workunit.client.1.smithi149.stdout:0/360: unlink d1/d9/d59/l72 0 2024-03-22T07:08:52.835 INFO:tasks.workunit.client.1.smithi149.stdout:0/361: fdatasync d1/f63 0 2024-03-22T07:08:52.839 INFO:tasks.workunit.client.1.smithi149.stdout:8/450: rename d1/d2/d9/d11/d4d/c79 to d1/d2/d9/d11/d8b/c90 0 2024-03-22T07:08:52.840 INFO:tasks.workunit.client.1.smithi149.stdout:9/490: dwrite db/d21/d37/d3a/f6c [0,4194304] 0 2024-03-22T07:08:52.851 INFO:tasks.workunit.client.1.smithi149.stdout:5/428: dwrite d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d81/f6a [0,4194304] 0 2024-03-22T07:08:52.859 INFO:tasks.workunit.client.1.smithi149.stdout:0/362: link d1/d31/f53 d1/d9/d14/f85 0 2024-03-22T07:08:52.860 INFO:tasks.workunit.client.1.smithi149.stdout:8/451: mkdir d1/d2/d9/d11/d8b/d8d/d91 0 2024-03-22T07:08:52.860 INFO:tasks.workunit.client.1.smithi149.stdout:8/452: chown d1/d2/d9/d11/f77 5981227 1 2024-03-22T07:08:52.861 INFO:tasks.workunit.client.1.smithi149.stdout:9/491: write db/d21/d37/f5c [2089019,27357] 0 2024-03-22T07:08:52.862 INFO:tasks.workunit.client.1.smithi149.stdout:5/429: mkdir d7/da/d20/d30/d3a/d3d/d2c/d63/d92 0 2024-03-22T07:08:52.863 INFO:tasks.workunit.client.1.smithi149.stdout:0/363: dread - d1/d9/d14/d1f/f38 zero size 2024-03-22T07:08:52.865 INFO:tasks.workunit.client.1.smithi149.stdout:5/430: mkdir d7/da/d20/d30/d3a/d93 0 2024-03-22T07:08:52.865 INFO:tasks.workunit.client.1.smithi149.stdout:5/431: write d7/da/d20/d45/f4f [656068,53893] 0 2024-03-22T07:08:52.865 INFO:tasks.workunit.client.1.smithi149.stdout:5/432: chown d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/f22 9 1 2024-03-22T07:08:52.874 INFO:tasks.workunit.client.1.smithi149.stdout:5/433: truncate d7/da/d20/d36/d44/f5d 1455330 0 2024-03-22T07:08:52.876 INFO:tasks.workunit.client.1.smithi149.stdout:5/434: mkdir d7/da/d20/d30/d3a/d93/d94 0 2024-03-22T07:08:52.876 INFO:tasks.workunit.client.1.smithi149.stdout:5/435: dread - d7/da/d20/d45/f7c zero size 2024-03-22T07:08:52.876 INFO:tasks.workunit.client.1.smithi149.stdout:5/436: creat d7/da/d20/d30/d3a/d3d/d2c/d63/f95 x:0 0 0 2024-03-22T07:08:52.895 INFO:tasks.workunit.client.1.smithi149.stdout:8/453: dwrite d1/d20/f66 [0,4194304] 0 2024-03-22T07:08:52.898 INFO:tasks.workunit.client.1.smithi149.stdout:0/364: dwrite d1/d9/f3a [0,4194304] 0 2024-03-22T07:08:52.898 INFO:tasks.workunit.client.1.smithi149.stdout:0/365: readlink d1/l2 0 2024-03-22T07:08:52.899 INFO:tasks.workunit.client.1.smithi149.stdout:0/366: readlink d1/d9/d14/l17 0 2024-03-22T07:08:52.899 INFO:tasks.workunit.client.1.smithi149.stdout:5/437: dwrite d7/f33 [0,4194304] 0 2024-03-22T07:08:52.899 INFO:tasks.workunit.client.1.smithi149.stdout:9/492: dwrite db/d22/f69 [4194304,4194304] 0 2024-03-22T07:08:52.899 INFO:tasks.workunit.client.1.smithi149.stdout:9/493: chown db/d42/d49/l5b 562485 1 2024-03-22T07:08:52.899 INFO:tasks.workunit.client.1.smithi149.stdout:9/494: dread - db/d33/f92 zero size 2024-03-22T07:08:52.899 INFO:tasks.workunit.client.1.smithi149.stdout:8/454: unlink d1/d2/d3/f1a 0 2024-03-22T07:08:52.900 INFO:tasks.workunit.client.1.smithi149.stdout:0/367: rename d1/d27/l80 to d1/d6b/l86 0 2024-03-22T07:08:52.904 INFO:tasks.workunit.client.1.smithi149.stdout:5/438: mknod d7/da/d20/d30/d3a/d3d/d2c/d63/d92/c96 0 2024-03-22T07:08:52.905 INFO:tasks.workunit.client.1.smithi149.stdout:8/455: mknod d1/c92 0 2024-03-22T07:08:52.906 INFO:tasks.workunit.client.1.smithi149.stdout:5/439: mknod d7/da/d20/d36/d44/c97 0 2024-03-22T07:08:52.906 INFO:tasks.workunit.client.1.smithi149.stdout:5/440: write d7/da/d20/d30/d3a/d3d/d2c/d63/f7f [159128,7595] 0 2024-03-22T07:08:52.909 INFO:tasks.workunit.client.1.smithi149.stdout:9/495: rmdir db/d21/d37 39 2024-03-22T07:08:52.909 INFO:tasks.workunit.client.1.smithi149.stdout:9/496: fdatasync db/d19/d28/f9a 0 2024-03-22T07:08:52.911 INFO:tasks.workunit.client.1.smithi149.stdout:5/441: getdents d7/da/d20/d30/d3a 0 2024-03-22T07:08:52.914 INFO:tasks.workunit.client.1.smithi149.stdout:8/456: mknod d1/d2/d9/d11/d4d/c93 0 2024-03-22T07:08:52.914 INFO:tasks.workunit.client.1.smithi149.stdout:9/497: getdents db/d42 0 2024-03-22T07:08:52.915 INFO:tasks.workunit.client.1.smithi149.stdout:9/498: write db/d19/d1c/f4f [2919175,18236] 0 2024-03-22T07:08:52.915 INFO:tasks.workunit.client.1.smithi149.stdout:8/457: mknod d1/d2/d9/df/c94 0 2024-03-22T07:08:52.916 INFO:tasks.workunit.client.1.smithi149.stdout:9/499: link db/f2f db/d21/d8c/f9f 0 2024-03-22T07:08:52.919 INFO:tasks.workunit.client.1.smithi149.stdout:9/500: symlink db/la0 0 2024-03-22T07:08:52.919 INFO:tasks.workunit.client.1.smithi149.stdout:9/501: fdatasync db/d12/f17 0 2024-03-22T07:08:52.919 INFO:tasks.workunit.client.1.smithi149.stdout:9/502: dread - db/d21/d8c/f96 zero size 2024-03-22T07:08:52.920 INFO:tasks.workunit.client.1.smithi149.stdout:9/503: symlink db/d21/d2e/la1 0 2024-03-22T07:08:52.921 INFO:tasks.workunit.client.1.smithi149.stdout:0/368: dwrite d1/d9/f29 [0,4194304] 0 2024-03-22T07:08:52.922 INFO:tasks.workunit.client.1.smithi149.stdout:8/458: dread d1/d2/d9/df/f2d [0,4194304] 0 2024-03-22T07:08:52.923 INFO:tasks.workunit.client.1.smithi149.stdout:9/504: unlink db/d21/d37/d3a/l9c 0 2024-03-22T07:08:52.923 INFO:tasks.workunit.client.1.smithi149.stdout:9/505: chown db/d19/l47 641995253 1 2024-03-22T07:08:52.929 INFO:tasks.workunit.client.1.smithi149.stdout:0/369: mkdir d1/d31/d51/d64/d87 0 2024-03-22T07:08:52.930 INFO:tasks.workunit.client.1.smithi149.stdout:5/442: dwrite d7/da/d20/d30/d3a/d3d/f34 [0,4194304] 0 2024-03-22T07:08:52.932 INFO:tasks.workunit.client.1.smithi149.stdout:9/506: dread db/d21/d37/d3a/f6c [0,4194304] 0 2024-03-22T07:08:52.935 INFO:tasks.workunit.client.1.smithi149.stdout:0/370: link d1/d31/d51/f47 d1/d31/d52/f88 0 2024-03-22T07:08:52.938 INFO:tasks.workunit.client.1.smithi149.stdout:5/443: creat d7/da/d20/d30/f98 x:0 0 0 2024-03-22T07:08:52.938 INFO:tasks.workunit.client.1.smithi149.stdout:8/459: dwrite d1/d20/d35/d22/d44/f6a [0,4194304] 0 2024-03-22T07:08:52.939 INFO:tasks.workunit.client.1.smithi149.stdout:9/507: rename db/d21/d37/d3a/c58 to db/d42/d49/d4b/d4d/ca2 0 2024-03-22T07:08:52.941 INFO:tasks.workunit.client.1.smithi149.stdout:5/444: rename d7/da/d20/d36/d44/f77 to d7/da/d20/f99 0 2024-03-22T07:08:52.944 INFO:tasks.workunit.client.1.smithi149.stdout:9/508: dread db/d12/f17 [0,4194304] 0 2024-03-22T07:08:52.946 INFO:tasks.workunit.client.1.smithi149.stdout:9/509: mkdir db/d21/d37/d7c/da3 0 2024-03-22T07:08:52.946 INFO:tasks.workunit.client.1.smithi149.stdout:9/510: chown db/d21/d37/d3a/f6c 19217 1 2024-03-22T07:08:52.957 INFO:tasks.workunit.client.1.smithi149.stdout:3/354: sync 2024-03-22T07:08:52.957 INFO:tasks.workunit.client.1.smithi149.stdout:1/390: sync 2024-03-22T07:08:52.957 INFO:tasks.workunit.client.1.smithi149.stdout:6/487: sync 2024-03-22T07:08:52.957 INFO:tasks.workunit.client.1.smithi149.stdout:4/444: sync 2024-03-22T07:08:52.957 INFO:tasks.workunit.client.1.smithi149.stdout:7/473: sync 2024-03-22T07:08:52.958 INFO:tasks.workunit.client.1.smithi149.stdout:4/445: link d0/c65 d0/c98 0 2024-03-22T07:08:52.959 INFO:tasks.workunit.client.1.smithi149.stdout:2/525: sync 2024-03-22T07:08:52.960 INFO:tasks.workunit.client.1.smithi149.stdout:2/526: creat d1/d4d/d69/d75/d89/fb7 x:0 0 0 2024-03-22T07:08:52.960 INFO:tasks.workunit.client.1.smithi149.stdout:1/391: mkdir d4/d20/d23/d3c/d85/d89 0 2024-03-22T07:08:52.961 INFO:tasks.workunit.client.1.smithi149.stdout:7/474: symlink d1/d18/d1b/d51/d7a/d7d/d5d/l96 0 2024-03-22T07:08:52.961 INFO:tasks.workunit.client.1.smithi149.stdout:4/446: symlink d0/d8/dc/d2e/d6f/l99 0 2024-03-22T07:08:52.962 INFO:tasks.workunit.client.1.smithi149.stdout:3/355: getdents d0/d2/d42 0 2024-03-22T07:08:52.962 INFO:tasks.workunit.client.1.smithi149.stdout:5/445: read d7/f8 [557962,105940] 0 2024-03-22T07:08:52.963 INFO:tasks.workunit.client.1.smithi149.stdout:2/527: mkdir d1/d4d/d69/d75/db8 0 2024-03-22T07:08:52.963 INFO:tasks.workunit.client.1.smithi149.stdout:2/528: chown d1/d3/d5/d1f 655 1 2024-03-22T07:08:52.964 INFO:tasks.workunit.client.1.smithi149.stdout:7/475: rmdir d1/d18/d1b/d51/d7a/d7d/d5d 39 2024-03-22T07:08:52.968 INFO:tasks.workunit.client.1.smithi149.stdout:5/446: creat d7/da/d20/d30/d3a/d3d/d2c/d47/f9a x:0 0 0 2024-03-22T07:08:52.968 INFO:tasks.workunit.client.1.smithi149.stdout:5/447: getdents d7/da/d20/d36/d84 0 2024-03-22T07:08:52.968 INFO:tasks.workunit.client.1.smithi149.stdout:9/511: dwrite db/d19/f7d [4194304,4194304] 0 2024-03-22T07:08:52.968 INFO:tasks.workunit.client.1.smithi149.stdout:8/460: dwrite d1/d2/d3/f86 [0,4194304] 0 2024-03-22T07:08:52.968 INFO:tasks.workunit.client.1.smithi149.stdout:8/461: write d1/f16 [1131495,44761] 0 2024-03-22T07:08:52.968 INFO:tasks.workunit.client.1.smithi149.stdout:8/462: truncate d1/d20/d35/d22/d44/d4f/f7e 465313 0 2024-03-22T07:08:52.968 INFO:tasks.workunit.client.1.smithi149.stdout:8/463: write d1/d20/f72 [445992,93379] 0 2024-03-22T07:08:52.968 INFO:tasks.workunit.client.1.smithi149.stdout:0/371: dwrite d1/d31/d52/f5a [0,4194304] 0 2024-03-22T07:08:52.969 INFO:tasks.workunit.client.1.smithi149.stdout:2/529: creat d1/d3/d5/d41/d9f/da7/fb9 x:0 0 0 2024-03-22T07:08:52.969 INFO:tasks.workunit.client.1.smithi149.stdout:2/530: chown d1/d3/f29 445589108 1 2024-03-22T07:08:52.970 INFO:tasks.workunit.client.1.smithi149.stdout:7/476: mkdir d1/d18/d1b/d51/d7a/d7d/d5b/d97 0 2024-03-22T07:08:52.970 INFO:tasks.workunit.client.1.smithi149.stdout:2/531: dread d1/d4d/d69/d75/d89/fa8 [0,4194304] 0 2024-03-22T07:08:52.973 INFO:tasks.workunit.client.1.smithi149.stdout:5/448: dread d7/da/d20/d30/d3a/d3d/d2c/d63/f7f [0,4194304] 0 2024-03-22T07:08:52.974 INFO:tasks.workunit.client.1.smithi149.stdout:5/449: fsync d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/f22 0 2024-03-22T07:08:52.977 INFO:tasks.workunit.client.1.smithi149.stdout:0/372: creat d1/d9/d14/d1f/f89 x:0 0 0 2024-03-22T07:08:52.978 INFO:tasks.workunit.client.1.smithi149.stdout:0/373: write d1/d9/d59/f6c [1027473,100516] 0 2024-03-22T07:08:52.978 INFO:tasks.workunit.client.1.smithi149.stdout:0/374: stat d1/d9/d14/f55 0 2024-03-22T07:08:52.978 INFO:tasks.workunit.client.1.smithi149.stdout:0/375: read d1/d31/f5c [19681,110119] 0 2024-03-22T07:08:52.979 INFO:tasks.workunit.client.1.smithi149.stdout:7/477: unlink d1/d1c/d2f/c4a 0 2024-03-22T07:08:52.980 INFO:tasks.workunit.client.1.smithi149.stdout:7/478: chown d1/f10 295875 1 2024-03-22T07:08:52.980 INFO:tasks.workunit.client.1.smithi149.stdout:4/447: read d0/d8/dc/d14/d3e/f51 [3749473,79658] 0 2024-03-22T07:08:52.980 INFO:tasks.workunit.client.1.smithi149.stdout:4/448: fsync d0/d8/dc/d2e/f62 0 2024-03-22T07:08:52.980 INFO:tasks.workunit.client.1.smithi149.stdout:4/449: readlink d0/l7 0 2024-03-22T07:08:52.980 INFO:tasks.workunit.client.1.smithi149.stdout:2/532: write d1/d3/f36 [2542336,64844] 0 2024-03-22T07:08:52.980 INFO:tasks.workunit.client.1.smithi149.stdout:5/450: rename d7/da/d20/f4c to d7/da/d20/d30/d3a/d93/f9b 0 2024-03-22T07:08:52.980 INFO:tasks.workunit.client.1.smithi149.stdout:5/451: creat d7/da/d20/d45/f9c x:0 0 0 2024-03-22T07:08:52.982 INFO:tasks.workunit.client.1.smithi149.stdout:2/533: symlink d1/d3/d5/d1f/d68/lba 0 2024-03-22T07:08:52.983 INFO:tasks.workunit.client.1.smithi149.stdout:4/450: rename d0/d29/c8b to d0/d8/dc/d4b/c9a 0 2024-03-22T07:08:52.986 INFO:tasks.workunit.client.1.smithi149.stdout:0/376: link d1/d9/d14/d1f/f24 d1/d31/f8a 0 2024-03-22T07:08:52.986 INFO:tasks.workunit.client.1.smithi149.stdout:0/377: chown d1/d31/c84 15650 1 2024-03-22T07:08:52.987 INFO:tasks.workunit.client.1.smithi149.stdout:7/479: link d1/f10 d1/d1c/f98 0 2024-03-22T07:08:52.987 INFO:tasks.workunit.client.1.smithi149.stdout:0/378: mknod d1/d27/d66/c8b 0 2024-03-22T07:08:52.990 INFO:tasks.workunit.client.1.smithi149.stdout:0/379: rmdir d1/d6b 39 2024-03-22T07:08:52.991 INFO:tasks.workunit.client.1.smithi149.stdout:8/464: read d1/d2/d9/d11/f55 [626904,89509] 0 2024-03-22T07:08:52.991 INFO:tasks.workunit.client.1.smithi149.stdout:1/392: dwrite d4/d10/d7c/d6a/d6b/f84 [0,4194304] 0 2024-03-22T07:08:52.992 INFO:tasks.workunit.client.1.smithi149.stdout:6/488: dwrite d9/df/d22/f87 [0,4194304] 0 2024-03-22T07:08:52.992 INFO:tasks.workunit.client.1.smithi149.stdout:1/393: stat d4/d20/d2d/d64/f86 0 2024-03-22T07:08:52.992 INFO:tasks.workunit.client.1.smithi149.stdout:1/394: fsync d4/d10/d7c/d6a/d6b/f84 0 2024-03-22T07:08:52.994 INFO:tasks.workunit.client.1.smithi149.stdout:2/534: dread d1/d8/d34/f57 [0,4194304] 0 2024-03-22T07:08:52.994 INFO:tasks.workunit.client.1.smithi149.stdout:8/465: creat d1/d20/f95 x:0 0 0 2024-03-22T07:08:53.013 INFO:tasks.workunit.client.1.smithi149.stdout:1/395: dread d4/d10/d28/f2a [0,4194304] 0 2024-03-22T07:08:53.014 INFO:tasks.workunit.client.1.smithi149.stdout:1/396: chown d4/d10/d28/f60 1012714860 1 2024-03-22T07:08:53.016 INFO:tasks.workunit.client.1.smithi149.stdout:6/489: write d9/df/d2e/f20 [81353,79900] 0 2024-03-22T07:08:53.016 INFO:tasks.workunit.client.1.smithi149.stdout:6/490: chown d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/c5d 167785 1 2024-03-22T07:08:53.016 INFO:tasks.workunit.client.1.smithi149.stdout:6/491: fsync d9/df/d22/f31 0 2024-03-22T07:08:53.016 INFO:tasks.workunit.client.1.smithi149.stdout:6/492: stat d9/df/d22/d6d/d7d/d96/d24/d34/d4f/f80 0 2024-03-22T07:08:53.016 INFO:tasks.workunit.client.1.smithi149.stdout:1/397: rename d4/d10/d28/f2e to d4/d10/d28/d49/f8a 0 2024-03-22T07:08:53.018 INFO:tasks.workunit.client.1.smithi149.stdout:1/398: dread d4/d20/d2d/f3a [0,4194304] 0 2024-03-22T07:08:53.018 INFO:tasks.workunit.client.1.smithi149.stdout:8/466: write d1/d2/d3/f86 [70661,43674] 0 2024-03-22T07:08:53.021 INFO:tasks.workunit.client.1.smithi149.stdout:0/380: dread d1/d9/d14/d1f/f4d [0,4194304] 0 2024-03-22T07:08:53.021 INFO:tasks.workunit.client.1.smithi149.stdout:0/381: write d1/d27/f62 [386092,62271] 0 2024-03-22T07:08:53.022 INFO:tasks.workunit.client.1.smithi149.stdout:0/382: creat d1/d31/d52/f8c x:0 0 0 2024-03-22T07:08:53.022 INFO:tasks.workunit.client.1.smithi149.stdout:0/383: read d1/d31/d51/f45 [12336,4314] 0 2024-03-22T07:08:53.024 INFO:tasks.workunit.client.1.smithi149.stdout:0/384: mkdir d1/d31/d51/d64/d8d 0 2024-03-22T07:08:53.024 INFO:tasks.workunit.client.1.smithi149.stdout:0/385: fdatasync d1/d31/f53 0 2024-03-22T07:08:53.024 INFO:tasks.workunit.client.1.smithi149.stdout:0/386: getdents d1/d31/d51/d64/d8d 0 2024-03-22T07:08:53.024 INFO:tasks.workunit.client.1.smithi149.stdout:0/387: rmdir d1/d20 39 2024-03-22T07:08:53.030 INFO:tasks.workunit.client.1.smithi149.stdout:1/399: write d4/d10/d28/f34 [1184787,108648] 0 2024-03-22T07:08:53.030 INFO:tasks.workunit.client.1.smithi149.stdout:1/400: write d4/d10/d58/f5b [2740677,1482] 0 2024-03-22T07:08:53.030 INFO:tasks.workunit.client.1.smithi149.stdout:1/401: write d4/d20/d23/f74 [353295,15031] 0 2024-03-22T07:08:53.031 INFO:tasks.workunit.client.1.smithi149.stdout:1/402: mknod d4/d7/d5d/c8b 0 2024-03-22T07:08:53.031 INFO:tasks.workunit.client.1.smithi149.stdout:1/403: chown d4/d10/d28/d49 517771865 1 2024-03-22T07:08:53.036 INFO:tasks.workunit.client.1.smithi149.stdout:6/493: write d9/df/d22/d6d/d7d/d96/f41 [157822,38311] 0 2024-03-22T07:08:53.036 INFO:tasks.workunit.client.1.smithi149.stdout:5/452: dwrite d7/da/d20/d36/d44/f5d [0,4194304] 0 2024-03-22T07:08:53.042 INFO:tasks.workunit.client.1.smithi149.stdout:7/480: dwrite d1/d1c/d2f/d74/f8a [0,4194304] 0 2024-03-22T07:08:53.044 INFO:tasks.workunit.client.1.smithi149.stdout:8/467: write d1/d20/d35/fb [394069,67083] 0 2024-03-22T07:08:53.044 INFO:tasks.workunit.client.1.smithi149.stdout:8/468: chown d1/d2/d9/d11/f58 578 1 2024-03-22T07:08:53.045 INFO:tasks.workunit.client.1.smithi149.stdout:8/469: readlink d1/d2/d3/l75 0 2024-03-22T07:08:53.045 INFO:tasks.workunit.client.1.smithi149.stdout:8/470: write d1/d2/d9/f3b [840316,12822] 0 2024-03-22T07:08:53.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/451: dwrite d0/d8/d1d/d24/d82/f88 [0,4194304] 0 2024-03-22T07:08:53.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/452: read d0/d8/f1f [3973209,13511] 0 2024-03-22T07:08:53.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/453: creat d0/d8/dc/d14/f9b x:0 0 0 2024-03-22T07:08:53.046 INFO:tasks.workunit.client.1.smithi149.stdout:2/535: dwrite d1/d4d/d69/d75/d89/faf [0,4194304] 0 2024-03-22T07:08:53.051 INFO:tasks.workunit.client.1.smithi149.stdout:4/454: mkdir d0/d8/d1d/d24/d9c 0 2024-03-22T07:08:53.051 INFO:tasks.workunit.client.1.smithi149.stdout:4/455: creat d0/d8/dc/f9d x:0 0 0 2024-03-22T07:08:53.055 INFO:tasks.workunit.client.1.smithi149.stdout:2/536: mkdir d1/d3/d5/d1f/d68/dbb 0 2024-03-22T07:08:53.056 INFO:tasks.workunit.client.1.smithi149.stdout:7/481: read d1/d1c/d4d/f68 [3009375,36068] 0 2024-03-22T07:08:53.060 INFO:tasks.workunit.client.1.smithi149.stdout:8/471: dread d1/d2/d3/f1f [0,4194304] 0 2024-03-22T07:08:53.060 INFO:tasks.workunit.client.1.smithi149.stdout:8/472: fdatasync d1/d2/d3/f76 0 2024-03-22T07:08:53.061 INFO:tasks.workunit.client.1.smithi149.stdout:1/404: dwrite d4/d10/d28/f65 [0,4194304] 0 2024-03-22T07:08:53.061 INFO:tasks.workunit.client.1.smithi149.stdout:1/405: chown d4/d20/c31 847 1 2024-03-22T07:08:53.061 INFO:tasks.workunit.client.1.smithi149.stdout:1/406: fdatasync d4/d20/d23/d2f/f62 0 2024-03-22T07:08:53.063 INFO:tasks.workunit.client.1.smithi149.stdout:8/473: rename d1/d20/f69 to d1/d2/d3/f96 0 2024-03-22T07:08:53.063 INFO:tasks.workunit.client.1.smithi149.stdout:8/474: readlink d1/d20/l6c 0 2024-03-22T07:08:53.063 INFO:tasks.workunit.client.1.smithi149.stdout:8/475: creat d1/d20/f97 x:0 0 0 2024-03-22T07:08:53.066 INFO:tasks.workunit.client.1.smithi149.stdout:1/407: rmdir d4/d10/d28/d88 0 2024-03-22T07:08:53.068 INFO:tasks.workunit.client.1.smithi149.stdout:8/476: unlink d1/d2/d3/f1f 0 2024-03-22T07:08:53.088 INFO:tasks.workunit.client.1.smithi149.stdout:5/453: dwrite d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/f22 [0,4194304] 0 2024-03-22T07:08:53.089 INFO:tasks.workunit.client.1.smithi149.stdout:5/454: rename d7/f9 to d7/da/d20/d36/f9d 0 2024-03-22T07:08:53.091 INFO:tasks.workunit.client.1.smithi149.stdout:5/455: creat d7/da/d7d/f9e x:0 0 0 2024-03-22T07:08:53.092 INFO:tasks.workunit.client.1.smithi149.stdout:5/456: dread d7/d35/f55 [0,4194304] 0 2024-03-22T07:08:53.092 INFO:tasks.workunit.client.1.smithi149.stdout:5/457: fdatasync d7/da/d20/f7e 0 2024-03-22T07:08:53.094 INFO:tasks.workunit.client.1.smithi149.stdout:5/458: rename d7/da/d20/d30/d3a/d3d/d2c/d63/f7f to d7/da/d7d/f9f 0 2024-03-22T07:08:53.095 INFO:tasks.workunit.client.1.smithi149.stdout:5/459: creat d7/da/d20/d30/d3a/d93/fa0 x:0 0 0 2024-03-22T07:08:53.095 INFO:tasks.workunit.client.1.smithi149.stdout:6/494: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f43 [0,4194304] 0 2024-03-22T07:08:53.096 INFO:tasks.workunit.client.1.smithi149.stdout:5/460: creat d7/da/d20/d89/fa1 x:0 0 0 2024-03-22T07:08:53.098 INFO:tasks.workunit.client.1.smithi149.stdout:6/495: symlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/l9b 0 2024-03-22T07:08:53.098 INFO:tasks.workunit.client.1.smithi149.stdout:6/496: chown d9/df/d22/d6d/d7d/d96/d24/d34/l6a 34965 1 2024-03-22T07:08:53.102 INFO:tasks.workunit.client.1.smithi149.stdout:6/497: mknod d9/df/d22/d6d/c9c 0 2024-03-22T07:08:53.106 INFO:tasks.workunit.client.1.smithi149.stdout:4/456: dwrite d0/d8/dc/f13 [0,4194304] 0 2024-03-22T07:08:53.107 INFO:tasks.workunit.client.1.smithi149.stdout:7/482: dwrite d1/f21 [0,4194304] 0 2024-03-22T07:08:53.108 INFO:tasks.workunit.client.1.smithi149.stdout:0/388: dwrite d1/d9/d59/f6c [0,4194304] 0 2024-03-22T07:08:53.108 INFO:tasks.workunit.client.1.smithi149.stdout:7/483: fsync d1/d18/d1b/d51/d5a/d6a/f6e 0 2024-03-22T07:08:53.108 INFO:tasks.workunit.client.1.smithi149.stdout:0/389: getdents d1/d31/d51/d64/d87 0 2024-03-22T07:08:53.108 INFO:tasks.workunit.client.1.smithi149.stdout:7/484: dread - d1/d1c/f94 zero size 2024-03-22T07:08:53.108 INFO:tasks.workunit.client.1.smithi149.stdout:4/457: rename d0/d8/dc/d2e/c41 to d0/d8/d1d/d24/d82/c9e 0 2024-03-22T07:08:53.108 INFO:tasks.workunit.client.1.smithi149.stdout:4/458: chown d0/d8/f34 0 1 2024-03-22T07:08:53.109 INFO:tasks.workunit.client.1.smithi149.stdout:8/477: dwrite d1/d2/d3/f96 [0,4194304] 0 2024-03-22T07:08:53.109 INFO:tasks.workunit.client.1.smithi149.stdout:8/478: stat d1/d2/d9 0 2024-03-22T07:08:53.110 INFO:tasks.workunit.client.1.smithi149.stdout:1/408: dwrite d4/d10/f30 [0,4194304] 0 2024-03-22T07:08:53.113 INFO:tasks.workunit.client.1.smithi149.stdout:7/485: rename d1/d1c/d4d to d1/d18/d1b/d99 0 2024-03-22T07:08:53.114 INFO:tasks.workunit.client.1.smithi149.stdout:4/459: mknod d0/d8/dc/d14/d85/d97/c9f 0 2024-03-22T07:08:53.116 INFO:tasks.workunit.client.1.smithi149.stdout:8/479: rmdir d1/d2/d9/d11/d8b/d8d 39 2024-03-22T07:08:53.116 INFO:tasks.workunit.client.1.smithi149.stdout:8/480: dread - d1/d20/d35/d22/d44/f52 zero size 2024-03-22T07:08:53.118 INFO:tasks.workunit.client.1.smithi149.stdout:7/486: mknod d1/d18/d1b/d1d/d72/c9a 0 2024-03-22T07:08:53.118 INFO:tasks.workunit.client.1.smithi149.stdout:7/487: truncate d1/d18/d1b/d99/f58 5795326 0 2024-03-22T07:08:53.119 INFO:tasks.workunit.client.1.smithi149.stdout:8/481: truncate d1/d20/d35/fb 3501556 0 2024-03-22T07:08:53.120 INFO:tasks.workunit.client.1.smithi149.stdout:7/488: rmdir d1/d1c/d49 39 2024-03-22T07:08:53.122 INFO:tasks.workunit.client.1.smithi149.stdout:7/489: link d1/d18/d1b/d51/d7a/d7d/l61 d1/d18/d1b/d51/d7a/d7d/d5b/d97/l9b 0 2024-03-22T07:08:53.124 INFO:tasks.workunit.client.1.smithi149.stdout:5/461: rmdir d7/da/d20/d30/d3a/d3d/d2c 39 2024-03-22T07:08:53.126 INFO:tasks.workunit.client.1.smithi149.stdout:7/490: mkdir d1/d1c/d49/d8e/d9c 0 2024-03-22T07:08:53.127 INFO:tasks.workunit.client.1.smithi149.stdout:7/491: write d1/d18/d1b/d1d/f40 [1959194,122343] 0 2024-03-22T07:08:53.128 INFO:tasks.workunit.client.1.smithi149.stdout:5/462: mkdir d7/da/d20/d89/da2 0 2024-03-22T07:08:53.128 INFO:tasks.workunit.client.1.smithi149.stdout:5/463: truncate d7/da/d20/d36/f91 162998 0 2024-03-22T07:08:53.128 INFO:tasks.workunit.client.1.smithi149.stdout:6/498: dwrite d9/df/d22/d6d/f78 [0,4194304] 0 2024-03-22T07:08:53.129 INFO:tasks.workunit.client.1.smithi149.stdout:6/499: read d9/d47/d52/f60 [605755,87939] 0 2024-03-22T07:08:53.129 INFO:tasks.workunit.client.1.smithi149.stdout:1/409: dwrite f1 [0,4194304] 0 2024-03-22T07:08:53.131 INFO:tasks.workunit.client.1.smithi149.stdout:7/492: rename d1/d18/d1b/d51/d7a/d7d/d5d/l96 to d1/d1c/d49/d8e/d53/l9d 0 2024-03-22T07:08:53.134 INFO:tasks.workunit.client.1.smithi149.stdout:6/500: link d9/df/d22/l28 d9/df/d22/d6d/d7d/d96/d24/d68/l9d 0 2024-03-22T07:08:53.134 INFO:tasks.workunit.client.1.smithi149.stdout:6/501: chown d9/df/d2e/f12 3307985 1 2024-03-22T07:08:53.135 INFO:tasks.workunit.client.1.smithi149.stdout:1/410: truncate d4/f3e 1820644 0 2024-03-22T07:08:53.135 INFO:tasks.workunit.client.1.smithi149.stdout:1/411: write d4/d10/d28/f83 [1807575,26286] 0 2024-03-22T07:08:53.135 INFO:tasks.workunit.client.1.smithi149.stdout:1/412: chown d4/d20/d23/f74 8455880 1 2024-03-22T07:08:53.138 INFO:tasks.workunit.client.1.smithi149.stdout:6/502: link d9/df/d22/d6d/d7d/d96/d24/d34/c76 d9/df/d22/d6d/d7d/c9e 0 2024-03-22T07:08:53.140 INFO:tasks.workunit.client.1.smithi149.stdout:1/413: mkdir d4/d20/d23/d2f/d8c 0 2024-03-22T07:08:53.141 INFO:tasks.workunit.client.1.smithi149.stdout:1/414: getdents d4/d10/d58 0 2024-03-22T07:08:53.142 INFO:tasks.workunit.client.1.smithi149.stdout:4/460: getdents d0/d8/dc/d14/d85/d97 0 2024-03-22T07:08:53.143 INFO:tasks.workunit.client.1.smithi149.stdout:1/415: symlink d4/d7/l8d 0 2024-03-22T07:08:53.144 INFO:tasks.workunit.client.1.smithi149.stdout:1/416: stat d4/l46 0 2024-03-22T07:08:53.145 INFO:tasks.workunit.client.1.smithi149.stdout:1/417: mkdir d4/d20/d23/d3c/d8e 0 2024-03-22T07:08:53.146 INFO:tasks.workunit.client.1.smithi149.stdout:8/482: dwrite d1/d20/d35/d22/d44/d4f/f56 [0,4194304] 0 2024-03-22T07:08:53.146 INFO:tasks.workunit.client.1.smithi149.stdout:8/483: chown d1/d20/d35/d22/d44 433302 1 2024-03-22T07:08:53.148 INFO:tasks.workunit.client.1.smithi149.stdout:1/418: creat d4/d20/d23/d3c/d8e/f8f x:0 0 0 2024-03-22T07:08:53.156 INFO:tasks.workunit.client.1.smithi149.stdout:6/503: write d9/df/d2e/f27 [1709318,91800] 0 2024-03-22T07:08:53.160 INFO:tasks.workunit.client.1.smithi149.stdout:6/504: rename d9/df/d22/d6d/d7d/d96/f48 to d9/df/d22/d6d/d7d/d96/d24/d68/f9f 0 2024-03-22T07:08:53.161 INFO:tasks.workunit.client.1.smithi149.stdout:6/505: getdents d9/d47/d52 0 2024-03-22T07:08:53.163 INFO:tasks.workunit.client.1.smithi149.stdout:5/464: dwrite d7/da/d20/d30/d3a/d3d/d2c/d63/f71 [0,4194304] 0 2024-03-22T07:08:53.163 INFO:tasks.workunit.client.1.smithi149.stdout:6/506: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/fa0 x:0 0 0 2024-03-22T07:08:53.165 INFO:tasks.workunit.client.1.smithi149.stdout:5/465: mknod d7/d35/ca3 0 2024-03-22T07:08:53.165 INFO:tasks.workunit.client.1.smithi149.stdout:7/493: dwrite d1/d1c/f73 [0,4194304] 0 2024-03-22T07:08:53.172 INFO:tasks.workunit.client.1.smithi149.stdout:6/507: getdents d9/df/d22/d6d 0 2024-03-22T07:08:53.174 INFO:tasks.workunit.client.1.smithi149.stdout:4/461: dwrite d0/d8/dc/d4b/f4d [4194304,4194304] 0 2024-03-22T07:08:53.174 INFO:tasks.workunit.client.1.smithi149.stdout:6/508: mkdir d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/da1 0 2024-03-22T07:08:53.174 INFO:tasks.workunit.client.1.smithi149.stdout:6/509: chown d9/c94 3 1 2024-03-22T07:08:53.185 INFO:tasks.workunit.client.1.smithi149.stdout:1/419: dwrite d4/d20/d2d/f3a [0,4194304] 0 2024-03-22T07:08:53.185 INFO:tasks.workunit.client.1.smithi149.stdout:1/420: write d4/f15 [5189426,81637] 0 2024-03-22T07:08:53.187 INFO:tasks.workunit.client.1.smithi149.stdout:8/484: dwrite d1/d2/d3/f96 [0,4194304] 0 2024-03-22T07:08:53.190 INFO:tasks.workunit.client.1.smithi149.stdout:6/510: dread d9/f17 [0,4194304] 0 2024-03-22T07:08:53.190 INFO:tasks.workunit.client.1.smithi149.stdout:6/511: chown d9/df/d2e/f20 7138836 1 2024-03-22T07:08:53.190 INFO:tasks.workunit.client.1.smithi149.stdout:4/462: dread d0/d29/f31 [0,4194304] 0 2024-03-22T07:08:53.190 INFO:tasks.workunit.client.1.smithi149.stdout:6/512: mkdir d9/da2 0 2024-03-22T07:08:53.191 INFO:tasks.workunit.client.1.smithi149.stdout:4/463: dread d0/d8/f21 [0,4194304] 0 2024-03-22T07:08:53.191 INFO:tasks.workunit.client.1.smithi149.stdout:6/513: creat d9/df/d22/d6d/d7d/d96/d24/d34/fa3 x:0 0 0 2024-03-22T07:08:53.193 INFO:tasks.workunit.client.1.smithi149.stdout:4/464: symlink d0/d8/d5c/la0 0 2024-03-22T07:08:53.193 INFO:tasks.workunit.client.1.smithi149.stdout:4/465: fsync d0/d8/dc/d14/d3e/f3d 0 2024-03-22T07:08:53.193 INFO:tasks.workunit.client.1.smithi149.stdout:4/466: creat d0/d8/d1d/d24/d82/fa1 x:0 0 0 2024-03-22T07:08:53.196 INFO:tasks.workunit.client.1.smithi149.stdout:8/485: dread d1/d20/d35/d22/d44/d84/f63 [0,4194304] 0 2024-03-22T07:08:53.198 INFO:tasks.workunit.client.1.smithi149.stdout:1/421: dread d4/d10/d7c/d6a/d6b/f84 [0,4194304] 0 2024-03-22T07:08:53.199 INFO:tasks.workunit.client.1.smithi149.stdout:7/494: dwrite d1/d18/d1b/d1d/f40 [4194304,4194304] 0 2024-03-22T07:08:53.200 INFO:tasks.workunit.client.1.smithi149.stdout:5/466: dwrite d7/da/d20/d45/f51 [0,4194304] 0 2024-03-22T07:08:53.200 INFO:tasks.workunit.client.1.smithi149.stdout:5/467: write d7/da/d20/d30/d3a/d3d/d2c/f88 [30509,120301] 0 2024-03-22T07:08:53.200 INFO:tasks.workunit.client.1.smithi149.stdout:1/422: symlink d4/d7/d5a/l90 0 2024-03-22T07:08:53.203 INFO:tasks.workunit.client.1.smithi149.stdout:7/495: mkdir d1/d18/d1b/d99/d56/d92/d9e 0 2024-03-22T07:08:53.204 INFO:tasks.workunit.client.1.smithi149.stdout:7/496: write d1/d18/d1b/d51/d5a/f78 [1696868,94542] 0 2024-03-22T07:08:53.207 INFO:tasks.workunit.client.1.smithi149.stdout:5/468: mkdir d7/da/d20/d30/d3a/d3d/d2c/d47/da4 0 2024-03-22T07:08:53.210 INFO:tasks.workunit.client.1.smithi149.stdout:1/423: dread d4/f15 [0,4194304] 0 2024-03-22T07:08:53.210 INFO:tasks.workunit.client.1.smithi149.stdout:1/424: write d4/d20/d23/d59/f5c [1476157,80732] 0 2024-03-22T07:08:53.210 INFO:tasks.workunit.client.1.smithi149.stdout:1/425: stat d4/d20/d23/c5f 0 2024-03-22T07:08:53.213 INFO:tasks.workunit.client.1.smithi149.stdout:1/426: creat d4/d20/d2d/f91 x:0 0 0 2024-03-22T07:08:53.213 INFO:tasks.workunit.client.1.smithi149.stdout:1/427: readlink d4/d10/l70 0 2024-03-22T07:08:53.213 INFO:tasks.workunit.client.1.smithi149.stdout:1/428: stat d4/d7/d5d/d82 0 2024-03-22T07:08:53.224 INFO:tasks.workunit.client.1.smithi149.stdout:1/429: dread d4/d10/d7c/d6a/d6b/f84 [0,4194304] 0 2024-03-22T07:08:53.224 INFO:tasks.workunit.client.1.smithi149.stdout:1/430: chown d4/d7/d5a/l90 31 1 2024-03-22T07:08:53.225 INFO:tasks.workunit.client.1.smithi149.stdout:5/469: write d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d81/f6a [2280144,117967] 0 2024-03-22T07:08:53.225 INFO:tasks.workunit.client.1.smithi149.stdout:6/514: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/f7a [0,4194304] 0 2024-03-22T07:08:53.225 INFO:tasks.workunit.client.1.smithi149.stdout:5/470: chown d7/da/c76 33 1 2024-03-22T07:08:53.227 INFO:tasks.workunit.client.1.smithi149.stdout:6/515: mknod d9/df/d22/d6d/d7d/d96/d24/d34/d37/ca4 0 2024-03-22T07:08:53.228 INFO:tasks.workunit.client.1.smithi149.stdout:5/471: symlink d7/da/d20/d36/d44/la5 0 2024-03-22T07:08:53.229 INFO:tasks.workunit.client.1.smithi149.stdout:6/516: mkdir d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5 0 2024-03-22T07:08:53.229 INFO:tasks.workunit.client.1.smithi149.stdout:8/486: dwrite d1/d2/d3/fd [0,4194304] 0 2024-03-22T07:08:53.229 INFO:tasks.workunit.client.1.smithi149.stdout:8/487: chown d1/d2/d9/f1d 300008 1 2024-03-22T07:08:53.230 INFO:tasks.workunit.client.1.smithi149.stdout:7/497: dwrite d1/d18/d1b/d1d/f13 [0,4194304] 0 2024-03-22T07:08:53.230 INFO:tasks.workunit.client.1.smithi149.stdout:5/472: creat d7/da/d20/d36/d84/fa6 x:0 0 0 2024-03-22T07:08:53.230 INFO:tasks.workunit.client.1.smithi149.stdout:5/473: chown d7/da/d20/d45/f9c 12 1 2024-03-22T07:08:53.233 INFO:tasks.workunit.client.1.smithi149.stdout:8/488: rename d1/d2/d9/d11/d45/f71 to d1/d20/d51/f98 0 2024-03-22T07:08:53.235 INFO:tasks.workunit.client.1.smithi149.stdout:5/474: mknod d7/da/d20/d36/ca7 0 2024-03-22T07:08:53.235 INFO:tasks.workunit.client.1.smithi149.stdout:5/475: fdatasync d7/da/d7d/f9f 0 2024-03-22T07:08:53.235 INFO:tasks.workunit.client.1.smithi149.stdout:5/476: chown d7/d35/f55 1 1 2024-03-22T07:08:53.235 INFO:tasks.workunit.client.1.smithi149.stdout:5/477: dread - d7/da/d20/d30/d3a/d3d/d2c/f8b zero size 2024-03-22T07:08:53.235 INFO:tasks.workunit.client.1.smithi149.stdout:5/478: truncate d7/da/d20/f5e 1662163 0 2024-03-22T07:08:53.236 INFO:tasks.workunit.client.1.smithi149.stdout:1/431: write d4/d7/fe [2776783,70844] 0 2024-03-22T07:08:53.236 INFO:tasks.workunit.client.1.smithi149.stdout:1/432: creat d4/d7/d5d/f92 x:0 0 0 2024-03-22T07:08:53.236 INFO:tasks.workunit.client.1.smithi149.stdout:1/433: stat d4/d10/d7c/d6a/d6b 0 2024-03-22T07:08:53.242 INFO:tasks.workunit.client.1.smithi149.stdout:1/434: mkdir d4/d20/d23/d2f/d93 0 2024-03-22T07:08:53.243 INFO:tasks.workunit.client.1.smithi149.stdout:8/489: rmdir d1/d2/d18 39 2024-03-22T07:08:53.245 INFO:tasks.workunit.client.1.smithi149.stdout:1/435: mkdir d4/d7/d94 0 2024-03-22T07:08:53.247 INFO:tasks.workunit.client.1.smithi149.stdout:6/517: dwrite f5 [0,4194304] 0 2024-03-22T07:08:53.248 INFO:tasks.workunit.client.1.smithi149.stdout:8/490: dread d1/d2/d18/f3c [0,4194304] 0 2024-03-22T07:08:53.249 INFO:tasks.workunit.client.1.smithi149.stdout:6/518: dread d9/df/f6c [0,4194304] 0 2024-03-22T07:08:53.250 INFO:tasks.workunit.client.1.smithi149.stdout:8/491: mknod d1/d2/d9/d11/d8b/d8d/c99 0 2024-03-22T07:08:53.251 INFO:tasks.workunit.client.1.smithi149.stdout:6/519: creat d9/df/d22/d6d/d7d/d96/d24/d68/d73/fa6 x:0 0 0 2024-03-22T07:08:53.254 INFO:tasks.workunit.client.1.smithi149.stdout:1/436: dread d4/d20/d2d/d64/f86 [0,4194304] 0 2024-03-22T07:08:53.264 INFO:tasks.workunit.client.1.smithi149.stdout:3/356: sync 2024-03-22T07:08:53.265 INFO:tasks.workunit.client.1.smithi149.stdout:2/537: sync 2024-03-22T07:08:53.265 INFO:tasks.workunit.client.1.smithi149.stdout:9/512: sync 2024-03-22T07:08:53.266 INFO:tasks.workunit.client.1.smithi149.stdout:6/520: read d9/df/d22/f89 [3379674,3084] 0 2024-03-22T07:08:53.266 INFO:tasks.workunit.client.1.smithi149.stdout:9/513: creat db/d21/d37/d3a/fa4 x:0 0 0 2024-03-22T07:08:53.267 INFO:tasks.workunit.client.1.smithi149.stdout:2/538: mkdir d1/d4d/d90/dbc 0 2024-03-22T07:08:53.268 INFO:tasks.workunit.client.1.smithi149.stdout:3/357: mkdir d0/d2/d30/d5d/d84 0 2024-03-22T07:08:53.268 INFO:tasks.workunit.client.1.smithi149.stdout:6/521: truncate d9/df/d22/d6d/d7d/d96/d24/d34/d37/f7a 3442014 0 2024-03-22T07:08:53.269 INFO:tasks.workunit.client.1.smithi149.stdout:9/514: creat db/d42/d49/d4b/fa5 x:0 0 0 2024-03-22T07:08:53.271 INFO:tasks.workunit.client.1.smithi149.stdout:6/522: link d9/df/d2e/f1c d9/df/d22/d6d/d7d/d96/d24/d42/fa7 0 2024-03-22T07:08:53.271 INFO:tasks.workunit.client.1.smithi149.stdout:9/515: mkdir db/d19/da6 0 2024-03-22T07:08:53.271 INFO:tasks.workunit.client.1.smithi149.stdout:9/516: chown db/d19/d28/f9a 240 1 2024-03-22T07:08:53.272 INFO:tasks.workunit.client.1.smithi149.stdout:2/539: symlink d1/d3/d6/lbd 0 2024-03-22T07:08:53.272 INFO:tasks.workunit.client.1.smithi149.stdout:1/437: dwrite d4/d20/f52 [4194304,4194304] 0 2024-03-22T07:08:53.272 INFO:tasks.workunit.client.1.smithi149.stdout:1/438: chown d4/d10/d28/l4e 3 1 2024-03-22T07:08:53.273 INFO:tasks.workunit.client.1.smithi149.stdout:9/517: creat db/d19/d39/d68/d6f/fa7 x:0 0 0 2024-03-22T07:08:53.273 INFO:tasks.workunit.client.1.smithi149.stdout:9/518: write db/d42/d49/d4b/f89 [121626,72543] 0 2024-03-22T07:08:53.273 INFO:tasks.workunit.client.1.smithi149.stdout:9/519: chown db/d21/d2e/c31 94500396 1 2024-03-22T07:08:53.274 INFO:tasks.workunit.client.1.smithi149.stdout:2/540: creat d1/d3/d7d/fbe x:0 0 0 2024-03-22T07:08:53.274 INFO:tasks.workunit.client.1.smithi149.stdout:2/541: chown d1/d8/l14 1482 1 2024-03-22T07:08:53.275 INFO:tasks.workunit.client.1.smithi149.stdout:1/439: symlink d4/d7/d5d/l95 0 2024-03-22T07:08:53.275 INFO:tasks.workunit.client.1.smithi149.stdout:1/440: read - d4/d20/d23/d3c/d8e/f8f zero size 2024-03-22T07:08:53.275 INFO:tasks.workunit.client.1.smithi149.stdout:1/441: read - d4/d10/f55 zero size 2024-03-22T07:08:53.276 INFO:tasks.workunit.client.1.smithi149.stdout:1/442: read d4/d20/d23/d59/f5c [851795,113307] 0 2024-03-22T07:08:53.276 INFO:tasks.workunit.client.1.smithi149.stdout:1/443: dread - d4/d7/d5a/f6d zero size 2024-03-22T07:08:53.277 INFO:tasks.workunit.client.1.smithi149.stdout:9/520: rename db/d22/c2a to db/d42/d49/d4b/d5f/ca8 0 2024-03-22T07:08:53.278 INFO:tasks.workunit.client.1.smithi149.stdout:2/542: unlink d1/d3/fb 0 2024-03-22T07:08:53.278 INFO:tasks.workunit.client.1.smithi149.stdout:2/543: write d1/d3/d5/d1f/f1e [1660544,89454] 0 2024-03-22T07:08:53.278 INFO:tasks.workunit.client.1.smithi149.stdout:2/544: write d1/d4d/f8a [236962,128469] 0 2024-03-22T07:08:53.280 INFO:tasks.workunit.client.1.smithi149.stdout:8/492: dwrite d1/d2/d3/f6 [4194304,4194304] 0 2024-03-22T07:08:53.280 INFO:tasks.workunit.client.1.smithi149.stdout:9/521: rename db/d19/d1c/l6a to db/d21/d37/d3a/la9 0 2024-03-22T07:08:53.281 INFO:tasks.workunit.client.1.smithi149.stdout:9/522: fdatasync db/d19/f1f 0 2024-03-22T07:08:53.281 INFO:tasks.workunit.client.1.smithi149.stdout:9/523: readlink db/d19/d1c/l63 0 2024-03-22T07:08:53.281 INFO:tasks.workunit.client.1.smithi149.stdout:2/545: symlink d1/d3/d5/d1f/d68/d97/lbf 0 2024-03-22T07:08:53.284 INFO:tasks.workunit.client.1.smithi149.stdout:2/546: symlink d1/d4d/d69/d75/db8/lc0 0 2024-03-22T07:08:53.286 INFO:tasks.workunit.client.1.smithi149.stdout:2/547: mknod d1/d3/d7d/cc1 0 2024-03-22T07:08:53.287 INFO:tasks.workunit.client.1.smithi149.stdout:2/548: symlink d1/d3/d6/d87/lc2 0 2024-03-22T07:08:53.288 INFO:tasks.workunit.client.1.smithi149.stdout:9/524: write db/d19/d39/d3b/f53 [3865105,125867] 0 2024-03-22T07:08:53.289 INFO:tasks.workunit.client.1.smithi149.stdout:9/525: write db/d33/f92 [1028976,20685] 0 2024-03-22T07:08:53.289 INFO:tasks.workunit.client.1.smithi149.stdout:9/526: write db/d19/d39/d68/d6f/fa7 [849611,113297] 0 2024-03-22T07:08:53.289 INFO:tasks.workunit.client.1.smithi149.stdout:9/527: creat db/d42/d49/faa x:0 0 0 2024-03-22T07:08:53.290 INFO:tasks.workunit.client.1.smithi149.stdout:2/549: link d1/d3/d5/d1f/d68/d97/lbf d1/d4d/d90/dbc/lc3 0 2024-03-22T07:08:53.290 INFO:tasks.workunit.client.1.smithi149.stdout:2/550: write d1/d3/d5/d1f/f93 [1766869,86778] 0 2024-03-22T07:08:53.291 INFO:tasks.workunit.client.1.smithi149.stdout:2/551: dread d1/d3/d5/d1f/d5e/d2f/fab [0,4194304] 0 2024-03-22T07:08:53.293 INFO:tasks.workunit.client.1.smithi149.stdout:1/444: rmdir d4/d7/d5d 39 2024-03-22T07:08:53.293 INFO:tasks.workunit.client.1.smithi149.stdout:1/445: fdatasync d4/d20/d23/d3c/f47 0 2024-03-22T07:08:53.293 INFO:tasks.workunit.client.1.smithi149.stdout:1/446: fsync d4/f3e 0 2024-03-22T07:08:53.294 INFO:tasks.workunit.client.1.smithi149.stdout:1/447: mkdir d4/d10/d58/d96 0 2024-03-22T07:08:53.296 INFO:tasks.workunit.client.1.smithi149.stdout:6/523: dwrite d9/d47/f5a [4194304,4194304] 0 2024-03-22T07:08:53.296 INFO:tasks.workunit.client.1.smithi149.stdout:6/524: fdatasync d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f53 0 2024-03-22T07:08:53.296 INFO:tasks.workunit.client.1.smithi149.stdout:6/525: chown d9/df/f1e 80369 1 2024-03-22T07:08:53.297 INFO:tasks.workunit.client.1.smithi149.stdout:1/448: link f1 d4/d20/d23/d3c/d85/d89/f97 0 2024-03-22T07:08:53.299 INFO:tasks.workunit.client.1.smithi149.stdout:1/449: creat d4/d10/d7c/f98 x:0 0 0 2024-03-22T07:08:53.299 INFO:tasks.workunit.client.1.smithi149.stdout:1/450: write d4/d10/d28/f60 [891001,117552] 0 2024-03-22T07:08:53.299 INFO:tasks.workunit.client.1.smithi149.stdout:1/451: dread - d4/d7/d5a/f87 zero size 2024-03-22T07:08:53.299 INFO:tasks.workunit.client.1.smithi149.stdout:1/452: stat d4/d10/d28/d49 0 2024-03-22T07:08:53.300 INFO:tasks.workunit.client.1.smithi149.stdout:1/453: symlink d4/d20/d23/d2f/d8c/l99 0 2024-03-22T07:08:53.300 INFO:tasks.workunit.client.1.smithi149.stdout:1/454: symlink d4/d20/d23/d2f/d93/l9a 0 2024-03-22T07:08:53.300 INFO:tasks.workunit.client.1.smithi149.stdout:1/455: write d4/d7/fe [312289,127032] 0 2024-03-22T07:08:53.301 INFO:tasks.workunit.client.1.smithi149.stdout:1/456: creat d4/d10/d58/f9b x:0 0 0 2024-03-22T07:08:53.302 INFO:tasks.workunit.client.1.smithi149.stdout:1/457: creat d4/d20/d23/d2f/d93/f9c x:0 0 0 2024-03-22T07:08:53.303 INFO:tasks.workunit.client.1.smithi149.stdout:1/458: mknod d4/d20/d23/c9d 0 2024-03-22T07:08:53.303 INFO:tasks.workunit.client.1.smithi149.stdout:8/493: dwrite d1/d2/d9/d11/d4d/f67 [0,4194304] 0 2024-03-22T07:08:53.304 INFO:tasks.workunit.client.1.smithi149.stdout:2/552: dread d1/d3/f13 [0,4194304] 0 2024-03-22T07:08:53.304 INFO:tasks.workunit.client.1.smithi149.stdout:1/459: symlink d4/d7/d5d/l9e 0 2024-03-22T07:08:53.304 INFO:tasks.workunit.client.1.smithi149.stdout:1/460: write d4/d20/f73 [421576,94304] 0 2024-03-22T07:08:53.307 INFO:tasks.workunit.client.1.smithi149.stdout:8/494: truncate d1/d2/d9/df/f2d 336493 0 2024-03-22T07:08:53.307 INFO:tasks.workunit.client.1.smithi149.stdout:6/526: write d9/df/d2e/f23 [3610019,98556] 0 2024-03-22T07:08:53.309 INFO:tasks.workunit.client.1.smithi149.stdout:2/553: symlink d1/d3/d6/d3f/d86/lc4 0 2024-03-22T07:08:53.317 INFO:tasks.workunit.client.1.smithi149.stdout:1/461: unlink d4/d10/d7c/f4f 0 2024-03-22T07:08:53.318 INFO:tasks.workunit.client.1.smithi149.stdout:2/554: symlink d1/d4d/d69/lc5 0 2024-03-22T07:08:53.318 INFO:tasks.workunit.client.1.smithi149.stdout:2/555: chown d1/d8/l14 328287 1 2024-03-22T07:08:53.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: pgmap v9: 65 pgs: 65 active+clean; 1.3 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 61 MiB/s rd, 80 MiB/s wr, 787 op/s 2024-03-22T07:08:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:53 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:53.320 INFO:tasks.workunit.client.1.smithi149.stdout:8/495: dread d1/d2/d3/f6 [4194304,4194304] 0 2024-03-22T07:08:53.320 INFO:tasks.workunit.client.1.smithi149.stdout:2/556: symlink d1/d3/d9a/lc6 0 2024-03-22T07:08:53.321 INFO:tasks.workunit.client.1.smithi149.stdout:1/462: write d4/d20/d23/d3c/f47 [1810227,24612] 0 2024-03-22T07:08:53.321 INFO:tasks.workunit.client.1.smithi149.stdout:1/463: chown l3 318565 1 2024-03-22T07:08:53.327 INFO:tasks.workunit.client.1.smithi149.stdout:8/496: dread d1/d20/d35/d22/d44/d84/f63 [0,4194304] 0 2024-03-22T07:08:53.328 INFO:tasks.workunit.client.1.smithi149.stdout:8/497: chown d1/d2/d9/d11/l80 0 1 2024-03-22T07:08:53.328 INFO:tasks.workunit.client.1.smithi149.stdout:8/498: write d1/d2/d9/df/f15 [976500,115052] 0 2024-03-22T07:08:53.328 INFO:tasks.workunit.client.1.smithi149.stdout:8/499: chown d1/d2/d9/d11/d4d/c6d 15849791 1 2024-03-22T07:08:53.328 INFO:tasks.workunit.client.1.smithi149.stdout:8/500: write d1/d20/d35/d22/d44/d84/f48 [1226566,104245] 0 2024-03-22T07:08:53.329 INFO:tasks.workunit.client.1.smithi149.stdout:8/501: mknod d1/d2/d9/df/c9a 0 2024-03-22T07:08:53.329 INFO:tasks.workunit.client.1.smithi149.stdout:6/527: dwrite f5 [0,4194304] 0 2024-03-22T07:08:53.329 INFO:tasks.workunit.client.1.smithi149.stdout:6/528: readlink d9/df/d22/d6d/d7d/d96/d24/l70 0 2024-03-22T07:08:53.330 INFO:tasks.workunit.client.1.smithi149.stdout:6/529: dread - d9/df/d22/d6d/d7d/d96/d24/d34/fa3 zero size 2024-03-22T07:08:53.330 INFO:tasks.workunit.client.1.smithi149.stdout:1/464: dread d4/d20/d23/d59/f5c [0,4194304] 0 2024-03-22T07:08:53.334 INFO:tasks.workunit.client.1.smithi149.stdout:2/557: dread d1/d3/d5/f5a [0,4194304] 0 2024-03-22T07:08:53.339 INFO:tasks.workunit.client.1.smithi149.stdout:6/530: link d9/df/d2e/d63/f69 d9/df/d22/d6d/d7d/fa8 0 2024-03-22T07:08:53.339 INFO:tasks.workunit.client.1.smithi149.stdout:6/531: write d9/df/f1e [2007459,3840] 0 2024-03-22T07:08:53.340 INFO:tasks.workunit.client.1.smithi149.stdout:8/502: getdents d1/d20 0 2024-03-22T07:08:53.343 INFO:tasks.workunit.client.1.smithi149.stdout:2/558: link d1/d3/d5/d41/l83 d1/d4d/d69/d75/d89/lc7 0 2024-03-22T07:08:53.343 INFO:tasks.workunit.client.1.smithi149.stdout:2/559: getdents d1/d3/d6/d32/d6f 0 2024-03-22T07:08:53.343 INFO:tasks.workunit.client.1.smithi149.stdout:2/560: chown d1/d3/d6/d32/l9d 0 1 2024-03-22T07:08:53.349 INFO:tasks.workunit.client.1.smithi149.stdout:1/465: getdents d4/d20/d2d 0 2024-03-22T07:08:53.351 INFO:tasks.workunit.client.1.smithi149.stdout:2/561: read d1/d8/f24 [2134544,22112] 0 2024-03-22T07:08:53.351 INFO:tasks.workunit.client.1.smithi149.stdout:2/562: creat d1/d3/d5/d1f/d68/d94/fc8 x:0 0 0 2024-03-22T07:08:53.351 INFO:tasks.workunit.client.1.smithi149.stdout:1/466: getdents d4/d20/d23/d2f/d8c 0 2024-03-22T07:08:53.351 INFO:tasks.workunit.client.1.smithi149.stdout:8/503: write d1/d2/d3/f96 [1773518,73591] 0 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: pgmap v9: 65 pgs: 65 active+clean; 1.3 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 61 MiB/s rd, 80 MiB/s wr, 787 op/s 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:53 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:53.355 INFO:tasks.workunit.client.1.smithi149.stdout:2/563: rename d1/d3/f25 to d1/d4d/d90/fc9 0 2024-03-22T07:08:53.355 INFO:tasks.workunit.client.1.smithi149.stdout:1/467: mkdir d4/d20/d2d/d64/d9f 0 2024-03-22T07:08:53.357 INFO:tasks.workunit.client.1.smithi149.stdout:2/564: rename d1/d3/d6/d3f/l58 to d1/d4d/d69/d75/db8/lca 0 2024-03-22T07:08:53.357 INFO:tasks.workunit.client.1.smithi149.stdout:2/565: read d1/d3/d6/f7 [2912718,123436] 0 2024-03-22T07:08:53.358 INFO:tasks.workunit.client.1.smithi149.stdout:6/532: dwrite d9/df/f9a [0,4194304] 0 2024-03-22T07:08:53.358 INFO:tasks.workunit.client.1.smithi149.stdout:1/468: mkdir d4/d20/da0 0 2024-03-22T07:08:53.360 INFO:tasks.workunit.client.1.smithi149.stdout:6/533: link d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f53 d9/df/d22/d6d/d7d/d96/d24/fa9 0 2024-03-22T07:08:53.361 INFO:tasks.workunit.client.1.smithi149.stdout:6/534: symlink d9/df/d22/d6d/d7d/d96/d24/d68/laa 0 2024-03-22T07:08:53.361 INFO:tasks.workunit.client.1.smithi149.stdout:6/535: write d9/df/d22/d6d/d7d/d96/f21 [960627,71896] 0 2024-03-22T07:08:53.365 INFO:tasks.workunit.client.1.smithi149.stdout:1/469: write d4/d10/d28/f2a [3677231,101167] 0 2024-03-22T07:08:53.365 INFO:tasks.workunit.client.1.smithi149.stdout:1/470: fdatasync d4/d10/d7c/f7d 0 2024-03-22T07:08:53.366 INFO:tasks.workunit.client.1.smithi149.stdout:4/467: sync 2024-03-22T07:08:53.366 INFO:tasks.workunit.client.1.smithi149.stdout:0/390: sync 2024-03-22T07:08:53.366 INFO:tasks.workunit.client.1.smithi149.stdout:0/391: readlink d1/d9/d14/l2e 0 2024-03-22T07:08:53.366 INFO:tasks.workunit.client.1.smithi149.stdout:4/468: write d0/f67 [1373813,67134] 0 2024-03-22T07:08:53.372 INFO:tasks.workunit.client.1.smithi149.stdout:1/471: creat d4/d20/d2d/fa1 x:0 0 0 2024-03-22T07:08:53.374 INFO:tasks.workunit.client.1.smithi149.stdout:1/472: symlink d4/d10/d7c/la2 0 2024-03-22T07:08:53.375 INFO:tasks.workunit.client.1.smithi149.stdout:1/473: symlink d4/d7/d94/la3 0 2024-03-22T07:08:53.378 INFO:tasks.workunit.client.1.smithi149.stdout:1/474: rename d4/d10/d28/d49/f8a to d4/d20/d23/d3c/d85/d89/fa4 0 2024-03-22T07:08:53.379 INFO:tasks.workunit.client.1.smithi149.stdout:1/475: write d4/d20/d23/f6f [1041410,90721] 0 2024-03-22T07:08:53.380 INFO:tasks.workunit.client.1.smithi149.stdout:8/504: dwrite d1/d2/d18/f3d [0,4194304] 0 2024-03-22T07:08:53.380 INFO:tasks.workunit.client.1.smithi149.stdout:8/505: write d1/d20/f72 [1146866,90469] 0 2024-03-22T07:08:53.383 INFO:tasks.workunit.client.1.smithi149.stdout:8/506: dread d1/d2/d9/d11/d45/f70 [0,4194304] 0 2024-03-22T07:08:53.384 INFO:tasks.workunit.client.1.smithi149.stdout:8/507: mknod d1/d2/d3/c9b 0 2024-03-22T07:08:53.388 INFO:tasks.workunit.client.1.smithi149.stdout:8/508: dread d1/d20/f54 [0,4194304] 0 2024-03-22T07:08:53.388 INFO:tasks.workunit.client.1.smithi149.stdout:8/509: chown d1/d2/d9/f1d 1852870 1 2024-03-22T07:08:53.397 INFO:tasks.workunit.client.1.smithi149.stdout:2/566: dwrite d1/d3/d7d/fbe [0,4194304] 0 2024-03-22T07:08:53.397 INFO:tasks.workunit.client.1.smithi149.stdout:2/567: fdatasync d1/d3/d5/d1f/d5e/f4a 0 2024-03-22T07:08:53.399 INFO:tasks.workunit.client.1.smithi149.stdout:2/568: mknod d1/d3/d6/d32/d6f/ccb 0 2024-03-22T07:08:53.399 INFO:tasks.workunit.client.1.smithi149.stdout:2/569: creat d1/d3/fcc x:0 0 0 2024-03-22T07:08:53.399 INFO:tasks.workunit.client.1.smithi149.stdout:2/570: fsync d1/d3/d5/d1f/f55 0 2024-03-22T07:08:53.401 INFO:tasks.workunit.client.1.smithi149.stdout:4/469: dwrite d0/f2d [0,4194304] 0 2024-03-22T07:08:53.401 INFO:tasks.workunit.client.1.smithi149.stdout:4/470: chown d0/d8/d1d/c52 367 1 2024-03-22T07:08:53.401 INFO:tasks.workunit.client.1.smithi149.stdout:4/471: write d0/d8/dc/f7c [1290158,15624] 0 2024-03-22T07:08:53.401 INFO:tasks.workunit.client.1.smithi149.stdout:0/392: dwrite d1/d31/d51/f45 [0,4194304] 0 2024-03-22T07:08:53.401 INFO:tasks.workunit.client.1.smithi149.stdout:6/536: dwrite d9/df/d2e/d63/f69 [0,4194304] 0 2024-03-22T07:08:53.401 INFO:tasks.workunit.client.1.smithi149.stdout:1/476: dread d4/d20/f73 [0,4194304] 0 2024-03-22T07:08:53.401 INFO:tasks.workunit.client.1.smithi149.stdout:1/477: truncate d4/d7/d5d/f92 622228 0 2024-03-22T07:08:53.401 INFO:tasks.workunit.client.1.smithi149.stdout:2/571: rename d1/d3/f29 to d1/d3/d6/d32/d6f/fcd 0 2024-03-22T07:08:53.401 INFO:tasks.workunit.client.1.smithi149.stdout:2/572: fsync d1/d3/d6/f92 0 2024-03-22T07:08:53.402 INFO:tasks.workunit.client.1.smithi149.stdout:8/510: dread d1/d2/d9/d11/d4d/f5e [0,4194304] 0 2024-03-22T07:08:53.410 INFO:tasks.workunit.client.1.smithi149.stdout:0/393: mknod d1/d31/d51/d64/c8e 0 2024-03-22T07:08:53.410 INFO:tasks.workunit.client.1.smithi149.stdout:0/394: fsync d1/d9/d59/f5d 0 2024-03-22T07:08:53.411 INFO:tasks.workunit.client.1.smithi149.stdout:6/537: dread d9/df/d22/d6d/d7d/d96/d24/d34/d37/f7a [0,4194304] 0 2024-03-22T07:08:53.411 INFO:tasks.workunit.client.1.smithi149.stdout:6/538: read - d9/df/d22/d6d/d7d/d96/d24/d34/d4f/d55/f8e zero size 2024-03-22T07:08:53.411 INFO:tasks.workunit.client.1.smithi149.stdout:6/539: creat d9/df/fab x:0 0 0 2024-03-22T07:08:53.411 INFO:tasks.workunit.client.1.smithi149.stdout:6/540: getdents d9/df/d22/d6d 0 2024-03-22T07:08:53.413 INFO:tasks.workunit.client.1.smithi149.stdout:2/573: link d1/d3/d6/d3f/l85 d1/d3/d5/d1f/d68/d94/lce 0 2024-03-22T07:08:53.416 INFO:tasks.workunit.client.1.smithi149.stdout:1/478: dwrite d4/f69 [0,4194304] 0 2024-03-22T07:08:53.419 INFO:tasks.workunit.client.1.smithi149.stdout:8/511: mknod d1/d2/d9/c9c 0 2024-03-22T07:08:53.419 INFO:tasks.workunit.client.1.smithi149.stdout:8/512: fdatasync d1/d20/d35/fb 0 2024-03-22T07:08:53.420 INFO:tasks.workunit.client.1.smithi149.stdout:0/395: mknod d1/d9/d14/d74/c8f 0 2024-03-22T07:08:53.420 INFO:tasks.workunit.client.1.smithi149.stdout:0/396: read d1/d31/f5c [3216121,65892] 0 2024-03-22T07:08:53.420 INFO:tasks.workunit.client.1.smithi149.stdout:0/397: chown d1/d9/d59/f5d 235 1 2024-03-22T07:08:53.422 INFO:tasks.workunit.client.1.smithi149.stdout:6/541: mkdir d9/df/d22/d6d/d7d/d96/d24/d34/dac 0 2024-03-22T07:08:53.423 INFO:tasks.workunit.client.1.smithi149.stdout:2/574: mknod d1/d3/d9a/ccf 0 2024-03-22T07:08:53.426 INFO:tasks.workunit.client.1.smithi149.stdout:8/513: mkdir d1/d20/d35/d9d 0 2024-03-22T07:08:53.427 INFO:tasks.workunit.client.1.smithi149.stdout:4/472: rmdir d0/d29/d55 0 2024-03-22T07:08:53.427 INFO:tasks.workunit.client.1.smithi149.stdout:0/398: creat d1/d1a/f90 x:0 0 0 2024-03-22T07:08:53.429 INFO:tasks.workunit.client.1.smithi149.stdout:2/575: rename d1/d3/l6c to d1/d3/d5/d1f/d68/d97/ld0 0 2024-03-22T07:08:53.429 INFO:tasks.workunit.client.1.smithi149.stdout:2/576: write d1/f60 [1548926,86291] 0 2024-03-22T07:08:53.432 INFO:tasks.workunit.client.1.smithi149.stdout:8/514: mkdir d1/d20/d35/d22/d44/d4f/d9e 0 2024-03-22T07:08:53.432 INFO:tasks.workunit.client.1.smithi149.stdout:8/515: readlink d1/d20/l6c 0 2024-03-22T07:08:53.432 INFO:tasks.workunit.client.1.smithi149.stdout:8/516: creat d1/d20/d35/d22/d44/f9f x:0 0 0 2024-03-22T07:08:53.433 INFO:tasks.workunit.client.1.smithi149.stdout:6/542: dread f5 [0,4194304] 0 2024-03-22T07:08:53.433 INFO:tasks.workunit.client.1.smithi149.stdout:8/517: write d1/d20/f54 [1190823,123638] 0 2024-03-22T07:08:53.433 INFO:tasks.workunit.client.1.smithi149.stdout:4/473: truncate d0/d8/dc/d14/d3e/f3d 3084801 0 2024-03-22T07:08:53.434 INFO:tasks.workunit.client.1.smithi149.stdout:4/474: read d0/d8/d1d/d24/f54 [242950,57989] 0 2024-03-22T07:08:53.434 INFO:tasks.workunit.client.1.smithi149.stdout:0/399: symlink d1/d31/d51/d64/d87/l91 0 2024-03-22T07:08:53.434 INFO:tasks.workunit.client.1.smithi149.stdout:0/400: write d1/d31/f53 [1136250,125235] 0 2024-03-22T07:08:53.435 INFO:tasks.workunit.client.1.smithi149.stdout:2/577: rmdir d1/d4d/d6a 39 2024-03-22T07:08:53.437 INFO:tasks.workunit.client.1.smithi149.stdout:1/479: rmdir d4/d10/d28/d49 0 2024-03-22T07:08:53.437 INFO:tasks.workunit.client.1.smithi149.stdout:1/480: readlink d4/d20/d23/d2f/d8c/l99 0 2024-03-22T07:08:53.440 INFO:tasks.workunit.client.1.smithi149.stdout:8/518: unlink d1/d20/d35/d22/d44/d84/f8c 0 2024-03-22T07:08:53.441 INFO:tasks.workunit.client.1.smithi149.stdout:8/519: fdatasync d1/d20/d35/d22/f38 0 2024-03-22T07:08:53.442 INFO:tasks.workunit.client.1.smithi149.stdout:6/543: creat d9/df/d22/d6d/d7d/d96/d24/d42/fad x:0 0 0 2024-03-22T07:08:53.442 INFO:tasks.workunit.client.1.smithi149.stdout:6/544: readlink d9/df/d22/d6d/l84 0 2024-03-22T07:08:53.445 INFO:tasks.workunit.client.1.smithi149.stdout:2/578: unlink d1/d3/d6/d3f/d86/lc4 0 2024-03-22T07:08:53.445 INFO:tasks.workunit.client.1.smithi149.stdout:2/579: read - d1/d4d/d69/d75/d89/fb7 zero size 2024-03-22T07:08:53.446 INFO:tasks.workunit.client.1.smithi149.stdout:1/481: chown d4/d20/d23/c56 114252 1 2024-03-22T07:08:53.447 INFO:tasks.workunit.client.1.smithi149.stdout:8/520: link d1/d2/d18/f78 d1/d20/d35/d22/d44/d82/fa0 0 2024-03-22T07:08:53.450 INFO:tasks.workunit.client.1.smithi149.stdout:6/545: rename d9/df/d22/d6d/d7d/d96/d24/d34/d37/f46 to d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/fae 0 2024-03-22T07:08:53.451 INFO:tasks.workunit.client.1.smithi149.stdout:2/580: creat d1/d4d/d6a/fd1 x:0 0 0 2024-03-22T07:08:53.454 INFO:tasks.workunit.client.1.smithi149.stdout:8/521: symlink d1/d20/d35/d9d/la1 0 2024-03-22T07:08:53.455 INFO:tasks.workunit.client.1.smithi149.stdout:6/546: mknod d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/caf 0 2024-03-22T07:08:53.455 INFO:tasks.workunit.client.1.smithi149.stdout:6/547: truncate d9/df/d2e/f1c 2276436 0 2024-03-22T07:08:53.456 INFO:tasks.workunit.client.1.smithi149.stdout:6/548: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/f3e [5061140,97127] 0 2024-03-22T07:08:53.457 INFO:tasks.workunit.client.1.smithi149.stdout:8/522: creat d1/d20/d51/fa2 x:0 0 0 2024-03-22T07:08:53.457 INFO:tasks.workunit.client.1.smithi149.stdout:8/523: stat d1/d20/d35/d9d/la1 0 2024-03-22T07:08:53.457 INFO:tasks.workunit.client.1.smithi149.stdout:8/524: creat d1/d2/d9/d11/fa3 x:0 0 0 2024-03-22T07:08:53.457 INFO:tasks.workunit.client.1.smithi149.stdout:6/549: truncate d9/df/d2e/d63/f69 3408581 0 2024-03-22T07:08:53.460 INFO:tasks.workunit.client.1.smithi149.stdout:8/525: rename d1/d2/d9/d11/d4d/c93 to d1/d2/d9/d11/d8b/d8d/ca4 0 2024-03-22T07:08:53.461 INFO:tasks.workunit.client.1.smithi149.stdout:8/526: rmdir d1/d20/d35/d22/d44/d82 39 2024-03-22T07:08:53.462 INFO:tasks.workunit.client.1.smithi149.stdout:8/527: symlink d1/d2/d9/d11/d8b/la5 0 2024-03-22T07:08:53.462 INFO:tasks.workunit.client.1.smithi149.stdout:8/528: write d1/d2/d9/d11/f55 [2731764,52662] 0 2024-03-22T07:08:53.463 INFO:tasks.workunit.client.1.smithi149.stdout:2/581: write d1/d4d/d69/d75/d89/faf [1952686,46115] 0 2024-03-22T07:08:53.463 INFO:tasks.workunit.client.1.smithi149.stdout:2/582: truncate d1/d3/d5/d1f/d68/d94/fac 505760 0 2024-03-22T07:08:53.463 INFO:tasks.workunit.client.1.smithi149.stdout:8/529: write d1/d20/d35/f6f [438461,19665] 0 2024-03-22T07:08:53.463 INFO:tasks.workunit.client.1.smithi149.stdout:8/530: creat d1/d20/d51/fa6 x:0 0 0 2024-03-22T07:08:53.464 INFO:tasks.workunit.client.1.smithi149.stdout:0/401: dwrite d1/f6a [0,4194304] 0 2024-03-22T07:08:53.464 INFO:tasks.workunit.client.1.smithi149.stdout:0/402: chown d1/d31/d51 66100152 1 2024-03-22T07:08:53.464 INFO:tasks.workunit.client.1.smithi149.stdout:4/475: dwrite d0/d8/f34 [0,4194304] 0 2024-03-22T07:08:53.465 INFO:tasks.workunit.client.1.smithi149.stdout:2/583: symlink d1/d4d/ld2 0 2024-03-22T07:08:53.468 INFO:tasks.workunit.client.1.smithi149.stdout:8/531: rmdir d1/d2/d9/df 39 2024-03-22T07:08:53.475 INFO:tasks.workunit.client.1.smithi149.stdout:2/584: getdents d1/d4d/d69 0 2024-03-22T07:08:53.475 INFO:tasks.workunit.client.1.smithi149.stdout:2/585: fdatasync d1/d3/d5/d1f/d68/d94/fc8 0 2024-03-22T07:08:53.477 INFO:tasks.workunit.client.1.smithi149.stdout:2/586: link d1/d4d/d69/d75/fa1 d1/d4d/d69/d75/fd3 0 2024-03-22T07:08:53.477 INFO:tasks.workunit.client.1.smithi149.stdout:2/587: fsync d1/d3/d5/d1f/f93 0 2024-03-22T07:08:53.477 INFO:tasks.workunit.client.1.smithi149.stdout:2/588: chown d1/d4d/d90/fc9 277 1 2024-03-22T07:08:53.478 INFO:tasks.workunit.client.1.smithi149.stdout:1/482: dwrite d4/f1b [0,4194304] 0 2024-03-22T07:08:53.479 INFO:tasks.workunit.client.1.smithi149.stdout:2/589: link d1/d4d/d69/d75/fa1 d1/d3/d5/d1f/d68/d94/fd4 0 2024-03-22T07:08:53.482 INFO:tasks.workunit.client.1.smithi149.stdout:8/532: write d1/f24 [366590,118769] 0 2024-03-22T07:08:53.484 INFO:tasks.workunit.client.1.smithi149.stdout:0/403: dread d1/d9/d14/d1f/f5f [0,4194304] 0 2024-03-22T07:08:53.485 INFO:tasks.workunit.client.1.smithi149.stdout:4/476: dwrite d0/d29/f31 [0,4194304] 0 2024-03-22T07:08:53.487 INFO:tasks.workunit.client.1.smithi149.stdout:8/533: mknod d1/d20/d51/ca7 0 2024-03-22T07:08:53.489 INFO:tasks.workunit.client.1.smithi149.stdout:0/404: creat d1/d6b/f92 x:0 0 0 2024-03-22T07:08:53.489 INFO:tasks.workunit.client.1.smithi149.stdout:2/590: getdents d1/d3/d7d 0 2024-03-22T07:08:53.489 INFO:tasks.workunit.client.1.smithi149.stdout:2/591: write d1/d3/d6/db2/f76 [542486,5416] 0 2024-03-22T07:08:53.489 INFO:tasks.workunit.client.1.smithi149.stdout:2/592: chown d1/d3/d5/d41/l83 1730608 1 2024-03-22T07:08:53.492 INFO:tasks.workunit.client.1.smithi149.stdout:0/405: symlink d1/d6b/l93 0 2024-03-22T07:08:53.492 INFO:tasks.workunit.client.1.smithi149.stdout:0/406: write d1/d31/f3f [209834,89995] 0 2024-03-22T07:08:53.493 INFO:tasks.workunit.client.1.smithi149.stdout:8/534: mknod d1/d2/d9/d11/d4d/ca8 0 2024-03-22T07:08:53.493 INFO:tasks.workunit.client.1.smithi149.stdout:8/535: chown d1/d2/d9/d11/f7a 29632 1 2024-03-22T07:08:53.493 INFO:tasks.workunit.client.1.smithi149.stdout:8/536: write d1/d20/d35/d22/d44/f52 [83833,58874] 0 2024-03-22T07:08:53.511 INFO:tasks.workunit.client.1.smithi149.stdout:5/479: sync 2024-03-22T07:08:53.511 INFO:tasks.workunit.client.1.smithi149.stdout:7/498: sync 2024-03-22T07:08:53.511 INFO:tasks.workunit.client.1.smithi149.stdout:7/499: write d1/d18/d1b/d51/d7a/f7f [145881,12962] 0 2024-03-22T07:08:53.512 INFO:tasks.workunit.client.1.smithi149.stdout:3/358: sync 2024-03-22T07:08:53.512 INFO:tasks.workunit.client.1.smithi149.stdout:9/528: sync 2024-03-22T07:08:53.514 INFO:tasks.workunit.client.1.smithi149.stdout:5/480: rename d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d2b/f68 to d7/da/d20/d89/da2/fa8 0 2024-03-22T07:08:53.515 INFO:tasks.workunit.client.1.smithi149.stdout:1/483: dwrite d4/d10/d7c/d6a/d6b/f81 [0,4194304] 0 2024-03-22T07:08:53.516 INFO:tasks.workunit.client.1.smithi149.stdout:4/477: dwrite d0/d8/dc/f18 [0,4194304] 0 2024-03-22T07:08:53.517 INFO:tasks.workunit.client.1.smithi149.stdout:9/529: rename db/d42/d49/d4b/d5f/d7f to db/d42/dab 0 2024-03-22T07:08:53.522 INFO:tasks.workunit.client.1.smithi149.stdout:4/478: rmdir d0/d6b/d93 0 2024-03-22T07:08:53.522 INFO:tasks.workunit.client.1.smithi149.stdout:4/479: readlink d0/d8/dc/l15 0 2024-03-22T07:08:53.523 INFO:tasks.workunit.client.1.smithi149.stdout:9/530: truncate db/d19/f7d 1880322 0 2024-03-22T07:08:53.524 INFO:tasks.workunit.client.1.smithi149.stdout:4/480: truncate d0/d8/d1d/f28 2310120 0 2024-03-22T07:08:53.524 INFO:tasks.workunit.client.1.smithi149.stdout:4/481: dread - d0/d8/dc/d4b/f70 zero size 2024-03-22T07:08:53.524 INFO:tasks.workunit.client.1.smithi149.stdout:4/482: chown d0/d29/f6e 18248 1 2024-03-22T07:08:53.526 INFO:tasks.workunit.client.1.smithi149.stdout:9/531: link db/d21/d8c/f98 db/d19/d39/d68/fac 0 2024-03-22T07:08:53.526 INFO:tasks.workunit.client.1.smithi149.stdout:9/532: truncate db/d19/d39/d3b/f48 4738730 0 2024-03-22T07:08:53.526 INFO:tasks.workunit.client.1.smithi149.stdout:4/483: creat d0/d8/fa2 x:0 0 0 2024-03-22T07:08:53.527 INFO:tasks.workunit.client.1.smithi149.stdout:7/500: write d1/d18/d1b/d51/d7a/d7d/f3b [2978448,98213] 0 2024-03-22T07:08:53.527 INFO:tasks.workunit.client.1.smithi149.stdout:7/501: dread - d1/d1c/f94 zero size 2024-03-22T07:08:53.528 INFO:tasks.workunit.client.1.smithi149.stdout:4/484: mknod d0/d8/dc/d14/d85/d97/ca3 0 2024-03-22T07:08:53.528 INFO:tasks.workunit.client.1.smithi149.stdout:4/485: readlink d0/d8/dc/l17 0 2024-03-22T07:08:53.530 INFO:tasks.workunit.client.1.smithi149.stdout:8/537: dwrite d1/d2/d9/d11/f17 [0,4194304] 0 2024-03-22T07:08:53.534 INFO:tasks.workunit.client.1.smithi149.stdout:4/486: mkdir d0/d8/d5c/da4 0 2024-03-22T07:08:53.535 INFO:tasks.workunit.client.1.smithi149.stdout:2/593: dwrite d1/d8/fb0 [0,4194304] 0 2024-03-22T07:08:53.536 INFO:tasks.workunit.client.1.smithi149.stdout:0/407: dwrite d1/d9/d14/d74/f81 [4194304,4194304] 0 2024-03-22T07:08:53.539 INFO:tasks.workunit.client.1.smithi149.stdout:3/359: dwrite d0/d2/d30/f73 [0,4194304] 0 2024-03-22T07:08:53.542 INFO:tasks.workunit.client.1.smithi149.stdout:4/487: dread d0/d8/dc/d2e/d6f/f83 [0,4194304] 0 2024-03-22T07:08:53.542 INFO:tasks.workunit.client.1.smithi149.stdout:4/488: readlink d0/d8/d5c/l63 0 2024-03-22T07:08:53.542 INFO:tasks.workunit.client.1.smithi149.stdout:2/594: creat d1/d3/d9a/fd5 x:0 0 0 2024-03-22T07:08:53.542 INFO:tasks.workunit.client.1.smithi149.stdout:2/595: fsync d1/d3/d5/d1f/d5e/d2f/f33 0 2024-03-22T07:08:53.543 INFO:tasks.workunit.client.1.smithi149.stdout:7/502: dread d1/f41 [0,4194304] 0 2024-03-22T07:08:53.545 INFO:tasks.workunit.client.1.smithi149.stdout:5/481: dwrite d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/fc [0,4194304] 0 2024-03-22T07:08:53.548 INFO:tasks.workunit.client.1.smithi149.stdout:3/360: mkdir d0/d4/d8/d24/d40/d54/d85 0 2024-03-22T07:08:53.548 INFO:tasks.workunit.client.1.smithi149.stdout:3/361: truncate d0/d2/d30/f70 237531 0 2024-03-22T07:08:53.554 INFO:tasks.workunit.client.1.smithi149.stdout:9/533: dwrite db/d42/d49/f8e [0,4194304] 0 2024-03-22T07:08:53.555 INFO:tasks.workunit.client.1.smithi149.stdout:2/596: creat d1/d4d/d69/d75/d7e/fd6 x:0 0 0 2024-03-22T07:08:53.555 INFO:tasks.workunit.client.1.smithi149.stdout:8/538: dwrite d1/d2/d9/d11/d45/f70 [0,4194304] 0 2024-03-22T07:08:53.555 INFO:tasks.workunit.client.1.smithi149.stdout:7/503: creat d1/d1c/f9f x:0 0 0 2024-03-22T07:08:53.555 INFO:tasks.workunit.client.1.smithi149.stdout:4/489: read d0/d8/dc/d14/f75 [1565589,118183] 0 2024-03-22T07:08:53.557 INFO:tasks.workunit.client.1.smithi149.stdout:0/408: link d1/d9/d14/d1f/f37 d1/d9/d14/f94 0 2024-03-22T07:08:53.558 INFO:tasks.workunit.client.1.smithi149.stdout:5/482: creat d7/da/d20/d30/d3a/d3d/d2c/d47/fa9 x:0 0 0 2024-03-22T07:08:53.559 INFO:tasks.workunit.client.1.smithi149.stdout:3/362: creat d0/d4/d8/d24/d40/d54/d85/f86 x:0 0 0 2024-03-22T07:08:53.559 INFO:tasks.workunit.client.1.smithi149.stdout:3/363: creat d0/d2/d36/d74/f87 x:0 0 0 2024-03-22T07:08:53.560 INFO:tasks.workunit.client.1.smithi149.stdout:9/534: truncate db/d22/f2b 560092 0 2024-03-22T07:08:53.563 INFO:tasks.workunit.client.1.smithi149.stdout:2/597: symlink d1/d4d/d69/d75/ld7 0 2024-03-22T07:08:53.564 INFO:tasks.workunit.client.1.smithi149.stdout:7/504: symlink d1/d85/la0 0 2024-03-22T07:08:53.565 INFO:tasks.workunit.client.1.smithi149.stdout:0/409: symlink d1/d9/d14/d74/d77/l95 0 2024-03-22T07:08:53.566 INFO:tasks.workunit.client.1.smithi149.stdout:5/483: symlink d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d2b/laa 0 2024-03-22T07:08:53.567 INFO:tasks.workunit.client.1.smithi149.stdout:5/484: chown d7/da/d20/d30/d3a/d3d/d2c/d63/d92 28718559 1 2024-03-22T07:08:53.567 INFO:tasks.workunit.client.1.smithi149.stdout:8/539: write d1/d2/d9/d11/d4d/f5e [1743932,15428] 0 2024-03-22T07:08:53.567 INFO:tasks.workunit.client.1.smithi149.stdout:8/540: write d1/d2/d3/f96 [455913,44361] 0 2024-03-22T07:08:53.570 INFO:tasks.workunit.client.1.smithi149.stdout:3/364: symlink d0/d4/d8/d24/d40/l88 0 2024-03-22T07:08:53.570 INFO:tasks.workunit.client.1.smithi149.stdout:3/365: chown d0/d4/d8/l26 1963240 1 2024-03-22T07:08:53.571 INFO:tasks.workunit.client.1.smithi149.stdout:7/505: creat d1/d1c/d49/d8e/fa1 x:0 0 0 2024-03-22T07:08:53.572 INFO:tasks.workunit.client.1.smithi149.stdout:0/410: chown d1/d1a/f4f 7 1 2024-03-22T07:08:53.572 INFO:tasks.workunit.client.1.smithi149.stdout:0/411: dread - d1/d9/d14/f55 zero size 2024-03-22T07:08:53.574 INFO:tasks.workunit.client.1.smithi149.stdout:3/366: rename d0/d2/d42/f4e to d0/d4/d8/d24/d39/f89 0 2024-03-22T07:08:53.574 INFO:tasks.workunit.client.1.smithi149.stdout:3/367: dread - d0/d2/d42/f7c zero size 2024-03-22T07:08:53.575 INFO:tasks.workunit.client.1.smithi149.stdout:4/490: write d0/d8/f46 [492348,50756] 0 2024-03-22T07:08:53.576 INFO:tasks.workunit.client.1.smithi149.stdout:7/506: symlink d1/d18/d1b/d51/d7a/d7d/d5b/d97/la2 0 2024-03-22T07:08:53.577 INFO:tasks.workunit.client.1.smithi149.stdout:7/507: mkdir d1/d18/d1b/d99/d56/da3 0 2024-03-22T07:08:53.579 INFO:tasks.workunit.client.1.smithi149.stdout:5/485: read d7/da/d20/d30/d3a/d3d/d2c/d63/f71 [2337744,110363] 0 2024-03-22T07:08:53.581 INFO:tasks.workunit.client.1.smithi149.stdout:7/508: unlink d1/d18/d1b/d1d/f38 0 2024-03-22T07:08:53.583 INFO:tasks.workunit.client.1.smithi149.stdout:5/486: creat d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d81/fab x:0 0 0 2024-03-22T07:08:53.583 INFO:tasks.workunit.client.1.smithi149.stdout:5/487: chown d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/c31 554 1 2024-03-22T07:08:53.584 INFO:tasks.workunit.client.1.smithi149.stdout:0/412: dread d1/d9/d14/f85 [0,4194304] 0 2024-03-22T07:08:53.585 INFO:tasks.workunit.client.1.smithi149.stdout:0/413: dread - d1/d9/d14/d1f/f38 zero size 2024-03-22T07:08:53.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/491: dread d0/d8/d1d/f8f [0,4194304] 0 2024-03-22T07:08:53.585 INFO:tasks.workunit.client.1.smithi149.stdout:4/492: fsync d0/d8/dc/d74/d76/f7b 0 2024-03-22T07:08:53.586 INFO:tasks.workunit.client.1.smithi149.stdout:7/509: link d1/d18/d1b/d99/d56/f81 d1/d18/fa4 0 2024-03-22T07:08:53.586 INFO:tasks.workunit.client.1.smithi149.stdout:5/488: dread d7/da/d20/d45/f4f [0,4194304] 0 2024-03-22T07:08:53.587 INFO:tasks.workunit.client.1.smithi149.stdout:9/535: dwrite db/d42/d49/faa [0,4194304] 0 2024-03-22T07:08:53.589 INFO:tasks.workunit.client.1.smithi149.stdout:0/414: unlink d1/d31/d52/f5a 0 2024-03-22T07:08:53.589 INFO:tasks.workunit.client.1.smithi149.stdout:0/415: write d1/d27/f6e [501487,122603] 0 2024-03-22T07:08:53.591 INFO:tasks.workunit.client.1.smithi149.stdout:4/493: unlink d0/d8/d5c/l72 0 2024-03-22T07:08:53.592 INFO:tasks.workunit.client.1.smithi149.stdout:7/510: mknod d1/d18/ca5 0 2024-03-22T07:08:53.593 INFO:tasks.workunit.client.1.smithi149.stdout:5/489: creat d7/da/d20/d30/d3a/d3d/d2c/d63/d92/fac x:0 0 0 2024-03-22T07:08:53.593 INFO:tasks.workunit.client.1.smithi149.stdout:7/511: write d1/d18/d1b/d51/d7a/d7d/d5b/f79 [697320,100770] 0 2024-03-22T07:08:53.593 INFO:tasks.workunit.client.1.smithi149.stdout:7/512: write d1/d18/d1b/d51/d7a/f7f [1091001,4849] 0 2024-03-22T07:08:53.595 INFO:tasks.workunit.client.1.smithi149.stdout:0/416: rename d1/d9/c58 to d1/d6b/c96 0 2024-03-22T07:08:53.596 INFO:tasks.workunit.client.1.smithi149.stdout:4/494: symlink d0/d8/dc/d4b/la5 0 2024-03-22T07:08:53.596 INFO:tasks.workunit.client.1.smithi149.stdout:4/495: write d0/d8/dc/d14/d3e/f61 [376069,7069] 0 2024-03-22T07:08:53.596 INFO:tasks.workunit.client.1.smithi149.stdout:4/496: write d0/d29/f58 [698961,71734] 0 2024-03-22T07:08:53.597 INFO:tasks.workunit.client.1.smithi149.stdout:0/417: mknod d1/d20/c97 0 2024-03-22T07:08:53.598 INFO:tasks.workunit.client.1.smithi149.stdout:5/490: rename d7/da/d20/d30/d3a/d3d/d2c/d63/f95 to d7/da/d20/d36/fad 0 2024-03-22T07:08:53.599 INFO:tasks.workunit.client.1.smithi149.stdout:9/536: write db/d21/d37/d3a/f6c [2071848,128077] 0 2024-03-22T07:08:53.599 INFO:tasks.workunit.client.1.smithi149.stdout:9/537: stat db/d21/d37/d7c/da3 0 2024-03-22T07:08:53.600 INFO:tasks.workunit.client.1.smithi149.stdout:7/513: rename d1/d1c/d49/d8e/d53/d5c to d1/d18/d1b/d51/d5a/d6a/da6 0 2024-03-22T07:08:53.605 INFO:tasks.workunit.client.1.smithi149.stdout:2/598: dwrite d1/d3/d9a/fd5 [0,4194304] 0 2024-03-22T07:08:53.605 INFO:tasks.workunit.client.1.smithi149.stdout:2/599: write d1/d8/f24 [276961,37009] 0 2024-03-22T07:08:53.605 INFO:tasks.workunit.client.1.smithi149.stdout:8/541: dwrite d1/d2/d9/d11/f77 [0,4194304] 0 2024-03-22T07:08:53.605 INFO:tasks.workunit.client.1.smithi149.stdout:3/368: dwrite d0/d2/d42/d66/f7d [0,4194304] 0 2024-03-22T07:08:53.605 INFO:tasks.workunit.client.1.smithi149.stdout:3/369: fsync d0/d4/d8/f25 0 2024-03-22T07:08:53.605 INFO:tasks.workunit.client.1.smithi149.stdout:9/538: creat db/d42/fad x:0 0 0 2024-03-22T07:08:53.605 INFO:tasks.workunit.client.1.smithi149.stdout:9/539: dread db/d42/d49/d4b/f89 [0,4194304] 0 2024-03-22T07:08:53.612 INFO:tasks.workunit.client.1.smithi149.stdout:8/542: dread d1/d2/d9/f29 [0,4194304] 0 2024-03-22T07:08:53.612 INFO:tasks.workunit.client.1.smithi149.stdout:8/543: stat d1/d2/d9/d11/c4c 0 2024-03-22T07:08:53.612 INFO:tasks.workunit.client.1.smithi149.stdout:2/600: mknod d1/d3/d5/d41/d9f/cd8 0 2024-03-22T07:08:53.621 INFO:tasks.workunit.client.1.smithi149.stdout:0/418: rename d1/d31/d51/d64/c8e to d1/d9/d14/d1f/c98 0 2024-03-22T07:08:53.621 INFO:tasks.workunit.client.1.smithi149.stdout:0/419: fsync d1/fb 0 2024-03-22T07:08:53.621 INFO:tasks.workunit.client.1.smithi149.stdout:0/420: write d1/d6b/f78 [11461,122172] 0 2024-03-22T07:08:53.629 INFO:tasks.workunit.client.1.smithi149.stdout:8/544: unlink d1/d2/d9/d11/d8b/c90 0 2024-03-22T07:08:53.629 INFO:tasks.workunit.client.1.smithi149.stdout:8/545: creat d1/d2/d3/fa9 x:0 0 0 2024-03-22T07:08:53.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/601: mknod d1/d4d/d90/dbc/cd9 0 2024-03-22T07:08:53.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/602: stat d1/d4d/d69/d75/d89/fa8 0 2024-03-22T07:08:53.631 INFO:tasks.workunit.client.1.smithi149.stdout:7/514: rename d1/d18/d1b/d1d/f62 to d1/fa7 0 2024-03-22T07:08:53.632 INFO:tasks.workunit.client.1.smithi149.stdout:5/491: dwrite d7/da/d20/d30/d3a/d3d/d2c/f8b [0,4194304] 0 2024-03-22T07:08:53.632 INFO:tasks.workunit.client.1.smithi149.stdout:5/492: dread - d7/da/d7d/f9e zero size 2024-03-22T07:08:53.633 INFO:tasks.workunit.client.1.smithi149.stdout:5/493: dread - d7/da/d20/d30/d3a/d3d/d2c/d63/d92/fac zero size 2024-03-22T07:08:53.633 INFO:tasks.workunit.client.1.smithi149.stdout:2/603: truncate d1/d3/d6/db2/f40 4415547 0 2024-03-22T07:08:53.637 INFO:tasks.workunit.client.1.smithi149.stdout:3/370: rename d0/d2/d30/d31/f33 to d0/d2/d30/f8a 0 2024-03-22T07:08:53.638 INFO:tasks.workunit.client.1.smithi149.stdout:9/540: dwrite db/d19/f7d [0,4194304] 0 2024-03-22T07:08:53.639 INFO:tasks.workunit.client.1.smithi149.stdout:8/546: read d1/d20/d35/d22/f38 [3389274,128907] 0 2024-03-22T07:08:53.639 INFO:tasks.workunit.client.1.smithi149.stdout:4/497: dwrite d0/d8/f34 [0,4194304] 0 2024-03-22T07:08:53.639 INFO:tasks.workunit.client.1.smithi149.stdout:4/498: rename d0/d8/dc/d14 to d0/d8/dc/d14/d85/d97/da6 22 2024-03-22T07:08:53.639 INFO:tasks.workunit.client.1.smithi149.stdout:4/499: truncate d0/d8/dc/f5b 549012 0 2024-03-22T07:08:53.640 INFO:tasks.workunit.client.1.smithi149.stdout:4/500: read d0/d8/f21 [110422,10752] 0 2024-03-22T07:08:53.640 INFO:tasks.workunit.client.1.smithi149.stdout:4/501: creat d0/d6b/fa7 x:0 0 0 2024-03-22T07:08:53.646 INFO:tasks.workunit.client.1.smithi149.stdout:7/515: creat d1/d1c/d2f/fa8 x:0 0 0 2024-03-22T07:08:53.646 INFO:tasks.workunit.client.1.smithi149.stdout:0/421: link d1/d9/d14/f94 d1/d9/d14/d1f/f99 0 2024-03-22T07:08:53.646 INFO:tasks.workunit.client.1.smithi149.stdout:7/516: fdatasync d1/d1c/f98 0 2024-03-22T07:08:53.646 INFO:tasks.workunit.client.1.smithi149.stdout:5/494: creat d7/da/d20/d30/d3a/d3d/d2c/d47/fae x:0 0 0 2024-03-22T07:08:53.648 INFO:tasks.workunit.client.1.smithi149.stdout:2/604: creat d1/d3/d6/db2/d71/fda x:0 0 0 2024-03-22T07:08:53.648 INFO:tasks.workunit.client.1.smithi149.stdout:3/371: dread d0/d2/d36/f20 [0,4194304] 0 2024-03-22T07:08:53.649 INFO:tasks.workunit.client.1.smithi149.stdout:9/541: unlink db/d42/d49/c9d 0 2024-03-22T07:08:53.649 INFO:tasks.workunit.client.1.smithi149.stdout:9/542: fsync db/d22/f36 0 2024-03-22T07:08:53.650 INFO:tasks.workunit.client.1.smithi149.stdout:8/547: truncate d1/d2/d9/d11/f55 4009305 0 2024-03-22T07:08:53.657 INFO:tasks.workunit.client.1.smithi149.stdout:4/502: dwrite d0/d6b/f81 [0,4194304] 0 2024-03-22T07:08:53.659 INFO:tasks.workunit.client.1.smithi149.stdout:7/517: creat d1/d18/d1b/d51/d7a/d7d/d5d/fa9 x:0 0 0 2024-03-22T07:08:53.659 INFO:tasks.workunit.client.1.smithi149.stdout:7/518: chown d1/d18/d1b/d51/d5a/d6a/f6e 370 1 2024-03-22T07:08:53.661 INFO:tasks.workunit.client.1.smithi149.stdout:0/422: rename d1/d31/d52/f8c to d1/d9/d14/d74/d77/f9a 0 2024-03-22T07:08:53.663 INFO:tasks.workunit.client.1.smithi149.stdout:5/495: truncate d7/f8 1113397 0 2024-03-22T07:08:53.663 INFO:tasks.workunit.client.1.smithi149.stdout:5/496: chown d7/da/d20/d45/f7c 44 1 2024-03-22T07:08:53.663 INFO:tasks.workunit.client.1.smithi149.stdout:5/497: fdatasync d7/d35/f55 0 2024-03-22T07:08:53.663 INFO:tasks.workunit.client.1.smithi149.stdout:5/498: readlink d7/da/d20/d36/d44/l5a 0 2024-03-22T07:08:53.663 INFO:tasks.workunit.client.1.smithi149.stdout:5/499: fdatasync d7/da/d20/d36/f9d 0 2024-03-22T07:08:53.664 INFO:tasks.workunit.client.1.smithi149.stdout:2/605: truncate d1/d3/f4b 821260 0 2024-03-22T07:08:53.664 INFO:tasks.workunit.client.1.smithi149.stdout:2/606: readlink d1/d4d/d69/d75/db8/lca 0 2024-03-22T07:08:53.665 INFO:tasks.workunit.client.1.smithi149.stdout:3/372: mknod d0/d4/d8/d24/d40/d62/c8b 0 2024-03-22T07:08:53.667 INFO:tasks.workunit.client.1.smithi149.stdout:8/548: getdents d1/d20/d35/d22/d44/d4f 0 2024-03-22T07:08:53.667 INFO:tasks.workunit.client.1.smithi149.stdout:7/519: write d1/d1c/d49/d8e/f50 [205574,69606] 0 2024-03-22T07:08:53.668 INFO:tasks.workunit.client.1.smithi149.stdout:9/543: dwrite db/d21/f23 [0,4194304] 0 2024-03-22T07:08:53.668 INFO:tasks.workunit.client.1.smithi149.stdout:7/520: read d1/d1c/d2f/f43 [3247481,119893] 0 2024-03-22T07:08:53.668 INFO:tasks.workunit.client.1.smithi149.stdout:7/521: readlink d1/d18/d1b/d99/l4f 0 2024-03-22T07:08:53.671 INFO:tasks.workunit.client.1.smithi149.stdout:4/503: symlink d0/d8/dc/d14/d85/la8 0 2024-03-22T07:08:53.672 INFO:tasks.workunit.client.1.smithi149.stdout:4/504: write d0/f32 [3232408,75907] 0 2024-03-22T07:08:53.673 INFO:tasks.workunit.client.1.smithi149.stdout:0/423: rename d1/d31/d51/l73 to d1/d31/d51/l9b 0 2024-03-22T07:08:53.674 INFO:tasks.workunit.client.1.smithi149.stdout:3/373: dread d0/d2/d36/f2e [0,4194304] 0 2024-03-22T07:08:53.674 INFO:tasks.workunit.client.1.smithi149.stdout:3/374: creat d0/d2/f8c x:0 0 0 2024-03-22T07:08:53.676 INFO:tasks.workunit.client.1.smithi149.stdout:2/607: rename d1/d4d/c4e to d1/d4d/d90/cdb 0 2024-03-22T07:08:53.676 INFO:tasks.workunit.client.1.smithi149.stdout:2/608: stat d1/d3/d6/db2/f40 0 2024-03-22T07:08:53.677 INFO:tasks.workunit.client.1.smithi149.stdout:0/424: dread d1/f39 [0,4194304] 0 2024-03-22T07:08:53.678 INFO:tasks.workunit.client.1.smithi149.stdout:8/549: truncate d1/d2/d3/f86 3922127 0 2024-03-22T07:08:53.680 INFO:tasks.workunit.client.1.smithi149.stdout:9/544: mkdir db/d22/dae 0 2024-03-22T07:08:53.686 INFO:tasks.workunit.client.1.smithi149.stdout:0/425: getdents d1/d27 0 2024-03-22T07:08:53.687 INFO:tasks.workunit.client.1.smithi149.stdout:0/426: readlink d1/d31/l3b 0 2024-03-22T07:08:53.687 INFO:tasks.workunit.client.1.smithi149.stdout:0/427: fdatasync d1/d9/d14/f18 0 2024-03-22T07:08:53.687 INFO:tasks.workunit.client.1.smithi149.stdout:0/428: stat d1/d31/l3b 0 2024-03-22T07:08:53.689 INFO:tasks.workunit.client.1.smithi149.stdout:9/545: rmdir db/d22/dae 0 2024-03-22T07:08:53.690 INFO:tasks.workunit.client.1.smithi149.stdout:7/522: rmdir d1/d1c/d2f 39 2024-03-22T07:08:53.691 INFO:tasks.workunit.client.1.smithi149.stdout:7/523: write d1/d18/d1b/d51/d5a/f7c [882892,83640] 0 2024-03-22T07:08:53.691 INFO:tasks.workunit.client.1.smithi149.stdout:7/524: creat d1/d18/d1b/d51/d7a/faa x:0 0 0 2024-03-22T07:08:53.695 INFO:tasks.workunit.client.1.smithi149.stdout:2/609: mknod d1/d3/d6/cdc 0 2024-03-22T07:08:53.696 INFO:tasks.workunit.client.1.smithi149.stdout:1/484: sync 2024-03-22T07:08:53.697 INFO:tasks.workunit.client.1.smithi149.stdout:0/429: mkdir d1/d31/d51/d64/d8d/d9c 0 2024-03-22T07:08:53.698 INFO:tasks.workunit.client.1.smithi149.stdout:6/550: sync 2024-03-22T07:08:53.698 INFO:tasks.workunit.client.1.smithi149.stdout:8/550: link d1/d20/d35/d22/d44/f52 d1/d2/d9/d11/d4d/faa 0 2024-03-22T07:08:53.698 INFO:tasks.workunit.client.1.smithi149.stdout:8/551: truncate d1/d2/f23 379104 0 2024-03-22T07:08:53.699 INFO:tasks.workunit.client.1.smithi149.stdout:8/552: stat d1/d2/d9/d11/d8b 0 2024-03-22T07:08:53.699 INFO:tasks.workunit.client.1.smithi149.stdout:9/546: write db/f2f [456183,65010] 0 2024-03-22T07:08:53.701 INFO:tasks.workunit.client.1.smithi149.stdout:7/525: mkdir d1/d18/d1b/d99/d56/d92/d9e/dab 0 2024-03-22T07:08:53.702 INFO:tasks.workunit.client.1.smithi149.stdout:7/526: write d1/d1c/d49/d8e/fa1 [422090,50976] 0 2024-03-22T07:08:53.702 INFO:tasks.workunit.client.1.smithi149.stdout:8/553: unlink d1/d2/d9/d11/d8b/la5 0 2024-03-22T07:08:53.702 INFO:tasks.workunit.client.1.smithi149.stdout:0/430: creat d1/d31/d51/d64/d87/f9d x:0 0 0 2024-03-22T07:08:53.702 INFO:tasks.workunit.client.1.smithi149.stdout:8/554: read d1/d2/d9/f4a [950557,10839] 0 2024-03-22T07:08:53.704 INFO:tasks.workunit.client.1.smithi149.stdout:9/547: rename db/d42/d49/d4b/f89 to db/d21/d74/faf 0 2024-03-22T07:08:53.704 INFO:tasks.workunit.client.1.smithi149.stdout:7/527: chown d1/d1c/d2f/d74/f8a 49483 1 2024-03-22T07:08:53.710 INFO:tasks.workunit.client.1.smithi149.stdout:9/548: mknod db/d21/d37/cb0 0 2024-03-22T07:08:53.716 INFO:tasks.workunit.client.1.smithi149.stdout:7/528: symlink d1/d1c/d2f/d74/lac 0 2024-03-22T07:08:53.716 INFO:tasks.workunit.client.1.smithi149.stdout:0/431: mkdir d1/d31/d51/d64/d8d/d9c/d9e 0 2024-03-22T07:08:53.716 INFO:tasks.workunit.client.1.smithi149.stdout:0/432: fsync d1/d31/f33 0 2024-03-22T07:08:53.717 INFO:tasks.workunit.client.1.smithi149.stdout:9/549: link db/d19/f94 db/d42/d49/fb1 0 2024-03-22T07:08:53.717 INFO:tasks.workunit.client.1.smithi149.stdout:9/550: truncate db/d33/f92 1617710 0 2024-03-22T07:08:53.717 INFO:tasks.workunit.client.1.smithi149.stdout:4/505: rmdir d0/d8/dc 39 2024-03-22T07:08:53.717 INFO:tasks.workunit.client.1.smithi149.stdout:8/555: symlink d1/d2/d9/d11/d4d/lab 0 2024-03-22T07:08:53.717 INFO:tasks.workunit.client.1.smithi149.stdout:8/556: write d1/d20/d35/d22/d44/d84/f63 [5007714,17814] 0 2024-03-22T07:08:53.719 INFO:tasks.workunit.client.1.smithi149.stdout:4/506: rename d0/d8/dc/d2e/c5d to d0/d8/dc/ca9 0 2024-03-22T07:08:53.719 INFO:tasks.workunit.client.1.smithi149.stdout:4/507: dread - d0/d8/dc/d2e/d6f/f7f zero size 2024-03-22T07:08:53.720 INFO:tasks.workunit.client.1.smithi149.stdout:0/433: mkdir d1/d20/d9f 0 2024-03-22T07:08:53.720 INFO:tasks.workunit.client.1.smithi149.stdout:9/551: write db/d42/d49/d4b/d4d/f55 [242547,111416] 0 2024-03-22T07:08:53.721 INFO:tasks.workunit.client.1.smithi149.stdout:9/552: readlink db/d12/l81 0 2024-03-22T07:08:53.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/375: getdents d0/d2 0 2024-03-22T07:08:53.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/376: fsync d0/d2/d30/f8a 0 2024-03-22T07:08:53.728 INFO:tasks.workunit.client.1.smithi149.stdout:0/434: rename d1/d9/d14/d74/d77/c7f to d1/d6b/ca0 0 2024-03-22T07:08:53.735 INFO:tasks.workunit.client.1.smithi149.stdout:9/553: link db/d12/c24 db/d19/d28/cb2 0 2024-03-22T07:08:53.735 INFO:tasks.workunit.client.1.smithi149.stdout:9/554: write db/d21/d37/d7c/f9b [894614,48850] 0 2024-03-22T07:08:53.735 INFO:tasks.workunit.client.1.smithi149.stdout:3/377: getdents d0/d2/d30/d5d 0 2024-03-22T07:08:53.742 INFO:tasks.workunit.client.1.smithi149.stdout:7/529: dwrite d1/d1c/d49/d8e/f87 [0,4194304] 0 2024-03-22T07:08:53.742 INFO:tasks.workunit.client.1.smithi149.stdout:7/530: creat d1/d18/d1b/d51/d5a/fad x:0 0 0 2024-03-22T07:08:53.746 INFO:tasks.workunit.client.1.smithi149.stdout:7/531: mknod d1/d1c/d2f/d74/cae 0 2024-03-22T07:08:53.746 INFO:tasks.workunit.client.1.smithi149.stdout:7/532: creat d1/d18/d1b/d99/faf x:0 0 0 2024-03-22T07:08:53.758 INFO:tasks.workunit.client.1.smithi149.stdout:5/500: truncate d7/f8 344409 0 2024-03-22T07:08:53.760 INFO:tasks.workunit.client.1.smithi149.stdout:7/533: dread d1/fa7 [0,4194304] 0 2024-03-22T07:08:53.760 INFO:tasks.workunit.client.1.smithi149.stdout:3/378: dread d0/d2/d30/f73 [0,4194304] 0 2024-03-22T07:08:53.760 INFO:tasks.workunit.client.1.smithi149.stdout:3/379: dread - d0/d2/f4d zero size 2024-03-22T07:08:53.760 INFO:tasks.workunit.client.1.smithi149.stdout:7/534: write d1/d18/d1b/d1d/fd [3848874,99438] 0 2024-03-22T07:08:53.760 INFO:tasks.workunit.client.1.smithi149.stdout:2/610: dwrite d1/d3/d5/d1f/f93 [0,4194304] 0 2024-03-22T07:08:53.760 INFO:tasks.workunit.client.1.smithi149.stdout:6/551: dwrite d9/df/f56 [4194304,4194304] 0 2024-03-22T07:08:53.761 INFO:tasks.workunit.client.1.smithi149.stdout:5/501: mkdir d7/da/d20/d30/d3a/d3d/d2c/d47/da4/daf 0 2024-03-22T07:08:53.764 INFO:tasks.workunit.client.1.smithi149.stdout:3/380: creat d0/d4/d8/f8d x:0 0 0 2024-03-22T07:08:53.764 INFO:tasks.workunit.client.1.smithi149.stdout:3/381: dread - d0/d4/d8/d24/f6e zero size 2024-03-22T07:08:53.764 INFO:tasks.workunit.client.1.smithi149.stdout:1/485: dwrite d4/d20/d23/f44 [0,4194304] 0 2024-03-22T07:08:53.764 INFO:tasks.workunit.client.1.smithi149.stdout:2/611: chown d1/d3/d5/f2b 1881856 1 2024-03-22T07:08:53.766 INFO:tasks.workunit.client.1.smithi149.stdout:6/552: unlink d9/df/f6c 0 2024-03-22T07:08:53.767 INFO:tasks.workunit.client.1.smithi149.stdout:8/557: dwrite d1/d20/f95 [0,4194304] 0 2024-03-22T07:08:53.767 INFO:tasks.workunit.client.1.smithi149.stdout:5/502: mkdir d7/da/d20/d30/d3a/d3d/db0 0 2024-03-22T07:08:53.768 INFO:tasks.workunit.client.1.smithi149.stdout:3/382: mkdir d0/d2/d30/d5d/d84/d8e 0 2024-03-22T07:08:53.768 INFO:tasks.workunit.client.1.smithi149.stdout:1/486: mkdir d4/d10/d58/d68/da5 0 2024-03-22T07:08:53.769 INFO:tasks.workunit.client.1.smithi149.stdout:1/487: write d4/d10/f55 [854249,24374] 0 2024-03-22T07:08:53.769 INFO:tasks.workunit.client.1.smithi149.stdout:1/488: fdatasync d4/d10/d28/f37 0 2024-03-22T07:08:53.771 INFO:tasks.workunit.client.1.smithi149.stdout:2/612: rename d1/f60 to d1/d3/fdd 0 2024-03-22T07:08:53.771 INFO:tasks.workunit.client.1.smithi149.stdout:6/553: creat d9/da2/fb0 x:0 0 0 2024-03-22T07:08:53.773 INFO:tasks.workunit.client.1.smithi149.stdout:1/489: link d4/ca d4/d20/d2d/d64/ca6 0 2024-03-22T07:08:53.776 INFO:tasks.workunit.client.1.smithi149.stdout:6/554: rename d9/df/d22/d6d/d7d/d96/d24/d34/d4f to d9/df/d22/d6d/d7d/d96/db1 0 2024-03-22T07:08:53.777 INFO:tasks.workunit.client.1.smithi149.stdout:1/490: getdents d4/d20/d23/d59 0 2024-03-22T07:08:53.778 INFO:tasks.workunit.client.1.smithi149.stdout:1/491: fdatasync d4/d20/d2d/f3a 0 2024-03-22T07:08:53.778 INFO:tasks.workunit.client.1.smithi149.stdout:1/492: write d4/d20/f73 [755568,62666] 0 2024-03-22T07:08:53.779 INFO:tasks.workunit.client.1.smithi149.stdout:0/435: dwrite d1/d9/d14/d1f/f24 [0,4194304] 0 2024-03-22T07:08:53.780 INFO:tasks.workunit.client.1.smithi149.stdout:4/508: dwrite d0/d8/dc/d4b/f70 [0,4194304] 0 2024-03-22T07:08:53.780 INFO:tasks.workunit.client.1.smithi149.stdout:4/509: truncate d0/d8/d1d/f28 3315410 0 2024-03-22T07:08:53.780 INFO:tasks.workunit.client.1.smithi149.stdout:4/510: read - d0/d8/dc/d2e/f62 zero size 2024-03-22T07:08:53.780 INFO:tasks.workunit.client.1.smithi149.stdout:4/511: write d0/d8/d1d/d24/d82/f91 [567286,54179] 0 2024-03-22T07:08:53.782 INFO:tasks.workunit.client.1.smithi149.stdout:9/555: dwrite db/d21/d37/f5c [0,4194304] 0 2024-03-22T07:08:53.784 INFO:tasks.workunit.client.1.smithi149.stdout:7/535: dwrite d1/d18/fa4 [0,4194304] 0 2024-03-22T07:08:53.784 INFO:tasks.workunit.client.1.smithi149.stdout:7/536: write d1/d1c/d2f/fa8 [1042199,81089] 0 2024-03-22T07:08:53.786 INFO:tasks.workunit.client.1.smithi149.stdout:5/503: dread d7/f33 [0,4194304] 0 2024-03-22T07:08:53.788 INFO:tasks.workunit.client.1.smithi149.stdout:2/613: dread d1/d3/d5/d1f/f55 [0,4194304] 0 2024-03-22T07:08:53.788 INFO:tasks.workunit.client.1.smithi149.stdout:4/512: write d0/d6b/f7d [115470,60232] 0 2024-03-22T07:08:53.794 INFO:tasks.workunit.client.1.smithi149.stdout:0/436: dread d1/d27/f6e [0,4194304] 0 2024-03-22T07:08:53.796 INFO:tasks.workunit.client.1.smithi149.stdout:3/383: rename d0/d2/d36 to d0/d2/d30/d5d/d84/d8f 0 2024-03-22T07:08:53.798 INFO:tasks.workunit.client.1.smithi149.stdout:9/556: dread db/d21/d74/f76 [0,4194304] 0 2024-03-22T07:08:53.803 INFO:tasks.workunit.client.1.smithi149.stdout:5/504: creat d7/da/d20/d30/d3a/d3d/d2c/fb1 x:0 0 0 2024-03-22T07:08:53.803 INFO:tasks.workunit.client.1.smithi149.stdout:5/505: write d7/da/d20/d30/d3a/d3d/d2c/d47/fae [102514,46447] 0 2024-03-22T07:08:53.803 INFO:tasks.workunit.client.1.smithi149.stdout:5/506: truncate d7/da/d20/d36/f66 924311 0 2024-03-22T07:08:53.803 INFO:tasks.workunit.client.1.smithi149.stdout:5/507: chown d7/da/c70 114605 1 2024-03-22T07:08:53.814 INFO:tasks.workunit.client.1.smithi149.stdout:1/493: rename d4/d10/d7c/d6a/c78 to d4/d10/d7c/d6a/d6b/ca7 0 2024-03-22T07:08:53.814 INFO:tasks.workunit.client.1.smithi149.stdout:9/557: unlink db/d12/l1b 0 2024-03-22T07:08:53.815 INFO:tasks.workunit.client.1.smithi149.stdout:5/508: mkdir d7/da/d20/d30/d3a/d3d/d2c/db2 0 2024-03-22T07:08:53.816 INFO:tasks.workunit.client.1.smithi149.stdout:1/494: link d4/d20/d2d/c36 d4/ca8 0 2024-03-22T07:08:53.816 INFO:tasks.workunit.client.1.smithi149.stdout:1/495: readlink d4/d7/d5d/l9e 0 2024-03-22T07:08:53.817 INFO:tasks.workunit.client.1.smithi149.stdout:0/437: dread d1/d9/d59/f61 [0,4194304] 0 2024-03-22T07:08:53.817 INFO:tasks.workunit.client.1.smithi149.stdout:7/537: rename d1/d18/d1b/d99/f58 to d1/d18/d1b/d51/d5a/d6a/fb0 0 2024-03-22T07:08:53.817 INFO:tasks.workunit.client.1.smithi149.stdout:7/538: readlink d1/d18/d1b/d51/l63 0 2024-03-22T07:08:53.817 INFO:tasks.workunit.client.1.smithi149.stdout:3/384: write d0/d2/d30/d5d/d84/d8f/f50 [1723443,79225] 0 2024-03-22T07:08:53.818 INFO:tasks.workunit.client.1.smithi149.stdout:9/558: truncate db/d19/d39/d3b/f48 421037 0 2024-03-22T07:08:53.819 INFO:tasks.workunit.client.1.smithi149.stdout:5/509: creat d7/da/d20/d30/d3a/d93/d94/fb3 x:0 0 0 2024-03-22T07:08:53.821 INFO:tasks.workunit.client.1.smithi149.stdout:1/496: dread d4/d10/f55 [0,4194304] 0 2024-03-22T07:08:53.821 INFO:tasks.workunit.client.1.smithi149.stdout:9/559: creat db/d19/d1c/d85/fb3 x:0 0 0 2024-03-22T07:08:53.822 INFO:tasks.workunit.client.1.smithi149.stdout:0/438: rename d1/d1a/f25 to d1/d9/d14/fa1 0 2024-03-22T07:08:53.824 INFO:tasks.workunit.client.1.smithi149.stdout:0/439: creat d1/d31/d51/d64/fa2 x:0 0 0 2024-03-22T07:08:53.824 INFO:tasks.workunit.client.1.smithi149.stdout:0/440: truncate d1/d31/d51/d64/d87/f9d 994303 0 2024-03-22T07:08:53.824 INFO:tasks.workunit.client.1.smithi149.stdout:5/510: dread d7/da/d20/d30/d3a/d93/f9b [0,4194304] 0 2024-03-22T07:08:53.826 INFO:tasks.workunit.client.1.smithi149.stdout:0/441: mkdir d1/d20/da3 0 2024-03-22T07:08:53.827 INFO:tasks.workunit.client.1.smithi149.stdout:0/442: stat d1/d6b/c79 0 2024-03-22T07:08:53.828 INFO:tasks.workunit.client.1.smithi149.stdout:5/511: truncate d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/fc 2272426 0 2024-03-22T07:08:53.829 INFO:tasks.workunit.client.1.smithi149.stdout:5/512: rmdir d7/da/d20/d36/d44/d4a 39 2024-03-22T07:08:53.830 INFO:tasks.workunit.client.1.smithi149.stdout:7/539: dread d1/d18/d1b/d1d/fd [0,4194304] 0 2024-03-22T07:08:53.830 INFO:tasks.workunit.client.1.smithi149.stdout:7/540: write d1/d18/d1b/d51/d7a/d7d/f77 [499630,29735] 0 2024-03-22T07:08:53.831 INFO:tasks.workunit.client.1.smithi149.stdout:6/555: dwrite d9/df/f1e [0,4194304] 0 2024-03-22T07:08:53.831 INFO:tasks.workunit.client.1.smithi149.stdout:6/556: write d9/df/f1d [4275391,39081] 0 2024-03-22T07:08:53.833 INFO:tasks.workunit.client.1.smithi149.stdout:7/541: truncate d1/f33 5347605 0 2024-03-22T07:08:53.837 INFO:tasks.workunit.client.1.smithi149.stdout:4/513: dwrite d0/d8/dc/d4b/f70 [0,4194304] 0 2024-03-22T07:08:53.837 INFO:tasks.workunit.client.1.smithi149.stdout:4/514: read d0/d8/dc/d14/f75 [3426393,54736] 0 2024-03-22T07:08:53.850 INFO:tasks.workunit.client.1.smithi149.stdout:7/542: dread d1/d18/d1b/f42 [0,4194304] 0 2024-03-22T07:08:53.854 INFO:tasks.workunit.client.1.smithi149.stdout:9/560: write db/d19/d1c/f20 [3055302,82650] 0 2024-03-22T07:08:53.858 INFO:tasks.workunit.client.1.smithi149.stdout:9/561: truncate db/d33/f67 4149192 0 2024-03-22T07:08:53.865 INFO:tasks.workunit.client.1.smithi149.stdout:7/543: dread d1/d18/d1b/d1d/f13 [0,4194304] 0 2024-03-22T07:08:53.868 INFO:tasks.workunit.client.1.smithi149.stdout:9/562: rmdir db/d19/d39/d68/d6f 39 2024-03-22T07:08:53.868 INFO:tasks.workunit.client.1.smithi149.stdout:9/563: readlink db/d21/d37/d3a/la9 0 2024-03-22T07:08:53.868 INFO:tasks.workunit.client.1.smithi149.stdout:7/544: mkdir d1/d1c/db1 0 2024-03-22T07:08:53.870 INFO:tasks.workunit.client.1.smithi149.stdout:9/564: read db/d19/d39/d3b/f6d [16591,106860] 0 2024-03-22T07:08:53.870 INFO:tasks.workunit.client.1.smithi149.stdout:9/565: read db/d12/f17 [1586342,81419] 0 2024-03-22T07:08:53.871 INFO:tasks.workunit.client.1.smithi149.stdout:2/614: dwrite d1/d3/d5/d1f/d5e/d2f/fab [0,4194304] 0 2024-03-22T07:08:53.872 INFO:tasks.workunit.client.1.smithi149.stdout:9/566: rename db/d21/l91 to db/d33/lb4 0 2024-03-22T07:08:53.876 INFO:tasks.workunit.client.1.smithi149.stdout:2/615: link d1/d3/d7d/fbe d1/d3/d5/d1f/fde 0 2024-03-22T07:08:53.876 INFO:tasks.workunit.client.1.smithi149.stdout:2/616: rename d1/d3/d6/d3f/d74 to d1/d3/d6/d3f/d74/ddf 22 2024-03-22T07:08:53.877 INFO:tasks.workunit.client.1.smithi149.stdout:3/385: dwrite d0/d4/d8/d24/f6e [0,4194304] 0 2024-03-22T07:08:53.878 INFO:tasks.workunit.client.1.smithi149.stdout:9/567: unlink db/d21/d37/c3e 0 2024-03-22T07:08:53.878 INFO:tasks.workunit.client.1.smithi149.stdout:2/617: mkdir d1/d3/de0 0 2024-03-22T07:08:53.881 INFO:tasks.workunit.client.1.smithi149.stdout:9/568: truncate db/d42/d49/d4b/d4d/f55 155354 0 2024-03-22T07:08:53.882 INFO:tasks.workunit.client.1.smithi149.stdout:9/569: write db/d12/f14 [589172,34024] 0 2024-03-22T07:08:53.887 INFO:tasks.workunit.client.1.smithi149.stdout:5/513: rmdir d7/da/d20/d30 39 2024-03-22T07:08:53.888 INFO:tasks.workunit.client.1.smithi149.stdout:3/386: dread d0/d4/f58 [0,4194304] 0 2024-03-22T07:08:53.890 INFO:tasks.workunit.client.1.smithi149.stdout:3/387: symlink d0/d4/d8/d24/d40/l90 0 2024-03-22T07:08:53.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/388: dread - d0/f72 zero size 2024-03-22T07:08:53.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/389: fsync d0/d2/f55 0 2024-03-22T07:08:53.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/390: chown d0/d2/d30/d5d/d84/d8f/d74/f87 86 1 2024-03-22T07:08:53.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/391: dread - d0/d4/d8/d24/d40/d62/f64 zero size 2024-03-22T07:08:53.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/392: dread - d0/d4/d8/f8d zero size 2024-03-22T07:08:53.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/393: fsync d0/d2/d30/f8a 0 2024-03-22T07:08:53.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/394: creat d0/d4/d8/d24/d40/d54/d85/f91 x:0 0 0 2024-03-22T07:08:53.891 INFO:tasks.workunit.client.1.smithi149.stdout:2/618: rename d1/d4d/d69/d75/d89 to d1/d4d/d6a/de1 0 2024-03-22T07:08:53.891 INFO:tasks.workunit.client.1.smithi149.stdout:2/619: write d1/d4d/d69/d75/fd3 [741019,90966] 0 2024-03-22T07:08:53.895 INFO:tasks.workunit.client.1.smithi149.stdout:2/620: rmdir d1/d3/d5/d41 39 2024-03-22T07:08:53.898 INFO:tasks.workunit.client.1.smithi149.stdout:2/621: creat d1/d4d/d90/dbc/fe2 x:0 0 0 2024-03-22T07:08:53.898 INFO:tasks.workunit.client.1.smithi149.stdout:2/622: chown d1/d3/d6/d3f/d74 413 1 2024-03-22T07:08:53.899 INFO:tasks.workunit.client.1.smithi149.stdout:2/623: chown d1/d3/d6/c12 96 1 2024-03-22T07:08:53.899 INFO:tasks.workunit.client.1.smithi149.stdout:2/624: fdatasync d1/d3/d5/d41/d9f/da7/fb9 0 2024-03-22T07:08:53.900 INFO:tasks.workunit.client.1.smithi149.stdout:0/443: dwrite d1/d9/d59/f61 [0,4194304] 0 2024-03-22T07:08:53.901 INFO:tasks.workunit.client.1.smithi149.stdout:6/557: dwrite d9/df/d22/d6d/d7d/d96/f41 [0,4194304] 0 2024-03-22T07:08:53.904 INFO:tasks.workunit.client.1.smithi149.stdout:2/625: mkdir d1/d4d/d69/d75/db6/de3 0 2024-03-22T07:08:53.904 INFO:tasks.workunit.client.1.smithi149.stdout:4/515: dwrite d0/d8/d1d/d24/f54 [0,4194304] 0 2024-03-22T07:08:53.904 INFO:tasks.workunit.client.1.smithi149.stdout:4/516: fdatasync d0/d29/f6e 0 2024-03-22T07:08:53.904 INFO:tasks.workunit.client.1.smithi149.stdout:1/497: dwrite d4/d10/d7c/f98 [0,4194304] 0 2024-03-22T07:08:53.904 INFO:tasks.workunit.client.1.smithi149.stdout:1/498: write d4/d7/f57 [4409506,31405] 0 2024-03-22T07:08:53.904 INFO:tasks.workunit.client.1.smithi149.stdout:7/545: dwrite d1/f5 [0,4194304] 0 2024-03-22T07:08:53.908 INFO:tasks.workunit.client.1.smithi149.stdout:2/626: creat d1/d3/d5/d1f/d5e/d2f/fe4 x:0 0 0 2024-03-22T07:08:53.908 INFO:tasks.workunit.client.1.smithi149.stdout:2/627: write d1/d4d/d69/d75/fd3 [1372816,95644] 0 2024-03-22T07:08:53.914 INFO:tasks.workunit.client.1.smithi149.stdout:4/517: mknod d0/d8/dc/d2e/d6f/caa 0 2024-03-22T07:08:53.920 INFO:tasks.workunit.client.1.smithi149.stdout:1/499: write d4/d10/f4d [2551507,80522] 0 2024-03-22T07:08:53.920 INFO:tasks.workunit.client.1.smithi149.stdout:3/395: dwrite d0/d2/d30/d7e/f69 [0,4194304] 0 2024-03-22T07:08:53.922 INFO:tasks.workunit.client.1.smithi149.stdout:5/514: dwrite d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/fc [0,4194304] 0 2024-03-22T07:08:53.925 INFO:tasks.workunit.client.1.smithi149.stdout:0/444: dwrite d1/d1a/f4f [4194304,4194304] 0 2024-03-22T07:08:53.928 INFO:tasks.workunit.client.1.smithi149.stdout:7/546: rmdir d1/d1c/d49 39 2024-03-22T07:08:53.928 INFO:tasks.workunit.client.1.smithi149.stdout:7/547: readlink d1/l12 0 2024-03-22T07:08:53.928 INFO:tasks.workunit.client.1.smithi149.stdout:7/548: chown d1/fa7 7190 1 2024-03-22T07:08:53.928 INFO:tasks.workunit.client.1.smithi149.stdout:4/518: link d0/d8/dc/f9d d0/d8/dc/d14/d3e/fab 0 2024-03-22T07:08:53.929 INFO:tasks.workunit.client.1.smithi149.stdout:1/500: symlink d4/d7/d94/la9 0 2024-03-22T07:08:53.929 INFO:tasks.workunit.client.1.smithi149.stdout:1/501: fsync d4/d20/d23/f27 0 2024-03-22T07:08:53.929 INFO:tasks.workunit.client.1.smithi149.stdout:2/628: dwrite d1/f10 [4194304,4194304] 0 2024-03-22T07:08:53.930 INFO:tasks.workunit.client.1.smithi149.stdout:5/515: creat d7/da/d20/d30/d3a/d3d/fb4 x:0 0 0 2024-03-22T07:08:53.931 INFO:tasks.workunit.client.1.smithi149.stdout:5/516: write d7/da/d20/d36/f66 [859239,74086] 0 2024-03-22T07:08:53.931 INFO:tasks.workunit.client.1.smithi149.stdout:5/517: write d7/da/f1b [4723320,58513] 0 2024-03-22T07:08:53.933 INFO:tasks.workunit.client.1.smithi149.stdout:3/396: dread d0/d2/d30/d7e/f69 [0,4194304] 0 2024-03-22T07:08:53.937 INFO:tasks.workunit.client.1.smithi149.stdout:0/445: symlink d1/d6b/la4 0 2024-03-22T07:08:53.938 INFO:tasks.workunit.client.1.smithi149.stdout:7/549: creat d1/d18/d1b/d99/d56/d92/d9e/dab/fb2 x:0 0 0 2024-03-22T07:08:53.939 INFO:tasks.workunit.client.1.smithi149.stdout:4/519: symlink d0/d8/dc/d14/d85/d97/lac 0 2024-03-22T07:08:53.943 INFO:tasks.workunit.client.1.smithi149.stdout:2/629: symlink d1/d4d/d69/d75/le5 0 2024-03-22T07:08:53.944 INFO:tasks.workunit.client.1.smithi149.stdout:5/518: creat d7/d35/d8e/fb5 x:0 0 0 2024-03-22T07:08:53.946 INFO:tasks.workunit.client.1.smithi149.stdout:3/397: rename d0/d2/c35 to d0/d2/d30/d5d/c92 0 2024-03-22T07:08:53.949 INFO:tasks.workunit.client.1.smithi149.stdout:4/520: creat d0/d8/dc/d74/d90/fad x:0 0 0 2024-03-22T07:08:53.950 INFO:tasks.workunit.client.1.smithi149.stdout:1/502: getdents d4/d20/d23 0 2024-03-22T07:08:53.950 INFO:tasks.workunit.client.1.smithi149.stdout:3/398: dread d0/d2/d30/f70 [0,4194304] 0 2024-03-22T07:08:53.950 INFO:tasks.workunit.client.1.smithi149.stdout:1/503: write d4/f42 [977622,104442] 0 2024-03-22T07:08:53.950 INFO:tasks.workunit.client.1.smithi149.stdout:1/504: chown d4/d20/d23/d2f/d93/f9c 1 1 2024-03-22T07:08:53.952 INFO:tasks.workunit.client.1.smithi149.stdout:2/630: link d1/d3/d6/db2/l5c d1/d3/d5/d41/le6 0 2024-03-22T07:08:53.952 INFO:tasks.workunit.client.1.smithi149.stdout:5/519: mknod d7/da/d20/d30/d3a/d93/cb6 0 2024-03-22T07:08:53.952 INFO:tasks.workunit.client.1.smithi149.stdout:8/558: sync 2024-03-22T07:08:53.952 INFO:tasks.workunit.client.1.smithi149.stdout:5/520: creat d7/da/d20/d45/fb7 x:0 0 0 2024-03-22T07:08:53.952 INFO:tasks.workunit.client.1.smithi149.stdout:5/521: dread - d7/da/d20/d45/fb7 zero size 2024-03-22T07:08:53.952 INFO:tasks.workunit.client.1.smithi149.stdout:5/522: truncate d7/da/d20/d45/f7c 1209998 0 2024-03-22T07:08:53.954 INFO:tasks.workunit.client.1.smithi149.stdout:0/446: rename d1/d31/d51/d64/c69 to d1/d27/d66/ca5 0 2024-03-22T07:08:53.956 INFO:tasks.workunit.client.1.smithi149.stdout:3/399: mkdir d0/d2/d5a/d93 0 2024-03-22T07:08:53.956 INFO:tasks.workunit.client.1.smithi149.stdout:3/400: write d0/d4/d8/d24/d39/f89 [4718706,103267] 0 2024-03-22T07:08:53.956 INFO:tasks.workunit.client.1.smithi149.stdout:7/550: write d1/d1c/f73 [1079667,96680] 0 2024-03-22T07:08:53.957 INFO:tasks.workunit.client.1.smithi149.stdout:1/505: mkdir d4/d20/d23/d2f/d8c/daa 0 2024-03-22T07:08:53.958 INFO:tasks.workunit.client.1.smithi149.stdout:2/631: creat d1/d3/d5/d1f/d68/d97/fe7 x:0 0 0 2024-03-22T07:08:53.959 INFO:tasks.workunit.client.1.smithi149.stdout:2/632: creat d1/d3/d6/db2/d71/fe8 x:0 0 0 2024-03-22T07:08:53.959 INFO:tasks.workunit.client.1.smithi149.stdout:2/633: fsync d1/d3/d5/d1f/d68/d94/fac 0 2024-03-22T07:08:53.961 INFO:tasks.workunit.client.1.smithi149.stdout:5/523: mknod d7/da/d20/cb8 0 2024-03-22T07:08:53.961 INFO:tasks.workunit.client.1.smithi149.stdout:5/524: fsync d7/da/d20/d30/d3a/d3d/d2c/f88 0 2024-03-22T07:08:53.963 INFO:tasks.workunit.client.1.smithi149.stdout:0/447: mkdir d1/d31/d51/d64/d87/da6 0 2024-03-22T07:08:53.965 INFO:tasks.workunit.client.1.smithi149.stdout:3/401: link d0/d2/f83 d0/d2/d42/d66/d82/f94 0 2024-03-22T07:08:53.966 INFO:tasks.workunit.client.1.smithi149.stdout:7/551: readlink d1/d1c/l64 0 2024-03-22T07:08:53.968 INFO:tasks.workunit.client.1.smithi149.stdout:1/506: mknod d4/d20/d2d/d64/d9f/cab 0 2024-03-22T07:08:53.968 INFO:tasks.workunit.client.1.smithi149.stdout:1/507: write d4/d20/d23/f74 [2585669,114896] 0 2024-03-22T07:08:53.978 INFO:tasks.workunit.client.1.smithi149.stdout:2/634: creat d1/d8/fe9 x:0 0 0 2024-03-22T07:08:53.978 INFO:tasks.workunit.client.1.smithi149.stdout:2/635: chown d1/d3/d5/d41/l83 699569230 1 2024-03-22T07:08:53.978 INFO:tasks.workunit.client.1.smithi149.stdout:2/636: truncate d1/d3/d5/d1f/d5e/f4a 1792118 0 2024-03-22T07:08:53.978 INFO:tasks.workunit.client.1.smithi149.stdout:0/448: mknod d1/d9/d59/ca7 0 2024-03-22T07:08:53.978 INFO:tasks.workunit.client.1.smithi149.stdout:0/449: fsync d1/d9/d14/d1f/f37 0 2024-03-22T07:08:53.978 INFO:tasks.workunit.client.1.smithi149.stdout:7/552: rename d1/d85 to d1/d1c/db3 0 2024-03-22T07:08:53.978 INFO:tasks.workunit.client.1.smithi149.stdout:7/553: fdatasync d1/d18/d1b/d99/f83 0 2024-03-22T07:08:53.978 INFO:tasks.workunit.client.1.smithi149.stdout:7/554: truncate d1/d18/d1b/d99/d56/d92/d9e/dab/fb2 466004 0 2024-03-22T07:08:53.978 INFO:tasks.workunit.client.1.smithi149.stdout:5/525: write d7/f58 [4916935,118540] 0 2024-03-22T07:08:53.979 INFO:tasks.workunit.client.1.smithi149.stdout:0/450: creat d1/d9/d14/fa8 x:0 0 0 2024-03-22T07:08:53.984 INFO:tasks.workunit.client.1.smithi149.stdout:0/451: mkdir d1/d31/d52/da9 0 2024-03-22T07:08:53.985 INFO:tasks.workunit.client.1.smithi149.stdout:7/555: mkdir d1/d18/d1b/d99/d56/db4 0 2024-03-22T07:08:53.985 INFO:tasks.workunit.client.1.smithi149.stdout:7/556: read d1/d1c/d2f/f43 [2297628,84798] 0 2024-03-22T07:08:53.985 INFO:tasks.workunit.client.1.smithi149.stdout:7/557: dread - d1/d18/d1b/d99/faf zero size 2024-03-22T07:08:53.985 INFO:tasks.workunit.client.1.smithi149.stdout:7/558: write d1/d18/d1b/d51/d5a/f78 [2367931,18449] 0 2024-03-22T07:08:53.985 INFO:tasks.workunit.client.1.smithi149.stdout:5/526: creat d7/da/d20/d30/d3a/d3d/d2c/fb9 x:0 0 0 2024-03-22T07:08:53.985 INFO:tasks.workunit.client.1.smithi149.stdout:5/527: chown d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d81/f90 9901724 1 2024-03-22T07:08:53.991 INFO:tasks.workunit.client.1.smithi149.stdout:0/452: dread d1/d27/f6e [0,4194304] 0 2024-03-22T07:08:53.995 INFO:tasks.workunit.client.1.smithi149.stdout:4/521: dwrite d0/f2d [0,4194304] 0 2024-03-22T07:08:53.995 INFO:tasks.workunit.client.1.smithi149.stdout:4/522: chown d0/d8/d1d/f8f 254 1 2024-03-22T07:08:53.995 INFO:tasks.workunit.client.1.smithi149.stdout:4/523: fsync d0/d8/f1c 0 2024-03-22T07:08:53.997 INFO:tasks.workunit.client.1.smithi149.stdout:7/559: dread d1/d1c/d2f/f43 [0,4194304] 0 2024-03-22T07:08:54.003 INFO:tasks.workunit.client.1.smithi149.stdout:7/560: mkdir d1/d18/d30/db5 0 2024-03-22T07:08:54.011 INFO:tasks.workunit.client.1.smithi149.stdout:3/402: dwrite d0/d4/d8/d24/f4f [0,4194304] 0 2024-03-22T07:08:54.017 INFO:tasks.workunit.client.1.smithi149.stdout:1/508: dwrite f1 [4194304,4194304] 0 2024-03-22T07:08:54.017 INFO:tasks.workunit.client.1.smithi149.stdout:3/403: rename d0/cf to d0/d2/d42/c95 0 2024-03-22T07:08:54.017 INFO:tasks.workunit.client.1.smithi149.stdout:3/404: truncate d0/d2/d42/d66/f7a 177674 0 2024-03-22T07:08:54.017 INFO:tasks.workunit.client.1.smithi149.stdout:3/405: dread - d0/d2/d30/d5d/d84/d8f/f37 zero size 2024-03-22T07:08:54.017 INFO:tasks.workunit.client.1.smithi149.stdout:3/406: write d0/d4/d8/d24/d39/f89 [1691275,32918] 0 2024-03-22T07:08:54.017 INFO:tasks.workunit.client.1.smithi149.stdout:3/407: write d0/d2/d42/d66/f7a [607306,56457] 0 2024-03-22T07:08:54.018 INFO:tasks.workunit.client.1.smithi149.stdout:7/561: write d1/f41 [2552524,80313] 0 2024-03-22T07:08:54.021 INFO:tasks.workunit.client.1.smithi149.stdout:7/562: rename d1/d1c/db1 to d1/d18/d1b/d51/d7a/d7d/d5b/d97/db6 0 2024-03-22T07:08:54.022 INFO:tasks.workunit.client.1.smithi149.stdout:7/563: symlink d1/d1c/d2f/d74/lb7 0 2024-03-22T07:08:54.023 INFO:tasks.workunit.client.1.smithi149.stdout:7/564: mkdir d1/db8 0 2024-03-22T07:08:54.023 INFO:tasks.workunit.client.1.smithi149.stdout:7/565: mknod d1/d18/d1b/d51/d7a/d7d/d5b/d97/db6/cb9 0 2024-03-22T07:08:54.024 INFO:tasks.workunit.client.1.smithi149.stdout:7/566: mknod d1/d18/d1b/d1d/d72/cba 0 2024-03-22T07:08:54.024 INFO:tasks.workunit.client.1.smithi149.stdout:7/567: write d1/d18/d1b/d51/d7a/faa [615283,102237] 0 2024-03-22T07:08:54.025 INFO:tasks.workunit.client.1.smithi149.stdout:7/568: creat d1/d1c/fbb x:0 0 0 2024-03-22T07:08:54.025 INFO:tasks.workunit.client.1.smithi149.stdout:7/569: truncate d1/d1c/d49/d8e/d53/f67 568258 0 2024-03-22T07:08:54.028 INFO:tasks.workunit.client.1.smithi149.stdout:2/637: dwrite d1/d4d/d69/d75/fd3 [0,4194304] 0 2024-03-22T07:08:54.031 INFO:tasks.workunit.client.1.smithi149.stdout:1/509: dread d4/d7/d5d/f72 [0,4194304] 0 2024-03-22T07:08:54.031 INFO:tasks.workunit.client.1.smithi149.stdout:1/510: creat d4/d20/d23/d3c/d8e/fac x:0 0 0 2024-03-22T07:08:54.032 INFO:tasks.workunit.client.1.smithi149.stdout:1/511: mknod d4/d20/d23/d2f/d8c/cad 0 2024-03-22T07:08:54.033 INFO:tasks.workunit.client.1.smithi149.stdout:1/512: readlink d4/d7/l2b 0 2024-03-22T07:08:54.033 INFO:tasks.workunit.client.1.smithi149.stdout:1/513: symlink d4/d20/d2d/lae 0 2024-03-22T07:08:54.033 INFO:tasks.workunit.client.1.smithi149.stdout:1/514: readlink d4/lf 0 2024-03-22T07:08:54.033 INFO:tasks.workunit.client.1.smithi149.stdout:1/515: creat d4/d20/d2d/faf x:0 0 0 2024-03-22T07:08:54.037 INFO:tasks.workunit.client.1.smithi149.stdout:5/528: dwrite d7/da/d20/d36/f66 [0,4194304] 0 2024-03-22T07:08:54.037 INFO:tasks.workunit.client.1.smithi149.stdout:5/529: stat d7/da/d20/l85 0 2024-03-22T07:08:54.037 INFO:tasks.workunit.client.1.smithi149.stdout:4/524: dwrite d0/d8/dc/f7c [0,4194304] 0 2024-03-22T07:08:54.037 INFO:tasks.workunit.client.1.smithi149.stdout:4/525: truncate d0/d8/d5c/f5e 460224 0 2024-03-22T07:08:54.038 INFO:tasks.workunit.client.1.smithi149.stdout:0/453: dwrite d1/d31/d51/d64/d87/f9d [0,4194304] 0 2024-03-22T07:08:54.038 INFO:tasks.workunit.client.1.smithi149.stdout:0/454: readlink d1/d6b/l86 0 2024-03-22T07:08:54.039 INFO:tasks.workunit.client.1.smithi149.stdout:2/638: write d1/f4 [7639565,83892] 0 2024-03-22T07:08:54.047 INFO:tasks.workunit.client.1.smithi149.stdout:5/530: dread d7/da/f37 [4194304,4194304] 0 2024-03-22T07:08:54.050 INFO:tasks.workunit.client.1.smithi149.stdout:0/455: creat d1/d31/d52/faa x:0 0 0 2024-03-22T07:08:54.056 INFO:tasks.workunit.client.1.smithi149.stdout:4/526: dwrite d0/d8/dc/d74/d90/fad [0,4194304] 0 2024-03-22T07:08:54.060 INFO:tasks.workunit.client.1.smithi149.stdout:7/570: rename d1/d1c/db3 to d1/d1c/dbc 0 2024-03-22T07:08:54.062 INFO:tasks.workunit.client.1.smithi149.stdout:0/456: creat d1/d20/da3/fab x:0 0 0 2024-03-22T07:08:54.062 INFO:tasks.workunit.client.1.smithi149.stdout:0/457: truncate d1/d9/d14/d1f/f24 5079575 0 2024-03-22T07:08:54.067 INFO:tasks.workunit.client.1.smithi149.stdout:5/531: dwrite d7/da/d20/d36/d84/fa6 [0,4194304] 0 2024-03-22T07:08:54.067 INFO:tasks.workunit.client.1.smithi149.stdout:4/527: symlink d0/d8/dc/d74/lae 0 2024-03-22T07:08:54.068 INFO:tasks.workunit.client.1.smithi149.stdout:0/458: dread d1/d31/f3f [0,4194304] 0 2024-03-22T07:08:54.070 INFO:tasks.workunit.client.1.smithi149.stdout:5/532: dread d7/da/d7d/f9f [0,4194304] 0 2024-03-22T07:08:54.073 INFO:tasks.workunit.client.1.smithi149.stdout:5/533: rmdir d7/d35 39 2024-03-22T07:08:54.080 INFO:tasks.workunit.client.1.smithi149.stdout:7/571: dwrite d1/f5 [0,4194304] 0 2024-03-22T07:08:54.081 INFO:tasks.workunit.client.1.smithi149.stdout:7/572: fdatasync d1/d18/d1b/d51/d5a/fad 0 2024-03-22T07:08:54.082 INFO:tasks.workunit.client.1.smithi149.stdout:7/573: creat d1/d1c/d49/d8e/d53/fbd x:0 0 0 2024-03-22T07:08:54.084 INFO:tasks.workunit.client.1.smithi149.stdout:7/574: symlink d1/d18/d30/db5/lbe 0 2024-03-22T07:08:54.084 INFO:tasks.workunit.client.1.smithi149.stdout:7/575: chown d1/d18/d30/f8b 58994037 1 2024-03-22T07:08:54.084 INFO:tasks.workunit.client.1.smithi149.stdout:7/576: write d1/d1c/f94 [836574,114079] 0 2024-03-22T07:08:54.085 INFO:tasks.workunit.client.1.smithi149.stdout:9/570: sync 2024-03-22T07:08:54.086 INFO:tasks.workunit.client.1.smithi149.stdout:9/571: creat db/d19/d39/d3b/fb5 x:0 0 0 2024-03-22T07:08:54.087 INFO:tasks.workunit.client.1.smithi149.stdout:9/572: chown ca 3199 1 2024-03-22T07:08:54.087 INFO:tasks.workunit.client.1.smithi149.stdout:9/573: fdatasync db/d42/f5a 0 2024-03-22T07:08:54.090 INFO:tasks.workunit.client.1.smithi149.stdout:4/528: dwrite d0/d29/f6e [0,4194304] 0 2024-03-22T07:08:54.090 INFO:tasks.workunit.client.1.smithi149.stdout:4/529: fsync d0/d8/dc/f5b 0 2024-03-22T07:08:54.103 INFO:tasks.workunit.client.1.smithi149.stdout:5/534: dwrite d7/da/d20/d30/d3a/d3d/d2c/f88 [0,4194304] 0 2024-03-22T07:08:54.110 INFO:tasks.workunit.client.1.smithi149.stdout:7/577: dwrite d1/d18/d1b/d1d/f16 [0,4194304] 0 2024-03-22T07:08:54.110 INFO:tasks.workunit.client.1.smithi149.stdout:7/578: fsync d1/d18/d1b/d99/f83 0 2024-03-22T07:08:54.111 INFO:tasks.workunit.client.1.smithi149.stdout:4/530: dwrite d0/d8/dc/f5b [0,4194304] 0 2024-03-22T07:08:54.118 INFO:tasks.workunit.client.1.smithi149.stdout:5/535: write d7/da/f1b [1899895,108917] 0 2024-03-22T07:08:54.118 INFO:tasks.workunit.client.1.smithi149.stdout:5/536: readlink d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/l42 0 2024-03-22T07:08:54.118 INFO:tasks.workunit.client.1.smithi149.stdout:5/537: chown d7/da/d7d/f9e 3491313 1 2024-03-22T07:08:54.121 INFO:tasks.workunit.client.1.smithi149.stdout:4/531: dread d0/d8/d1d/d24/d82/f88 [0,4194304] 0 2024-03-22T07:08:54.121 INFO:tasks.workunit.client.1.smithi149.stdout:5/538: creat d7/da/d20/d36/fba x:0 0 0 2024-03-22T07:08:54.121 INFO:tasks.workunit.client.1.smithi149.stdout:4/532: write d0/f32 [4970638,69406] 0 2024-03-22T07:08:54.121 INFO:tasks.workunit.client.1.smithi149.stdout:4/533: creat d0/d8/d4c/faf x:0 0 0 2024-03-22T07:08:54.122 INFO:tasks.workunit.client.1.smithi149.stdout:5/539: mknod d7/da/d20/cbb 0 2024-03-22T07:08:54.125 INFO:tasks.workunit.client.1.smithi149.stdout:4/534: mkdir d0/d6b/db0 0 2024-03-22T07:08:54.137 INFO:tasks.workunit.client.1.smithi149.stdout:7/579: dwrite d1/d18/d1b/d1d/f1f [0,4194304] 0 2024-03-22T07:08:54.151 INFO:tasks.workunit.client.1.smithi149.stdout:4/535: dwrite d0/d6b/f94 [0,4194304] 0 2024-03-22T07:08:54.151 INFO:tasks.workunit.client.1.smithi149.stdout:5/540: dwrite d7/da/d20/d30/d3a/d93/f9b [0,4194304] 0 2024-03-22T07:08:54.151 INFO:tasks.workunit.client.1.smithi149.stdout:7/580: dread d1/d1c/d49/f80 [0,4194304] 0 2024-03-22T07:08:54.156 INFO:tasks.workunit.client.1.smithi149.stdout:5/541: link d7/da/d20/l85 d7/da/lbc 0 2024-03-22T07:08:54.156 INFO:tasks.workunit.client.1.smithi149.stdout:5/542: creat d7/da/d20/d36/fbd x:0 0 0 2024-03-22T07:08:54.156 INFO:tasks.workunit.client.1.smithi149.stdout:5/543: fdatasync d7/f33 0 2024-03-22T07:08:54.156 INFO:tasks.workunit.client.1.smithi149.stdout:5/544: read d7/da/d20/d45/f7c [523483,33555] 0 2024-03-22T07:08:54.156 INFO:tasks.workunit.client.1.smithi149.stdout:7/581: creat d1/d1c/d2f/d74/fbf x:0 0 0 2024-03-22T07:08:54.159 INFO:tasks.workunit.client.1.smithi149.stdout:5/545: symlink d7/da/d20/d30/d3a/d3d/d2c/d47/lbe 0 2024-03-22T07:08:54.159 INFO:tasks.workunit.client.1.smithi149.stdout:5/546: chown d7/da/d20/c43 0 1 2024-03-22T07:08:54.160 INFO:tasks.workunit.client.1.smithi149.stdout:6/558: sync 2024-03-22T07:08:54.160 INFO:tasks.workunit.client.1.smithi149.stdout:3/408: sync 2024-03-22T07:08:54.160 INFO:tasks.workunit.client.1.smithi149.stdout:1/516: sync 2024-03-22T07:08:54.160 INFO:tasks.workunit.client.1.smithi149.stdout:8/559: sync 2024-03-22T07:08:54.160 INFO:tasks.workunit.client.1.smithi149.stdout:2/639: sync 2024-03-22T07:08:54.160 INFO:tasks.workunit.client.1.smithi149.stdout:1/517: write d4/d7/f50 [299534,47880] 0 2024-03-22T07:08:54.160 INFO:tasks.workunit.client.1.smithi149.stdout:1/518: dread - d4/d20/d23/d3c/d8e/fac zero size 2024-03-22T07:08:54.161 INFO:tasks.workunit.client.1.smithi149.stdout:6/559: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 [235568,25775] 0 2024-03-22T07:08:54.161 INFO:tasks.workunit.client.1.smithi149.stdout:6/560: creat d9/df/d22/fb2 x:0 0 0 2024-03-22T07:08:54.161 INFO:tasks.workunit.client.1.smithi149.stdout:5/547: symlink d7/lbf 0 2024-03-22T07:08:54.161 INFO:tasks.workunit.client.1.smithi149.stdout:2/640: symlink d1/d4d/d90/lea 0 2024-03-22T07:08:54.162 INFO:tasks.workunit.client.1.smithi149.stdout:8/560: symlink d1/d20/d35/d22/d44/d4f/lac 0 2024-03-22T07:08:54.162 INFO:tasks.workunit.client.1.smithi149.stdout:8/561: chown d1/d2/d9/c53 544396493 1 2024-03-22T07:08:54.162 INFO:tasks.workunit.client.1.smithi149.stdout:3/409: symlink d0/d2/d42/d66/l96 0 2024-03-22T07:08:54.163 INFO:tasks.workunit.client.1.smithi149.stdout:1/519: truncate d4/d10/d7c/f98 2187949 0 2024-03-22T07:08:54.163 INFO:tasks.workunit.client.1.smithi149.stdout:7/582: dread d1/d18/d1b/d51/d7a/d7d/f77 [0,4194304] 0 2024-03-22T07:08:54.164 INFO:tasks.workunit.client.1.smithi149.stdout:7/583: stat d1/d1c/f7b 0 2024-03-22T07:08:54.164 INFO:tasks.workunit.client.1.smithi149.stdout:7/584: readlink d1/d1c/d49/d8e/l66 0 2024-03-22T07:08:54.164 INFO:tasks.workunit.client.1.smithi149.stdout:7/585: readlink d1/d18/d1b/l69 0 2024-03-22T07:08:54.165 INFO:tasks.workunit.client.1.smithi149.stdout:5/548: truncate d7/da/d20/d30/d3a/d3d/d2c/f8b 3454079 0 2024-03-22T07:08:54.166 INFO:tasks.workunit.client.1.smithi149.stdout:2/641: mknod d1/d3/d6/d3f/d74/ceb 0 2024-03-22T07:08:54.167 INFO:tasks.workunit.client.1.smithi149.stdout:3/410: symlink d0/d2/d5a/l97 0 2024-03-22T07:08:54.168 INFO:tasks.workunit.client.1.smithi149.stdout:1/520: creat d4/d20/da0/fb0 x:0 0 0 2024-03-22T07:08:54.168 INFO:tasks.workunit.client.1.smithi149.stdout:1/521: chown d4/d10/d28 3221 1 2024-03-22T07:08:54.169 INFO:tasks.workunit.client.1.smithi149.stdout:6/561: rename d9/df/d22/fb2 to d9/df/d2e/d63/fb3 0 2024-03-22T07:08:54.169 INFO:tasks.workunit.client.1.smithi149.stdout:6/562: stat d9/df/d22/d6d/d7d/d96/d24/d34/f4c 0 2024-03-22T07:08:54.172 INFO:tasks.workunit.client.1.smithi149.stdout:2/642: symlink d1/d4d/d6a/lec 0 2024-03-22T07:08:54.173 INFO:tasks.workunit.client.1.smithi149.stdout:1/522: symlink d4/d20/d23/d3c/lb1 0 2024-03-22T07:08:54.174 INFO:tasks.workunit.client.1.smithi149.stdout:5/549: getdents d7/da/d7d 0 2024-03-22T07:08:54.177 INFO:tasks.workunit.client.1.smithi149.stdout:4/536: dwrite d0/d29/f6e [0,4194304] 0 2024-03-22T07:08:54.177 INFO:tasks.workunit.client.1.smithi149.stdout:2/643: rename d1/f10 to d1/d4d/d69/d75/fed 0 2024-03-22T07:08:54.181 INFO:tasks.workunit.client.1.smithi149.stdout:5/550: unlink d7/da/d20/d30/d3a/d3d/d2c/l62 0 2024-03-22T07:08:54.181 INFO:tasks.workunit.client.1.smithi149.stdout:5/551: readlink d7/da/d20/d36/d44/l65 0 2024-03-22T07:08:54.182 INFO:tasks.workunit.client.1.smithi149.stdout:4/537: unlink d0/d8/dc/d14/d85/d97/ca3 0 2024-03-22T07:08:54.184 INFO:tasks.workunit.client.1.smithi149.stdout:3/411: write d0/d2/d30/f3f [747660,113960] 0 2024-03-22T07:08:54.184 INFO:tasks.workunit.client.1.smithi149.stdout:3/412: dread - d0/f72 zero size 2024-03-22T07:08:54.185 INFO:tasks.workunit.client.1.smithi149.stdout:4/538: dread d0/d8/dc/d2e/d6f/f83 [0,4194304] 0 2024-03-22T07:08:54.185 INFO:tasks.workunit.client.1.smithi149.stdout:5/552: creat d7/da/d20/d30/d3a/fc0 x:0 0 0 2024-03-22T07:08:54.185 INFO:tasks.workunit.client.1.smithi149.stdout:2/644: rmdir d1/d4d/d69/d75/d7e 39 2024-03-22T07:08:54.186 INFO:tasks.workunit.client.1.smithi149.stdout:2/645: fdatasync d1/d4d/d6a/de1/faf 0 2024-03-22T07:08:54.187 INFO:tasks.workunit.client.1.smithi149.stdout:3/413: symlink d0/d4/d8/d24/d40/d62/l98 0 2024-03-22T07:08:54.187 INFO:tasks.workunit.client.1.smithi149.stdout:3/414: creat d0/d4/d8/f99 x:0 0 0 2024-03-22T07:08:54.189 INFO:tasks.workunit.client.1.smithi149.stdout:5/553: creat d7/da/d20/d30/d3a/d3d/d2c/d63/fc1 x:0 0 0 2024-03-22T07:08:54.191 INFO:tasks.workunit.client.1.smithi149.stdout:3/415: mknod d0/d4/d8/d24/d39/c9a 0 2024-03-22T07:08:54.192 INFO:tasks.workunit.client.1.smithi149.stdout:3/416: link d0/d2/f2d d0/d4/d6c/f9b 0 2024-03-22T07:08:54.193 INFO:tasks.workunit.client.1.smithi149.stdout:8/562: dwrite d1/d2/d3/f86 [0,4194304] 0 2024-03-22T07:08:54.193 INFO:tasks.workunit.client.1.smithi149.stdout:4/539: write d0/d29/f4a [140383,39295] 0 2024-03-22T07:08:54.193 INFO:tasks.workunit.client.1.smithi149.stdout:8/563: chown d1/d20/c7d 0 1 2024-03-22T07:08:54.193 INFO:tasks.workunit.client.1.smithi149.stdout:4/540: write d0/f5a [2223655,101435] 0 2024-03-22T07:08:54.194 INFO:tasks.workunit.client.1.smithi149.stdout:3/417: getdents d0/d2/d30 0 2024-03-22T07:08:54.196 INFO:tasks.workunit.client.1.smithi149.stdout:4/541: unlink d0/d8/dc/d14/d3e/f3d 0 2024-03-22T07:08:54.197 INFO:tasks.workunit.client.1.smithi149.stdout:3/418: symlink d0/d4/d8/d24/d40/d54/l9c 0 2024-03-22T07:08:54.197 INFO:tasks.workunit.client.1.smithi149.stdout:3/419: readlink d0/d4/l6 0 2024-03-22T07:08:54.199 INFO:tasks.workunit.client.1.smithi149.stdout:4/542: creat d0/d8/dc/d14/d3e/fb1 x:0 0 0 2024-03-22T07:08:54.199 INFO:tasks.workunit.client.1.smithi149.stdout:4/543: read d0/d8/d4c/f64 [4030016,72844] 0 2024-03-22T07:08:54.199 INFO:tasks.workunit.client.1.smithi149.stdout:4/544: fsync d0/d8/dc/d74/f95 0 2024-03-22T07:08:54.199 INFO:tasks.workunit.client.1.smithi149.stdout:4/545: chown d0/d6b/f7d 7 1 2024-03-22T07:08:54.199 INFO:tasks.workunit.client.1.smithi149.stdout:6/563: dwrite d9/df/f1e [0,4194304] 0 2024-03-22T07:08:54.199 INFO:tasks.workunit.client.1.smithi149.stdout:3/420: mkdir d0/d2/d30/d5d/d84/d8e/d9d 0 2024-03-22T07:08:54.201 INFO:tasks.workunit.client.1.smithi149.stdout:4/546: symlink d0/d6b/lb2 0 2024-03-22T07:08:54.203 INFO:tasks.workunit.client.1.smithi149.stdout:1/523: dwrite d4/d7/f39 [0,4194304] 0 2024-03-22T07:08:54.206 INFO:tasks.workunit.client.1.smithi149.stdout:6/564: rename d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f53 to d9/d47/fb4 0 2024-03-22T07:08:54.206 INFO:tasks.workunit.client.1.smithi149.stdout:6/565: creat d9/da2/fb5 x:0 0 0 2024-03-22T07:08:54.207 INFO:tasks.workunit.client.1.smithi149.stdout:4/547: mkdir d0/d8/dc/d4b/db3 0 2024-03-22T07:08:54.207 INFO:tasks.workunit.client.1.smithi149.stdout:4/548: write d0/d6b/f94 [4818944,62205] 0 2024-03-22T07:08:54.207 INFO:tasks.workunit.client.1.smithi149.stdout:4/549: readlink d0/d8/dc/d14/d85/la8 0 2024-03-22T07:08:54.207 INFO:tasks.workunit.client.1.smithi149.stdout:4/550: dread - d0/d8/d4c/faf zero size 2024-03-22T07:08:54.207 INFO:tasks.workunit.client.1.smithi149.stdout:1/524: truncate d4/f1b 3951979 0 2024-03-22T07:08:54.209 INFO:tasks.workunit.client.1.smithi149.stdout:4/551: getdents d0/d8/d1d/d24 0 2024-03-22T07:08:54.209 INFO:tasks.workunit.client.1.smithi149.stdout:4/552: dread - d0/d8/fa2 zero size 2024-03-22T07:08:54.209 INFO:tasks.workunit.client.1.smithi149.stdout:4/553: readlink d0/la 0 2024-03-22T07:08:54.211 INFO:tasks.workunit.client.1.smithi149.stdout:1/525: mkdir d4/d10/d28/db2 0 2024-03-22T07:08:54.211 INFO:tasks.workunit.client.1.smithi149.stdout:1/526: read - d4/d20/d23/d2f/f62 zero size 2024-03-22T07:08:54.212 INFO:tasks.workunit.client.1.smithi149.stdout:1/527: chown d4/d10/f12 59 1 2024-03-22T07:08:54.212 INFO:tasks.workunit.client.1.smithi149.stdout:8/564: dwrite d1/d20/f66 [0,4194304] 0 2024-03-22T07:08:54.219 INFO:tasks.workunit.client.1.smithi149.stdout:6/566: write d9/df/f15 [1428575,84027] 0 2024-03-22T07:08:54.219 INFO:tasks.workunit.client.1.smithi149.stdout:1/528: creat d4/d20/d2d/d64/fb3 x:0 0 0 2024-03-22T07:08:54.220 INFO:tasks.workunit.client.1.smithi149.stdout:8/565: mknod d1/d2/d9/d11/d8b/d8d/cad 0 2024-03-22T07:08:54.223 INFO:tasks.workunit.client.1.smithi149.stdout:1/529: dread d4/f11 [0,4194304] 0 2024-03-22T07:08:54.223 INFO:tasks.workunit.client.1.smithi149.stdout:8/566: mknod d1/d2/d9/d11/d8b/cae 0 2024-03-22T07:08:54.225 INFO:tasks.workunit.client.1.smithi149.stdout:3/421: dwrite d0/d2/d30/f3f [0,4194304] 0 2024-03-22T07:08:54.225 INFO:tasks.workunit.client.1.smithi149.stdout:1/530: rmdir d4/d10/d58/d68/da5 0 2024-03-22T07:08:54.229 INFO:tasks.workunit.client.1.smithi149.stdout:3/422: mkdir d0/d4/d8/d24/d40/d54/d9e 0 2024-03-22T07:08:54.230 INFO:tasks.workunit.client.1.smithi149.stdout:3/423: creat d0/d2/d30/f9f x:0 0 0 2024-03-22T07:08:54.230 INFO:tasks.workunit.client.1.smithi149.stdout:3/424: dread - d0/d4/d8/d24/d40/d54/d85/f86 zero size 2024-03-22T07:08:54.230 INFO:tasks.workunit.client.1.smithi149.stdout:3/425: write d0/d4/d8/d24/d40/d54/d85/f91 [298279,8521] 0 2024-03-22T07:08:54.230 INFO:tasks.workunit.client.1.smithi149.stdout:3/426: fsync d0/d2/f2d 0 2024-03-22T07:08:54.230 INFO:tasks.workunit.client.1.smithi149.stdout:4/554: dwrite d0/d8/d4c/f57 [0,4194304] 0 2024-03-22T07:08:54.232 INFO:tasks.workunit.client.1.smithi149.stdout:6/567: dread d9/df/d22/d6d/f78 [0,4194304] 0 2024-03-22T07:08:54.232 INFO:tasks.workunit.client.1.smithi149.stdout:4/555: dread d0/d8/dc/d2e/d6f/f83 [0,4194304] 0 2024-03-22T07:08:54.232 INFO:tasks.workunit.client.1.smithi149.stdout:3/427: mkdir d0/d4/d8/d24/d40/d54/da0 0 2024-03-22T07:08:54.237 INFO:tasks.workunit.client.1.smithi149.stdout:6/568: rename d9/df/d2e/f8d to d9/df/d22/d6d/d7d/fb6 0 2024-03-22T07:08:54.237 INFO:tasks.workunit.client.1.smithi149.stdout:6/569: creat d9/df/d22/d6d/d7d/d96/d24/d42/fb7 x:0 0 0 2024-03-22T07:08:54.238 INFO:tasks.workunit.client.1.smithi149.stdout:3/428: mknod d0/d4/d8/d24/d40/d54/d85/ca1 0 2024-03-22T07:08:54.238 INFO:tasks.workunit.client.1.smithi149.stdout:3/429: chown d0/d2/d30/d31/c4b 110902 1 2024-03-22T07:08:54.242 INFO:tasks.workunit.client.1.smithi149.stdout:6/570: symlink d9/lb8 0 2024-03-22T07:08:54.242 INFO:tasks.workunit.client.1.smithi149.stdout:3/430: creat d0/d2/d42/fa2 x:0 0 0 2024-03-22T07:08:54.244 INFO:tasks.workunit.client.1.smithi149.stdout:3/431: mkdir d0/d2/d30/d7e/da3 0 2024-03-22T07:08:54.244 INFO:tasks.workunit.client.1.smithi149.stdout:3/432: chown d0/d4/d8/l44 1808 1 2024-03-22T07:08:54.244 INFO:tasks.workunit.client.1.smithi149.stdout:0/459: sync 2024-03-22T07:08:54.245 INFO:tasks.workunit.client.1.smithi149.stdout:3/433: mknod d0/d2/d42/d66/ca4 0 2024-03-22T07:08:54.246 INFO:tasks.workunit.client.1.smithi149.stdout:0/460: mknod d1/d9/d59/cac 0 2024-03-22T07:08:54.247 INFO:tasks.workunit.client.1.smithi149.stdout:3/434: creat d0/d4/d8/d24/d40/fa5 x:0 0 0 2024-03-22T07:08:54.251 INFO:tasks.workunit.client.1.smithi149.stdout:0/461: mknod d1/d31/d51/d64/d87/da6/cad 0 2024-03-22T07:08:54.251 INFO:tasks.workunit.client.1.smithi149.stdout:0/462: chown d1/d6b/f92 36401753 1 2024-03-22T07:08:54.251 INFO:tasks.workunit.client.1.smithi149.stdout:0/463: creat d1/d20/da3/fae x:0 0 0 2024-03-22T07:08:54.251 INFO:tasks.workunit.client.1.smithi149.stdout:0/464: readlink d1/d31/l3b 0 2024-03-22T07:08:54.252 INFO:tasks.workunit.client.1.smithi149.stdout:1/531: dread d4/f1b [0,4194304] 0 2024-03-22T07:08:54.252 INFO:tasks.workunit.client.1.smithi149.stdout:4/556: dwrite d0/d8/d4c/f57 [0,4194304] 0 2024-03-22T07:08:54.252 INFO:tasks.workunit.client.1.smithi149.stdout:4/557: getdents d0/d8/dc/d4b/db3 0 2024-03-22T07:08:54.252 INFO:tasks.workunit.client.1.smithi149.stdout:3/435: symlink d0/d2/d30/d7e/la6 0 2024-03-22T07:08:54.254 INFO:tasks.workunit.client.1.smithi149.stdout:6/571: write d9/f19 [707966,43144] 0 2024-03-22T07:08:54.254 INFO:tasks.workunit.client.1.smithi149.stdout:8/567: dwrite d1/d2/d9/f4a [0,4194304] 0 2024-03-22T07:08:54.257 INFO:tasks.workunit.client.1.smithi149.stdout:1/532: unlink d4/d20/d23/d2f/d8c/cad 0 2024-03-22T07:08:54.258 INFO:tasks.workunit.client.1.smithi149.stdout:3/436: symlink d0/d4/d8/d24/d40/la7 0 2024-03-22T07:08:54.261 INFO:tasks.workunit.client.1.smithi149.stdout:6/572: truncate d9/df/d2e/f1c 1178628 0 2024-03-22T07:08:54.263 INFO:tasks.workunit.client.1.smithi149.stdout:6/573: write d9/df/f91 [261405,73925] 0 2024-03-22T07:08:54.263 INFO:tasks.workunit.client.1.smithi149.stdout:6/574: chown d9/f17 0 1 2024-03-22T07:08:54.263 INFO:tasks.workunit.client.1.smithi149.stdout:6/575: readlink d9/df/d22/d6d/d7d/l81 0 2024-03-22T07:08:54.263 INFO:tasks.workunit.client.1.smithi149.stdout:8/568: mkdir d1/d2/d9/d11/d8b/daf 0 2024-03-22T07:08:54.263 INFO:tasks.workunit.client.1.smithi149.stdout:6/576: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/fb9 x:0 0 0 2024-03-22T07:08:54.263 INFO:tasks.workunit.client.1.smithi149.stdout:8/569: chown d1/d2/d9/d11/d45/f70 432552 1 2024-03-22T07:08:54.267 INFO:tasks.workunit.client.1.smithi149.stdout:0/465: write d1/d9/d14/f76 [4110459,123801] 0 2024-03-22T07:08:54.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/577: mknod d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/cba 0 2024-03-22T07:08:54.269 INFO:tasks.workunit.client.1.smithi149.stdout:4/558: dread d0/d8/dc/f18 [0,4194304] 0 2024-03-22T07:08:54.269 INFO:tasks.workunit.client.1.smithi149.stdout:4/559: dread - d0/d8/fa2 zero size 2024-03-22T07:08:54.269 INFO:tasks.workunit.client.1.smithi149.stdout:8/570: rename d1/d2/f23 to d1/d2/d3/fb0 0 2024-03-22T07:08:54.269 INFO:tasks.workunit.client.1.smithi149.stdout:8/571: write d1/d2/d9/d11/f58 [4477280,97487] 0 2024-03-22T07:08:54.270 INFO:tasks.workunit.client.1.smithi149.stdout:4/560: mkdir d0/d8/d4c/db4 0 2024-03-22T07:08:54.274 INFO:tasks.workunit.client.1.smithi149.stdout:4/561: symlink d0/d8/d1d/lb5 0 2024-03-22T07:08:54.274 INFO:tasks.workunit.client.1.smithi149.stdout:4/562: fdatasync d0/d8/dc/d14/d3e/fab 0 2024-03-22T07:08:54.277 INFO:tasks.workunit.client.1.smithi149.stdout:0/466: rename d1/d6b/ca0 to d1/d1a/caf 0 2024-03-22T07:08:54.280 INFO:tasks.workunit.client.1.smithi149.stdout:6/578: rename d9/df/d22/d6d/d7d/d96/d24/d34/f7e to d9/df/d22/d6d/d7d/d96/db1/fbb 0 2024-03-22T07:08:54.280 INFO:tasks.workunit.client.1.smithi149.stdout:6/579: stat d9/df/d22/c3d 0 2024-03-22T07:08:54.284 INFO:tasks.workunit.client.1.smithi149.stdout:8/572: rename d1/d2/d9/f65 to d1/d2/d9/d11/d8b/fb1 0 2024-03-22T07:08:54.284 INFO:tasks.workunit.client.1.smithi149.stdout:8/573: write d1/d2/d9/d11/d4d/f5e [1446413,5654] 0 2024-03-22T07:08:54.284 INFO:tasks.workunit.client.1.smithi149.stdout:8/574: getdents d1/d20/d35/d22/d44/d4f/d9e 0 2024-03-22T07:08:54.286 INFO:tasks.workunit.client.1.smithi149.stdout:8/575: mkdir d1/d2/d9/db2 0 2024-03-22T07:08:54.288 INFO:tasks.workunit.client.1.smithi149.stdout:8/576: mknod d1/d2/d9/d11/d8b/daf/cb3 0 2024-03-22T07:08:54.289 INFO:tasks.workunit.client.1.smithi149.stdout:6/580: write d9/df/d22/d6d/d7d/d96/f41 [2725418,86818] 0 2024-03-22T07:08:54.289 INFO:tasks.workunit.client.1.smithi149.stdout:6/581: fdatasync d9/df/d22/d6d/d7d/d96/d24/d68/d73/f8b 0 2024-03-22T07:08:54.290 INFO:tasks.workunit.client.1.smithi149.stdout:6/582: getdents d9/df/d22/d6d/d7d/d96/d24/d68 0 2024-03-22T07:08:54.290 INFO:tasks.workunit.client.1.smithi149.stdout:6/583: chown d9/df/d22/d6d/d7d/d96/db1/fbb 3541049 1 2024-03-22T07:08:54.290 INFO:tasks.workunit.client.1.smithi149.stdout:6/584: fsync d9/df/d22/d6d/d7d/d96/d24/d42/fad 0 2024-03-22T07:08:54.290 INFO:tasks.workunit.client.1.smithi149.stdout:0/467: dread d1/d31/d51/d64/d87/f9d [0,4194304] 0 2024-03-22T07:08:54.290 INFO:tasks.workunit.client.1.smithi149.stdout:9/574: sync 2024-03-22T07:08:54.294 INFO:tasks.workunit.client.1.smithi149.stdout:0/468: rmdir d1/d31/d51/d64 39 2024-03-22T07:08:54.296 INFO:tasks.workunit.client.1.smithi149.stdout:0/469: truncate d1/d1a/f2d 180519 0 2024-03-22T07:08:54.299 INFO:tasks.workunit.client.1.smithi149.stdout:0/470: link d1/f4a d1/d31/d51/d64/d87/fb0 0 2024-03-22T07:08:54.301 INFO:tasks.workunit.client.1.smithi149.stdout:0/471: getdents d1/d31/d51 0 2024-03-22T07:08:54.302 INFO:tasks.workunit.client.1.smithi149.stdout:0/472: fdatasync d1/d1a/f50 0 2024-03-22T07:08:54.302 INFO:tasks.workunit.client.1.smithi149.stdout:0/473: readlink d1/d20/l7d 0 2024-03-22T07:08:54.306 INFO:tasks.workunit.client.1.smithi149.stdout:3/437: dwrite d0/d2/d30/d5d/d84/d8f/f37 [0,4194304] 0 2024-03-22T07:08:54.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:54 smithi081 ceph-mon[29245]: Adjusting osd_memory_target on smithi081 to 1557M 2024-03-22T07:08:54.325 INFO:tasks.workunit.client.1.smithi149.stdout:4/563: dwrite d0/d8/d1d/d24/d82/fa1 [0,4194304] 0 2024-03-22T07:08:54.325 INFO:tasks.workunit.client.1.smithi149.stdout:1/533: dwrite d4/d7/d5a/f6d [0,4194304] 0 2024-03-22T07:08:54.325 INFO:tasks.workunit.client.1.smithi149.stdout:1/534: stat d4/d20/d23/d3c/d8e/fac 0 2024-03-22T07:08:54.325 INFO:tasks.workunit.client.1.smithi149.stdout:4/564: truncate d0/d8/dc/f9d 282624 0 2024-03-22T07:08:54.325 INFO:tasks.workunit.client.1.smithi149.stdout:1/535: dread - d4/d20/d23/d2f/d93/f9c zero size 2024-03-22T07:08:54.325 INFO:tasks.workunit.client.1.smithi149.stdout:1/536: write d4/d20/f73 [986526,94375] 0 2024-03-22T07:08:54.325 INFO:tasks.workunit.client.1.smithi149.stdout:1/537: creat d4/d7/fb4 x:0 0 0 2024-03-22T07:08:54.329 INFO:tasks.workunit.client.1.smithi149.stdout:6/585: dwrite d9/df/d22/d6d/d7d/d96/f6b [0,4194304] 0 2024-03-22T07:08:54.330 INFO:tasks.workunit.client.1.smithi149.stdout:8/577: dwrite d1/d20/d35/d22/f38 [4194304,4194304] 0 2024-03-22T07:08:54.330 INFO:tasks.workunit.client.1.smithi149.stdout:9/575: dwrite db/d42/f4c [0,4194304] 0 2024-03-22T07:08:54.330 INFO:tasks.workunit.client.1.smithi149.stdout:9/576: stat db/d19/d28 0 2024-03-22T07:08:54.332 INFO:tasks.workunit.client.1.smithi149.stdout:0/474: dwrite d1/d27/f62 [0,4194304] 0 2024-03-22T07:08:54.332 INFO:tasks.workunit.client.1.smithi149.stdout:0/475: fdatasync d1/d9/d14/d74/d77/f9a 0 2024-03-22T07:08:54.334 INFO:tasks.workunit.client.1.smithi149.stdout:8/578: read d1/d2/d9/f37 [43244,19543] 0 2024-03-22T07:08:54.334 INFO:tasks.workunit.client.1.smithi149.stdout:8/579: write d1/d20/d35/d22/d44/d4f/f56 [5218928,28872] 0 2024-03-22T07:08:54.334 INFO:tasks.workunit.client.1.smithi149.stdout:8/580: write d1/d2/d18/f3c [622361,41263] 0 2024-03-22T07:08:54.341 INFO:tasks.workunit.client.1.smithi149.stdout:9/577: dread db/d19/f3c [0,4194304] 0 2024-03-22T07:08:54.345 INFO:tasks.workunit.client.1.smithi149.stdout:5/554: sync 2024-03-22T07:08:54.345 INFO:tasks.workunit.client.1.smithi149.stdout:0/476: creat d1/fb1 x:0 0 0 2024-03-22T07:08:54.345 INFO:tasks.workunit.client.1.smithi149.stdout:2/646: sync 2024-03-22T07:08:54.345 INFO:tasks.workunit.client.1.smithi149.stdout:0/477: chown d1/d6b/la4 4401636 1 2024-03-22T07:08:54.345 INFO:tasks.workunit.client.1.smithi149.stdout:7/586: sync 2024-03-22T07:08:54.346 INFO:tasks.workunit.client.1.smithi149.stdout:4/565: dwrite d0/d8/d1d/f8f [0,4194304] 0 2024-03-22T07:08:54.346 INFO:tasks.workunit.client.1.smithi149.stdout:6/586: dread d9/df/d22/f89 [0,4194304] 0 2024-03-22T07:08:54.346 INFO:tasks.workunit.client.1.smithi149.stdout:6/587: creat d9/da2/fbc x:0 0 0 2024-03-22T07:08:54.346 INFO:tasks.workunit.client.1.smithi149.stdout:8/581: mkdir d1/d20/db4 0 2024-03-22T07:08:54.346 INFO:tasks.workunit.client.1.smithi149.stdout:8/582: chown d1/d20/d51/c81 14 1 2024-03-22T07:08:54.347 INFO:tasks.workunit.client.1.smithi149.stdout:9/578: unlink db/d21/d37/d3a/fa4 0 2024-03-22T07:08:54.348 INFO:tasks.workunit.client.1.smithi149.stdout:9/579: write db/d19/d28/f84 [851714,95692] 0 2024-03-22T07:08:54.351 INFO:tasks.workunit.client.1.smithi149.stdout:5/555: mkdir d7/da/d20/d30/d3a/d93/d94/dc2 0 2024-03-22T07:08:54.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:54 smithi149 ceph-mon[36068]: Adjusting osd_memory_target on smithi081 to 1557M 2024-03-22T07:08:54.352 INFO:tasks.workunit.client.1.smithi149.stdout:2/647: mkdir d1/d3/d9a/dee 0 2024-03-22T07:08:54.353 INFO:tasks.workunit.client.1.smithi149.stdout:0/478: creat d1/d9/d14/d74/d77/fb2 x:0 0 0 2024-03-22T07:08:54.355 INFO:tasks.workunit.client.1.smithi149.stdout:7/587: mkdir d1/d1c/d49/dc0 0 2024-03-22T07:08:54.355 INFO:tasks.workunit.client.1.smithi149.stdout:7/588: truncate d1/d18/d30/f45 301549 0 2024-03-22T07:08:54.356 INFO:tasks.workunit.client.1.smithi149.stdout:4/566: unlink d0/c6c 0 2024-03-22T07:08:54.358 INFO:tasks.workunit.client.1.smithi149.stdout:6/588: symlink d9/df/d22/d6d/d7d/d96/lbd 0 2024-03-22T07:08:54.358 INFO:tasks.workunit.client.1.smithi149.stdout:6/589: write d9/df/fab [1039876,59204] 0 2024-03-22T07:08:54.359 INFO:tasks.workunit.client.1.smithi149.stdout:4/567: write d0/d8/dc/fe [1027017,28409] 0 2024-03-22T07:08:54.359 INFO:tasks.workunit.client.1.smithi149.stdout:4/568: stat d0/d6b/l89 0 2024-03-22T07:08:54.359 INFO:tasks.workunit.client.1.smithi149.stdout:4/569: write d0/d8/f21 [212739,31600] 0 2024-03-22T07:08:54.360 INFO:tasks.workunit.client.1.smithi149.stdout:8/583: mknod d1/d20/cb5 0 2024-03-22T07:08:54.360 INFO:tasks.workunit.client.1.smithi149.stdout:8/584: chown d1/d2/d9/df/c7b 3683 1 2024-03-22T07:08:54.362 INFO:tasks.workunit.client.1.smithi149.stdout:1/538: rename d4/d10/d28 to d4/d20/d23/d3c/d8e/db5 0 2024-03-22T07:08:54.362 INFO:tasks.workunit.client.1.smithi149.stdout:5/556: mknod d7/da/d20/d30/d3a/d3d/d2c/d63/d92/cc3 0 2024-03-22T07:08:54.364 INFO:tasks.workunit.client.1.smithi149.stdout:2/648: creat d1/d3/d9a/fef x:0 0 0 2024-03-22T07:08:54.364 INFO:tasks.workunit.client.1.smithi149.stdout:2/649: write d1/d8/fb5 [251453,4597] 0 2024-03-22T07:08:54.364 INFO:tasks.workunit.client.1.smithi149.stdout:2/650: truncate d1/d3/d6/f92 739277 0 2024-03-22T07:08:54.364 INFO:tasks.workunit.client.1.smithi149.stdout:2/651: getdents d1/d3/d9a/dee 0 2024-03-22T07:08:54.364 INFO:tasks.workunit.client.1.smithi149.stdout:2/652: stat d1/d4d/d69/d75/fd3 0 2024-03-22T07:08:54.364 INFO:tasks.workunit.client.1.smithi149.stdout:2/653: readlink d1/d3/d5/d41/le6 0 2024-03-22T07:08:54.366 INFO:tasks.workunit.client.1.smithi149.stdout:0/479: unlink d1/d9/d14/l2e 0 2024-03-22T07:08:54.366 INFO:tasks.workunit.client.1.smithi149.stdout:9/580: dwrite db/d21/f23 [0,4194304] 0 2024-03-22T07:08:54.366 INFO:tasks.workunit.client.1.smithi149.stdout:0/480: write d1/fb1 [810778,23228] 0 2024-03-22T07:08:54.366 INFO:tasks.workunit.client.1.smithi149.stdout:9/581: creat db/d19/d1c/d85/fb6 x:0 0 0 2024-03-22T07:08:54.366 INFO:tasks.workunit.client.1.smithi149.stdout:9/582: write db/d19/d39/d3b/f53 [4898114,11194] 0 2024-03-22T07:08:54.366 INFO:tasks.workunit.client.1.smithi149.stdout:9/583: write db/d12/f14 [1758430,13861] 0 2024-03-22T07:08:54.368 INFO:tasks.workunit.client.1.smithi149.stdout:7/589: truncate d1/d1c/f7b 348414 0 2024-03-22T07:08:54.370 INFO:tasks.workunit.client.1.smithi149.stdout:6/590: truncate d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/fae 2840655 0 2024-03-22T07:08:54.371 INFO:tasks.workunit.client.1.smithi149.stdout:6/591: truncate d9/da2/fbc 548220 0 2024-03-22T07:08:54.373 INFO:tasks.workunit.client.1.smithi149.stdout:8/585: symlink d1/d2/d9/lb6 0 2024-03-22T07:08:54.373 INFO:tasks.workunit.client.1.smithi149.stdout:1/539: mknod d4/d20/d23/d3c/cb6 0 2024-03-22T07:08:54.376 INFO:tasks.workunit.client.1.smithi149.stdout:2/654: dread d1/d3/fdd [0,4194304] 0 2024-03-22T07:08:54.376 INFO:tasks.workunit.client.1.smithi149.stdout:2/655: chown d1/d8/d34/c67 3804 1 2024-03-22T07:08:54.376 INFO:tasks.workunit.client.1.smithi149.stdout:6/592: link d9/df/d22/d6d/d7d/d96/d24/d34/d37/c83 d9/df/d2e/d4e/cbe 0 2024-03-22T07:08:54.377 INFO:tasks.workunit.client.1.smithi149.stdout:6/593: symlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/lbf 0 2024-03-22T07:08:54.380 INFO:tasks.workunit.client.1.smithi149.stdout:8/586: dread d1/d2/d9/df/f2d [0,4194304] 0 2024-03-22T07:08:54.382 INFO:tasks.workunit.client.1.smithi149.stdout:8/587: creat d1/d20/d35/d9d/fb7 x:0 0 0 2024-03-22T07:08:54.384 INFO:tasks.workunit.client.1.smithi149.stdout:6/594: read d9/df/d22/d6d/d7d/fb6 [87785,14667] 0 2024-03-22T07:08:54.384 INFO:tasks.workunit.client.1.smithi149.stdout:6/595: chown d9/df/d22/d6d/d7d/d96/d24/d42/l64 5939978 1 2024-03-22T07:08:54.384 INFO:tasks.workunit.client.1.smithi149.stdout:6/596: chown d9/df/d22/d6d/d7d/d96/d24/d68/laa 15 1 2024-03-22T07:08:54.384 INFO:tasks.workunit.client.1.smithi149.stdout:8/588: mknod d1/d2/d9/d11/d4d/cb8 0 2024-03-22T07:08:54.384 INFO:tasks.workunit.client.1.smithi149.stdout:8/589: fsync d1/d20/d35/d22/f38 0 2024-03-22T07:08:54.395 INFO:tasks.workunit.client.1.smithi149.stdout:1/540: dread d4/d20/d23/f74 [0,4194304] 0 2024-03-22T07:08:54.395 INFO:tasks.workunit.client.1.smithi149.stdout:1/541: truncate d4/d20/f22 66187 0 2024-03-22T07:08:54.399 INFO:tasks.workunit.client.1.smithi149.stdout:1/542: link d4/d20/d2d/l38 d4/d7/d5a/lb7 0 2024-03-22T07:08:54.399 INFO:tasks.workunit.client.1.smithi149.stdout:1/543: write d4/d20/d23/d2f/f35 [309874,78712] 0 2024-03-22T07:08:54.400 INFO:tasks.workunit.client.1.smithi149.stdout:1/544: read - d4/d20/d2d/faf zero size 2024-03-22T07:08:54.401 INFO:tasks.workunit.client.1.smithi149.stdout:0/481: write d1/d9/d14/fa1 [3039204,102227] 0 2024-03-22T07:08:54.402 INFO:tasks.workunit.client.1.smithi149.stdout:1/545: link d4/d20/d23/f6f d4/d20/d23/d3c/d8e/db5/fb8 0 2024-03-22T07:08:54.405 INFO:tasks.workunit.client.1.smithi149.stdout:0/482: mknod d1/d9/d14/d1f/cb3 0 2024-03-22T07:08:54.406 INFO:tasks.workunit.client.1.smithi149.stdout:0/483: mkdir d1/db4 0 2024-03-22T07:08:54.406 INFO:tasks.workunit.client.1.smithi149.stdout:0/484: dread - d1/d9/d14/d1f/f38 zero size 2024-03-22T07:08:54.407 INFO:tasks.workunit.client.1.smithi149.stdout:0/485: getdents d1/d27 0 2024-03-22T07:08:54.407 INFO:tasks.workunit.client.1.smithi149.stdout:0/486: creat d1/d31/d51/d64/d8d/d9c/fb5 x:0 0 0 2024-03-22T07:08:54.407 INFO:tasks.workunit.client.1.smithi149.stdout:0/487: dread - d1/d31/d52/faa zero size 2024-03-22T07:08:54.410 INFO:tasks.workunit.client.1.smithi149.stdout:1/546: dread d4/d10/d7c/f98 [0,4194304] 0 2024-03-22T07:08:54.410 INFO:tasks.workunit.client.1.smithi149.stdout:1/547: write d4/d20/d23/d2f/f62 [739050,93322] 0 2024-03-22T07:08:54.410 INFO:tasks.workunit.client.1.smithi149.stdout:1/548: fdatasync d4/d7/f33 0 2024-03-22T07:08:54.411 INFO:tasks.workunit.client.1.smithi149.stdout:1/549: mknod d4/d7/cb9 0 2024-03-22T07:08:54.412 INFO:tasks.workunit.client.1.smithi149.stdout:1/550: unlink d4/d20/d23/d2f/d8c/l99 0 2024-03-22T07:08:54.413 INFO:tasks.workunit.client.1.smithi149.stdout:1/551: rmdir d4/d20/d23/d3c/d85 39 2024-03-22T07:08:54.413 INFO:tasks.workunit.client.1.smithi149.stdout:1/552: write d4/d10/f12 [456064,28712] 0 2024-03-22T07:08:54.414 INFO:tasks.workunit.client.1.smithi149.stdout:1/553: link d4/d20/d23/d59/c7b d4/d20/d23/d2f/d93/cba 0 2024-03-22T07:08:54.414 INFO:tasks.workunit.client.1.smithi149.stdout:1/554: chown c2 1318694263 1 2024-03-22T07:08:54.417 INFO:tasks.workunit.client.1.smithi149.stdout:1/555: dread d4/d10/d7c/f98 [0,4194304] 0 2024-03-22T07:08:54.419 INFO:tasks.workunit.client.1.smithi149.stdout:1/556: rename d4/d20/d23/d3c/d8e/db5/f2a to d4/d20/d2d/d64/d9f/fbb 0 2024-03-22T07:08:54.419 INFO:tasks.workunit.client.1.smithi149.stdout:1/557: chown d4/f1b 220831456 1 2024-03-22T07:08:54.420 INFO:tasks.workunit.client.1.smithi149.stdout:0/488: read d1/d9/f1e [3008436,129220] 0 2024-03-22T07:08:54.421 INFO:tasks.workunit.client.1.smithi149.stdout:0/489: chown d1/d9/d14/c30 9 1 2024-03-22T07:08:54.421 INFO:tasks.workunit.client.1.smithi149.stdout:9/584: dwrite db/d21/d2e/f99 [0,4194304] 0 2024-03-22T07:08:54.421 INFO:tasks.workunit.client.1.smithi149.stdout:9/585: write db/d42/d49/d4b/d4d/f66 [718664,47110] 0 2024-03-22T07:08:54.423 INFO:tasks.workunit.client.1.smithi149.stdout:0/490: mknod d1/d31/d52/da9/cb6 0 2024-03-22T07:08:54.425 INFO:tasks.workunit.client.1.smithi149.stdout:9/586: unlink db/d21/d74/faf 0 2024-03-22T07:08:54.426 INFO:tasks.workunit.client.1.smithi149.stdout:0/491: symlink d1/d9/d14/lb7 0 2024-03-22T07:08:54.426 INFO:tasks.workunit.client.1.smithi149.stdout:0/492: fdatasync d1/d9/d14/d74/f81 0 2024-03-22T07:08:54.426 INFO:tasks.workunit.client.1.smithi149.stdout:9/587: stat db/d19/d28/cb2 0 2024-03-22T07:08:54.428 INFO:tasks.workunit.client.1.smithi149.stdout:9/588: creat db/d19/d39/d68/d6f/fb7 x:0 0 0 2024-03-22T07:08:54.428 INFO:tasks.workunit.client.1.smithi149.stdout:9/589: chown db/d21/d37/d3a/c41 11412 1 2024-03-22T07:08:54.428 INFO:tasks.workunit.client.1.smithi149.stdout:9/590: write db/d19/f94 [1742665,106876] 0 2024-03-22T07:08:54.428 INFO:tasks.workunit.client.1.smithi149.stdout:4/570: dwrite d0/d8/dc/d14/f75 [0,4194304] 0 2024-03-22T07:08:54.429 INFO:tasks.workunit.client.1.smithi149.stdout:2/656: dwrite d1/d4d/d69/d75/fd3 [0,4194304] 0 2024-03-22T07:08:54.429 INFO:tasks.workunit.client.1.smithi149.stdout:5/557: dwrite d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d2b/f73 [0,4194304] 0 2024-03-22T07:08:54.429 INFO:tasks.workunit.client.1.smithi149.stdout:5/558: read - d7/da/d20/d30/d3a/d3d/fb4 zero size 2024-03-22T07:08:54.429 INFO:tasks.workunit.client.1.smithi149.stdout:7/590: dwrite d1/d1c/f9f [0,4194304] 0 2024-03-22T07:08:54.429 INFO:tasks.workunit.client.1.smithi149.stdout:5/559: write d7/da/d20/d30/d3a/d3d/d2c/d63/fc1 [956997,35566] 0 2024-03-22T07:08:54.430 INFO:tasks.workunit.client.1.smithi149.stdout:6/597: dwrite d9/df/d22/d6d/d7d/d96/d24/fa9 [0,4194304] 0 2024-03-22T07:08:54.430 INFO:tasks.workunit.client.1.smithi149.stdout:6/598: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 [1164182,13273] 0 2024-03-22T07:08:54.433 INFO:tasks.workunit.client.1.smithi149.stdout:8/590: dwrite d1/d2/d9/d11/f77 [4194304,4194304] 0 2024-03-22T07:08:54.434 INFO:tasks.workunit.client.1.smithi149.stdout:2/657: unlink d1/d4d/d90/lea 0 2024-03-22T07:08:54.434 INFO:tasks.workunit.client.1.smithi149.stdout:2/658: readlink d1/d4d/d69/d75/db8/lc0 0 2024-03-22T07:08:54.440 INFO:tasks.workunit.client.1.smithi149.stdout:6/599: truncate d9/df/d22/d6d/d7d/d96/db1/fbb 1518343 0 2024-03-22T07:08:54.444 INFO:tasks.workunit.client.1.smithi149.stdout:7/591: write d1/f5 [158830,30263] 0 2024-03-22T07:08:54.446 INFO:tasks.workunit.client.1.smithi149.stdout:9/591: dread db/d21/d74/f76 [0,4194304] 0 2024-03-22T07:08:54.446 INFO:tasks.workunit.client.1.smithi149.stdout:9/592: fdatasync db/d19/d39/d3b/fb5 0 2024-03-22T07:08:54.451 INFO:tasks.workunit.client.1.smithi149.stdout:4/571: dread d0/d8/dc/f7c [0,4194304] 0 2024-03-22T07:08:54.461 INFO:tasks.workunit.client.1.smithi149.stdout:1/558: dwrite d4/d10/d58/f9b [0,4194304] 0 2024-03-22T07:08:54.463 INFO:tasks.workunit.client.1.smithi149.stdout:2/659: mknod d1/d4d/cf0 0 2024-03-22T07:08:54.463 INFO:tasks.workunit.client.1.smithi149.stdout:2/660: chown d1/d4d/d69/d75/le5 67976083 1 2024-03-22T07:08:54.464 INFO:tasks.workunit.client.1.smithi149.stdout:5/560: rmdir d7/d35/d8e 39 2024-03-22T07:08:54.464 INFO:tasks.workunit.client.1.smithi149.stdout:5/561: dread - d7/da/d20/d45/f69 zero size 2024-03-22T07:08:54.466 INFO:tasks.workunit.client.1.smithi149.stdout:7/592: mknod d1/d18/d1b/d99/d56/db4/cc1 0 2024-03-22T07:08:54.471 INFO:tasks.workunit.client.1.smithi149.stdout:8/591: rmdir d1/d2 39 2024-03-22T07:08:54.472 INFO:tasks.workunit.client.1.smithi149.stdout:1/559: dread d4/d20/d23/d3c/f47 [0,4194304] 0 2024-03-22T07:08:54.473 INFO:tasks.workunit.client.1.smithi149.stdout:8/592: rename d1/d2/d9/d11/d4d/f67 to d1/d2/d9/d11/d8b/daf/fb9 0 2024-03-22T07:08:54.473 INFO:tasks.workunit.client.1.smithi149.stdout:8/593: truncate d1/d20/d51/fa6 801285 0 2024-03-22T07:08:54.473 INFO:tasks.workunit.client.1.smithi149.stdout:8/594: truncate d1/d20/d51/fa2 540968 0 2024-03-22T07:08:54.474 INFO:tasks.workunit.client.1.smithi149.stdout:8/595: mknod d1/cba 0 2024-03-22T07:08:54.477 INFO:tasks.workunit.client.1.smithi149.stdout:1/560: rename d4/d20/c31 to d4/d20/d2d/cbc 0 2024-03-22T07:08:54.477 INFO:tasks.workunit.client.1.smithi149.stdout:1/561: readlink d4/d20/d23/d2f/l7f 0 2024-03-22T07:08:54.477 INFO:tasks.workunit.client.1.smithi149.stdout:8/596: truncate d1/d2/d18/f3d 307240 0 2024-03-22T07:08:54.477 INFO:tasks.workunit.client.1.smithi149.stdout:8/597: write d1/d2/d9/f1d [1862604,65534] 0 2024-03-22T07:08:54.482 INFO:tasks.workunit.client.1.smithi149.stdout:0/493: dwrite d1/d27/f62 [0,4194304] 0 2024-03-22T07:08:54.482 INFO:tasks.workunit.client.1.smithi149.stdout:8/598: dread d1/f16 [0,4194304] 0 2024-03-22T07:08:54.488 INFO:tasks.workunit.client.1.smithi149.stdout:4/572: dwrite d0/d8/f34 [0,4194304] 0 2024-03-22T07:08:54.492 INFO:tasks.workunit.client.1.smithi149.stdout:9/593: dwrite db/d42/fad [0,4194304] 0 2024-03-22T07:08:54.493 INFO:tasks.workunit.client.1.smithi149.stdout:4/573: creat d0/d8/d4c/db4/fb6 x:0 0 0 2024-03-22T07:08:54.493 INFO:tasks.workunit.client.1.smithi149.stdout:1/562: write d4/d10/d7c/d6a/d6b/f81 [828243,34321] 0 2024-03-22T07:08:54.493 INFO:tasks.workunit.client.1.smithi149.stdout:1/563: creat d4/d10/fbd x:0 0 0 2024-03-22T07:08:54.493 INFO:tasks.workunit.client.1.smithi149.stdout:1/564: write d4/d10/d7c/f98 [2887405,21710] 0 2024-03-22T07:08:54.495 INFO:tasks.workunit.client.1.smithi149.stdout:9/594: mknod db/d21/d9e/cb8 0 2024-03-22T07:08:54.496 INFO:tasks.workunit.client.1.smithi149.stdout:4/574: creat d0/d8/dc/d2e/fb7 x:0 0 0 2024-03-22T07:08:54.497 INFO:tasks.workunit.client.1.smithi149.stdout:9/595: write db/d19/d39/d68/d6f/fa7 [268221,22970] 0 2024-03-22T07:08:54.497 INFO:tasks.workunit.client.1.smithi149.stdout:9/596: readlink db/d12/l6e 0 2024-03-22T07:08:54.497 INFO:tasks.workunit.client.1.smithi149.stdout:9/597: read - db/d19/d1c/d85/fb3 zero size 2024-03-22T07:08:54.497 INFO:tasks.workunit.client.1.smithi149.stdout:9/598: write db/d19/d1c/d85/fb3 [501008,124452] 0 2024-03-22T07:08:54.497 INFO:tasks.workunit.client.1.smithi149.stdout:9/599: stat db/d21/d2e/la1 0 2024-03-22T07:08:54.498 INFO:tasks.workunit.client.1.smithi149.stdout:0/494: dread d1/d9/d14/f18 [0,4194304] 0 2024-03-22T07:08:54.499 INFO:tasks.workunit.client.1.smithi149.stdout:6/600: dwrite d9/df/d22/d6d/f78 [0,4194304] 0 2024-03-22T07:08:54.500 INFO:tasks.workunit.client.1.smithi149.stdout:1/565: mkdir d4/d20/d2d/d64/d9f/dbe 0 2024-03-22T07:08:54.502 INFO:tasks.workunit.client.1.smithi149.stdout:4/575: rename d0/d8/d1d/f28 to d0/d8/dc/d74/d76/fb8 0 2024-03-22T07:08:54.502 INFO:tasks.workunit.client.1.smithi149.stdout:4/576: creat d0/d6b/fb9 x:0 0 0 2024-03-22T07:08:54.502 INFO:tasks.workunit.client.1.smithi149.stdout:4/577: creat d0/d8/d1d/fba x:0 0 0 2024-03-22T07:08:54.503 INFO:tasks.workunit.client.1.smithi149.stdout:4/578: chown d0/d8/dc/d2e/d6f/f8e 214769 1 2024-03-22T07:08:54.503 INFO:tasks.workunit.client.1.smithi149.stdout:3/438: sync 2024-03-22T07:08:54.503 INFO:tasks.workunit.client.1.smithi149.stdout:8/599: dwrite d1/d2/d9/d11/f17 [0,4194304] 0 2024-03-22T07:08:54.512 INFO:tasks.workunit.client.1.smithi149.stdout:8/600: dread d1/d2/d9/d11/d8b/daf/fb9 [0,4194304] 0 2024-03-22T07:08:54.512 INFO:tasks.workunit.client.1.smithi149.stdout:8/601: stat d1/d2/d18/f78 0 2024-03-22T07:08:54.512 INFO:tasks.workunit.client.1.smithi149.stdout:8/602: write d1/d2/f8f [683605,77715] 0 2024-03-22T07:08:54.514 INFO:tasks.workunit.client.1.smithi149.stdout:8/603: dread d1/d20/d35/f6f [0,4194304] 0 2024-03-22T07:08:54.515 INFO:tasks.workunit.client.1.smithi149.stdout:0/495: rename d1/d9/d14/d74/d77/l95 to d1/d9/lb8 0 2024-03-22T07:08:54.515 INFO:tasks.workunit.client.1.smithi149.stdout:0/496: write d1/f39 [1184854,55768] 0 2024-03-22T07:08:54.518 INFO:tasks.workunit.client.1.smithi149.stdout:4/579: link d0/d8/dc/d14/d3e/f6a d0/d8/d4c/fbb 0 2024-03-22T07:08:54.521 INFO:tasks.workunit.client.1.smithi149.stdout:8/604: link d1/d2/d9/l74 d1/d20/d35/d22/d44/d4f/d9e/lbb 0 2024-03-22T07:08:54.522 INFO:tasks.workunit.client.1.smithi149.stdout:1/566: dread d4/d7/d5a/f5e [0,4194304] 0 2024-03-22T07:08:54.522 INFO:tasks.workunit.client.1.smithi149.stdout:1/567: truncate d4/d10/f4c 828025 0 2024-03-22T07:08:54.522 INFO:tasks.workunit.client.1.smithi149.stdout:1/568: chown d4/d7/d5d/l77 770887086 1 2024-03-22T07:08:54.522 INFO:tasks.workunit.client.1.smithi149.stdout:1/569: readlink l3 0 2024-03-22T07:08:54.522 INFO:tasks.workunit.client.1.smithi149.stdout:1/570: chown d4/l46 1519 1 2024-03-22T07:08:54.523 INFO:tasks.workunit.client.1.smithi149.stdout:4/580: rmdir d0/d8/d5c/da4 0 2024-03-22T07:08:54.523 INFO:tasks.workunit.client.1.smithi149.stdout:8/605: mknod d1/d20/d35/d22/d44/d4f/cbc 0 2024-03-22T07:08:54.523 INFO:tasks.workunit.client.1.smithi149.stdout:8/606: write d1/d2/d18/f3c [1785714,51094] 0 2024-03-22T07:08:54.525 INFO:tasks.workunit.client.1.smithi149.stdout:8/607: mknod d1/d20/d35/d22/d44/cbd 0 2024-03-22T07:08:54.527 INFO:tasks.workunit.client.1.smithi149.stdout:1/571: getdents d4/d10/d7c/d6a/d6b 0 2024-03-22T07:08:54.527 INFO:tasks.workunit.client.1.smithi149.stdout:1/572: chown d4/d20/d23/d3c/d8e/db5/f37 225343445 1 2024-03-22T07:08:54.530 INFO:tasks.workunit.client.1.smithi149.stdout:8/608: rename d1/d20/d35/c34 to d1/d2/d9/d11/cbe 0 2024-03-22T07:08:54.530 INFO:tasks.workunit.client.1.smithi149.stdout:8/609: truncate d1/d20/d35/d22/d44/d4f/f6e 683586 0 2024-03-22T07:08:54.530 INFO:tasks.workunit.client.1.smithi149.stdout:1/573: mkdir d4/d20/d23/d3c/d8e/db5/db2/dbf 0 2024-03-22T07:08:54.531 INFO:tasks.workunit.client.1.smithi149.stdout:8/610: symlink d1/d2/d9/d11/d45/lbf 0 2024-03-22T07:08:54.531 INFO:tasks.workunit.client.1.smithi149.stdout:8/611: read d1/d20/d35/d22/d44/f52 [83071,111773] 0 2024-03-22T07:08:54.532 INFO:tasks.workunit.client.1.smithi149.stdout:6/601: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f43 [0,4194304] 0 2024-03-22T07:08:54.532 INFO:tasks.workunit.client.1.smithi149.stdout:1/574: mkdir d4/d20/da0/dc0 0 2024-03-22T07:08:54.534 INFO:tasks.workunit.client.1.smithi149.stdout:3/439: dwrite d0/d4/d8/d24/d40/d54/f6f [0,4194304] 0 2024-03-22T07:08:54.536 INFO:tasks.workunit.client.1.smithi149.stdout:1/575: link d4/d10/c19 d4/d20/cc1 0 2024-03-22T07:08:54.536 INFO:tasks.workunit.client.1.smithi149.stdout:1/576: truncate d4/d10/f4c 1256028 0 2024-03-22T07:08:54.536 INFO:tasks.workunit.client.1.smithi149.stdout:1/577: fsync d4/d10/f55 0 2024-03-22T07:08:54.538 INFO:tasks.workunit.client.1.smithi149.stdout:6/602: dread d9/df/d2e/f1c [0,4194304] 0 2024-03-22T07:08:54.538 INFO:tasks.workunit.client.1.smithi149.stdout:3/440: dread d0/d4/d8/d24/f47 [0,4194304] 0 2024-03-22T07:08:54.541 INFO:tasks.workunit.client.1.smithi149.stdout:6/603: rename d9/df/d2e/d4e to d9/df/d22/dc0 0 2024-03-22T07:08:54.544 INFO:tasks.workunit.client.1.smithi149.stdout:4/581: dwrite d0/d8/dc/d4b/f70 [4194304,4194304] 0 2024-03-22T07:08:54.544 INFO:tasks.workunit.client.1.smithi149.stdout:4/582: readlink d0/d8/d4c/l84 0 2024-03-22T07:08:54.544 INFO:tasks.workunit.client.1.smithi149.stdout:0/497: dwrite d1/d9/f3a [4194304,4194304] 0 2024-03-22T07:08:54.557 INFO:tasks.workunit.client.1.smithi149.stdout:4/583: getdents d0/d8/d1d/d24/d82 0 2024-03-22T07:08:54.557 INFO:tasks.workunit.client.1.smithi149.stdout:4/584: dread - d0/d8/fa2 zero size 2024-03-22T07:08:54.559 INFO:tasks.workunit.client.1.smithi149.stdout:4/585: getdents d0/d8/dc/d14/d3e 0 2024-03-22T07:08:54.560 INFO:tasks.workunit.client.1.smithi149.stdout:4/586: mkdir d0/d8/dc/d4b/dbc 0 2024-03-22T07:08:54.562 INFO:tasks.workunit.client.1.smithi149.stdout:1/578: dwrite d4/d20/d23/f44 [0,4194304] 0 2024-03-22T07:08:54.575 INFO:tasks.workunit.client.1.smithi149.stdout:4/587: dread d0/f32 [0,4194304] 0 2024-03-22T07:08:54.575 INFO:tasks.workunit.client.1.smithi149.stdout:3/441: dwrite d0/d4/d8/f19 [0,4194304] 0 2024-03-22T07:08:54.575 INFO:tasks.workunit.client.1.smithi149.stdout:4/588: readlink d0/d8/dc/d14/l47 0 2024-03-22T07:08:54.575 INFO:tasks.workunit.client.1.smithi149.stdout:4/589: mkdir d0/d8/dc/dbd 0 2024-03-22T07:08:54.576 INFO:tasks.workunit.client.1.smithi149.stdout:3/442: symlink d0/d4/d8/d24/la8 0 2024-03-22T07:08:54.577 INFO:tasks.workunit.client.1.smithi149.stdout:4/590: rename d0/d8/dc/d14/f75 to d0/d8/dc/dbd/fbe 0 2024-03-22T07:08:54.578 INFO:tasks.workunit.client.1.smithi149.stdout:3/443: mkdir d0/d2/d5a/d93/da9 0 2024-03-22T07:08:54.578 INFO:tasks.workunit.client.1.smithi149.stdout:3/444: stat d0/d4/l6 0 2024-03-22T07:08:54.579 INFO:tasks.workunit.client.1.smithi149.stdout:3/445: symlink d0/d2/d30/d5d/d84/d8f/laa 0 2024-03-22T07:08:54.579 INFO:tasks.workunit.client.1.smithi149.stdout:3/446: stat d0/d4/d8/d24/d40/la7 0 2024-03-22T07:08:54.580 INFO:tasks.workunit.client.1.smithi149.stdout:3/447: link d0/d2/d30/d5d/d84/d8f/f59 d0/d4/d8/d24/d40/d54/d9e/fab 0 2024-03-22T07:08:54.581 INFO:tasks.workunit.client.1.smithi149.stdout:3/448: mknod d0/d4/d8/cac 0 2024-03-22T07:08:54.582 INFO:tasks.workunit.client.1.smithi149.stdout:3/449: mknod d0/cad 0 2024-03-22T07:08:54.582 INFO:tasks.workunit.client.1.smithi149.stdout:3/450: creat d0/d4/d8/d24/fae x:0 0 0 2024-03-22T07:08:54.585 INFO:tasks.workunit.client.1.smithi149.stdout:6/604: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/fb9 [0,4194304] 0 2024-03-22T07:08:54.588 INFO:tasks.workunit.client.1.smithi149.stdout:6/605: truncate d9/df/d2e/f1b 1499612 0 2024-03-22T07:08:54.588 INFO:tasks.workunit.client.1.smithi149.stdout:6/606: chown d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/f97 0 1 2024-03-22T07:08:54.588 INFO:tasks.workunit.client.1.smithi149.stdout:6/607: chown d9/df/d22/d6d/d7d/d96/db1/d55/l6e 15222 1 2024-03-22T07:08:54.588 INFO:tasks.workunit.client.1.smithi149.stdout:6/608: chown d9/df/d22/d6d/d7d/d96/d24/d68/f9f 16 1 2024-03-22T07:08:54.590 INFO:tasks.workunit.client.1.smithi149.stdout:6/609: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/fc1 x:0 0 0 2024-03-22T07:08:54.590 INFO:tasks.workunit.client.1.smithi149.stdout:6/610: write d9/df/d22/d6d/d7d/d96/db1/f80 [689001,105222] 0 2024-03-22T07:08:54.590 INFO:tasks.workunit.client.1.smithi149.stdout:6/611: write d9/df/d2e/d63/fb3 [798656,110239] 0 2024-03-22T07:08:54.590 INFO:tasks.workunit.client.1.smithi149.stdout:6/612: creat d9/da2/fc2 x:0 0 0 2024-03-22T07:08:54.590 INFO:tasks.workunit.client.1.smithi149.stdout:6/613: chown d9/df/d22/d6d/d7d/d96 914 1 2024-03-22T07:08:54.592 INFO:tasks.workunit.client.1.smithi149.stdout:0/498: dwrite d1/d1a/f50 [0,4194304] 0 2024-03-22T07:08:54.592 INFO:tasks.workunit.client.1.smithi149.stdout:0/499: write d1/d9/d14/d1f/f89 [945105,49621] 0 2024-03-22T07:08:54.592 INFO:tasks.workunit.client.1.smithi149.stdout:0/500: write d1/d9/d14/d1f/f38 [181823,102139] 0 2024-03-22T07:08:54.592 INFO:tasks.workunit.client.1.smithi149.stdout:4/591: dread d0/d8/d1d/d24/f54 [0,4194304] 0 2024-03-22T07:08:54.593 INFO:tasks.workunit.client.1.smithi149.stdout:1/579: dwrite d4/d10/f12 [0,4194304] 0 2024-03-22T07:08:54.593 INFO:tasks.workunit.client.1.smithi149.stdout:6/614: creat d9/df/d22/d6d/d7d/d96/d24/d68/fc3 x:0 0 0 2024-03-22T07:08:54.593 INFO:tasks.workunit.client.1.smithi149.stdout:8/612: dwrite d1/d2/d18/f3c [0,4194304] 0 2024-03-22T07:08:54.597 INFO:tasks.workunit.client.1.smithi149.stdout:4/592: rename d0/d8/dc/d14/f44 to d0/d8/dc/d2e/fbf 0 2024-03-22T07:08:54.597 INFO:tasks.workunit.client.1.smithi149.stdout:4/593: stat d0/d8/dc/dbd/fbe 0 2024-03-22T07:08:54.597 INFO:tasks.workunit.client.1.smithi149.stdout:4/594: chown d0/d8/d1d/d24/c69 1714647070 1 2024-03-22T07:08:54.601 INFO:tasks.workunit.client.1.smithi149.stdout:1/580: creat d4/d10/d58/d96/fc2 x:0 0 0 2024-03-22T07:08:54.604 INFO:tasks.workunit.client.1.smithi149.stdout:0/501: write d1/d9/d14/d74/f81 [7837365,34201] 0 2024-03-22T07:08:54.604 INFO:tasks.workunit.client.1.smithi149.stdout:0/502: chown d1/d9/d14/lb7 34684 1 2024-03-22T07:08:54.604 INFO:tasks.workunit.client.1.smithi149.stdout:0/503: truncate d1/d9/d14/f55 26609 0 2024-03-22T07:08:54.605 INFO:tasks.workunit.client.1.smithi149.stdout:0/504: read - d1/d31/d51/d64/d8d/d9c/fb5 zero size 2024-03-22T07:08:54.608 INFO:tasks.workunit.client.1.smithi149.stdout:3/451: dwrite d0/d2/d30/d31/f3d [0,4194304] 0 2024-03-22T07:08:54.614 INFO:tasks.workunit.client.1.smithi149.stdout:6/615: dwrite d9/da2/fb5 [0,4194304] 0 2024-03-22T07:08:54.614 INFO:tasks.workunit.client.1.smithi149.stdout:6/616: write d9/da2/fc2 [295103,126296] 0 2024-03-22T07:08:54.614 INFO:tasks.workunit.client.1.smithi149.stdout:8/613: creat d1/d20/d35/d22/d44/d82/fc0 x:0 0 0 2024-03-22T07:08:54.615 INFO:tasks.workunit.client.1.smithi149.stdout:8/614: write d1/f24 [4745772,6627] 0 2024-03-22T07:08:54.616 INFO:tasks.workunit.client.1.smithi149.stdout:4/595: mknod d0/d8/dc/d2e/cc0 0 2024-03-22T07:08:54.619 INFO:tasks.workunit.client.1.smithi149.stdout:1/581: write d4/d10/d7c/d6a/d6b/f84 [2816271,83755] 0 2024-03-22T07:08:54.622 INFO:tasks.workunit.client.1.smithi149.stdout:3/452: mkdir d0/d2/d5a/daf 0 2024-03-22T07:08:54.625 INFO:tasks.workunit.client.1.smithi149.stdout:4/596: mkdir d0/d8/d5c/dc1 0 2024-03-22T07:08:54.628 INFO:tasks.workunit.client.1.smithi149.stdout:6/617: write d9/df/d22/d6d/d7d/d96/f6b [2850154,98694] 0 2024-03-22T07:08:54.628 INFO:tasks.workunit.client.1.smithi149.stdout:2/661: sync 2024-03-22T07:08:54.629 INFO:tasks.workunit.client.1.smithi149.stdout:0/505: dwrite d1/d9/d59/f6c [0,4194304] 0 2024-03-22T07:08:54.629 INFO:tasks.workunit.client.1.smithi149.stdout:0/506: truncate d1/d31/d51/f5b 1529382 0 2024-03-22T07:08:54.629 INFO:tasks.workunit.client.1.smithi149.stdout:0/507: write d1/d6b/f92 [426539,20855] 0 2024-03-22T07:08:54.630 INFO:tasks.workunit.client.1.smithi149.stdout:6/618: creat d9/df/d22/d6d/fc4 x:0 0 0 2024-03-22T07:08:54.634 INFO:tasks.workunit.client.1.smithi149.stdout:0/508: rename d1/d1a/f4f to d1/db4/fb9 0 2024-03-22T07:08:54.634 INFO:tasks.workunit.client.1.smithi149.stdout:0/509: write d1/d31/d51/d64/d8d/d9c/fb5 [145050,4601] 0 2024-03-22T07:08:54.634 INFO:tasks.workunit.client.1.smithi149.stdout:0/510: chown d1/f63 51914 1 2024-03-22T07:08:54.634 INFO:tasks.workunit.client.1.smithi149.stdout:6/619: symlink d9/d47/lc5 0 2024-03-22T07:08:54.634 INFO:tasks.workunit.client.1.smithi149.stdout:6/620: chown d9/d47/d52/f8a 236 1 2024-03-22T07:08:54.634 INFO:tasks.workunit.client.1.smithi149.stdout:6/621: readlink d9/df/d22/d6d/d7d/l81 0 2024-03-22T07:08:54.634 INFO:tasks.workunit.client.1.smithi149.stdout:4/597: write d0/d8/dc/f18 [336972,111825] 0 2024-03-22T07:08:54.635 INFO:tasks.workunit.client.1.smithi149.stdout:2/662: rmdir d1/d4d/d69/d75 39 2024-03-22T07:08:54.635 INFO:tasks.workunit.client.1.smithi149.stdout:4/598: stat d0/d8/d1d/d24/d82/f91 0 2024-03-22T07:08:54.636 INFO:tasks.workunit.client.1.smithi149.stdout:0/511: creat d1/d9/d59/fba x:0 0 0 2024-03-22T07:08:54.636 INFO:tasks.workunit.client.1.smithi149.stdout:6/622: getdents d9/df/d22/d6d/d7d/d96/db1 0 2024-03-22T07:08:54.638 INFO:tasks.workunit.client.1.smithi149.stdout:7/593: sync 2024-03-22T07:08:54.638 INFO:tasks.workunit.client.1.smithi149.stdout:5/562: sync 2024-03-22T07:08:54.639 INFO:tasks.workunit.client.1.smithi149.stdout:9/600: sync 2024-03-22T07:08:54.641 INFO:tasks.workunit.client.1.smithi149.stdout:4/599: symlink d0/d6b/db0/lc2 0 2024-03-22T07:08:54.644 INFO:tasks.workunit.client.1.smithi149.stdout:0/512: mknod d1/d9/cbb 0 2024-03-22T07:08:54.644 INFO:tasks.workunit.client.1.smithi149.stdout:0/513: write d1/fb [2784229,83769] 0 2024-03-22T07:08:54.646 INFO:tasks.workunit.client.1.smithi149.stdout:5/563: unlink d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d2b/f73 0 2024-03-22T07:08:54.648 INFO:tasks.workunit.client.1.smithi149.stdout:9/601: mkdir db/d19/d93/db9 0 2024-03-22T07:08:54.648 INFO:tasks.workunit.client.1.smithi149.stdout:4/600: creat d0/fc3 x:0 0 0 2024-03-22T07:08:54.648 INFO:tasks.workunit.client.1.smithi149.stdout:4/601: truncate d0/d6b/fb9 684806 0 2024-03-22T07:08:54.648 INFO:tasks.workunit.client.1.smithi149.stdout:4/602: write d0/d8/dc/f9d [322608,129453] 0 2024-03-22T07:08:54.648 INFO:tasks.workunit.client.1.smithi149.stdout:4/603: write d0/d8/dc/d14/f9b [702051,49950] 0 2024-03-22T07:08:54.648 INFO:tasks.workunit.client.1.smithi149.stdout:4/604: fdatasync d0/f5a 0 2024-03-22T07:08:54.649 INFO:tasks.workunit.client.1.smithi149.stdout:0/514: mknod d1/d27/d66/cbc 0 2024-03-22T07:08:54.651 INFO:tasks.workunit.client.1.smithi149.stdout:5/564: truncate d7/da/d20/d36/d84/fa6 1926732 0 2024-03-22T07:08:54.651 INFO:tasks.workunit.client.1.smithi149.stdout:0/515: creat d1/d31/d51/d64/fbd x:0 0 0 2024-03-22T07:08:54.653 INFO:tasks.workunit.client.1.smithi149.stdout:5/565: unlink d7/c1a 0 2024-03-22T07:08:54.654 INFO:tasks.workunit.client.1.smithi149.stdout:0/516: symlink d1/d9/d14/d1f/lbe 0 2024-03-22T07:08:54.656 INFO:tasks.workunit.client.1.smithi149.stdout:7/594: write d1/d18/f1e [2991254,35331] 0 2024-03-22T07:08:54.656 INFO:tasks.workunit.client.1.smithi149.stdout:0/517: link d1/d6b/c96 d1/d31/d51/d64/d8d/d9c/d9e/cbf 0 2024-03-22T07:08:54.658 INFO:tasks.workunit.client.1.smithi149.stdout:5/566: write d7/da/d20/d30/d3a/d3d/d2c/d47/f52 [451984,51547] 0 2024-03-22T07:08:54.658 INFO:tasks.workunit.client.1.smithi149.stdout:5/567: chown d7/da/d20/d30/d3a/d3d/d2c/d63/c8a 58441759 1 2024-03-22T07:08:54.658 INFO:tasks.workunit.client.1.smithi149.stdout:7/595: rename d1/d18/d1b/d51/l63 to d1/d18/d1b/d51/d7a/d7d/d5b/lc2 0 2024-03-22T07:08:54.658 INFO:tasks.workunit.client.1.smithi149.stdout:7/596: readlink d1/d18/d1b/d51/d7a/d7d/d5b/lc2 0 2024-03-22T07:08:54.658 INFO:tasks.workunit.client.1.smithi149.stdout:7/597: getdents d1/db8 0 2024-03-22T07:08:54.660 INFO:tasks.workunit.client.1.smithi149.stdout:5/568: symlink d7/lc4 0 2024-03-22T07:08:54.660 INFO:tasks.workunit.client.1.smithi149.stdout:5/569: fsync d7/da/d20/d30/d3a/fc0 0 2024-03-22T07:08:54.661 INFO:tasks.workunit.client.1.smithi149.stdout:3/453: dwrite d0/d4/d8/d24/f47 [0,4194304] 0 2024-03-22T07:08:54.661 INFO:tasks.workunit.client.1.smithi149.stdout:2/663: dwrite d1/d3/d5/d1f/d68/d97/fe7 [0,4194304] 0 2024-03-22T07:08:54.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/623: dwrite d9/df/d22/d6d/d7d/fb6 [0,4194304] 0 2024-03-22T07:08:54.662 INFO:tasks.workunit.client.1.smithi149.stdout:5/570: mknod d7/da/d20/d30/d3a/d3d/d2c/d47/da4/daf/cc5 0 2024-03-22T07:08:54.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/624: write d9/df/d22/d6d/d7d/d96/d24/d42/fad [173881,26013] 0 2024-03-22T07:08:54.663 INFO:tasks.workunit.client.1.smithi149.stdout:6/625: fsync d9/df/d22/f58 0 2024-03-22T07:08:54.665 INFO:tasks.workunit.client.1.smithi149.stdout:0/518: truncate d1/d9/d59/f6c 4176763 0 2024-03-22T07:08:54.668 INFO:tasks.workunit.client.1.smithi149.stdout:5/571: dread d7/da/d7d/f9f [0,4194304] 0 2024-03-22T07:08:54.671 INFO:tasks.workunit.client.1.smithi149.stdout:6/626: truncate d9/d47/d52/f60 199170 0 2024-03-22T07:08:54.671 INFO:tasks.workunit.client.1.smithi149.stdout:6/627: chown d9/lb8 33071 1 2024-03-22T07:08:54.671 INFO:tasks.workunit.client.1.smithi149.stdout:6/628: chown d9/df/d2e/d63/fb3 6494 1 2024-03-22T07:08:54.673 INFO:tasks.workunit.client.1.smithi149.stdout:0/519: mkdir d1/d9/d59/dc0 0 2024-03-22T07:08:54.677 INFO:tasks.workunit.client.1.smithi149.stdout:3/454: write d0/d2/d42/d66/f7d [4059960,115048] 0 2024-03-22T07:08:54.679 INFO:tasks.workunit.client.1.smithi149.stdout:3/455: rename d0/d4/fe to d0/d4/d8/d24/d40/d54/d85/fb0 0 2024-03-22T07:08:54.679 INFO:tasks.workunit.client.1.smithi149.stdout:3/456: creat d0/d4/d8/d24/d39/fb1 x:0 0 0 2024-03-22T07:08:54.681 INFO:tasks.workunit.client.1.smithi149.stdout:2/664: read d1/d8/fb0 [2524013,106672] 0 2024-03-22T07:08:54.681 INFO:tasks.workunit.client.1.smithi149.stdout:2/665: write d1/d3/d9a/fef [259775,90027] 0 2024-03-22T07:08:54.681 INFO:tasks.workunit.client.1.smithi149.stdout:2/666: creat d1/d4d/ff1 x:0 0 0 2024-03-22T07:08:54.681 INFO:tasks.workunit.client.1.smithi149.stdout:2/667: fsync d1/d3/d9a/fad 0 2024-03-22T07:08:54.682 INFO:tasks.workunit.client.1.smithi149.stdout:3/457: symlink d0/d2/d30/d7e/lb2 0 2024-03-22T07:08:54.682 INFO:tasks.workunit.client.1.smithi149.stdout:3/458: chown d0/d4/d8 166685 1 2024-03-22T07:08:54.683 INFO:tasks.workunit.client.1.smithi149.stdout:4/605: dwrite d0/d8/d1d/d24/d82/fa1 [0,4194304] 0 2024-03-22T07:08:54.683 INFO:tasks.workunit.client.1.smithi149.stdout:4/606: fdatasync d0/d29/f31 0 2024-03-22T07:08:54.685 INFO:tasks.workunit.client.1.smithi149.stdout:7/598: dwrite d1/d18/f34 [0,4194304] 0 2024-03-22T07:08:54.686 INFO:tasks.workunit.client.1.smithi149.stdout:3/459: mknod d0/d4/cb3 0 2024-03-22T07:08:54.688 INFO:tasks.workunit.client.1.smithi149.stdout:2/668: getdents d1/d8/d34 0 2024-03-22T07:08:54.689 INFO:tasks.workunit.client.1.smithi149.stdout:5/572: dread d7/da/d20/d30/d3a/d3d/d2c/f4d [0,4194304] 0 2024-03-22T07:08:54.690 INFO:tasks.workunit.client.1.smithi149.stdout:7/599: mknod d1/d1c/dbc/cc3 0 2024-03-22T07:08:54.690 INFO:tasks.workunit.client.1.smithi149.stdout:7/600: write d1/d18/f1e [1436904,108620] 0 2024-03-22T07:08:54.692 INFO:tasks.workunit.client.1.smithi149.stdout:9/602: dwrite db/d21/d37/f64 [0,4194304] 0 2024-03-22T07:08:54.693 INFO:tasks.workunit.client.1.smithi149.stdout:7/601: creat d1/d18/d1b/d51/fc4 x:0 0 0 2024-03-22T07:08:54.695 INFO:tasks.workunit.client.1.smithi149.stdout:5/573: getdents d7/da/d20/d30/d3a 0 2024-03-22T07:08:54.698 INFO:tasks.workunit.client.1.smithi149.stdout:7/602: creat d1/d18/d1b/d51/d7a/d7d/d5b/d97/fc5 x:0 0 0 2024-03-22T07:08:54.698 INFO:tasks.workunit.client.1.smithi149.stdout:7/603: fsync d1/d18/d1b/d99/f68 0 2024-03-22T07:08:54.698 INFO:tasks.workunit.client.1.smithi149.stdout:7/604: chown d1/d18/d1b/d99/d56/d92/d9e/dab/fb2 1393384 1 2024-03-22T07:08:54.698 INFO:tasks.workunit.client.1.smithi149.stdout:7/605: chown d1/d1c/d49/d8e/f87 3902176 1 2024-03-22T07:08:54.699 INFO:tasks.workunit.client.1.smithi149.stdout:5/574: link d7/da/d20/l6b d7/da/d20/d30/d3a/d3d/db0/lc6 0 2024-03-22T07:08:54.702 INFO:tasks.workunit.client.1.smithi149.stdout:3/460: write d0/f1a [521283,119699] 0 2024-03-22T07:08:54.702 INFO:tasks.workunit.client.1.smithi149.stdout:3/461: readlink d0/d2/d30/d5d/d84/d8f/l28 0 2024-03-22T07:08:54.704 INFO:tasks.workunit.client.1.smithi149.stdout:5/575: link d7/da/d20/d36/ca7 d7/d35/d8e/cc7 0 2024-03-22T07:08:54.704 INFO:tasks.workunit.client.1.smithi149.stdout:5/576: chown d7/da/d20/d30/d3a/d3d/c75 5376167 1 2024-03-22T07:08:54.704 INFO:tasks.workunit.client.1.smithi149.stdout:3/462: unlink d0/d2/d5a/c6a 0 2024-03-22T07:08:54.705 INFO:tasks.workunit.client.1.smithi149.stdout:3/463: rmdir d0/d2/d5a/d93 39 2024-03-22T07:08:54.707 INFO:tasks.workunit.client.1.smithi149.stdout:3/464: mknod d0/d4/d6c/cb4 0 2024-03-22T07:08:54.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/629: dwrite d9/df/d22/f87 [0,4194304] 0 2024-03-22T07:08:54.708 INFO:tasks.workunit.client.1.smithi149.stdout:6/630: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/fc1 [88346,25889] 0 2024-03-22T07:08:54.717 INFO:tasks.workunit.client.1.smithi149.stdout:0/520: dwrite d1/d9/d14/f28 [0,4194304] 0 2024-03-22T07:08:54.717 INFO:tasks.workunit.client.1.smithi149.stdout:4/607: dwrite d0/d8/d4c/f64 [0,4194304] 0 2024-03-22T07:08:54.719 INFO:tasks.workunit.client.1.smithi149.stdout:0/521: mkdir d1/d9/d14/d74/dc1 0 2024-03-22T07:08:54.720 INFO:tasks.workunit.client.1.smithi149.stdout:9/603: write f3 [476898,67227] 0 2024-03-22T07:08:54.720 INFO:tasks.workunit.client.1.smithi149.stdout:9/604: chown db/d42/d49/d4b/l7e 108876 1 2024-03-22T07:08:54.722 INFO:tasks.workunit.client.1.smithi149.stdout:4/608: mknod d0/d8/dc/d74/d76/cc4 0 2024-03-22T07:08:54.723 INFO:tasks.workunit.client.1.smithi149.stdout:9/605: unlink db/d19/d1c/f4f 0 2024-03-22T07:08:54.724 INFO:tasks.workunit.client.1.smithi149.stdout:4/609: creat d0/d8/dc/d74/fc5 x:0 0 0 2024-03-22T07:08:54.726 INFO:tasks.workunit.client.1.smithi149.stdout:9/606: mknod db/d19/d39/d68/cba 0 2024-03-22T07:08:54.726 INFO:tasks.workunit.client.1.smithi149.stdout:9/607: chown db/d19/d1c 19000468 1 2024-03-22T07:08:54.726 INFO:tasks.workunit.client.1.smithi149.stdout:9/608: write db/d21/d37/f5c [5080310,76772] 0 2024-03-22T07:08:54.726 INFO:tasks.workunit.client.1.smithi149.stdout:9/609: read db/d33/f92 [1249607,58428] 0 2024-03-22T07:08:54.729 INFO:tasks.workunit.client.1.smithi149.stdout:9/610: mknod db/d19/d93/cbb 0 2024-03-22T07:08:54.732 INFO:tasks.workunit.client.1.smithi149.stdout:9/611: rename db/d42/d49/fb1 to db/d21/d37/d7c/fbc 0 2024-03-22T07:08:54.732 INFO:tasks.workunit.client.1.smithi149.stdout:9/612: creat db/d22/fbd x:0 0 0 2024-03-22T07:08:54.735 INFO:tasks.workunit.client.1.smithi149.stdout:5/577: dread d7/da/d20/d30/d3a/d3d/d2c/d63/f72 [0,4194304] 0 2024-03-22T07:08:54.739 INFO:tasks.workunit.client.1.smithi149.stdout:0/522: dread d1/d31/d51/d64/d87/fb0 [0,4194304] 0 2024-03-22T07:08:54.741 INFO:tasks.workunit.client.1.smithi149.stdout:4/610: dread d0/d6b/f81 [0,4194304] 0 2024-03-22T07:08:54.745 INFO:tasks.workunit.client.1.smithi149.stdout:4/611: unlink d0/d8/dc/d74/fc5 0 2024-03-22T07:08:54.754 INFO:tasks.workunit.client.1.smithi149.stdout:2/669: dwrite d1/d3/d5/d1f/f84 [0,4194304] 0 2024-03-22T07:08:54.755 INFO:tasks.workunit.client.1.smithi149.stdout:6/631: dwrite d9/da2/fb5 [0,4194304] 0 2024-03-22T07:08:54.758 INFO:tasks.workunit.client.1.smithi149.stdout:3/465: dwrite d0/d2/d30/f8a [0,4194304] 0 2024-03-22T07:08:54.758 INFO:tasks.workunit.client.1.smithi149.stdout:3/466: truncate d0/d2/d30/d5d/d84/d8f/d74/f75 772174 0 2024-03-22T07:08:54.758 INFO:tasks.workunit.client.1.smithi149.stdout:3/467: fdatasync d0/d2/d30/d5d/d84/d8f/d74/f75 0 2024-03-22T07:08:54.758 INFO:tasks.workunit.client.1.smithi149.stdout:3/468: fdatasync d0/d4/d8/d24/d39/f89 0 2024-03-22T07:08:54.758 INFO:tasks.workunit.client.1.smithi149.stdout:3/469: chown d0/d2/d30/d5d/d84/d8f/c6b 106110 1 2024-03-22T07:08:54.758 INFO:tasks.workunit.client.1.smithi149.stdout:0/523: dread d1/d9/d59/f61 [0,4194304] 0 2024-03-22T07:08:54.765 INFO:tasks.workunit.client.1.smithi149.stdout:2/670: creat d1/d3/d5/d1f/d5e/ff2 x:0 0 0 2024-03-22T07:08:54.768 INFO:tasks.workunit.client.1.smithi149.stdout:2/671: dread d1/d4d/f8a [0,4194304] 0 2024-03-22T07:08:54.768 INFO:tasks.workunit.client.1.smithi149.stdout:2/672: creat d1/d3/ff3 x:0 0 0 2024-03-22T07:08:54.769 INFO:tasks.workunit.client.1.smithi149.stdout:3/470: mkdir d0/d4/db5 0 2024-03-22T07:08:54.769 INFO:tasks.workunit.client.1.smithi149.stdout:3/471: write d0/f38 [180150,41296] 0 2024-03-22T07:08:54.769 INFO:tasks.workunit.client.1.smithi149.stdout:3/472: dread - d0/d2/f8c zero size 2024-03-22T07:08:54.769 INFO:tasks.workunit.client.1.smithi149.stdout:6/632: getdents d9/df/d22/d6d/d7d/d96/d24/d68 0 2024-03-22T07:08:54.769 INFO:tasks.workunit.client.1.smithi149.stdout:6/633: stat d9/df/d22/c3d 0 2024-03-22T07:08:54.773 INFO:tasks.workunit.client.1.smithi149.stdout:2/673: getdents d1/d3/d6/d3f 0 2024-03-22T07:08:54.773 INFO:tasks.workunit.client.1.smithi149.stdout:2/674: chown d1/d4d/d69/d75/fa1 101792 1 2024-03-22T07:08:54.773 INFO:tasks.workunit.client.1.smithi149.stdout:2/675: write d1/d3/d9a/fad [223798,39717] 0 2024-03-22T07:08:54.773 INFO:tasks.workunit.client.1.smithi149.stdout:9/613: dwrite db/d42/d49/d4b/d4d/f55 [0,4194304] 0 2024-03-22T07:08:54.774 INFO:tasks.workunit.client.1.smithi149.stdout:3/473: mkdir d0/d4/d8/d24/db6 0 2024-03-22T07:08:54.779 INFO:tasks.workunit.client.1.smithi149.stdout:5/578: dwrite d7/da/d20/d30/d3a/d3d/d2c/d47/fae [0,4194304] 0 2024-03-22T07:08:54.783 INFO:tasks.workunit.client.1.smithi149.stdout:5/579: readlink d7/l16 0 2024-03-22T07:08:54.785 INFO:tasks.workunit.client.1.smithi149.stdout:5/580: mkdir d7/da/d20/d30/d3a/d3d/d2c/db2/dc8 0 2024-03-22T07:08:54.786 INFO:tasks.workunit.client.1.smithi149.stdout:5/581: creat d7/da/d20/d30/d3a/d93/d94/dc2/fc9 x:0 0 0 2024-03-22T07:08:54.787 INFO:tasks.workunit.client.1.smithi149.stdout:5/582: write d7/da/d7d/f9e [435610,129220] 0 2024-03-22T07:08:54.791 INFO:tasks.workunit.client.1.smithi149.stdout:5/583: getdents d7/da/d20/d30/d3a/d3d 0 2024-03-22T07:08:54.791 INFO:tasks.workunit.client.1.smithi149.stdout:2/676: write d1/d4d/d69/d75/fd3 [856571,40326] 0 2024-03-22T07:08:54.791 INFO:tasks.workunit.client.1.smithi149.stdout:0/524: dwrite d1/fb [0,4194304] 0 2024-03-22T07:08:54.792 INFO:tasks.workunit.client.1.smithi149.stdout:4/612: dwrite d0/d8/dc/d74/d76/fb8 [0,4194304] 0 2024-03-22T07:08:54.797 INFO:tasks.workunit.client.1.smithi149.stdout:2/677: unlink d1/d3/d5/d1f/l30 0 2024-03-22T07:08:54.797 INFO:tasks.workunit.client.1.smithi149.stdout:2/678: write d1/d8/f19 [1299318,5746] 0 2024-03-22T07:08:54.797 INFO:tasks.workunit.client.1.smithi149.stdout:2/679: chown d1/d3/d5/f5a 84546203 1 2024-03-22T07:08:54.797 INFO:tasks.workunit.client.1.smithi149.stdout:2/680: write d1/d4d/d69/d75/fd3 [1780148,86981] 0 2024-03-22T07:08:54.797 INFO:tasks.workunit.client.1.smithi149.stdout:0/525: rename d1/l6f to d1/d20/da3/lc2 0 2024-03-22T07:08:54.799 INFO:tasks.workunit.client.1.smithi149.stdout:4/613: truncate d0/d8/dc/f7c 1301042 0 2024-03-22T07:08:54.801 INFO:tasks.workunit.client.1.smithi149.stdout:2/681: mkdir d1/d4d/d69/d75/df4 0 2024-03-22T07:08:54.802 INFO:tasks.workunit.client.1.smithi149.stdout:5/584: dread d7/da/d20/d30/d3a/d3d/f34 [0,4194304] 0 2024-03-22T07:08:54.802 INFO:tasks.workunit.client.1.smithi149.stdout:0/526: link d1/c8 d1/d9/d14/d74/cc3 0 2024-03-22T07:08:54.803 INFO:tasks.workunit.client.1.smithi149.stdout:4/614: mknod d0/d8/dc/d14/cc6 0 2024-03-22T07:08:54.806 INFO:tasks.workunit.client.1.smithi149.stdout:5/585: link d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/d81/f6a d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/fca 0 2024-03-22T07:08:54.807 INFO:tasks.workunit.client.1.smithi149.stdout:2/682: getdents d1/d3/d6/db2/d71 0 2024-03-22T07:08:54.809 INFO:tasks.workunit.client.1.smithi149.stdout:5/586: rename d7/da/d20/d45/f69 to d7/da/d20/d30/d3a/d3d/d2c/d47/da4/daf/fcb 0 2024-03-22T07:08:54.809 INFO:tasks.workunit.client.1.smithi149.stdout:5/587: truncate d7/da/d20/d89/da2/fa8 402122 0 2024-03-22T07:08:54.809 INFO:tasks.workunit.client.1.smithi149.stdout:5/588: read - d7/da/d20/d30/f98 zero size 2024-03-22T07:08:54.810 INFO:tasks.workunit.client.1.smithi149.stdout:6/634: dwrite d9/d47/d52/f93 [0,4194304] 0 2024-03-22T07:08:54.810 INFO:tasks.workunit.client.1.smithi149.stdout:8/615: sync 2024-03-22T07:08:54.810 INFO:tasks.workunit.client.1.smithi149.stdout:2/683: creat d1/d4d/d90/ff5 x:0 0 0 2024-03-22T07:08:54.814 INFO:tasks.workunit.client.1.smithi149.stdout:3/474: dwrite d0/d2/d42/d66/f7a [0,4194304] 0 2024-03-22T07:08:54.816 INFO:tasks.workunit.client.1.smithi149.stdout:0/527: dread d1/d31/f8a [0,4194304] 0 2024-03-22T07:08:54.816 INFO:tasks.workunit.client.1.smithi149.stdout:0/528: creat d1/d9/d14/fc4 x:0 0 0 2024-03-22T07:08:54.816 INFO:tasks.workunit.client.1.smithi149.stdout:5/589: creat d7/da/d20/d30/d3a/d93/fcc x:0 0 0 2024-03-22T07:08:54.816 INFO:tasks.workunit.client.1.smithi149.stdout:9/614: dwrite db/d19/f3c [0,4194304] 0 2024-03-22T07:08:54.818 INFO:tasks.workunit.client.1.smithi149.stdout:9/615: write db/d12/f17 [368765,67213] 0 2024-03-22T07:08:54.818 INFO:tasks.workunit.client.1.smithi149.stdout:8/616: getdents d1/d2/d9/d11 0 2024-03-22T07:08:54.818 INFO:tasks.workunit.client.1.smithi149.stdout:8/617: fsync d1/d20/d35/d22/d44/d82/fc0 0 2024-03-22T07:08:54.818 INFO:tasks.workunit.client.1.smithi149.stdout:8/618: write d1/d2/d9/d11/d4d/f5e [3800306,12644] 0 2024-03-22T07:08:54.818 INFO:tasks.workunit.client.1.smithi149.stdout:6/635: creat d9/fc6 x:0 0 0 2024-03-22T07:08:54.818 INFO:tasks.workunit.client.1.smithi149.stdout:3/475: link d0/d2/d30/l5b d0/lb7 0 2024-03-22T07:08:54.818 INFO:tasks.workunit.client.1.smithi149.stdout:3/476: chown d0/d4/d8/f23 13683161 1 2024-03-22T07:08:54.823 INFO:tasks.workunit.client.1.smithi149.stdout:9/616: link db/d21/d37/d3a/la9 db/d21/d37/d7c/lbe 0 2024-03-22T07:08:54.825 INFO:tasks.workunit.client.1.smithi149.stdout:1/582: sync 2024-03-22T07:08:54.826 INFO:tasks.workunit.client.1.smithi149.stdout:8/619: creat d1/d2/d9/db2/fc1 x:0 0 0 2024-03-22T07:08:54.827 INFO:tasks.workunit.client.1.smithi149.stdout:9/617: dread db/d19/d39/d68/d6f/fa7 [0,4194304] 0 2024-03-22T07:08:54.828 INFO:tasks.workunit.client.1.smithi149.stdout:6/636: rename d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f54 to d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/fc7 0 2024-03-22T07:08:54.829 INFO:tasks.workunit.client.1.smithi149.stdout:5/590: write d7/da/d20/d30/d3a/d3d/d2c/f4d [7274,72148] 0 2024-03-22T07:08:54.830 INFO:tasks.workunit.client.1.smithi149.stdout:1/583: link d4/d20/d23/d59/c7b d4/d20/d2d/cc3 0 2024-03-22T07:08:54.830 INFO:tasks.workunit.client.1.smithi149.stdout:1/584: write d4/f6 [5489265,10097] 0 2024-03-22T07:08:54.830 INFO:tasks.workunit.client.1.smithi149.stdout:1/585: fdatasync d4/d10/d7c/f7d 0 2024-03-22T07:08:54.833 INFO:tasks.workunit.client.1.smithi149.stdout:4/615: unlink d0/d8/dc/f7c 0 2024-03-22T07:08:54.833 INFO:tasks.workunit.client.1.smithi149.stdout:8/620: getdents d1/d2/d9/df 0 2024-03-22T07:08:54.833 INFO:tasks.workunit.client.1.smithi149.stdout:8/621: write d1/d20/d35/fb [3324549,14424] 0 2024-03-22T07:08:54.836 INFO:tasks.workunit.client.1.smithi149.stdout:5/591: symlink d7/da/d20/d36/d44/lcd 0 2024-03-22T07:08:54.836 INFO:tasks.workunit.client.1.smithi149.stdout:3/477: dread d0/d2/d30/f3f [0,4194304] 0 2024-03-22T07:08:54.838 INFO:tasks.workunit.client.1.smithi149.stdout:1/586: symlink d4/d7/d94/lc4 0 2024-03-22T07:08:54.838 INFO:tasks.workunit.client.1.smithi149.stdout:1/587: chown d4/d20/d23/d3c/d85/d89 9895 1 2024-03-22T07:08:54.838 INFO:tasks.workunit.client.1.smithi149.stdout:1/588: dread - d4/d7/fb4 zero size 2024-03-22T07:08:54.840 INFO:tasks.workunit.client.1.smithi149.stdout:2/684: dwrite d1/d8/d34/f7a [0,4194304] 0 2024-03-22T07:08:54.841 INFO:tasks.workunit.client.1.smithi149.stdout:0/529: dwrite d1/d31/f3f [0,4194304] 0 2024-03-22T07:08:54.841 INFO:tasks.workunit.client.1.smithi149.stdout:0/530: write d1/d31/d52/f88 [2296255,44470] 0 2024-03-22T07:08:54.842 INFO:tasks.workunit.client.1.smithi149.stdout:0/531: read d1/d31/d51/f5b [613672,93781] 0 2024-03-22T07:08:54.842 INFO:tasks.workunit.client.1.smithi149.stdout:1/589: symlink d4/d7/lc5 0 2024-03-22T07:08:54.842 INFO:tasks.workunit.client.1.smithi149.stdout:1/590: truncate d4/d20/d23/d2f/d93/f9c 159991 0 2024-03-22T07:08:54.845 INFO:tasks.workunit.client.1.smithi149.stdout:8/622: rmdir d1/d20/d35/d22/d44/d84 39 2024-03-22T07:08:54.845 INFO:tasks.workunit.client.1.smithi149.stdout:8/623: chown d1/d2/d9/d11/cbe 47829 1 2024-03-22T07:08:54.845 INFO:tasks.workunit.client.1.smithi149.stdout:8/624: chown d1/d2/d9/f4a 3 1 2024-03-22T07:08:54.848 INFO:tasks.workunit.client.1.smithi149.stdout:6/637: dread d9/d47/f5a [0,4194304] 0 2024-03-22T07:08:54.848 INFO:tasks.workunit.client.1.smithi149.stdout:2/685: mkdir d1/d8/d59/df6 0 2024-03-22T07:08:54.848 INFO:tasks.workunit.client.1.smithi149.stdout:2/686: chown d1/d3/d6/d87 159 1 2024-03-22T07:08:54.851 INFO:tasks.workunit.client.1.smithi149.stdout:3/478: getdents d0/d4/d8/d24/d40/d54/d85 0 2024-03-22T07:08:54.854 INFO:tasks.workunit.client.1.smithi149.stdout:1/591: link f1 d4/d20/d23/d2f/d93/fc6 0 2024-03-22T07:08:54.856 INFO:tasks.workunit.client.1.smithi149.stdout:8/625: dread d1/d2/d9/d11/d45/f70 [0,4194304] 0 2024-03-22T07:08:54.859 INFO:tasks.workunit.client.1.smithi149.stdout:7/606: sync 2024-03-22T07:08:54.861 INFO:tasks.workunit.client.1.smithi149.stdout:6/638: write d9/df/d22/f87 [3432696,36613] 0 2024-03-22T07:08:54.861 INFO:tasks.workunit.client.1.smithi149.stdout:2/687: symlink d1/d8/d34/lf7 0 2024-03-22T07:08:54.861 INFO:tasks.workunit.client.1.smithi149.stdout:2/688: fdatasync d1/d3/f13 0 2024-03-22T07:08:54.861 INFO:tasks.workunit.client.1.smithi149.stdout:2/689: chown d1/d3/d6/d3f/d74/l96 13739 1 2024-03-22T07:08:54.863 INFO:tasks.workunit.client.1.smithi149.stdout:8/626: unlink d1/d2/d9/f37 0 2024-03-22T07:08:54.864 INFO:tasks.workunit.client.1.smithi149.stdout:0/532: dwrite d1/d9/d14/f18 [0,4194304] 0 2024-03-22T07:08:54.864 INFO:tasks.workunit.client.1.smithi149.stdout:4/616: dwrite d0/d29/f3c [4194304,4194304] 0 2024-03-22T07:08:54.865 INFO:tasks.workunit.client.1.smithi149.stdout:4/617: chown d0/d8/d1d/d24/d82 4745 1 2024-03-22T07:08:54.865 INFO:tasks.workunit.client.1.smithi149.stdout:7/607: mknod d1/d18/d1b/d99/d56/d92/d9e/dab/cc6 0 2024-03-22T07:08:54.867 INFO:tasks.workunit.client.1.smithi149.stdout:6/639: mkdir d9/df/d22/dc0/dc8 0 2024-03-22T07:08:54.868 INFO:tasks.workunit.client.1.smithi149.stdout:6/640: creat d9/fc9 x:0 0 0 2024-03-22T07:08:54.872 INFO:tasks.workunit.client.1.smithi149.stdout:2/690: rename d1/d4d/d69/d75/ld7 to d1/d3/d6/d32/d6f/lf8 0 2024-03-22T07:08:54.872 INFO:tasks.workunit.client.1.smithi149.stdout:2/691: fsync d1/d3/d9a/fd5 0 2024-03-22T07:08:54.872 INFO:tasks.workunit.client.1.smithi149.stdout:3/479: getdents d0/d4/d8/d24/d40 0 2024-03-22T07:08:54.873 INFO:tasks.workunit.client.1.smithi149.stdout:8/627: mknod d1/d20/d35/d22/d44/cc2 0 2024-03-22T07:08:54.876 INFO:tasks.workunit.client.1.smithi149.stdout:0/533: symlink d1/db4/lc5 0 2024-03-22T07:08:54.876 INFO:tasks.workunit.client.1.smithi149.stdout:4/618: link d0/d8/d4c/f6d d0/d8/dc/d14/fc7 0 2024-03-22T07:08:54.877 INFO:tasks.workunit.client.1.smithi149.stdout:6/641: mknod d9/df/cca 0 2024-03-22T07:08:54.878 INFO:tasks.workunit.client.1.smithi149.stdout:3/480: unlink d0/d2/c48 0 2024-03-22T07:08:54.878 INFO:tasks.workunit.client.1.smithi149.stdout:3/481: chown d0/d4/d8/l44 16796 1 2024-03-22T07:08:54.881 INFO:tasks.workunit.client.1.smithi149.stdout:6/642: rename d9/df/d22/d6d/d7d/d96/d24/d34/fa3 to d9/da2/fcb 0 2024-03-22T07:08:54.883 INFO:tasks.workunit.client.1.smithi149.stdout:3/482: mkdir d0/d2/d30/d7e/db8 0 2024-03-22T07:08:54.883 INFO:tasks.workunit.client.1.smithi149.stdout:3/483: write d0/d2/d30/d5d/d84/d8f/d74/f87 [526918,96937] 0 2024-03-22T07:08:54.884 INFO:tasks.workunit.client.1.smithi149.stdout:6/643: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/da1/fcc x:0 0 0 2024-03-22T07:08:54.887 INFO:tasks.workunit.client.1.smithi149.stdout:3/484: creat d0/d2/d30/d5d/d84/d8e/d9d/fb9 x:0 0 0 2024-03-22T07:08:54.889 INFO:tasks.workunit.client.1.smithi149.stdout:3/485: truncate d0/d2/d30/d5d/d84/d8f/f59 1702564 0 2024-03-22T07:08:54.891 INFO:tasks.workunit.client.1.smithi149.stdout:4/619: dread d0/d8/dc/d4b/f86 [0,4194304] 0 2024-03-22T07:08:54.891 INFO:tasks.workunit.client.1.smithi149.stdout:4/620: creat d0/d6b/fc8 x:0 0 0 2024-03-22T07:08:54.891 INFO:tasks.workunit.client.1.smithi149.stdout:4/621: stat d0/d8/dc/dbd 0 2024-03-22T07:08:54.891 INFO:tasks.workunit.client.1.smithi149.stdout:3/486: dread d0/d2/d30/f70 [0,4194304] 0 2024-03-22T07:08:54.892 INFO:tasks.workunit.client.1.smithi149.stdout:1/592: dwrite d4/d10/d7c/d6a/d6b/f84 [0,4194304] 0 2024-03-22T07:08:54.898 INFO:tasks.workunit.client.1.smithi149.stdout:1/593: rename d4/d20/d23/d2f/d93/cba to d4/d7/cc7 0 2024-03-22T07:08:54.898 INFO:tasks.workunit.client.1.smithi149.stdout:7/608: dwrite d1/d1c/d49/d8e/f87 [4194304,4194304] 0 2024-03-22T07:08:54.899 INFO:tasks.workunit.client.1.smithi149.stdout:7/609: fdatasync d1/d18/d1b/d51/fc4 0 2024-03-22T07:08:54.900 INFO:tasks.workunit.client.1.smithi149.stdout:0/534: write d1/d31/f33 [2881768,116100] 0 2024-03-22T07:08:54.900 INFO:tasks.workunit.client.1.smithi149.stdout:7/610: symlink d1/d18/d1b/d51/lc7 0 2024-03-22T07:08:54.902 INFO:tasks.workunit.client.1.smithi149.stdout:7/611: write d1/d18/d1b/d51/d7a/d7d/f86 [153930,61451] 0 2024-03-22T07:08:54.902 INFO:tasks.workunit.client.1.smithi149.stdout:7/612: write d1/d18/d1b/d1d/f8 [2813744,76143] 0 2024-03-22T07:08:54.902 INFO:tasks.workunit.client.1.smithi149.stdout:7/613: write d1/d18/d1b/d1d/f13 [4292917,130819] 0 2024-03-22T07:08:54.906 INFO:tasks.workunit.client.1.smithi149.stdout:8/628: dwrite d1/d2/d3/fb0 [0,4194304] 0 2024-03-22T07:08:54.906 INFO:tasks.workunit.client.1.smithi149.stdout:8/629: truncate d1/d20/d51/fa6 1239741 0 2024-03-22T07:08:54.910 INFO:tasks.workunit.client.1.smithi149.stdout:8/630: creat d1/fc3 x:0 0 0 2024-03-22T07:08:54.910 INFO:tasks.workunit.client.1.smithi149.stdout:8/631: write d1/d2/d3/f76 [978456,24719] 0 2024-03-22T07:08:54.912 INFO:tasks.workunit.client.1.smithi149.stdout:4/622: dread d0/d8/dc/d14/d3e/f51 [0,4194304] 0 2024-03-22T07:08:54.912 INFO:tasks.workunit.client.1.smithi149.stdout:4/623: chown d0/d8/dc/d2e/d6f/f8e 22336 1 2024-03-22T07:08:54.912 INFO:tasks.workunit.client.1.smithi149.stdout:8/632: creat d1/d20/d35/d22/d44/d84/fc4 x:0 0 0 2024-03-22T07:08:54.913 INFO:tasks.workunit.client.1.smithi149.stdout:4/624: symlink d0/d8/dc/d14/d85/lc9 0 2024-03-22T07:08:54.915 INFO:tasks.workunit.client.1.smithi149.stdout:8/633: rename d1/d2/d9/l74 to d1/lc5 0 2024-03-22T07:08:54.917 INFO:tasks.workunit.client.1.smithi149.stdout:7/614: read d1/d18/d1b/d1d/f16 [1111634,27256] 0 2024-03-22T07:08:54.917 INFO:tasks.workunit.client.1.smithi149.stdout:2/692: dwrite d1/d3/fdd [0,4194304] 0 2024-03-22T07:08:54.917 INFO:tasks.workunit.client.1.smithi149.stdout:4/625: symlink d0/lca 0 2024-03-22T07:08:54.918 INFO:tasks.workunit.client.1.smithi149.stdout:6/644: dwrite d9/df/d22/d6d/d7d/d96/d24/d68/fc3 [0,4194304] 0 2024-03-22T07:08:54.918 INFO:tasks.workunit.client.1.smithi149.stdout:8/634: creat d1/d2/d9/d11/d8b/d8d/d91/fc6 x:0 0 0 2024-03-22T07:08:54.919 INFO:tasks.workunit.client.1.smithi149.stdout:8/635: chown d1/d20/l6c 327843 1 2024-03-22T07:08:54.920 INFO:tasks.workunit.client.1.smithi149.stdout:2/693: rename d1/d3/d6/db2/ca4 to d1/d72/cf9 0 2024-03-22T07:08:54.920 INFO:tasks.workunit.client.1.smithi149.stdout:2/694: chown d1/d4d/d90/dbc 10944913 1 2024-03-22T07:08:54.920 INFO:tasks.workunit.client.1.smithi149.stdout:4/626: symlink d0/d8/dc/d74/d76/lcb 0 2024-03-22T07:08:54.923 INFO:tasks.workunit.client.1.smithi149.stdout:1/594: dread d4/d10/f12 [0,4194304] 0 2024-03-22T07:08:54.923 INFO:tasks.workunit.client.1.smithi149.stdout:8/636: link d1/d20/d35/d22/f38 d1/d20/fc7 0 2024-03-22T07:08:54.927 INFO:tasks.workunit.client.1.smithi149.stdout:2/695: creat d1/d3/d7d/ffa x:0 0 0 2024-03-22T07:08:54.927 INFO:tasks.workunit.client.1.smithi149.stdout:2/696: chown d1/d3/d5/d1f/d68/d97/ld0 2 1 2024-03-22T07:08:54.927 INFO:tasks.workunit.client.1.smithi149.stdout:2/697: write d1/d3/d5/d41/d9f/da7/fb9 [598928,125901] 0 2024-03-22T07:08:54.930 INFO:tasks.workunit.client.1.smithi149.stdout:4/627: rename d0/d8/dc/f9d to d0/d8/dc/d14/d3e/fcc 0 2024-03-22T07:08:54.933 INFO:tasks.workunit.client.1.smithi149.stdout:1/595: mknod d4/d20/d2d/d64/d9f/cc8 0 2024-03-22T07:08:54.933 INFO:tasks.workunit.client.1.smithi149.stdout:1/596: read - d4/d20/d2d/d64/fb3 zero size 2024-03-22T07:08:54.933 INFO:tasks.workunit.client.1.smithi149.stdout:1/597: chown d4/f15 266 1 2024-03-22T07:08:54.933 INFO:tasks.workunit.client.1.smithi149.stdout:1/598: creat d4/d20/d23/d2f/d93/fc9 x:0 0 0 2024-03-22T07:08:54.933 INFO:tasks.workunit.client.1.smithi149.stdout:1/599: chown d4/d10/c19 859663473 1 2024-03-22T07:08:54.935 INFO:tasks.workunit.client.1.smithi149.stdout:4/628: mknod d0/d8/dc/d14/d3e/ccd 0 2024-03-22T07:08:54.935 INFO:tasks.workunit.client.1.smithi149.stdout:6/645: dread d9/df/d2e/f20 [0,4194304] 0 2024-03-22T07:08:54.935 INFO:tasks.workunit.client.1.smithi149.stdout:6/646: read - d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/f97 zero size 2024-03-22T07:08:54.935 INFO:tasks.workunit.client.1.smithi149.stdout:3/487: dwrite d0/d4/d8/d24/d39/fb1 [0,4194304] 0 2024-03-22T07:08:54.935 INFO:tasks.workunit.client.1.smithi149.stdout:3/488: chown d0/d2/d30/d5d/d84/d8f/f2e 0 1 2024-03-22T07:08:54.935 INFO:tasks.workunit.client.1.smithi149.stdout:3/489: write d0/d2/d42/d66/d82/f94 [815310,51550] 0 2024-03-22T07:08:54.938 INFO:tasks.workunit.client.1.smithi149.stdout:4/629: creat d0/d8/d1d/d24/d9c/fce x:0 0 0 2024-03-22T07:08:54.938 INFO:tasks.workunit.client.1.smithi149.stdout:0/535: dwrite d1/d31/f8a [0,4194304] 0 2024-03-22T07:08:54.938 INFO:tasks.workunit.client.1.smithi149.stdout:0/536: write d1/d9/f1e [1352081,62506] 0 2024-03-22T07:08:54.938 INFO:tasks.workunit.client.1.smithi149.stdout:0/537: chown d1/d1a/f2d 138256862 1 2024-03-22T07:08:54.942 INFO:tasks.workunit.client.1.smithi149.stdout:7/615: dwrite d1/f41 [0,4194304] 0 2024-03-22T07:08:54.949 INFO:tasks.workunit.client.1.smithi149.stdout:8/637: dwrite d1/d20/fc7 [4194304,4194304] 0 2024-03-22T07:08:54.952 INFO:tasks.workunit.client.1.smithi149.stdout:3/490: creat d0/d2/d30/d31/fba x:0 0 0 2024-03-22T07:08:54.952 INFO:tasks.workunit.client.1.smithi149.stdout:3/491: readlink d0/d4/d8/d24/d40/l88 0 2024-03-22T07:08:54.953 INFO:tasks.workunit.client.1.smithi149.stdout:1/600: mkdir d4/d7/d5a/dca 0 2024-03-22T07:08:54.953 INFO:tasks.workunit.client.1.smithi149.stdout:3/492: read d0/d4/d6c/f9b [242001,67098] 0 2024-03-22T07:08:54.953 INFO:tasks.workunit.client.1.smithi149.stdout:3/493: write d0/d4/d8/f99 [974943,2123] 0 2024-03-22T07:08:54.954 INFO:tasks.workunit.client.1.smithi149.stdout:4/630: symlink d0/d8/d4c/lcf 0 2024-03-22T07:08:54.954 INFO:tasks.workunit.client.1.smithi149.stdout:4/631: dread - d0/d8/dc/d2e/d6f/f7f zero size 2024-03-22T07:08:54.954 INFO:tasks.workunit.client.1.smithi149.stdout:4/632: write d0/d8/dc/d2e/f62 [881636,85456] 0 2024-03-22T07:08:54.955 INFO:tasks.workunit.client.1.smithi149.stdout:0/538: mkdir d1/d20/d9f/dc6 0 2024-03-22T07:08:54.957 INFO:tasks.workunit.client.1.smithi149.stdout:7/616: creat d1/d18/d1b/d51/d5a/d6a/da6/d93/fc8 x:0 0 0 2024-03-22T07:08:54.957 INFO:tasks.workunit.client.1.smithi149.stdout:8/638: rename d1/d2/d9/d11/d45/c89 to d1/d2/d9/d11/d45/cc8 0 2024-03-22T07:08:54.957 INFO:tasks.workunit.client.1.smithi149.stdout:1/601: truncate d4/d20/d23/f44 768891 0 2024-03-22T07:08:54.959 INFO:tasks.workunit.client.1.smithi149.stdout:3/494: creat d0/d4/db5/fbb x:0 0 0 2024-03-22T07:08:54.959 INFO:tasks.workunit.client.1.smithi149.stdout:1/602: dread d4/d20/f22 [0,4194304] 0 2024-03-22T07:08:54.959 INFO:tasks.workunit.client.1.smithi149.stdout:2/698: dwrite d1/d72/f9e [0,4194304] 0 2024-03-22T07:08:54.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/539: symlink d1/d20/da3/lc7 0 2024-03-22T07:08:54.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/540: read - d1/d9/d14/fa8 zero size 2024-03-22T07:08:54.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/541: chown d1/d31/d51/d64/fa2 60871 1 2024-03-22T07:08:54.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/542: write d1/d20/da3/fab [379830,102707] 0 2024-03-22T07:08:54.961 INFO:tasks.workunit.client.1.smithi149.stdout:7/617: creat d1/d18/d1b/d51/d5a/d6a/da6/fc9 x:0 0 0 2024-03-22T07:08:54.961 INFO:tasks.workunit.client.1.smithi149.stdout:2/699: dread d1/d3/d9a/fef [0,4194304] 0 2024-03-22T07:08:54.964 INFO:tasks.workunit.client.1.smithi149.stdout:3/495: rename d0/d2/d30/d5d/d84/d8f/d74 to d0/d4/d8/d24/d40/d54/d85/dbc 0 2024-03-22T07:08:54.964 INFO:tasks.workunit.client.1.smithi149.stdout:6/647: dwrite d9/da2/fc2 [0,4194304] 0 2024-03-22T07:08:54.968 INFO:tasks.workunit.client.1.smithi149.stdout:2/700: truncate d1/d3/d5/d1f/f55 423488 0 2024-03-22T07:08:54.970 INFO:tasks.workunit.client.1.smithi149.stdout:8/639: getdents d1 0 2024-03-22T07:08:54.975 INFO:tasks.workunit.client.1.smithi149.stdout:1/603: rename d4/ca to d4/d20/d23/d3c/d8e/db5/ccb 0 2024-03-22T07:08:54.975 INFO:tasks.workunit.client.1.smithi149.stdout:1/604: write d4/d7/fe [4233753,103323] 0 2024-03-22T07:08:54.975 INFO:tasks.workunit.client.1.smithi149.stdout:0/543: read d1/fb [688359,15774] 0 2024-03-22T07:08:54.975 INFO:tasks.workunit.client.1.smithi149.stdout:0/544: dread - d1/d9/d14/fc4 zero size 2024-03-22T07:08:54.975 INFO:tasks.workunit.client.1.smithi149.stdout:0/545: fsync d1/d1a/f50 0 2024-03-22T07:08:54.976 INFO:tasks.workunit.client.1.smithi149.stdout:2/701: symlink d1/d3/d7d/lfb 0 2024-03-22T07:08:54.976 INFO:tasks.workunit.client.1.smithi149.stdout:0/546: write d1/d9/d14/d74/d77/f9a [532734,80223] 0 2024-03-22T07:08:54.976 INFO:tasks.workunit.client.1.smithi149.stdout:4/633: dwrite d0/d8/d1d/d24/d82/f88 [4194304,4194304] 0 2024-03-22T07:08:54.976 INFO:tasks.workunit.client.1.smithi149.stdout:7/618: fdatasync d1/d18/d1b/d1d/f8 0 2024-03-22T07:08:54.976 INFO:tasks.workunit.client.1.smithi149.stdout:8/640: creat d1/d20/d35/d22/d44/d4f/fc9 x:0 0 0 2024-03-22T07:08:54.978 INFO:tasks.workunit.client.1.smithi149.stdout:1/605: rmdir d4/d20/d23 39 2024-03-22T07:08:54.978 INFO:tasks.workunit.client.1.smithi149.stdout:2/702: mkdir d1/d3/d5/d1f/dfc 0 2024-03-22T07:08:54.979 INFO:tasks.workunit.client.1.smithi149.stdout:0/547: symlink d1/d27/d66/lc8 0 2024-03-22T07:08:54.980 INFO:tasks.workunit.client.1.smithi149.stdout:2/703: dread d1/d3/d6/f92 [0,4194304] 0 2024-03-22T07:08:54.981 INFO:tasks.workunit.client.1.smithi149.stdout:7/619: symlink d1/d18/d1b/d99/d56/db4/lca 0 2024-03-22T07:08:54.983 INFO:tasks.workunit.client.1.smithi149.stdout:8/641: symlink d1/d20/d35/d22/d44/lca 0 2024-03-22T07:08:54.984 INFO:tasks.workunit.client.1.smithi149.stdout:0/548: mkdir d1/d9/d59/dc0/dc9 0 2024-03-22T07:08:54.985 INFO:tasks.workunit.client.1.smithi149.stdout:7/620: read d1/d1c/f94 [207693,106224] 0 2024-03-22T07:08:54.988 INFO:tasks.workunit.client.1.smithi149.stdout:8/642: mknod d1/d20/db4/ccb 0 2024-03-22T07:08:54.988 INFO:tasks.workunit.client.1.smithi149.stdout:7/621: creat d1/d1c/dbc/fcb x:0 0 0 2024-03-22T07:08:54.991 INFO:tasks.workunit.client.1.smithi149.stdout:4/634: read d0/d8/d1d/f8f [2311032,9287] 0 2024-03-22T07:08:54.993 INFO:tasks.workunit.client.1.smithi149.stdout:6/648: dwrite d9/d47/d52/f8a [0,4194304] 0 2024-03-22T07:08:54.993 INFO:tasks.workunit.client.1.smithi149.stdout:6/649: chown d9/df/f1e 2334 1 2024-03-22T07:08:54.993 INFO:tasks.workunit.client.1.smithi149.stdout:1/606: dread d4/d10/d7c/d6a/d6b/f84 [0,4194304] 0 2024-03-22T07:08:54.993 INFO:tasks.workunit.client.1.smithi149.stdout:0/549: creat d1/d31/d51/d64/d7e/fca x:0 0 0 2024-03-22T07:08:54.995 INFO:tasks.workunit.client.1.smithi149.stdout:9/618: sync 2024-03-22T07:08:54.996 INFO:tasks.workunit.client.1.smithi149.stdout:5/592: sync 2024-03-22T07:08:55.001 INFO:tasks.workunit.client.1.smithi149.stdout:4/635: creat d0/d8/dc/d14/d3e/fd0 x:0 0 0 2024-03-22T07:08:55.001 INFO:tasks.workunit.client.1.smithi149.stdout:4/636: chown d0/d8/dc/d4b/dbc 625993753 1 2024-03-22T07:08:55.001 INFO:tasks.workunit.client.1.smithi149.stdout:4/637: write d0/d8/dc/d14/d3e/f6a [522733,53277] 0 2024-03-22T07:08:55.001 INFO:tasks.workunit.client.1.smithi149.stdout:6/650: mknod d9/df/d22/d6d/d7d/ccd 0 2024-03-22T07:08:55.004 INFO:tasks.workunit.client.1.smithi149.stdout:0/550: creat d1/d31/d51/d64/d87/da6/fcb x:0 0 0 2024-03-22T07:08:55.005 INFO:tasks.workunit.client.1.smithi149.stdout:3/496: dwrite d0/d2/d30/d31/f3d [0,4194304] 0 2024-03-22T07:08:55.006 INFO:tasks.workunit.client.1.smithi149.stdout:9/619: dread db/d19/f1f [0,4194304] 0 2024-03-22T07:08:55.007 INFO:tasks.workunit.client.1.smithi149.stdout:5/593: rename d7/da/d20/d30/d3a/d3d/db0/lc6 to d7/da/d20/d30/d3a/lce 0 2024-03-22T07:08:55.007 INFO:tasks.workunit.client.1.smithi149.stdout:3/497: write d0/d4/d8/f21 [4422809,16261] 0 2024-03-22T07:08:55.007 INFO:tasks.workunit.client.1.smithi149.stdout:6/651: stat d9/df/d22/d6d/d7d/d96/l33 0 2024-03-22T07:08:55.008 INFO:tasks.workunit.client.1.smithi149.stdout:0/551: mkdir d1/d9/d59/dc0/dc9/dcc 0 2024-03-22T07:08:55.010 INFO:tasks.workunit.client.1.smithi149.stdout:5/594: mknod d7/da/d20/d30/d3a/d3d/db0/ccf 0 2024-03-22T07:08:55.010 INFO:tasks.workunit.client.1.smithi149.stdout:4/638: rmdir d0/d8/d5c 39 2024-03-22T07:08:55.010 INFO:tasks.workunit.client.1.smithi149.stdout:4/639: chown d0/d6b/l89 27996040 1 2024-03-22T07:08:55.011 INFO:tasks.workunit.client.1.smithi149.stdout:1/607: dread d4/d7/d5a/f6d [0,4194304] 0 2024-03-22T07:08:55.011 INFO:tasks.workunit.client.1.smithi149.stdout:5/595: getdents d7/da/d20/d30/d3a/d3d/d2c/d47 0 2024-03-22T07:08:55.012 INFO:tasks.workunit.client.1.smithi149.stdout:4/640: symlink d0/d8/dc/d4b/ld1 0 2024-03-22T07:08:55.013 INFO:tasks.workunit.client.1.smithi149.stdout:1/608: mknod d4/d7/d5a/dca/ccc 0 2024-03-22T07:08:55.013 INFO:tasks.workunit.client.1.smithi149.stdout:1/609: dread - d4/d7/fb4 zero size 2024-03-22T07:08:55.013 INFO:tasks.workunit.client.1.smithi149.stdout:1/610: chown d4/d20/d23/d3c/d85 1578873320 1 2024-03-22T07:08:55.014 INFO:tasks.workunit.client.1.smithi149.stdout:5/596: stat d7/da/d20/d36/c60 0 2024-03-22T07:08:55.014 INFO:tasks.workunit.client.1.smithi149.stdout:0/552: truncate d1/d31/f33 2011329 0 2024-03-22T07:08:55.015 INFO:tasks.workunit.client.1.smithi149.stdout:0/553: chown d1/d9/d14/d74/d77/l83 3411223 1 2024-03-22T07:08:55.015 INFO:tasks.workunit.client.1.smithi149.stdout:0/554: stat d1/d1a 0 2024-03-22T07:08:55.015 INFO:tasks.workunit.client.1.smithi149.stdout:0/555: fsync d1/d31/d51/d64/d87/f9d 0 2024-03-22T07:08:55.015 INFO:tasks.workunit.client.1.smithi149.stdout:0/556: truncate d1/d31/d51/d64/d87/fb0 4904903 0 2024-03-22T07:08:55.015 INFO:tasks.workunit.client.1.smithi149.stdout:4/641: mkdir d0/d8/d5c/dc1/dd2 0 2024-03-22T07:08:55.016 INFO:tasks.workunit.client.1.smithi149.stdout:0/557: dread d1/d9/d14/f55 [0,4194304] 0 2024-03-22T07:08:55.016 INFO:tasks.workunit.client.1.smithi149.stdout:0/558: write d1/d31/d51/d64/fa2 [557698,14489] 0 2024-03-22T07:08:55.017 INFO:tasks.workunit.client.1.smithi149.stdout:1/611: mknod d4/ccd 0 2024-03-22T07:08:55.017 INFO:tasks.workunit.client.1.smithi149.stdout:1/612: truncate d4/d10/f55 1627512 0 2024-03-22T07:08:55.017 INFO:tasks.workunit.client.1.smithi149.stdout:8/643: dwrite d1/d20/d35/d22/d44/d84/f48 [0,4194304] 0 2024-03-22T07:08:55.017 INFO:tasks.workunit.client.1.smithi149.stdout:2/704: dwrite d1/d3/d9a/fd5 [0,4194304] 0 2024-03-22T07:08:55.017 INFO:tasks.workunit.client.1.smithi149.stdout:9/620: dread db/fc [0,4194304] 0 2024-03-22T07:08:55.019 INFO:tasks.workunit.client.1.smithi149.stdout:5/597: rename d7/da/d20/l85 to d7/da/d20/d30/d3a/d3d/ld0 0 2024-03-22T07:08:55.023 INFO:tasks.workunit.client.1.smithi149.stdout:5/598: dread d7/da/d20/d36/d44/f6f [0,4194304] 0 2024-03-22T07:08:55.029 INFO:tasks.workunit.client.1.smithi149.stdout:4/642: truncate d0/d8/dc/f18 3011581 0 2024-03-22T07:08:55.029 INFO:tasks.workunit.client.1.smithi149.stdout:4/643: readlink d0/d8/dc/d14/l47 0 2024-03-22T07:08:55.029 INFO:tasks.workunit.client.1.smithi149.stdout:4/644: creat d0/d8/d1d/d24/d9c/fd3 x:0 0 0 2024-03-22T07:08:55.029 INFO:tasks.workunit.client.1.smithi149.stdout:4/645: stat d0/lf 0 2024-03-22T07:08:55.033 INFO:tasks.workunit.client.1.smithi149.stdout:1/613: truncate d4/d7/f57 2924763 0 2024-03-22T07:08:55.043 INFO:tasks.workunit.client.1.smithi149.stdout:2/705: link d1/d4d/d90/ff5 d1/d3/d6/d32/d6f/ffd 0 2024-03-22T07:08:55.044 INFO:tasks.workunit.client.1.smithi149.stdout:9/621: rename db/d12/c8f to db/d19/d39/cbf 0 2024-03-22T07:08:55.049 INFO:tasks.workunit.client.1.smithi149.stdout:6/652: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/fc7 [0,4194304] 0 2024-03-22T07:08:55.049 INFO:tasks.workunit.client.1.smithi149.stdout:6/653: write d9/df/d2e/f12 [1082254,111057] 0 2024-03-22T07:08:55.049 INFO:tasks.workunit.client.1.smithi149.stdout:4/646: mknod d0/cd4 0 2024-03-22T07:08:55.049 INFO:tasks.workunit.client.1.smithi149.stdout:4/647: creat d0/d8/fd5 x:0 0 0 2024-03-22T07:08:55.049 INFO:tasks.workunit.client.1.smithi149.stdout:4/648: fdatasync d0/d8/dc/d14/f9b 0 2024-03-22T07:08:55.049 INFO:tasks.workunit.client.1.smithi149.stdout:3/498: dwrite d0/d2/d30/d5d/d84/d8f/f59 [0,4194304] 0 2024-03-22T07:08:55.050 INFO:tasks.workunit.client.1.smithi149.stdout:1/614: mkdir d4/d20/d2d/d64/dce 0 2024-03-22T07:08:55.050 INFO:tasks.workunit.client.1.smithi149.stdout:1/615: stat d4/d7/d94/lc4 0 2024-03-22T07:08:55.050 INFO:tasks.workunit.client.1.smithi149.stdout:5/599: dwrite d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/f22 [0,4194304] 0 2024-03-22T07:08:55.051 INFO:tasks.workunit.client.1.smithi149.stdout:5/600: truncate d7/da/d20/d30/d3a/d3d/fb4 520341 0 2024-03-22T07:08:55.051 INFO:tasks.workunit.client.1.smithi149.stdout:2/706: dread d1/d3/d6/db2/f40 [0,4194304] 0 2024-03-22T07:08:55.052 INFO:tasks.workunit.client.1.smithi149.stdout:8/644: rmdir d1/d2/d9 39 2024-03-22T07:08:55.053 INFO:tasks.workunit.client.1.smithi149.stdout:8/645: dread - d1/d20/d35/d22/d44/d84/fc4 zero size 2024-03-22T07:08:55.053 INFO:tasks.workunit.client.1.smithi149.stdout:9/622: mknod db/d21/d37/d7c/da3/cc0 0 2024-03-22T07:08:55.054 INFO:tasks.workunit.client.1.smithi149.stdout:9/623: creat db/d19/d28/fc1 x:0 0 0 2024-03-22T07:08:55.057 INFO:tasks.workunit.client.1.smithi149.stdout:4/649: mknod d0/d8/d1d/cd6 0 2024-03-22T07:08:55.058 INFO:tasks.workunit.client.1.smithi149.stdout:3/499: symlink d0/d2/d30/d5d/d84/d8e/lbd 0 2024-03-22T07:08:55.058 INFO:tasks.workunit.client.1.smithi149.stdout:3/500: readlink d0/d2/l5 0 2024-03-22T07:08:55.063 INFO:tasks.workunit.client.1.smithi149.stdout:5/601: truncate d7/da/fd 206595 0 2024-03-22T07:08:55.065 INFO:tasks.workunit.client.1.smithi149.stdout:8/646: creat d1/d20/d35/d22/fcc x:0 0 0 2024-03-22T07:08:55.067 INFO:tasks.workunit.client.1.smithi149.stdout:9/624: unlink db/d21/d2e/c31 0 2024-03-22T07:08:55.068 INFO:tasks.workunit.client.1.smithi149.stdout:4/650: read d0/d29/f31 [2199582,95981] 0 2024-03-22T07:08:55.069 INFO:tasks.workunit.client.1.smithi149.stdout:3/501: unlink d0/d4/d8/c77 0 2024-03-22T07:08:55.070 INFO:tasks.workunit.client.1.smithi149.stdout:1/616: rmdir d4/d20/d23/d2f/d93 39 2024-03-22T07:08:55.071 INFO:tasks.workunit.client.1.smithi149.stdout:5/602: unlink d7/da/d20/cb8 0 2024-03-22T07:08:55.072 INFO:tasks.workunit.client.1.smithi149.stdout:8/647: link d1/d2/d9/d11/l80 d1/d2/d9/d11/d45/lcd 0 2024-03-22T07:08:55.073 INFO:tasks.workunit.client.1.smithi149.stdout:9/625: rename db/d22/f36 to db/d21/d37/d7c/da3/fc2 0 2024-03-22T07:08:55.075 INFO:tasks.workunit.client.1.smithi149.stdout:1/617: truncate d4/d7/d5d/f92 235000 0 2024-03-22T07:08:55.077 INFO:tasks.workunit.client.1.smithi149.stdout:5/603: rmdir d7/da/d20/d30/d3a/d3d/d2c/db2/dc8 0 2024-03-22T07:08:55.078 INFO:tasks.workunit.client.1.smithi149.stdout:2/707: dwrite d1/d3/d6/d3f/f7f [0,4194304] 0 2024-03-22T07:08:55.078 INFO:tasks.workunit.client.1.smithi149.stdout:6/654: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/f7a [0,4194304] 0 2024-03-22T07:08:55.079 INFO:tasks.workunit.client.1.smithi149.stdout:8/648: mkdir d1/d20/db4/dce 0 2024-03-22T07:08:55.080 INFO:tasks.workunit.client.1.smithi149.stdout:9/626: mknod db/d42/d49/d4b/d5f/cc3 0 2024-03-22T07:08:55.083 INFO:tasks.workunit.client.1.smithi149.stdout:1/618: link d4/d20/da0/fb0 d4/d20/d23/d3c/d8e/fcf 0 2024-03-22T07:08:55.083 INFO:tasks.workunit.client.1.smithi149.stdout:1/619: chown d4/d10/d7c/d6a/d6b/f81 99401 1 2024-03-22T07:08:55.083 INFO:tasks.workunit.client.1.smithi149.stdout:2/708: dread d1/d3/d5/d1f/d5e/f4a [0,4194304] 0 2024-03-22T07:08:55.083 INFO:tasks.workunit.client.1.smithi149.stdout:2/709: write d1/d3/d6/db2/f40 [2725181,108181] 0 2024-03-22T07:08:55.083 INFO:tasks.workunit.client.1.smithi149.stdout:2/710: creat d1/ffe x:0 0 0 2024-03-22T07:08:55.084 INFO:tasks.workunit.client.1.smithi149.stdout:5/604: symlink d7/da/d20/d36/d44/ld1 0 2024-03-22T07:08:55.088 INFO:tasks.workunit.client.1.smithi149.stdout:9/627: mknod db/d21/d37/d7c/cc4 0 2024-03-22T07:08:55.090 INFO:tasks.workunit.client.1.smithi149.stdout:1/620: mknod d4/d20/cd0 0 2024-03-22T07:08:55.090 INFO:tasks.workunit.client.1.smithi149.stdout:1/621: truncate d4/d20/d2d/faf 524617 0 2024-03-22T07:08:55.093 INFO:tasks.workunit.client.1.smithi149.stdout:4/651: dwrite d0/d8/dc/d2e/f62 [0,4194304] 0 2024-03-22T07:08:55.095 INFO:tasks.workunit.client.1.smithi149.stdout:5/605: rename d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/f22 to d7/da/d20/d30/d3a/d3d/d2c/d47/da4/fd2 0 2024-03-22T07:08:55.095 INFO:tasks.workunit.client.1.smithi149.stdout:5/606: chown d7/da/d20/d36 34390851 1 2024-03-22T07:08:55.097 INFO:tasks.workunit.client.1.smithi149.stdout:8/649: dread d1/d2/d9/d11/f77 [0,4194304] 0 2024-03-22T07:08:55.097 INFO:tasks.workunit.client.1.smithi149.stdout:8/650: stat d1/d20/d35/d22/d44/d4f/f7e 0 2024-03-22T07:08:55.097 INFO:tasks.workunit.client.1.smithi149.stdout:4/652: dread d0/d8/f1f [4194304,4194304] 0 2024-03-22T07:08:55.097 INFO:tasks.workunit.client.1.smithi149.stdout:8/651: fsync d1/d2/d9/d11/d8b/fb1 0 2024-03-22T07:08:55.097 INFO:tasks.workunit.client.1.smithi149.stdout:8/652: chown d1/d20/d35/d22/l83 103 1 2024-03-22T07:08:55.097 INFO:tasks.workunit.client.1.smithi149.stdout:6/655: dread d9/f17 [0,4194304] 0 2024-03-22T07:08:55.097 INFO:tasks.workunit.client.1.smithi149.stdout:9/628: unlink db/d22/fbd 0 2024-03-22T07:08:55.097 INFO:tasks.workunit.client.1.smithi149.stdout:4/653: write d0/d8/d5c/f5e [459525,84037] 0 2024-03-22T07:08:55.097 INFO:tasks.workunit.client.1.smithi149.stdout:4/654: fdatasync d0/d8/f34 0 2024-03-22T07:08:55.098 INFO:tasks.workunit.client.1.smithi149.stdout:1/622: creat d4/d7/d61/fd1 x:0 0 0 2024-03-22T07:08:55.098 INFO:tasks.workunit.client.1.smithi149.stdout:1/623: dread d4/d10/f4c [0,4194304] 0 2024-03-22T07:08:55.101 INFO:tasks.workunit.client.1.smithi149.stdout:1/624: dread d4/d20/d23/d2f/f62 [0,4194304] 0 2024-03-22T07:08:55.101 INFO:tasks.workunit.client.1.smithi149.stdout:1/625: fsync d4/d20/d23/d3c/d8e/db5/f53 0 2024-03-22T07:08:55.102 INFO:tasks.workunit.client.1.smithi149.stdout:4/655: dread d0/d8/dc/d14/f16 [0,4194304] 0 2024-03-22T07:08:55.102 INFO:tasks.workunit.client.1.smithi149.stdout:1/626: dread d4/d20/d23/d2f/d93/f9c [0,4194304] 0 2024-03-22T07:08:55.104 INFO:tasks.workunit.client.1.smithi149.stdout:1/627: dread d4/d20/d23/d2f/f62 [0,4194304] 0 2024-03-22T07:08:55.104 INFO:tasks.workunit.client.1.smithi149.stdout:5/607: creat d7/da/fd3 x:0 0 0 2024-03-22T07:08:55.104 INFO:tasks.workunit.client.1.smithi149.stdout:5/608: dread - d7/da/d20/d89/fa1 zero size 2024-03-22T07:08:55.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/628: read d4/d20/d2d/faf [233890,27668] 0 2024-03-22T07:08:55.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/629: chown d4/d20/d23/d3c/d85 2 1 2024-03-22T07:08:55.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/630: fsync d4/d10/f30 0 2024-03-22T07:08:55.105 INFO:tasks.workunit.client.1.smithi149.stdout:8/653: link d1/d2/d9/d11/d8b/cae d1/d20/d35/ccf 0 2024-03-22T07:08:55.106 INFO:tasks.workunit.client.1.smithi149.stdout:5/609: dread d7/da/d20/d36/f41 [0,4194304] 0 2024-03-22T07:08:55.108 INFO:tasks.workunit.client.1.smithi149.stdout:2/711: dwrite d1/d3/d5/d1f/f39 [0,4194304] 0 2024-03-22T07:08:55.108 INFO:tasks.workunit.client.1.smithi149.stdout:6/656: rename d9/df/d22/f31 to d9/df/d2e/fce 0 2024-03-22T07:08:55.108 INFO:tasks.workunit.client.1.smithi149.stdout:6/657: stat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/caf 0 2024-03-22T07:08:55.111 INFO:tasks.workunit.client.1.smithi149.stdout:9/629: mknod db/d21/d74/cc5 0 2024-03-22T07:08:55.111 INFO:tasks.workunit.client.1.smithi149.stdout:4/656: write d0/d8/d1d/d24/d82/fa1 [413017,93300] 0 2024-03-22T07:08:55.112 INFO:tasks.workunit.client.1.smithi149.stdout:1/631: symlink d4/d10/d7c/ld2 0 2024-03-22T07:08:55.114 INFO:tasks.workunit.client.1.smithi149.stdout:8/654: symlink d1/d2/d18/ld0 0 2024-03-22T07:08:55.116 INFO:tasks.workunit.client.1.smithi149.stdout:8/655: write d1/d2/d9/f3b [495736,45635] 0 2024-03-22T07:08:55.116 INFO:tasks.workunit.client.1.smithi149.stdout:8/656: chown d1/lc5 15054 1 2024-03-22T07:08:55.116 INFO:tasks.workunit.client.1.smithi149.stdout:5/610: mknod d7/da/d20/d45/cd4 0 2024-03-22T07:08:55.116 INFO:tasks.workunit.client.1.smithi149.stdout:1/632: write d4/d20/d23/f6f [116294,60004] 0 2024-03-22T07:08:55.116 INFO:tasks.workunit.client.1.smithi149.stdout:5/611: fdatasync d7/da/d20/d30/d3a/d3d/d2c/fb9 0 2024-03-22T07:08:55.116 INFO:tasks.workunit.client.1.smithi149.stdout:5/612: creat d7/da/d20/d36/fd5 x:0 0 0 2024-03-22T07:08:55.118 INFO:tasks.workunit.client.1.smithi149.stdout:6/658: dread d9/d47/f5a [0,4194304] 0 2024-03-22T07:08:55.118 INFO:tasks.workunit.client.1.smithi149.stdout:6/659: chown d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f71 28647 1 2024-03-22T07:08:55.118 INFO:tasks.workunit.client.1.smithi149.stdout:6/660: readlink d9/df/d22/d6d/l84 0 2024-03-22T07:08:55.118 INFO:tasks.workunit.client.1.smithi149.stdout:6/661: stat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f71 0 2024-03-22T07:08:55.119 INFO:tasks.workunit.client.1.smithi149.stdout:2/712: rename d1/d3/d5/d41/le6 to d1/d72/lff 0 2024-03-22T07:08:55.122 INFO:tasks.workunit.client.1.smithi149.stdout:9/630: mknod db/d42/d49/d4b/d5f/cc6 0 2024-03-22T07:08:55.124 INFO:tasks.workunit.client.1.smithi149.stdout:4/657: unlink d0/c98 0 2024-03-22T07:08:55.124 INFO:tasks.workunit.client.1.smithi149.stdout:4/658: write d0/d8/dc/d14/f16 [1686584,2] 0 2024-03-22T07:08:55.124 INFO:tasks.workunit.client.1.smithi149.stdout:4/659: truncate d0/d8/f1c 1146241 0 2024-03-22T07:08:55.126 INFO:tasks.workunit.client.1.smithi149.stdout:8/657: symlink d1/d2/d18/ld1 0 2024-03-22T07:08:55.126 INFO:tasks.workunit.client.1.smithi149.stdout:8/658: chown d1/d20/d35/d22/f38 868388851 1 2024-03-22T07:08:55.126 INFO:tasks.workunit.client.1.smithi149.stdout:8/659: truncate d1/d20/d35/d22/d44/d4f/f6e 1192265 0 2024-03-22T07:08:55.126 INFO:tasks.workunit.client.1.smithi149.stdout:1/633: read d4/d10/d58/f9b [1733311,78522] 0 2024-03-22T07:08:55.134 INFO:tasks.workunit.client.1.smithi149.stdout:2/713: creat d1/d3/d6/f100 x:0 0 0 2024-03-22T07:08:55.134 INFO:tasks.workunit.client.1.smithi149.stdout:6/662: write d9/d47/d52/f93 [2527707,62332] 0 2024-03-22T07:08:55.134 INFO:tasks.workunit.client.1.smithi149.stdout:4/660: symlink d0/d8/d4c/db4/ld7 0 2024-03-22T07:08:55.134 INFO:tasks.workunit.client.1.smithi149.stdout:1/634: mkdir d4/d7/dd3 0 2024-03-22T07:08:55.135 INFO:tasks.workunit.client.1.smithi149.stdout:5/613: symlink d7/ld6 0 2024-03-22T07:08:55.136 INFO:tasks.workunit.client.1.smithi149.stdout:5/614: write d7/da/d20/d89/fa1 [299398,73011] 0 2024-03-22T07:08:55.136 INFO:tasks.workunit.client.1.smithi149.stdout:5/615: write d7/da/d20/d30/d3a/d3d/d2c/d47/d7b/fc [2838720,40846] 0 2024-03-22T07:08:55.138 INFO:tasks.workunit.client.1.smithi149.stdout:9/631: truncate db/d33/f3d 735061 0 2024-03-22T07:08:55.138 INFO:tasks.workunit.client.1.smithi149.stdout:9/632: chown db/d21/d37/d7c/fbc 12 1 2024-03-22T07:08:55.142 INFO:tasks.workunit.client.1.smithi149.stdout:4/661: symlink d0/d8/dc/d4b/db3/ld8 0 2024-03-22T07:08:55.142 INFO:tasks.workunit.client.1.smithi149.stdout:4/662: chown d0/d8/d4c/l84 47031297 1 2024-03-22T07:08:55.142 INFO:tasks.workunit.client.1.smithi149.stdout:4/663: stat d0 0 2024-03-22T07:08:55.145 INFO:tasks.workunit.client.1.smithi149.stdout:1/635: mkdir d4/d7/d94/dd4 0 2024-03-22T07:08:55.146 INFO:tasks.workunit.client.1.smithi149.stdout:1/636: truncate d4/d7/d5d/f92 937289 0 2024-03-22T07:08:55.146 INFO:tasks.workunit.client.1.smithi149.stdout:5/616: rename d7/da/d20/d30/d3a/d3d/d2c/d47 to d7/da/d20/d30/d3a/d3d/d2c/d63/dd7 0 2024-03-22T07:08:55.147 INFO:tasks.workunit.client.1.smithi149.stdout:5/617: mknod d7/da/d20/d30/d3a/d3d/db0/cd8 0 2024-03-22T07:08:55.151 INFO:tasks.workunit.client.1.smithi149.stdout:5/618: dread d7/d35/f55 [0,4194304] 0 2024-03-22T07:08:55.151 INFO:tasks.workunit.client.1.smithi149.stdout:5/619: write d7/da/d20/d36/f9d [13192672,6937] 0 2024-03-22T07:08:55.153 INFO:tasks.workunit.client.1.smithi149.stdout:9/633: write db/fc [1159096,121757] 0 2024-03-22T07:08:55.154 INFO:tasks.workunit.client.1.smithi149.stdout:5/620: mkdir d7/da/d20/d45/dd9 0 2024-03-22T07:08:55.156 INFO:tasks.workunit.client.1.smithi149.stdout:9/634: mkdir db/d21/d37/d7c/dc7 0 2024-03-22T07:08:55.156 INFO:tasks.workunit.client.1.smithi149.stdout:9/635: write db/d19/d39/f5e [1330464,21197] 0 2024-03-22T07:08:55.156 INFO:tasks.workunit.client.1.smithi149.stdout:9/636: stat db/d21/d37 0 2024-03-22T07:08:55.159 INFO:tasks.workunit.client.1.smithi149.stdout:5/621: mkdir d7/da/d20/d89/da2/dda 0 2024-03-22T07:08:55.159 INFO:tasks.workunit.client.1.smithi149.stdout:5/622: dread - d7/da/d20/d30/d3a/d3d/d2c/fb9 zero size 2024-03-22T07:08:55.172 INFO:tasks.workunit.client.1.smithi149.stdout:8/660: dwrite d1/d20/f95 [0,4194304] 0 2024-03-22T07:08:55.175 INFO:tasks.workunit.client.1.smithi149.stdout:8/661: rename d1/d2/d9/df/l2f to d1/d2/ld2 0 2024-03-22T07:08:55.176 INFO:tasks.workunit.client.1.smithi149.stdout:7/622: sync 2024-03-22T07:08:55.177 INFO:tasks.workunit.client.1.smithi149.stdout:5/623: write d7/da/d20/d36/f9d [10669627,131065] 0 2024-03-22T07:08:55.177 INFO:tasks.workunit.client.1.smithi149.stdout:5/624: stat d7/da/c70 0 2024-03-22T07:08:55.177 INFO:tasks.workunit.client.1.smithi149.stdout:5/625: creat d7/da/d20/fdb x:0 0 0 2024-03-22T07:08:55.177 INFO:tasks.workunit.client.1.smithi149.stdout:5/626: read d7/da/d20/d30/d3a/d3d/d2c/d63/dd7/d7b/fc [2172078,35177] 0 2024-03-22T07:08:55.180 INFO:tasks.workunit.client.1.smithi149.stdout:5/627: readlink d7/d35/l50 0 2024-03-22T07:08:55.181 INFO:tasks.workunit.client.1.smithi149.stdout:5/628: mkdir d7/da/d20/d30/d3a/d93/ddc 0 2024-03-22T07:08:55.183 INFO:tasks.workunit.client.1.smithi149.stdout:5/629: mknod d7/cdd 0 2024-03-22T07:08:55.183 INFO:tasks.workunit.client.1.smithi149.stdout:5/630: chown d7/da/d20/d45/f7c 19975111 1 2024-03-22T07:08:55.184 INFO:tasks.workunit.client.1.smithi149.stdout:5/631: mknod d7/da/d20/d89/cde 0 2024-03-22T07:08:55.189 INFO:tasks.workunit.client.1.smithi149.stdout:2/714: dwrite d1/d3/d5/d1f/f1e [0,4194304] 0 2024-03-22T07:08:55.189 INFO:tasks.workunit.client.1.smithi149.stdout:5/632: symlink d7/da/d20/d89/da2/dda/ldf 0 2024-03-22T07:08:55.189 INFO:tasks.workunit.client.1.smithi149.stdout:2/715: symlink d1/d3/d5/d41/l101 0 2024-03-22T07:08:55.190 INFO:tasks.workunit.client.1.smithi149.stdout:6/663: dwrite d9/df/f15 [0,4194304] 0 2024-03-22T07:08:55.192 INFO:tasks.workunit.client.1.smithi149.stdout:4/664: dwrite d0/d8/dc/d14/d3e/fab [0,4194304] 0 2024-03-22T07:08:55.192 INFO:tasks.workunit.client.1.smithi149.stdout:1/637: dwrite d4/d20/d23/d2f/d93/fc6 [0,4194304] 0 2024-03-22T07:08:55.192 INFO:tasks.workunit.client.1.smithi149.stdout:9/637: dwrite db/d19/d39/d3b/f48 [0,4194304] 0 2024-03-22T07:08:55.192 INFO:tasks.workunit.client.1.smithi149.stdout:9/638: dread - db/d19/d28/fc1 zero size 2024-03-22T07:08:55.195 INFO:tasks.workunit.client.1.smithi149.stdout:2/716: dread d1/d3/d5/d1f/f55 [0,4194304] 0 2024-03-22T07:08:55.197 INFO:tasks.workunit.client.1.smithi149.stdout:6/664: symlink d9/d47/lcf 0 2024-03-22T07:08:55.198 INFO:tasks.workunit.client.1.smithi149.stdout:8/662: dread d1/d2/d9/f3b [0,4194304] 0 2024-03-22T07:08:55.200 INFO:tasks.workunit.client.1.smithi149.stdout:4/665: getdents d0/d8/dc/d14 0 2024-03-22T07:08:55.200 INFO:tasks.workunit.client.1.smithi149.stdout:4/666: write d0/d29/f3c [9078812,40501] 0 2024-03-22T07:08:55.200 INFO:tasks.workunit.client.1.smithi149.stdout:4/667: readlink d0/d8/dc/d74/l8a 0 2024-03-22T07:08:55.201 INFO:tasks.workunit.client.1.smithi149.stdout:7/623: dwrite d1/d18/d1b/d51/d5a/d6a/fb0 [4194304,4194304] 0 2024-03-22T07:08:55.203 INFO:tasks.workunit.client.1.smithi149.stdout:8/663: creat d1/fd3 x:0 0 0 2024-03-22T07:08:55.203 INFO:tasks.workunit.client.1.smithi149.stdout:8/664: chown d1/l3f 70105178 1 2024-03-22T07:08:55.203 INFO:tasks.workunit.client.1.smithi149.stdout:8/665: readlink d1/d20/d35/d22/d44/d4f/lac 0 2024-03-22T07:08:55.203 INFO:tasks.workunit.client.1.smithi149.stdout:8/666: write d1/f24 [713129,90357] 0 2024-03-22T07:08:55.206 INFO:tasks.workunit.client.1.smithi149.stdout:5/633: rename d7/da/d20/d30/d3a/d3d/d2c/d63 to d7/d35/de0 0 2024-03-22T07:08:55.208 INFO:tasks.workunit.client.1.smithi149.stdout:9/639: write db/d21/f23 [1512149,121445] 0 2024-03-22T07:08:55.209 INFO:tasks.workunit.client.1.smithi149.stdout:9/640: chown db/d19/d39/d3b/c95 95371590 1 2024-03-22T07:08:55.210 INFO:tasks.workunit.client.1.smithi149.stdout:7/624: mknod d1/d18/d1b/d1d/d72/ccc 0 2024-03-22T07:08:55.210 INFO:tasks.workunit.client.1.smithi149.stdout:7/625: readlink d1/d1c/dbc/la0 0 2024-03-22T07:08:55.210 INFO:tasks.workunit.client.1.smithi149.stdout:7/626: chown d1/d18/d1b/d51/d7a/faa 314876 1 2024-03-22T07:08:55.211 INFO:tasks.workunit.client.1.smithi149.stdout:6/665: getdents d9/df/d22/d6d/d7d/d96/d24/d34 0 2024-03-22T07:08:55.214 INFO:tasks.workunit.client.1.smithi149.stdout:8/667: getdents d1/d2 0 2024-03-22T07:08:55.214 INFO:tasks.workunit.client.1.smithi149.stdout:5/634: symlink d7/da/d20/d89/da2/dda/le1 0 2024-03-22T07:08:55.215 INFO:tasks.workunit.client.1.smithi149.stdout:5/635: fdatasync d7/da/d20/f99 0 2024-03-22T07:08:55.216 INFO:tasks.workunit.client.1.smithi149.stdout:2/717: rename d1/d3/d6/d3f/d74/la6 to d1/d3/de0/l102 0 2024-03-22T07:08:55.218 INFO:tasks.workunit.client.1.smithi149.stdout:9/641: truncate db/d21/d37/f5c 1977105 0 2024-03-22T07:08:55.218 INFO:tasks.workunit.client.1.smithi149.stdout:9/642: write db/d21/d8c/f96 [192892,66393] 0 2024-03-22T07:08:55.218 INFO:tasks.workunit.client.1.smithi149.stdout:9/643: chown db/d12 158836507 1 2024-03-22T07:08:55.218 INFO:tasks.workunit.client.1.smithi149.stdout:9/644: fsync db/d21/d37/d7c/fbc 0 2024-03-22T07:08:55.219 INFO:tasks.workunit.client.1.smithi149.stdout:7/627: mkdir d1/d1c/d49/dcd 0 2024-03-22T07:08:55.220 INFO:tasks.workunit.client.1.smithi149.stdout:4/668: dwrite d0/d8/dc/d74/d90/fad [0,4194304] 0 2024-03-22T07:08:55.220 INFO:tasks.workunit.client.1.smithi149.stdout:8/668: symlink d1/d20/db4/ld4 0 2024-03-22T07:08:55.222 INFO:tasks.workunit.client.1.smithi149.stdout:2/718: unlink d1/d3/d7d/fbe 0 2024-03-22T07:08:55.224 INFO:tasks.workunit.client.1.smithi149.stdout:0/559: sync 2024-03-22T07:08:55.224 INFO:tasks.workunit.client.1.smithi149.stdout:3/502: sync 2024-03-22T07:08:55.224 INFO:tasks.workunit.client.1.smithi149.stdout:3/503: fsync d0/d2/d30/d5d/d84/d8f/f50 0 2024-03-22T07:08:55.225 INFO:tasks.workunit.client.1.smithi149.stdout:4/669: write d0/d8/d4c/fbb [436970,61320] 0 2024-03-22T07:08:55.225 INFO:tasks.workunit.client.1.smithi149.stdout:8/669: truncate d1/d20/fc7 5938879 0 2024-03-22T07:08:55.229 INFO:tasks.workunit.client.1.smithi149.stdout:5/636: rmdir d7/da/d20/d45 39 2024-03-22T07:08:55.229 INFO:tasks.workunit.client.1.smithi149.stdout:5/637: read - d7/da/d20/d45/fb7 zero size 2024-03-22T07:08:55.229 INFO:tasks.workunit.client.1.smithi149.stdout:2/719: unlink d1/d3/d5/d41/c62 0 2024-03-22T07:08:55.229 INFO:tasks.workunit.client.1.smithi149.stdout:2/720: chown d1/d4d/d90 2676145 1 2024-03-22T07:08:55.229 INFO:tasks.workunit.client.1.smithi149.stdout:2/721: dread - d1/d4d/d6a/fd1 zero size 2024-03-22T07:08:55.231 INFO:tasks.workunit.client.1.smithi149.stdout:0/560: creat d1/d9/d59/dc0/dc9/fcd x:0 0 0 2024-03-22T07:08:55.233 INFO:tasks.workunit.client.1.smithi149.stdout:3/504: link d0/d2/d42/d66/d82/f94 d0/fbe 0 2024-03-22T07:08:55.233 INFO:tasks.workunit.client.1.smithi149.stdout:3/505: fdatasync d0/d2/d30/d7e/f69 0 2024-03-22T07:08:55.236 INFO:tasks.workunit.client.1.smithi149.stdout:4/670: mknod d0/d8/d1d/d24/d82/cd9 0 2024-03-22T07:08:55.236 INFO:tasks.workunit.client.1.smithi149.stdout:2/722: dread d1/d3/f4b [0,4194304] 0 2024-03-22T07:08:55.238 INFO:tasks.workunit.client.1.smithi149.stdout:5/638: rename d7/d35/de0/fc1 to d7/d35/de0/dd7/d7b/fe2 0 2024-03-22T07:08:55.242 INFO:tasks.workunit.client.1.smithi149.stdout:0/561: rmdir d1/d9 39 2024-03-22T07:08:55.242 INFO:tasks.workunit.client.1.smithi149.stdout:3/506: link d0/d2/d30/d5d/f65 d0/d2/d5a/fbf 0 2024-03-22T07:08:55.242 INFO:tasks.workunit.client.1.smithi149.stdout:2/723: unlink d1/d3/d5/la3 0 2024-03-22T07:08:55.242 INFO:tasks.workunit.client.1.smithi149.stdout:2/724: write d1/d3/d6/d32/d6f/ffd [538101,6923] 0 2024-03-22T07:08:55.245 INFO:tasks.workunit.client.1.smithi149.stdout:2/725: creat d1/d3/d6/d3f/f103 x:0 0 0 2024-03-22T07:08:55.247 INFO:tasks.workunit.client.1.smithi149.stdout:2/726: getdents d1/d72 0 2024-03-22T07:08:55.249 INFO:tasks.workunit.client.1.smithi149.stdout:2/727: unlink d1/d3/c15 0 2024-03-22T07:08:55.249 INFO:tasks.workunit.client.1.smithi149.stdout:2/728: stat d1/d3/d6/db2/d71/lb3 0 2024-03-22T07:08:55.254 INFO:tasks.workunit.client.1.smithi149.stdout:2/729: mkdir d1/d4d/d8b/d104 0 2024-03-22T07:08:55.254 INFO:tasks.workunit.client.1.smithi149.stdout:9/645: dwrite db/d19/f94 [0,4194304] 0 2024-03-22T07:08:55.254 INFO:tasks.workunit.client.1.smithi149.stdout:9/646: chown db/d12/l32 15059112 1 2024-03-22T07:08:55.267 INFO:tasks.workunit.client.1.smithi149.stdout:0/562: dread d1/d9/f3a [0,4194304] 0 2024-03-22T07:08:55.267 INFO:tasks.workunit.client.1.smithi149.stdout:0/563: chown d1/d9/d59/f6c 465097259 1 2024-03-22T07:08:55.269 INFO:tasks.workunit.client.1.smithi149.stdout:0/564: creat d1/d31/d52/da9/fce x:0 0 0 2024-03-22T07:08:55.269 INFO:tasks.workunit.client.1.smithi149.stdout:0/565: chown d1/l2 1522856435 1 2024-03-22T07:08:55.269 INFO:tasks.workunit.client.1.smithi149.stdout:0/566: readlink d1/d20/l75 0 2024-03-22T07:08:55.269 INFO:tasks.workunit.client.1.smithi149.stdout:0/567: dread - d1/d31/d52/da9/fce zero size 2024-03-22T07:08:55.269 INFO:tasks.workunit.client.1.smithi149.stdout:0/568: fdatasync d1/d31/d51/f5b 0 2024-03-22T07:08:55.270 INFO:tasks.workunit.client.1.smithi149.stdout:0/569: creat d1/d20/d9f/dc6/fcf x:0 0 0 2024-03-22T07:08:55.270 INFO:tasks.workunit.client.1.smithi149.stdout:4/671: dread d0/d6b/f81 [0,4194304] 0 2024-03-22T07:08:55.272 INFO:tasks.workunit.client.1.smithi149.stdout:3/507: dread d0/d2/d30/d5d/d84/d8f/f37 [0,4194304] 0 2024-03-22T07:08:55.272 INFO:tasks.workunit.client.1.smithi149.stdout:0/570: rename d1/d31/d51/d64/d87/fb0 to d1/d9/d14/d74/dc1/fd0 0 2024-03-22T07:08:55.272 INFO:tasks.workunit.client.1.smithi149.stdout:0/571: chown d1/d9/d14/l17 52 1 2024-03-22T07:08:55.273 INFO:tasks.workunit.client.1.smithi149.stdout:3/508: mknod d0/d2/d42/d66/cc0 0 2024-03-22T07:08:55.273 INFO:tasks.workunit.client.1.smithi149.stdout:0/572: write d1/d31/f53 [679141,110976] 0 2024-03-22T07:08:55.274 INFO:tasks.workunit.client.1.smithi149.stdout:0/573: stat d1/d20/d9f 0 2024-03-22T07:08:55.274 INFO:tasks.workunit.client.1.smithi149.stdout:0/574: dread - d1/d20/d9f/dc6/fcf zero size 2024-03-22T07:08:55.275 INFO:tasks.workunit.client.1.smithi149.stdout:0/575: mknod d1/d20/d9f/cd1 0 2024-03-22T07:08:55.290 INFO:tasks.workunit.client.1.smithi149.stdout:7/628: dwrite d1/d18/d1b/d51/d7a/faa [0,4194304] 0 2024-03-22T07:08:55.290 INFO:tasks.workunit.client.1.smithi149.stdout:7/629: readlink d1/d18/d1b/d51/d7a/d7d/d5b/d97/la2 0 2024-03-22T07:08:55.290 INFO:tasks.workunit.client.1.smithi149.stdout:7/630: creat d1/d18/d1b/d99/d56/d92/d9e/dab/fce x:0 0 0 2024-03-22T07:08:55.292 INFO:tasks.workunit.client.1.smithi149.stdout:5/639: dwrite d7/da/d20/d36/f41 [0,4194304] 0 2024-03-22T07:08:55.292 INFO:tasks.workunit.client.1.smithi149.stdout:5/640: fdatasync d7/da/d20/d45/f9c 0 2024-03-22T07:08:55.297 INFO:tasks.workunit.client.1.smithi149.stdout:2/730: dwrite d1/d4d/d6a/fd1 [0,4194304] 0 2024-03-22T07:08:55.297 INFO:tasks.workunit.client.1.smithi149.stdout:8/670: dwrite d1/d2/d9/d11/fa3 [0,4194304] 0 2024-03-22T07:08:55.297 INFO:tasks.workunit.client.1.smithi149.stdout:4/672: dwrite d0/d8/dc/d4b/f71 [0,4194304] 0 2024-03-22T07:08:55.298 INFO:tasks.workunit.client.1.smithi149.stdout:7/631: creat d1/d1c/fcf x:0 0 0 2024-03-22T07:08:55.301 INFO:tasks.workunit.client.1.smithi149.stdout:7/632: dread d1/d1c/f94 [0,4194304] 0 2024-03-22T07:08:55.301 INFO:tasks.workunit.client.1.smithi149.stdout:5/641: mknod d7/da/d20/d30/d3a/d93/d94/ce3 0 2024-03-22T07:08:55.306 INFO:tasks.workunit.client.1.smithi149.stdout:4/673: dread d0/d29/f3c [0,4194304] 0 2024-03-22T07:08:55.306 INFO:tasks.workunit.client.1.smithi149.stdout:8/671: rmdir d1/d20/d35 39 2024-03-22T07:08:55.306 INFO:tasks.workunit.client.1.smithi149.stdout:8/672: write d1/d20/d35/d22/d44/f6a [5794167,4179] 0 2024-03-22T07:08:55.307 INFO:tasks.workunit.client.1.smithi149.stdout:4/674: getdents d0/d8/dc/dbd 0 2024-03-22T07:08:55.308 INFO:tasks.workunit.client.1.smithi149.stdout:8/673: mkdir d1/d2/d9/df/dd5 0 2024-03-22T07:08:55.310 INFO:tasks.workunit.client.1.smithi149.stdout:5/642: rename d7/da/d20/d36/d44/c97 to d7/d35/ce4 0 2024-03-22T07:08:55.311 INFO:tasks.workunit.client.1.smithi149.stdout:4/675: unlink d0/d8/d1d/d24/d9c/fce 0 2024-03-22T07:08:55.313 INFO:tasks.workunit.client.1.smithi149.stdout:0/576: dwrite d1/d31/f5c [0,4194304] 0 2024-03-22T07:08:55.313 INFO:tasks.workunit.client.1.smithi149.stdout:0/577: truncate d1/d31/d52/f88 4694304 0 2024-03-22T07:08:55.313 INFO:tasks.workunit.client.1.smithi149.stdout:4/676: unlink d0/d6b/fa7 0 2024-03-22T07:08:55.314 INFO:tasks.workunit.client.1.smithi149.stdout:0/578: write d1/fb1 [306911,6054] 0 2024-03-22T07:08:55.314 INFO:tasks.workunit.client.1.smithi149.stdout:0/579: chown d1/d9/d14/d1f/cb3 118973843 1 2024-03-22T07:08:55.315 INFO:tasks.workunit.client.1.smithi149.stdout:3/509: dwrite d0/d4/d8/d24/d40/d62/f64 [0,4194304] 0 2024-03-22T07:08:55.315 INFO:tasks.workunit.client.1.smithi149.stdout:3/510: write d0/d4/d8/d24/fae [87027,24072] 0 2024-03-22T07:08:55.317 INFO:tasks.workunit.client.1.smithi149.stdout:4/677: unlink d0/d8/dc/d4b/f80 0 2024-03-22T07:08:55.321 INFO:tasks.workunit.client.1.smithi149.stdout:0/580: creat d1/d9/fd2 x:0 0 0 2024-03-22T07:08:55.322 INFO:tasks.workunit.client.1.smithi149.stdout:0/581: write d1/d31/d51/d64/d87/da6/fcb [177248,100872] 0 2024-03-22T07:08:55.322 INFO:tasks.workunit.client.1.smithi149.stdout:3/511: symlink d0/d4/d8/lc1 0 2024-03-22T07:08:55.326 INFO:tasks.workunit.client.1.smithi149.stdout:5/643: dread d7/d35/de0/dd7/da4/fd2 [0,4194304] 0 2024-03-22T07:08:55.326 INFO:tasks.workunit.client.1.smithi149.stdout:5/644: stat d7/da/d20/d36/d44/l5a 0 2024-03-22T07:08:55.330 INFO:tasks.workunit.client.1.smithi149.stdout:8/674: rename d1/d20 to d1/d2/d9/d11/d4d/dd6 0 2024-03-22T07:08:55.332 INFO:tasks.workunit.client.1.smithi149.stdout:4/678: link d0/d8/d1d/c78 d0/d29/cda 0 2024-03-22T07:08:55.334 INFO:tasks.workunit.client.1.smithi149.stdout:7/633: dwrite d1/d18/d1b/d51/d5a/f7c [0,4194304] 0 2024-03-22T07:08:55.334 INFO:tasks.workunit.client.1.smithi149.stdout:7/634: creat d1/d18/d1b/d1d/d72/fd0 x:0 0 0 2024-03-22T07:08:55.334 INFO:tasks.workunit.client.1.smithi149.stdout:7/635: write d1/d18/d1b/d51/d7a/d7d/d5b/d97/fc5 [3334,127078] 0 2024-03-22T07:08:55.334 INFO:tasks.workunit.client.1.smithi149.stdout:0/582: mkdir d1/d9/d14/dd3 0 2024-03-22T07:08:55.334 INFO:tasks.workunit.client.1.smithi149.stdout:0/583: fsync d1/d31/d51/f47 0 2024-03-22T07:08:55.334 INFO:tasks.workunit.client.1.smithi149.stdout:0/584: readlink d1/l2 0 2024-03-22T07:08:55.335 INFO:tasks.workunit.client.1.smithi149.stdout:1/638: sync 2024-03-22T07:08:55.335 INFO:tasks.workunit.client.1.smithi149.stdout:1/639: write d4/d7/d5a/f6d [4734011,85940] 0 2024-03-22T07:08:55.335 INFO:tasks.workunit.client.1.smithi149.stdout:1/640: truncate d4/d10/d58/d96/fc2 505247 0 2024-03-22T07:08:55.335 INFO:tasks.workunit.client.1.smithi149.stdout:1/641: readlink d4/d7/lc5 0 2024-03-22T07:08:55.335 INFO:tasks.workunit.client.1.smithi149.stdout:2/731: dwrite d1/d3/d5/d1f/f55 [0,4194304] 0 2024-03-22T07:08:55.335 INFO:tasks.workunit.client.1.smithi149.stdout:3/512: creat d0/d4/d8/d24/d40/d54/d85/dbc/fc2 x:0 0 0 2024-03-22T07:08:55.335 INFO:tasks.workunit.client.1.smithi149.stdout:3/513: chown d0/d2/d30/d5d/d84/d8f 1372 1 2024-03-22T07:08:55.340 INFO:tasks.workunit.client.1.smithi149.stdout:5/645: write d7/da/d20/d30/d3a/d3d/d2c/f8b [896653,55421] 0 2024-03-22T07:08:55.340 INFO:tasks.workunit.client.1.smithi149.stdout:5/646: creat d7/da/d20/d36/d84/fe5 x:0 0 0 2024-03-22T07:08:55.340 INFO:tasks.workunit.client.1.smithi149.stdout:4/679: creat d0/d8/dc/d14/d85/d97/fdb x:0 0 0 2024-03-22T07:08:55.340 INFO:tasks.workunit.client.1.smithi149.stdout:4/680: chown d0/d8/dc/d74/d76/cc4 154908 1 2024-03-22T07:08:55.340 INFO:tasks.workunit.client.1.smithi149.stdout:4/681: chown d0/d8/d1d/d24/c92 403 1 2024-03-22T07:08:55.345 INFO:tasks.workunit.client.1.smithi149.stdout:7/636: mkdir d1/d1c/d49/d8e/dd1 0 2024-03-22T07:08:55.347 INFO:tasks.workunit.client.1.smithi149.stdout:1/642: creat d4/d20/d2d/d64/d9f/dbe/fd5 x:0 0 0 2024-03-22T07:08:55.347 INFO:tasks.workunit.client.1.smithi149.stdout:1/643: write d4/d10/f63 [1956160,129231] 0 2024-03-22T07:08:55.347 INFO:tasks.workunit.client.1.smithi149.stdout:7/637: read d1/d1c/d49/d8e/fa1 [148279,37263] 0 2024-03-22T07:08:55.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:55 smithi149 ceph-mon[36068]: pgmap v10: 65 pgs: 65 active+clean; 1.3 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 54 MiB/s rd, 72 MiB/s wr, 702 op/s 2024-03-22T07:08:55.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:55 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:55.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:55 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:55.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:55 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:08:55.352 INFO:tasks.workunit.client.1.smithi149.stdout:5/647: write d7/da/d20/d36/f41 [3580048,8297] 0 2024-03-22T07:08:55.357 INFO:tasks.workunit.client.1.smithi149.stdout:1/644: truncate d4/d20/d23/f74 427967 0 2024-03-22T07:08:55.358 INFO:tasks.workunit.client.1.smithi149.stdout:7/638: unlink d1/d18/f34 0 2024-03-22T07:08:55.358 INFO:tasks.workunit.client.1.smithi149.stdout:8/675: dwrite d1/fc3 [0,4194304] 0 2024-03-22T07:08:55.358 INFO:tasks.workunit.client.1.smithi149.stdout:8/676: chown d1/d2/d18/f3c 0 1 2024-03-22T07:08:55.359 INFO:tasks.workunit.client.1.smithi149.stdout:2/732: mknod d1/d3/de0/c105 0 2024-03-22T07:08:55.361 INFO:tasks.workunit.client.1.smithi149.stdout:1/645: creat d4/d20/d23/d3c/d85/fd6 x:0 0 0 2024-03-22T07:08:55.362 INFO:tasks.workunit.client.1.smithi149.stdout:4/682: dwrite d0/d8/d4c/f57 [4194304,4194304] 0 2024-03-22T07:08:55.366 INFO:tasks.workunit.client.1.smithi149.stdout:5/648: dread d7/d35/de0/dd7/d7b/d81/f6a [0,4194304] 0 2024-03-22T07:08:55.367 INFO:tasks.workunit.client.1.smithi149.stdout:2/733: mknod d1/d4d/d8b/c106 0 2024-03-22T07:08:55.368 INFO:tasks.workunit.client.1.smithi149.stdout:3/514: rename d0/d2/d30/d31 to d0/d2/d30/d5d/d84/dc3 0 2024-03-22T07:08:55.369 INFO:tasks.workunit.client.1.smithi149.stdout:7/639: dread d1/d18/d1b/f2c [0,4194304] 0 2024-03-22T07:08:55.369 INFO:tasks.workunit.client.1.smithi149.stdout:7/640: creat d1/d18/d1b/d1d/d72/fd2 x:0 0 0 2024-03-22T07:08:55.370 INFO:tasks.workunit.client.1.smithi149.stdout:1/646: creat d4/d10/d58/d68/fd7 x:0 0 0 2024-03-22T07:08:55.374 INFO:tasks.workunit.client.1.smithi149.stdout:5/649: link d7/da/d20/d36/d44/l5a d7/da/d20/d30/d3a/d3d/le6 0 2024-03-22T07:08:55.374 INFO:tasks.workunit.client.1.smithi149.stdout:8/677: dread d1/d2/d9/d11/d4d/dd6/fc7 [0,4194304] 0 2024-03-22T07:08:55.374 INFO:tasks.workunit.client.1.smithi149.stdout:2/734: write d1/d3/d5/d1f/d68/d94/fd4 [3693254,28105] 0 2024-03-22T07:08:55.376 INFO:tasks.workunit.client.1.smithi149.stdout:3/515: mknod d0/d4/d8/d24/d40/d62/cc4 0 2024-03-22T07:08:55.377 INFO:tasks.workunit.client.1.smithi149.stdout:3/516: write d0/d2/d30/d5d/d84/d8e/d9d/fb9 [947035,86333] 0 2024-03-22T07:08:55.379 INFO:tasks.workunit.client.1.smithi149.stdout:7/641: mknod d1/d18/d1b/d51/d5a/d6a/da6/cd3 0 2024-03-22T07:08:55.379 INFO:tasks.workunit.client.1.smithi149.stdout:7/642: readlink d1/d18/d1b/d99/d56/db4/lca 0 2024-03-22T07:08:55.382 INFO:tasks.workunit.client.1.smithi149.stdout:5/650: mkdir d7/da/d20/d89/da2/de7 0 2024-03-22T07:08:55.383 INFO:tasks.workunit.client.1.smithi149.stdout:8/678: mkdir d1/d2/d9/d11/d4d/dd6/d35/d22/dd7 0 2024-03-22T07:08:55.386 INFO:tasks.workunit.client.1.smithi149.stdout:7/643: truncate d1/d18/fa4 341188 0 2024-03-22T07:08:55.386 INFO:tasks.workunit.client.1.smithi149.stdout:4/683: dwrite d0/d8/d1d/f8f [0,4194304] 0 2024-03-22T07:08:55.389 INFO:tasks.workunit.client.1.smithi149.stdout:1/647: rename d4/d20/d23 to d4/d10/dd8 0 2024-03-22T07:08:55.390 INFO:tasks.workunit.client.1.smithi149.stdout:3/517: read d0/d2/d42/d66/f7a [1810215,24253] 0 2024-03-22T07:08:55.390 INFO:tasks.workunit.client.1.smithi149.stdout:8/679: unlink d1/d2/d9/d11/d4d/dd6/d35/d22/c68 0 2024-03-22T07:08:55.390 INFO:tasks.workunit.client.1.smithi149.stdout:8/680: readlink d1/d2/l4e 0 2024-03-22T07:08:55.392 INFO:tasks.workunit.client.1.smithi149.stdout:2/735: dwrite d1/d3/d9a/fad [0,4194304] 0 2024-03-22T07:08:55.392 INFO:tasks.workunit.client.1.smithi149.stdout:2/736: write d1/d4d/d69/d75/fa1 [3166883,88481] 0 2024-03-22T07:08:55.392 INFO:tasks.workunit.client.1.smithi149.stdout:2/737: fsync d1/d3/d5/d1f/f55 0 2024-03-22T07:08:55.393 INFO:tasks.workunit.client.1.smithi149.stdout:1/648: link d4/c16 d4/d20/d2d/cd9 0 2024-03-22T07:08:55.393 INFO:tasks.workunit.client.1.smithi149.stdout:1/649: fsync d4/d10/dd8/d3c/d85/d89/f97 0 2024-03-22T07:08:55.394 INFO:tasks.workunit.client.1.smithi149.stdout:6/666: sync 2024-03-22T07:08:55.395 INFO:tasks.workunit.client.1.smithi149.stdout:9/647: sync 2024-03-22T07:08:55.395 INFO:tasks.workunit.client.1.smithi149.stdout:9/648: stat db/d19/d39/d68/d6f 0 2024-03-22T07:08:55.395 INFO:tasks.workunit.client.1.smithi149.stdout:6/667: readlink d9/df/d22/d6d/d7d/d96/d24/d34/l8f 0 2024-03-22T07:08:55.395 INFO:tasks.workunit.client.1.smithi149.stdout:7/644: dread d1/d18/d1b/d99/f68 [0,4194304] 0 2024-03-22T07:08:55.397 INFO:tasks.workunit.client.1.smithi149.stdout:2/738: fsync d1/d4d/d69/d75/fd3 0 2024-03-22T07:08:55.399 INFO:tasks.workunit.client.1.smithi149.stdout:1/650: mknod d4/d7/d5d/cda 0 2024-03-22T07:08:55.399 INFO:tasks.workunit.client.1.smithi149.stdout:1/651: chown d4/d20/d2d/cc3 239 1 2024-03-22T07:08:55.400 INFO:tasks.workunit.client.1.smithi149.stdout:7/645: dread d1/d18/d1b/d1d/f13 [0,4194304] 0 2024-03-22T07:08:55.401 INFO:tasks.workunit.client.1.smithi149.stdout:6/668: creat d9/df/d22/d6d/d7d/d96/db1/d55/fd0 x:0 0 0 2024-03-22T07:08:55.402 INFO:tasks.workunit.client.1.smithi149.stdout:2/739: creat d1/d3/d6/d3f/d86/f107 x:0 0 0 2024-03-22T07:08:55.404 INFO:tasks.workunit.client.1.smithi149.stdout:1/652: mkdir d4/d10/dd8/d3c/d8e/db5/ddb 0 2024-03-22T07:08:55.404 INFO:tasks.workunit.client.1.smithi149.stdout:1/653: read d4/d10/dd8/d3c/d8e/db5/f83 [2011667,125230] 0 2024-03-22T07:08:55.404 INFO:tasks.workunit.client.1.smithi149.stdout:1/654: chown d4/d10/dd8/f27 739614 1 2024-03-22T07:08:55.404 INFO:tasks.workunit.client.1.smithi149.stdout:1/655: truncate d4/d10/dd8/d3c/f47 2632283 0 2024-03-22T07:08:55.405 INFO:tasks.workunit.client.1.smithi149.stdout:4/684: dread d0/d8/dc/d4b/f86 [0,4194304] 0 2024-03-22T07:08:55.405 INFO:tasks.workunit.client.1.smithi149.stdout:4/685: dread - d0/d8/fa2 zero size 2024-03-22T07:08:55.405 INFO:tasks.workunit.client.1.smithi149.stdout:4/686: write d0/d8/f56 [2738064,54807] 0 2024-03-22T07:08:55.405 INFO:tasks.workunit.client.1.smithi149.stdout:9/649: symlink db/lc8 0 2024-03-22T07:08:55.405 INFO:tasks.workunit.client.1.smithi149.stdout:9/650: readlink db/d21/d37/d3a/la9 0 2024-03-22T07:08:55.406 INFO:tasks.workunit.client.1.smithi149.stdout:6/669: mknod d9/df/d2e/d63/cd1 0 2024-03-22T07:08:55.406 INFO:tasks.workunit.client.1.smithi149.stdout:6/670: creat d9/df/d22/d6d/d7d/d96/db1/d55/fd2 x:0 0 0 2024-03-22T07:08:55.406 INFO:tasks.workunit.client.1.smithi149.stdout:6/671: truncate d9/df/d2e/d63/fb3 1400444 0 2024-03-22T07:08:55.407 INFO:tasks.workunit.client.1.smithi149.stdout:2/740: creat d1/d72/f108 x:0 0 0 2024-03-22T07:08:55.410 INFO:tasks.workunit.client.1.smithi149.stdout:1/656: symlink d4/d7/d5d/ldc 0 2024-03-22T07:08:55.410 INFO:tasks.workunit.client.1.smithi149.stdout:1/657: write d4/d10/d7c/f7d [21348,16720] 0 2024-03-22T07:08:55.413 INFO:tasks.workunit.client.1.smithi149.stdout:2/741: truncate d1/d3/d5/d1f/d5e/ff2 245507 0 2024-03-22T07:08:55.416 INFO:tasks.workunit.client.1.smithi149.stdout:1/658: creat d4/d20/d2d/d64/dce/fdd x:0 0 0 2024-03-22T07:08:55.416 INFO:tasks.workunit.client.1.smithi149.stdout:9/651: dread db/d33/f92 [0,4194304] 0 2024-03-22T07:08:55.416 INFO:tasks.workunit.client.1.smithi149.stdout:4/687: write d0/d29/f6e [3405255,24393] 0 2024-03-22T07:08:55.417 INFO:tasks.workunit.client.1.smithi149.stdout:1/659: symlink d4/d7/d5a/dca/lde 0 2024-03-22T07:08:55.418 INFO:tasks.workunit.client.1.smithi149.stdout:9/652: mkdir db/d21/d8c/dc9 0 2024-03-22T07:08:55.419 INFO:tasks.workunit.client.1.smithi149.stdout:9/653: readlink db/d19/d28/l65 0 2024-03-22T07:08:55.420 INFO:tasks.workunit.client.1.smithi149.stdout:9/654: unlink db/d22/c25 0 2024-03-22T07:08:55.430 INFO:tasks.workunit.client.1.smithi149.stdout:6/672: dwrite d9/df/d22/d6d/fc4 [0,4194304] 0 2024-03-22T07:08:55.430 INFO:tasks.workunit.client.1.smithi149.stdout:6/673: fdatasync d9/da2/fbc 0 2024-03-22T07:08:55.431 INFO:tasks.workunit.client.1.smithi149.stdout:2/742: write d1/d3/d6/d3f/f7f [2492371,109236] 0 2024-03-22T07:08:55.432 INFO:tasks.workunit.client.1.smithi149.stdout:7/646: dwrite d1/d18/d1b/d51/d7a/d7d/d5b/d97/fc5 [0,4194304] 0 2024-03-22T07:08:55.432 INFO:tasks.workunit.client.1.smithi149.stdout:2/743: write d1/d3/d6/d3f/f103 [859929,123755] 0 2024-03-22T07:08:55.436 INFO:tasks.workunit.client.1.smithi149.stdout:6/674: mknod d9/df/d22/d6d/d7d/d96/d24/d68/d73/cd3 0 2024-03-22T07:08:55.439 INFO:tasks.workunit.client.1.smithi149.stdout:6/675: symlink d9/df/d22/d6d/d7d/d96/d24/d34/ld4 0 2024-03-22T07:08:55.441 INFO:tasks.workunit.client.1.smithi149.stdout:6/676: mknod d9/df/d22/d6d/d7d/cd5 0 2024-03-22T07:08:55.446 INFO:tasks.workunit.client.1.smithi149.stdout:2/744: dread d1/d8/fb0 [0,4194304] 0 2024-03-22T07:08:55.447 INFO:tasks.workunit.client.1.smithi149.stdout:9/655: dwrite db/d19/f94 [0,4194304] 0 2024-03-22T07:08:55.452 INFO:tasks.workunit.client.1.smithi149.stdout:2/745: rename d1/d4d/d69/d75/fed to d1/d3/d5/f109 0 2024-03-22T07:08:55.454 INFO:tasks.workunit.client.1.smithi149.stdout:2/746: rmdir d1/d3/d5/d1f/d68/dbb 0 2024-03-22T07:08:55.454 INFO:tasks.workunit.client.1.smithi149.stdout:2/747: creat d1/d3/f10a x:0 0 0 2024-03-22T07:08:55.456 INFO:tasks.workunit.client.1.smithi149.stdout:2/748: read d1/d8/f19 [653516,123726] 0 2024-03-22T07:08:55.457 INFO:tasks.workunit.client.1.smithi149.stdout:4/688: dwrite d0/d8/dc/d14/f9b [0,4194304] 0 2024-03-22T07:08:55.457 INFO:tasks.workunit.client.1.smithi149.stdout:2/749: rename d1/d3/d6/d3f/d74/l96 to d1/d4d/l10b 0 2024-03-22T07:08:55.459 INFO:tasks.workunit.client.1.smithi149.stdout:9/656: write db/d42/fad [1505905,57438] 0 2024-03-22T07:08:55.459 INFO:tasks.workunit.client.1.smithi149.stdout:9/657: fdatasync db/d22/f2b 0 2024-03-22T07:08:55.461 INFO:tasks.workunit.client.1.smithi149.stdout:9/658: write db/d19/d39/f5e [68864,70963] 0 2024-03-22T07:08:55.462 INFO:tasks.workunit.client.1.smithi149.stdout:2/750: symlink d1/d3/d5/l10c 0 2024-03-22T07:08:55.462 INFO:tasks.workunit.client.1.smithi149.stdout:2/751: creat d1/d3/d6/d3f/f10d x:0 0 0 2024-03-22T07:08:55.462 INFO:tasks.workunit.client.1.smithi149.stdout:2/752: truncate d1/d3/d6/d32/d6f/ffd 646306 0 2024-03-22T07:08:55.463 INFO:tasks.workunit.client.1.smithi149.stdout:2/753: dread d1/d3/f4b [0,4194304] 0 2024-03-22T07:08:55.465 INFO:tasks.workunit.client.1.smithi149.stdout:2/754: getdents d1/d3/d7d 0 2024-03-22T07:08:55.466 INFO:tasks.workunit.client.1.smithi149.stdout:1/660: dwrite d4/d20/d2d/d64/dce/fdd [0,4194304] 0 2024-03-22T07:08:55.466 INFO:tasks.workunit.client.1.smithi149.stdout:1/661: stat d4/d10/f4d 0 2024-03-22T07:08:55.466 INFO:tasks.workunit.client.1.smithi149.stdout:6/677: dwrite d9/df/d22/f74 [0,4194304] 0 2024-03-22T07:08:55.466 INFO:tasks.workunit.client.1.smithi149.stdout:6/678: rename d9/df to d9/df/d22/d6d/d7d/d96/db1/d55/dd6 22 2024-03-22T07:08:55.467 INFO:tasks.workunit.client.1.smithi149.stdout:7/647: dwrite d1/fa7 [4194304,4194304] 0 2024-03-22T07:08:55.467 INFO:tasks.workunit.client.1.smithi149.stdout:7/648: chown d1/l12 200682 1 2024-03-22T07:08:55.467 INFO:tasks.workunit.client.1.smithi149.stdout:2/755: mknod d1/d8/c10e 0 2024-03-22T07:08:55.467 INFO:tasks.workunit.client.1.smithi149.stdout:4/689: write d0/d8/dc/d4b/f71 [2742520,60043] 0 2024-03-22T07:08:55.467 INFO:tasks.workunit.client.1.smithi149.stdout:4/690: fdatasync d0/f67 0 2024-03-22T07:08:55.470 INFO:tasks.workunit.client.1.smithi149.stdout:1/662: symlink d4/d10/dd8/d3c/ldf 0 2024-03-22T07:08:55.470 INFO:tasks.workunit.client.1.smithi149.stdout:1/663: truncate d4/d10/dd8/d3c/d8e/db5/f53 4596296 0 2024-03-22T07:08:55.473 INFO:tasks.workunit.client.1.smithi149.stdout:6/679: rmdir d9/df/d22/d6d/d7d/d96/d24/d34 39 2024-03-22T07:08:55.474 INFO:tasks.workunit.client.1.smithi149.stdout:6/680: truncate d9/df/d22/d6d/d7d/d96/d24/d68/d73/fa6 687132 0 2024-03-22T07:08:55.474 INFO:tasks.workunit.client.1.smithi149.stdout:6/681: chown d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/da1 3 1 2024-03-22T07:08:55.478 INFO:tasks.workunit.client.1.smithi149.stdout:9/659: dwrite db/d21/f23 [0,4194304] 0 2024-03-22T07:08:55.479 INFO:tasks.workunit.client.1.smithi149.stdout:2/756: mknod d1/d4d/d6a/c10f 0 2024-03-22T07:08:55.480 INFO:tasks.workunit.client.1.smithi149.stdout:2/757: write d1/d3/f4b [453941,88194] 0 2024-03-22T07:08:55.481 INFO:tasks.workunit.client.1.smithi149.stdout:6/682: creat d9/d47/d52/fd7 x:0 0 0 2024-03-22T07:08:55.481 INFO:tasks.workunit.client.1.smithi149.stdout:6/683: fdatasync d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 0 2024-03-22T07:08:55.481 INFO:tasks.workunit.client.1.smithi149.stdout:6/684: fsync d9/da2/fb0 0 2024-03-22T07:08:55.486 INFO:tasks.workunit.client.1.smithi149.stdout:7/649: rmdir d1/d18/d1b/d51 39 2024-03-22T07:08:55.489 INFO:tasks.workunit.client.1.smithi149.stdout:9/660: mknod db/d19/d93/cca 0 2024-03-22T07:08:55.489 INFO:tasks.workunit.client.1.smithi149.stdout:1/664: dwrite d4/d7/d61/fd1 [0,4194304] 0 2024-03-22T07:08:55.490 INFO:tasks.workunit.client.1.smithi149.stdout:1/665: fdatasync d4/d20/d2d/d64/d9f/dbe/fd5 0 2024-03-22T07:08:55.491 INFO:tasks.workunit.client.1.smithi149.stdout:4/691: mkdir d0/d8/dc/d74/ddc 0 2024-03-22T07:08:55.491 INFO:tasks.workunit.client.1.smithi149.stdout:4/692: creat d0/d8/dc/d74/fdd x:0 0 0 2024-03-22T07:08:55.493 INFO:tasks.workunit.client.1.smithi149.stdout:1/666: mknod d4/d10/d58/d68/ce0 0 2024-03-22T07:08:55.493 INFO:tasks.workunit.client.1.smithi149.stdout:1/667: chown d4/d7/l8d 54184 1 2024-03-22T07:08:55.494 INFO:tasks.workunit.client.1.smithi149.stdout:9/661: getdents db/d19 0 2024-03-22T07:08:55.494 INFO:tasks.workunit.client.1.smithi149.stdout:9/662: chown db/d42/d49/d4b/d5f 4286 1 2024-03-22T07:08:55.496 INFO:tasks.workunit.client.1.smithi149.stdout:4/693: symlink d0/d8/lde 0 2024-03-22T07:08:55.496 INFO:tasks.workunit.client.1.smithi149.stdout:4/694: stat d0/c12 0 2024-03-22T07:08:55.499 INFO:tasks.workunit.client.1.smithi149.stdout:1/668: rename d4/d10/dd8/d2f/d93/fc6 to d4/d10/dd8/d3c/d8e/db5/fe1 0 2024-03-22T07:08:55.502 INFO:tasks.workunit.client.1.smithi149.stdout:9/663: rename db/d22/l57 to db/d42/d49/d4b/d5f/lcb 0 2024-03-22T07:08:55.505 INFO:tasks.workunit.client.1.smithi149.stdout:7/650: write d1/f41 [2306834,20283] 0 2024-03-22T07:08:55.506 INFO:tasks.workunit.client.1.smithi149.stdout:2/758: dwrite d1/d3/d5/d1f/d5e/ff2 [0,4194304] 0 2024-03-22T07:08:55.509 INFO:tasks.workunit.client.1.smithi149.stdout:7/651: mknod d1/d18/d1b/d99/d56/d92/d9e/cd4 0 2024-03-22T07:08:55.511 INFO:tasks.workunit.client.1.smithi149.stdout:6/685: dwrite d9/df/d2e/fce [0,4194304] 0 2024-03-22T07:08:55.515 INFO:tasks.workunit.client.1.smithi149.stdout:7/652: truncate d1/d18/d1b/d1d/f8 582714 0 2024-03-22T07:08:55.517 INFO:tasks.workunit.client.1.smithi149.stdout:6/686: link d9/df/d22/d6d/d7d/d96/d24/d68/f9f d9/df/d22/d6d/d7d/d96/d24/d68/d73/fd8 0 2024-03-22T07:08:55.521 INFO:tasks.workunit.client.1.smithi149.stdout:6/687: dread d9/df/d2e/f1c [0,4194304] 0 2024-03-22T07:08:55.522 INFO:tasks.workunit.client.1.smithi149.stdout:6/688: mknod d9/df/d22/d6d/d7d/d96/d24/d34/d37/cd9 0 2024-03-22T07:08:55.525 INFO:tasks.workunit.client.1.smithi149.stdout:6/689: getdents d9/df/d22 0 2024-03-22T07:08:55.527 INFO:tasks.workunit.client.1.smithi149.stdout:6/690: rename d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/fae to d9/df/fda 0 2024-03-22T07:08:55.527 INFO:tasks.workunit.client.1.smithi149.stdout:6/691: chown d9/df/d22/d6d/d7d/fb6 230295172 1 2024-03-22T07:08:55.527 INFO:tasks.workunit.client.1.smithi149.stdout:6/692: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/da1/fdb x:0 0 0 2024-03-22T07:08:55.531 INFO:tasks.workunit.client.1.smithi149.stdout:2/759: dread d1/d3/d6/db2/f65 [0,4194304] 0 2024-03-22T07:08:55.536 INFO:tasks.workunit.client.1.smithi149.stdout:6/693: write d9/f3c [3706795,1837] 0 2024-03-22T07:08:55.536 INFO:tasks.workunit.client.1.smithi149.stdout:6/694: fdatasync d9/df/d22/d6d/d7d/d96/d24/d34/f4c 0 2024-03-22T07:08:55.536 INFO:tasks.workunit.client.1.smithi149.stdout:6/695: write d9/df/d22/d6d/d7d/d96/db1/d55/fd2 [803109,121078] 0 2024-03-22T07:08:55.536 INFO:tasks.workunit.client.1.smithi149.stdout:6/696: fsync d9/df/d22/f87 0 2024-03-22T07:08:55.538 INFO:tasks.workunit.client.1.smithi149.stdout:1/669: dwrite d4/d20/d2d/d64/d9f/dbe/fd5 [0,4194304] 0 2024-03-22T07:08:55.538 INFO:tasks.workunit.client.1.smithi149.stdout:4/695: dwrite d0/d8/dc/d14/d3e/f6a [0,4194304] 0 2024-03-22T07:08:55.538 INFO:tasks.workunit.client.1.smithi149.stdout:1/670: chown d4/d10/dd8/c5f 3391 1 2024-03-22T07:08:55.538 INFO:tasks.workunit.client.1.smithi149.stdout:6/697: symlink d9/df/d2e/d63/ldc 0 2024-03-22T07:08:55.538 INFO:tasks.workunit.client.1.smithi149.stdout:9/664: dwrite db/d19/f3c [0,4194304] 0 2024-03-22T07:08:55.540 INFO:tasks.workunit.client.1.smithi149.stdout:7/653: dwrite d1/d18/f1e [0,4194304] 0 2024-03-22T07:08:55.540 INFO:tasks.workunit.client.1.smithi149.stdout:7/654: fdatasync d1/d18/d1b/d1d/fd 0 2024-03-22T07:08:55.542 INFO:tasks.workunit.client.1.smithi149.stdout:1/671: truncate d4/d7/d5d/f72 1311505 0 2024-03-22T07:08:55.543 INFO:tasks.workunit.client.1.smithi149.stdout:1/672: stat d4/d10/d7c/f98 0 2024-03-22T07:08:55.544 INFO:tasks.workunit.client.1.smithi149.stdout:6/698: mkdir d9/df/d22/d6d/ddd 0 2024-03-22T07:08:55.544 INFO:tasks.workunit.client.1.smithi149.stdout:6/699: fdatasync d9/df/f15 0 2024-03-22T07:08:55.544 INFO:tasks.workunit.client.1.smithi149.stdout:6/700: truncate d9/f32 936901 0 2024-03-22T07:08:55.544 INFO:tasks.workunit.client.1.smithi149.stdout:6/701: creat d9/fde x:0 0 0 2024-03-22T07:08:55.549 INFO:tasks.workunit.client.1.smithi149.stdout:7/655: creat d1/d18/d1b/d99/d56/db4/fd5 x:0 0 0 2024-03-22T07:08:55.551 INFO:tasks.workunit.client.1.smithi149.stdout:4/696: mkdir d0/d8/dc/d2e/d6f/ddf 0 2024-03-22T07:08:55.551 INFO:tasks.workunit.client.1.smithi149.stdout:9/665: read db/d22/f69 [47694,22468] 0 2024-03-22T07:08:55.551 INFO:tasks.workunit.client.1.smithi149.stdout:9/666: readlink db/l27 0 2024-03-22T07:08:55.551 INFO:tasks.workunit.client.1.smithi149.stdout:9/667: fsync db/d19/d39/d68/fac 0 2024-03-22T07:08:55.553 INFO:tasks.workunit.client.1.smithi149.stdout:1/673: creat d4/d10/dd8/d3c/d85/d89/fe2 x:0 0 0 2024-03-22T07:08:55.553 INFO:tasks.workunit.client.1.smithi149.stdout:1/674: fsync d4/d10/f4d 0 2024-03-22T07:08:55.556 INFO:tasks.workunit.client.1.smithi149.stdout:4/697: symlink d0/d8/dc/d2e/d6f/le0 0 2024-03-22T07:08:55.556 INFO:tasks.workunit.client.1.smithi149.stdout:4/698: creat d0/d8/dc/d74/fe1 x:0 0 0 2024-03-22T07:08:55.557 INFO:tasks.workunit.client.1.smithi149.stdout:4/699: write d0/d29/f58 [320630,49328] 0 2024-03-22T07:08:55.558 INFO:tasks.workunit.client.1.smithi149.stdout:1/675: creat d4/d7/d5d/d82/fe3 x:0 0 0 2024-03-22T07:08:55.558 INFO:tasks.workunit.client.1.smithi149.stdout:1/676: creat d4/d20/d2d/d64/fe4 x:0 0 0 2024-03-22T07:08:55.558 INFO:tasks.workunit.client.1.smithi149.stdout:1/677: write d4/d10/d58/f9b [5218959,93868] 0 2024-03-22T07:08:55.560 INFO:tasks.workunit.client.1.smithi149.stdout:9/668: symlink db/lcc 0 2024-03-22T07:08:55.560 INFO:tasks.workunit.client.1.smithi149.stdout:9/669: write db/d22/f2b [1218026,17486] 0 2024-03-22T07:08:55.561 INFO:tasks.workunit.client.1.smithi149.stdout:4/700: creat d0/d8/dc/d2e/d6f/ddf/fe2 x:0 0 0 2024-03-22T07:08:55.561 INFO:tasks.workunit.client.1.smithi149.stdout:4/701: truncate d0/d8/d1d/d24/d82/f91 779926 0 2024-03-22T07:08:55.563 INFO:tasks.workunit.client.1.smithi149.stdout:0/585: sync 2024-03-22T07:08:55.563 INFO:tasks.workunit.client.1.smithi149.stdout:0/586: chown d1/db4/lc5 0 1 2024-03-22T07:08:55.564 INFO:tasks.workunit.client.1.smithi149.stdout:0/587: dread - d1/d31/d51/d64/d7e/fca zero size 2024-03-22T07:08:55.564 INFO:tasks.workunit.client.1.smithi149.stdout:0/588: fsync d1/f39 0 2024-03-22T07:08:55.564 INFO:tasks.workunit.client.1.smithi149.stdout:8/681: sync 2024-03-22T07:08:55.564 INFO:tasks.workunit.client.1.smithi149.stdout:5/651: sync 2024-03-22T07:08:55.564 INFO:tasks.workunit.client.1.smithi149.stdout:3/518: sync 2024-03-22T07:08:55.564 INFO:tasks.workunit.client.1.smithi149.stdout:9/670: creat db/d21/fcd x:0 0 0 2024-03-22T07:08:55.565 INFO:tasks.workunit.client.1.smithi149.stdout:0/589: mkdir d1/d31/d52/dd4 0 2024-03-22T07:08:55.567 INFO:tasks.workunit.client.1.smithi149.stdout:8/682: truncate d1/d2/d3/f76 54066 0 2024-03-22T07:08:55.567 INFO:tasks.workunit.client.1.smithi149.stdout:8/683: chown d1/d2/d9 395 1 2024-03-22T07:08:55.567 INFO:tasks.workunit.client.1.smithi149.stdout:8/684: readlink d1/d2/d3/l25 0 2024-03-22T07:08:55.568 INFO:tasks.workunit.client.1.smithi149.stdout:5/652: link d7/c27 d7/da/d20/d30/d3a/d93/ce8 0 2024-03-22T07:08:55.568 INFO:tasks.workunit.client.1.smithi149.stdout:5/653: dread - d7/da/fd3 zero size 2024-03-22T07:08:55.568 INFO:tasks.workunit.client.1.smithi149.stdout:3/519: truncate d0/d4/d8/d24/f6e 2430613 0 2024-03-22T07:08:55.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:55 smithi081 ceph-mon[29245]: pgmap v10: 65 pgs: 65 active+clean; 1.3 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 54 MiB/s rd, 72 MiB/s wr, 702 op/s 2024-03-22T07:08:55.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:55 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:55.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:55 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:55.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:55 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:08:55.569 INFO:tasks.workunit.client.1.smithi149.stdout:8/685: rmdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d82 39 2024-03-22T07:08:55.569 INFO:tasks.workunit.client.1.smithi149.stdout:8/686: write d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f6e [2123698,117985] 0 2024-03-22T07:08:55.569 INFO:tasks.workunit.client.1.smithi149.stdout:0/590: dread d1/d20/da3/fab [0,4194304] 0 2024-03-22T07:08:55.570 INFO:tasks.workunit.client.1.smithi149.stdout:5/654: symlink d7/da/d20/d89/da2/dda/le9 0 2024-03-22T07:08:55.570 INFO:tasks.workunit.client.1.smithi149.stdout:3/520: unlink d0/d2/d30/d5d/d84/d8f/f59 0 2024-03-22T07:08:55.571 INFO:tasks.workunit.client.1.smithi149.stdout:3/521: stat d0/d2/d30/f8a 0 2024-03-22T07:08:55.572 INFO:tasks.workunit.client.1.smithi149.stdout:8/687: unlink d1/d2/d9/d11/f7a 0 2024-03-22T07:08:55.573 INFO:tasks.workunit.client.1.smithi149.stdout:0/591: symlink d1/d31/d51/d64/d7e/ld5 0 2024-03-22T07:08:55.573 INFO:tasks.workunit.client.1.smithi149.stdout:0/592: fdatasync d1/d9/f71 0 2024-03-22T07:08:55.573 INFO:tasks.workunit.client.1.smithi149.stdout:0/593: creat d1/d31/d51/d64/d87/da6/fd6 x:0 0 0 2024-03-22T07:08:55.573 INFO:tasks.workunit.client.1.smithi149.stdout:0/594: fsync d1/d9/d14/d1f/f4d 0 2024-03-22T07:08:55.573 INFO:tasks.workunit.client.1.smithi149.stdout:5/655: mkdir d7/d35/de0/dd7/dea 0 2024-03-22T07:08:55.574 INFO:tasks.workunit.client.1.smithi149.stdout:3/522: link d0/d4/d8/f21 d0/d2/d30/d5d/d84/d8f/fc5 0 2024-03-22T07:08:55.576 INFO:tasks.workunit.client.1.smithi149.stdout:8/688: creat d1/d2/fd8 x:0 0 0 2024-03-22T07:08:55.576 INFO:tasks.workunit.client.1.smithi149.stdout:8/689: chown d1/d2/d9/f3b 569 1 2024-03-22T07:08:55.578 INFO:tasks.workunit.client.1.smithi149.stdout:0/595: getdents d1/d31/d51/d64/d87 0 2024-03-22T07:08:55.578 INFO:tasks.workunit.client.1.smithi149.stdout:0/596: creat d1/fd7 x:0 0 0 2024-03-22T07:08:55.578 INFO:tasks.workunit.client.1.smithi149.stdout:0/597: chown d1/f63 0 1 2024-03-22T07:08:55.578 INFO:tasks.workunit.client.1.smithi149.stdout:3/523: symlink d0/d4/d6c/lc6 0 2024-03-22T07:08:55.580 INFO:tasks.workunit.client.1.smithi149.stdout:8/690: link d1/d2/f8f d1/d2/d9/d11/d4d/dd6/d35/d22/dd7/fd9 0 2024-03-22T07:08:55.580 INFO:tasks.workunit.client.1.smithi149.stdout:4/702: write d0/d8/dc/dbd/fbe [4073678,21189] 0 2024-03-22T07:08:55.580 INFO:tasks.workunit.client.1.smithi149.stdout:4/703: creat d0/d8/dc/d74/fe3 x:0 0 0 2024-03-22T07:08:55.580 INFO:tasks.workunit.client.1.smithi149.stdout:4/704: creat d0/d8/d4c/fe4 x:0 0 0 2024-03-22T07:08:55.580 INFO:tasks.workunit.client.1.smithi149.stdout:4/705: chown d0/d8/dc/d14/d3e/fb1 6735899 1 2024-03-22T07:08:55.580 INFO:tasks.workunit.client.1.smithi149.stdout:0/598: creat d1/d31/d51/d64/d87/fd8 x:0 0 0 2024-03-22T07:08:55.581 INFO:tasks.workunit.client.1.smithi149.stdout:0/599: readlink d1/d6b/la4 0 2024-03-22T07:08:55.581 INFO:tasks.workunit.client.1.smithi149.stdout:3/524: creat d0/d2/d30/d5d/d84/fc7 x:0 0 0 2024-03-22T07:08:55.584 INFO:tasks.workunit.client.1.smithi149.stdout:4/706: link d0/d8/d4c/f57 d0/d8/dc/d2e/d6f/ddf/fe5 0 2024-03-22T07:08:55.584 INFO:tasks.workunit.client.1.smithi149.stdout:6/702: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f43 [0,4194304] 0 2024-03-22T07:08:55.584 INFO:tasks.workunit.client.1.smithi149.stdout:6/703: chown d9/df/fda 78688756 1 2024-03-22T07:08:55.584 INFO:tasks.workunit.client.1.smithi149.stdout:7/656: dwrite d1/d18/d1b/d51/d5a/f7c [0,4194304] 0 2024-03-22T07:08:55.584 INFO:tasks.workunit.client.1.smithi149.stdout:7/657: read - d1/d18/d1b/d51/d7a/d7d/d5d/fa9 zero size 2024-03-22T07:08:55.584 INFO:tasks.workunit.client.1.smithi149.stdout:0/600: mkdir d1/d20/d9f/dc6/dd9 0 2024-03-22T07:08:55.585 INFO:tasks.workunit.client.1.smithi149.stdout:0/601: dread d1/d20/f3d [0,4194304] 0 2024-03-22T07:08:55.585 INFO:tasks.workunit.client.1.smithi149.stdout:3/525: link d0/d2/d30/d7e/f69 d0/d4/d8/fc8 0 2024-03-22T07:08:55.586 INFO:tasks.workunit.client.1.smithi149.stdout:3/526: fsync d0/d4/f58 0 2024-03-22T07:08:55.586 INFO:tasks.workunit.client.1.smithi149.stdout:3/527: write d0/d2/d42/d66/f7a [120995,14467] 0 2024-03-22T07:08:55.586 INFO:tasks.workunit.client.1.smithi149.stdout:9/671: dwrite db/d19/d28/fc1 [0,4194304] 0 2024-03-22T07:08:55.587 INFO:tasks.workunit.client.1.smithi149.stdout:8/691: symlink d1/d2/d9/d11/d4d/lda 0 2024-03-22T07:08:55.589 INFO:tasks.workunit.client.1.smithi149.stdout:5/656: read d7/d35/de0/dd7/fae [4182821,54368] 0 2024-03-22T07:08:55.590 INFO:tasks.workunit.client.1.smithi149.stdout:5/657: stat d7/da/d20/d30/d3a/d93/f9b 0 2024-03-22T07:08:55.590 INFO:tasks.workunit.client.1.smithi149.stdout:4/707: rename d0/d8/dc/d14/f16 to d0/d8/d1d/fe6 0 2024-03-22T07:08:55.590 INFO:tasks.workunit.client.1.smithi149.stdout:5/658: fsync d7/da/d20/d36/f66 0 2024-03-22T07:08:55.590 INFO:tasks.workunit.client.1.smithi149.stdout:4/708: truncate d0/d8/dc/d74/d76/f7b 602668 0 2024-03-22T07:08:55.590 INFO:tasks.workunit.client.1.smithi149.stdout:4/709: chown d0/f32 160 1 2024-03-22T07:08:55.590 INFO:tasks.workunit.client.1.smithi149.stdout:6/704: getdents d9/df/d22/d6d/d7d/d96/d24/d34 0 2024-03-22T07:08:55.595 INFO:tasks.workunit.client.1.smithi149.stdout:7/658: unlink d1/d18/d1b/d1d/d72/f88 0 2024-03-22T07:08:55.595 INFO:tasks.workunit.client.1.smithi149.stdout:9/672: dread db/d19/d39/d3b/f48 [0,4194304] 0 2024-03-22T07:08:55.595 INFO:tasks.workunit.client.1.smithi149.stdout:9/673: stat db/d21/d8c/f98 0 2024-03-22T07:08:55.598 INFO:tasks.workunit.client.1.smithi149.stdout:3/528: mkdir d0/d4/db5/dc9 0 2024-03-22T07:08:55.598 INFO:tasks.workunit.client.1.smithi149.stdout:3/529: dread - d0/d4/d8/d24/d40/d54/d85/dbc/fc2 zero size 2024-03-22T07:08:55.601 INFO:tasks.workunit.client.1.smithi149.stdout:8/692: mknod d1/d2/d18/cdb 0 2024-03-22T07:08:55.601 INFO:tasks.workunit.client.1.smithi149.stdout:8/693: fdatasync d1/d2/d9/f29 0 2024-03-22T07:08:55.601 INFO:tasks.workunit.client.1.smithi149.stdout:5/659: truncate d7/da/d20/d36/f9d 12034196 0 2024-03-22T07:08:55.603 INFO:tasks.workunit.client.1.smithi149.stdout:7/659: creat d1/d18/d1b/d51/d5a/d6a/fd6 x:0 0 0 2024-03-22T07:08:55.605 INFO:tasks.workunit.client.1.smithi149.stdout:9/674: truncate db/d21/f23 1472130 0 2024-03-22T07:08:55.605 INFO:tasks.workunit.client.1.smithi149.stdout:9/675: fsync db/d21/d8c/f96 0 2024-03-22T07:08:55.607 INFO:tasks.workunit.client.1.smithi149.stdout:0/602: write d1/f6a [3647624,3441] 0 2024-03-22T07:08:55.607 INFO:tasks.workunit.client.1.smithi149.stdout:3/530: rename d0/d2/l5 to d0/d4/d8/d24/d40/d62/lca 0 2024-03-22T07:08:55.607 INFO:tasks.workunit.client.1.smithi149.stdout:3/531: dread - d0/f72 zero size 2024-03-22T07:08:55.607 INFO:tasks.workunit.client.1.smithi149.stdout:3/532: creat d0/d2/d30/fcb x:0 0 0 2024-03-22T07:08:55.607 INFO:tasks.workunit.client.1.smithi149.stdout:3/533: readlink d0/lb7 0 2024-03-22T07:08:55.607 INFO:tasks.workunit.client.1.smithi149.stdout:8/694: mknod d1/d2/d9/df/cdc 0 2024-03-22T07:08:55.608 INFO:tasks.workunit.client.1.smithi149.stdout:5/660: mkdir d7/da/d20/d36/deb 0 2024-03-22T07:08:55.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/710: getdents d0/d6b 0 2024-03-22T07:08:55.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/711: write d0/d6b/f7d [70223,71152] 0 2024-03-22T07:08:55.609 INFO:tasks.workunit.client.1.smithi149.stdout:4/712: stat d0/d6b/l89 0 2024-03-22T07:08:55.612 INFO:tasks.workunit.client.1.smithi149.stdout:7/660: mkdir d1/d18/d1b/d99/d56/da3/dd7 0 2024-03-22T07:08:55.614 INFO:tasks.workunit.client.1.smithi149.stdout:0/603: link d1/d9/d14/d1f/f4d d1/d20/d9f/dc6/dd9/fda 0 2024-03-22T07:08:55.616 INFO:tasks.workunit.client.1.smithi149.stdout:3/534: mkdir d0/d2/d30/d7e/dcc 0 2024-03-22T07:08:55.617 INFO:tasks.workunit.client.1.smithi149.stdout:8/695: creat d1/d2/d9/d11/d8b/fdd x:0 0 0 2024-03-22T07:08:55.619 INFO:tasks.workunit.client.1.smithi149.stdout:5/661: link d7/da/d20/d45/fb7 d7/da/d20/d89/da2/fec 0 2024-03-22T07:08:55.619 INFO:tasks.workunit.client.1.smithi149.stdout:5/662: stat d7/lbf 0 2024-03-22T07:08:55.619 INFO:tasks.workunit.client.1.smithi149.stdout:5/663: readlink d7/da/d20/l2f 0 2024-03-22T07:08:55.619 INFO:tasks.workunit.client.1.smithi149.stdout:4/713: creat d0/d8/dc/d74/ddc/fe7 x:0 0 0 2024-03-22T07:08:55.623 INFO:tasks.workunit.client.1.smithi149.stdout:7/661: unlink d1/d18/d30/f45 0 2024-03-22T07:08:55.623 INFO:tasks.workunit.client.1.smithi149.stdout:7/662: chown d1/d18/d1b/d51/d7a/d7d/d5b 227805565 1 2024-03-22T07:08:55.625 INFO:tasks.workunit.client.1.smithi149.stdout:0/604: readlink d1/d9/d14/d1f/l7b 0 2024-03-22T07:08:55.627 INFO:tasks.workunit.client.1.smithi149.stdout:6/705: dwrite d9/df/d22/dc0/f5e [0,4194304] 0 2024-03-22T07:08:55.627 INFO:tasks.workunit.client.1.smithi149.stdout:6/706: fsync d9/da2/fbc 0 2024-03-22T07:08:55.627 INFO:tasks.workunit.client.1.smithi149.stdout:9/676: dwrite db/d42/f4c [0,4194304] 0 2024-03-22T07:08:55.627 INFO:tasks.workunit.client.1.smithi149.stdout:9/677: chown db/d19/da6 5043 1 2024-03-22T07:08:55.628 INFO:tasks.workunit.client.1.smithi149.stdout:3/535: mkdir d0/d4/d8/d24/db6/dcd 0 2024-03-22T07:08:55.629 INFO:tasks.workunit.client.1.smithi149.stdout:8/696: truncate d1/d2/d9/d11/d4d/dd6/d51/fa6 330506 0 2024-03-22T07:08:55.631 INFO:tasks.workunit.client.1.smithi149.stdout:5/664: mkdir d7/da/ded 0 2024-03-22T07:08:55.631 INFO:tasks.workunit.client.1.smithi149.stdout:5/665: creat d7/da/fee x:0 0 0 2024-03-22T07:08:55.632 INFO:tasks.workunit.client.1.smithi149.stdout:4/714: stat d0/d8/dc/f13 0 2024-03-22T07:08:55.634 INFO:tasks.workunit.client.1.smithi149.stdout:7/663: rename d1/d18/d1b/d99/f83 to d1/d1c/d49/dcd/fd8 0 2024-03-22T07:08:55.634 INFO:tasks.workunit.client.1.smithi149.stdout:7/664: write d1/d18/d1b/d99/d56/db4/fd5 [199325,110206] 0 2024-03-22T07:08:55.639 INFO:tasks.workunit.client.1.smithi149.stdout:6/707: unlink d9/df/d22/d6d/d7d/d96/d24/d42/fad 0 2024-03-22T07:08:55.639 INFO:tasks.workunit.client.1.smithi149.stdout:6/708: creat d9/d47/d52/fdf x:0 0 0 2024-03-22T07:08:55.644 INFO:tasks.workunit.client.1.smithi149.stdout:9/678: unlink db/d21/d37/d7c/cc4 0 2024-03-22T07:08:55.645 INFO:tasks.workunit.client.1.smithi149.stdout:0/605: dread d1/d9/d14/d1f/f2a [0,4194304] 0 2024-03-22T07:08:55.645 INFO:tasks.workunit.client.1.smithi149.stdout:0/606: creat d1/d20/d9f/dc6/fdb x:0 0 0 2024-03-22T07:08:55.646 INFO:tasks.workunit.client.1.smithi149.stdout:8/697: unlink d1/d2/d18/ld0 0 2024-03-22T07:08:55.646 INFO:tasks.workunit.client.1.smithi149.stdout:3/536: dwrite d0/d4/d8/d24/d40/d54/d85/dbc/f80 [0,4194304] 0 2024-03-22T07:08:55.647 INFO:tasks.workunit.client.1.smithi149.stdout:8/698: dread d1/d2/d9/df/f2d [0,4194304] 0 2024-03-22T07:08:55.648 INFO:tasks.workunit.client.1.smithi149.stdout:5/666: creat d7/da/d20/d30/fef x:0 0 0 2024-03-22T07:08:55.649 INFO:tasks.workunit.client.1.smithi149.stdout:4/715: mknod d0/d8/dc/d74/ce8 0 2024-03-22T07:08:55.651 INFO:tasks.workunit.client.1.smithi149.stdout:7/665: mknod d1/d18/d1b/d51/d7a/d7d/d5d/cd9 0 2024-03-22T07:08:55.651 INFO:tasks.workunit.client.1.smithi149.stdout:6/709: write d9/df/d22/d6d/d7d/d96/f6b [1601493,117946] 0 2024-03-22T07:08:55.654 INFO:tasks.workunit.client.1.smithi149.stdout:9/679: mknod db/d21/d37/cce 0 2024-03-22T07:08:55.656 INFO:tasks.workunit.client.1.smithi149.stdout:8/699: dread d1/d2/d9/d11/d4d/dd6/d35/d22/d44/f6a [0,4194304] 0 2024-03-22T07:08:55.656 INFO:tasks.workunit.client.1.smithi149.stdout:0/607: mkdir d1/d20/d9f/dc6/ddc 0 2024-03-22T07:08:55.665 INFO:tasks.workunit.client.1.smithi149.stdout:3/537: mkdir d0/d2/d5a/daf/dce 0 2024-03-22T07:08:55.667 INFO:tasks.workunit.client.1.smithi149.stdout:5/667: link d7/d35/de0/dd7/d7b/d2b/laa d7/da/d20/lf0 0 2024-03-22T07:08:55.668 INFO:tasks.workunit.client.1.smithi149.stdout:7/666: truncate d1/d18/d1b/d1d/f13 2985366 0 2024-03-22T07:08:55.668 INFO:tasks.workunit.client.1.smithi149.stdout:7/667: write d1/f41 [1048379,27531] 0 2024-03-22T07:08:55.671 INFO:tasks.workunit.client.1.smithi149.stdout:6/710: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/fe0 x:0 0 0 2024-03-22T07:08:55.672 INFO:tasks.workunit.client.1.smithi149.stdout:9/680: symlink db/d21/d37/d3a/lcf 0 2024-03-22T07:08:55.673 INFO:tasks.workunit.client.1.smithi149.stdout:8/700: creat d1/d2/d9/df/fde x:0 0 0 2024-03-22T07:08:55.674 INFO:tasks.workunit.client.1.smithi149.stdout:3/538: dread d0/d4/d8/f19 [0,4194304] 0 2024-03-22T07:08:55.674 INFO:tasks.workunit.client.1.smithi149.stdout:3/539: fdatasync d0/d4/d8/f19 0 2024-03-22T07:08:55.674 INFO:tasks.workunit.client.1.smithi149.stdout:3/540: creat d0/d4/d8/d24/d40/d54/d85/dbc/fcf x:0 0 0 2024-03-22T07:08:55.674 INFO:tasks.workunit.client.1.smithi149.stdout:0/608: truncate d1/d9/d14/f76 69066 0 2024-03-22T07:08:55.675 INFO:tasks.workunit.client.1.smithi149.stdout:5/668: creat d7/d35/d8e/ff1 x:0 0 0 2024-03-22T07:08:55.676 INFO:tasks.workunit.client.1.smithi149.stdout:5/669: fsync d7/da/d20/d30/d3a/d3d/d2c/f8b 0 2024-03-22T07:08:55.676 INFO:tasks.workunit.client.1.smithi149.stdout:5/670: readlink d7/da/d20/d89/da2/dda/le9 0 2024-03-22T07:08:55.677 INFO:tasks.workunit.client.1.smithi149.stdout:6/711: creat d9/df/d22/dc0/fe1 x:0 0 0 2024-03-22T07:08:55.677 INFO:tasks.workunit.client.1.smithi149.stdout:6/712: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f71 [377556,19631] 0 2024-03-22T07:08:55.678 INFO:tasks.workunit.client.1.smithi149.stdout:9/681: rmdir db/d21/d9e 39 2024-03-22T07:08:55.680 INFO:tasks.workunit.client.1.smithi149.stdout:8/701: unlink d1/d2/d3/l75 0 2024-03-22T07:08:55.681 INFO:tasks.workunit.client.1.smithi149.stdout:5/671: dread d7/da/d20/d89/da2/fa8 [0,4194304] 0 2024-03-22T07:08:55.681 INFO:tasks.workunit.client.1.smithi149.stdout:5/672: write d7/da/fee [121802,37253] 0 2024-03-22T07:08:55.681 INFO:tasks.workunit.client.1.smithi149.stdout:5/673: dread - d7/da/d20/d36/fd5 zero size 2024-03-22T07:08:55.681 INFO:tasks.workunit.client.1.smithi149.stdout:5/674: write d7/da/d20/d30/d3a/f79 [143588,35002] 0 2024-03-22T07:08:55.681 INFO:tasks.workunit.client.1.smithi149.stdout:5/675: write d7/da/d20/f99 [810941,38564] 0 2024-03-22T07:08:55.681 INFO:tasks.workunit.client.1.smithi149.stdout:3/541: rmdir d0/d2/d30/d5d 39 2024-03-22T07:08:55.685 INFO:tasks.workunit.client.1.smithi149.stdout:6/713: unlink d9/df/d22/d6d/d7d/d96/db1/f80 0 2024-03-22T07:08:55.686 INFO:tasks.workunit.client.1.smithi149.stdout:6/714: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/fe2 x:0 0 0 2024-03-22T07:08:55.686 INFO:tasks.workunit.client.1.smithi149.stdout:6/715: fsync d9/df/d22/f87 0 2024-03-22T07:08:55.686 INFO:tasks.workunit.client.1.smithi149.stdout:7/668: dwrite d1/d18/d1b/d1d/f16 [0,4194304] 0 2024-03-22T07:08:55.686 INFO:tasks.workunit.client.1.smithi149.stdout:9/682: creat db/d21/d9e/fd0 x:0 0 0 2024-03-22T07:08:55.687 INFO:tasks.workunit.client.1.smithi149.stdout:7/669: read d1/d18/d1b/d51/d7a/f7f [95549,19785] 0 2024-03-22T07:08:55.687 INFO:tasks.workunit.client.1.smithi149.stdout:8/702: rmdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/d9e 39 2024-03-22T07:08:55.687 INFO:tasks.workunit.client.1.smithi149.stdout:8/703: creat d1/d2/d9/df/fdf x:0 0 0 2024-03-22T07:08:55.690 INFO:tasks.workunit.client.1.smithi149.stdout:6/716: creat d9/d47/fe3 x:0 0 0 2024-03-22T07:08:55.690 INFO:tasks.workunit.client.1.smithi149.stdout:6/717: creat d9/da2/fe4 x:0 0 0 2024-03-22T07:08:55.690 INFO:tasks.workunit.client.1.smithi149.stdout:6/718: chown d9/df/d22/d6d/d7d/d96/d24/l70 2002416504 1 2024-03-22T07:08:55.690 INFO:tasks.workunit.client.1.smithi149.stdout:6/719: chown d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 149965 1 2024-03-22T07:08:55.690 INFO:tasks.workunit.client.1.smithi149.stdout:6/720: chown d9/f17 7818699 1 2024-03-22T07:08:55.691 INFO:tasks.workunit.client.1.smithi149.stdout:9/683: symlink db/d19/d28/ld1 0 2024-03-22T07:08:55.694 INFO:tasks.workunit.client.1.smithi149.stdout:9/684: dread db/d19/f1f [0,4194304] 0 2024-03-22T07:08:55.694 INFO:tasks.workunit.client.1.smithi149.stdout:9/685: write db/d21/d37/d7c/da3/fc2 [1320629,125545] 0 2024-03-22T07:08:55.694 INFO:tasks.workunit.client.1.smithi149.stdout:7/670: mkdir d1/d18/dda 0 2024-03-22T07:08:55.697 INFO:tasks.workunit.client.1.smithi149.stdout:0/609: dwrite d1/d9/f3a [4194304,4194304] 0 2024-03-22T07:08:55.701 INFO:tasks.workunit.client.1.smithi149.stdout:6/721: rename d9/df/d22/d6d/d7d/d96/d24/d42/fa7 to d9/df/d22/d6d/d7d/d96/d24/d42/fe5 0 2024-03-22T07:08:55.701 INFO:tasks.workunit.client.1.smithi149.stdout:6/722: readlink d9/df/d2e/l82 0 2024-03-22T07:08:55.701 INFO:tasks.workunit.client.1.smithi149.stdout:6/723: creat d9/d47/d52/fe6 x:0 0 0 2024-03-22T07:08:55.701 INFO:tasks.workunit.client.1.smithi149.stdout:6/724: write d9/df/d2e/f27 [1225169,72913] 0 2024-03-22T07:08:55.701 INFO:tasks.workunit.client.1.smithi149.stdout:6/725: chown d9/df/d2e/fce 1120774 1 2024-03-22T07:08:55.703 INFO:tasks.workunit.client.1.smithi149.stdout:1/678: sync 2024-03-22T07:08:55.703 INFO:tasks.workunit.client.1.smithi149.stdout:2/760: sync 2024-03-22T07:08:55.703 INFO:tasks.workunit.client.1.smithi149.stdout:8/704: getdents d1/d2 0 2024-03-22T07:08:55.705 INFO:tasks.workunit.client.1.smithi149.stdout:6/726: write d9/df/d2e/f1c [897755,42208] 0 2024-03-22T07:08:55.705 INFO:tasks.workunit.client.1.smithi149.stdout:0/610: creat d1/d20/d9f/dc6/ddc/fdd x:0 0 0 2024-03-22T07:08:55.707 INFO:tasks.workunit.client.1.smithi149.stdout:9/686: symlink db/d33/ld2 0 2024-03-22T07:08:55.708 INFO:tasks.workunit.client.1.smithi149.stdout:1/679: mkdir d4/d10/dd8/d3c/d8e/db5/db2/dbf/de5 0 2024-03-22T07:08:55.711 INFO:tasks.workunit.client.1.smithi149.stdout:8/705: rename d1/d2/d9/d11/d4d/dd6/d35/c4b to d1/d2/d9/d11/ce0 0 2024-03-22T07:08:55.712 INFO:tasks.workunit.client.1.smithi149.stdout:6/727: symlink d9/df/d22/d6d/d7d/le7 0 2024-03-22T07:08:55.713 INFO:tasks.workunit.client.1.smithi149.stdout:9/687: mkdir db/d19/d39/dd3 0 2024-03-22T07:08:55.714 INFO:tasks.workunit.client.1.smithi149.stdout:9/688: write db/f2f [2994556,88326] 0 2024-03-22T07:08:55.715 INFO:tasks.workunit.client.1.smithi149.stdout:6/728: mknod d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/ce8 0 2024-03-22T07:08:55.716 INFO:tasks.workunit.client.1.smithi149.stdout:1/680: rmdir d4/d20/d2d/d64/d9f/dbe 39 2024-03-22T07:08:55.717 INFO:tasks.workunit.client.1.smithi149.stdout:0/611: rename d1/d9/d14/d1f/l60 to d1/d20/d9f/dc6/ddc/lde 0 2024-03-22T07:08:55.717 INFO:tasks.workunit.client.1.smithi149.stdout:8/706: mknod d1/d2/d9/d11/d4d/ce1 0 2024-03-22T07:08:55.719 INFO:tasks.workunit.client.1.smithi149.stdout:3/542: dwrite d0/d4/d8/f19 [0,4194304] 0 2024-03-22T07:08:55.719 INFO:tasks.workunit.client.1.smithi149.stdout:9/689: dread db/fc [0,4194304] 0 2024-03-22T07:08:55.719 INFO:tasks.workunit.client.1.smithi149.stdout:9/690: readlink db/d19/d28/l29 0 2024-03-22T07:08:55.719 INFO:tasks.workunit.client.1.smithi149.stdout:6/729: mkdir d9/df/d22/dc0/dc8/de9 0 2024-03-22T07:08:55.719 INFO:tasks.workunit.client.1.smithi149.stdout:6/730: creat d9/df/d22/d6d/d7d/d96/d24/d68/d73/fea x:0 0 0 2024-03-22T07:08:55.722 INFO:tasks.workunit.client.1.smithi149.stdout:1/681: rmdir d4/d7/d5d 39 2024-03-22T07:08:55.722 INFO:tasks.workunit.client.1.smithi149.stdout:0/612: symlink d1/d20/ldf 0 2024-03-22T07:08:55.722 INFO:tasks.workunit.client.1.smithi149.stdout:0/613: write d1/d27/f6e [849637,78170] 0 2024-03-22T07:08:55.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/543: unlink d0/d4/d8/f23 0 2024-03-22T07:08:55.730 INFO:tasks.workunit.client.1.smithi149.stdout:7/671: dwrite d1/d18/d1b/d51/d5a/d6a/fd6 [0,4194304] 0 2024-03-22T07:08:55.732 INFO:tasks.workunit.client.1.smithi149.stdout:2/761: dwrite d1/d4d/f8a [0,4194304] 0 2024-03-22T07:08:55.733 INFO:tasks.workunit.client.1.smithi149.stdout:3/544: rmdir d0/d2/d5a/d93/da9 0 2024-03-22T07:08:55.733 INFO:tasks.workunit.client.1.smithi149.stdout:5/676: dwrite d7/f58 [4194304,4194304] 0 2024-03-22T07:08:55.733 INFO:tasks.workunit.client.1.smithi149.stdout:8/707: dread d1/d2/d9/f4a [0,4194304] 0 2024-03-22T07:08:55.741 INFO:tasks.workunit.client.1.smithi149.stdout:2/762: symlink d1/d4d/d69/l110 0 2024-03-22T07:08:55.742 INFO:tasks.workunit.client.1.smithi149.stdout:1/682: dwrite d4/d10/dd8/d3c/d8e/db5/f60 [0,4194304] 0 2024-03-22T07:08:55.742 INFO:tasks.workunit.client.1.smithi149.stdout:1/683: creat d4/d20/fe6 x:0 0 0 2024-03-22T07:08:55.742 INFO:tasks.workunit.client.1.smithi149.stdout:6/731: rename d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/da1 to d9/df/d22/d6d/deb 0 2024-03-22T07:08:55.746 INFO:tasks.workunit.client.1.smithi149.stdout:3/545: symlink d0/d2/d30/d5d/d84/ld0 0 2024-03-22T07:08:55.748 INFO:tasks.workunit.client.1.smithi149.stdout:2/763: dread d1/d3/d5/f2b [0,4194304] 0 2024-03-22T07:08:55.749 INFO:tasks.workunit.client.1.smithi149.stdout:5/677: symlink d7/d35/de0/dd7/dea/lf2 0 2024-03-22T07:08:55.750 INFO:tasks.workunit.client.1.smithi149.stdout:8/708: creat d1/d2/d9/d11/d4d/dd6/d51/fe2 x:0 0 0 2024-03-22T07:08:55.752 INFO:tasks.workunit.client.1.smithi149.stdout:7/672: dwrite d1/d18/d1b/d1d/f40 [4194304,4194304] 0 2024-03-22T07:08:55.752 INFO:tasks.workunit.client.1.smithi149.stdout:7/673: write d1/d1c/f7b [13729,83213] 0 2024-03-22T07:08:55.759 INFO:tasks.workunit.client.1.smithi149.stdout:0/614: rename d1/d31/d51/d64/d87/l91 to d1/d9/d14/le0 0 2024-03-22T07:08:55.759 INFO:tasks.workunit.client.1.smithi149.stdout:0/615: fsync d1/d9/d59/f5d 0 2024-03-22T07:08:55.759 INFO:tasks.workunit.client.1.smithi149.stdout:0/616: creat d1/d20/d9f/dc6/fe1 x:0 0 0 2024-03-22T07:08:55.759 INFO:tasks.workunit.client.1.smithi149.stdout:6/732: truncate d9/f17 592310 0 2024-03-22T07:08:55.761 INFO:tasks.workunit.client.1.smithi149.stdout:8/709: mknod d1/d2/d9/d11/d8b/d8d/ce3 0 2024-03-22T07:08:55.764 INFO:tasks.workunit.client.1.smithi149.stdout:7/674: creat d1/fdb x:0 0 0 2024-03-22T07:08:55.764 INFO:tasks.workunit.client.1.smithi149.stdout:7/675: stat d1/d18/d1b/d51/d7a/d7d/d5b/d97/fc5 0 2024-03-22T07:08:55.764 INFO:tasks.workunit.client.1.smithi149.stdout:7/676: chown d1/d18/d1b/l69 8102 1 2024-03-22T07:08:55.764 INFO:tasks.workunit.client.1.smithi149.stdout:7/677: getdents d1/db8 0 2024-03-22T07:08:55.765 INFO:tasks.workunit.client.1.smithi149.stdout:7/678: truncate d1/d18/d1b/f42 4969869 0 2024-03-22T07:08:55.765 INFO:tasks.workunit.client.1.smithi149.stdout:7/679: read - d1/d18/d1b/d1d/d72/fd0 zero size 2024-03-22T07:08:55.766 INFO:tasks.workunit.client.1.smithi149.stdout:1/684: rename d4/d10/dd8/d3c/d8e/db5/f60 to d4/d20/fe7 0 2024-03-22T07:08:55.768 INFO:tasks.workunit.client.1.smithi149.stdout:6/733: mknod d9/da2/cec 0 2024-03-22T07:08:55.769 INFO:tasks.workunit.client.1.smithi149.stdout:3/546: getdents d0/d4/d8/d24/d39 0 2024-03-22T07:08:55.771 INFO:tasks.workunit.client.1.smithi149.stdout:2/764: creat d1/d3/d5/d1f/dfc/f111 x:0 0 0 2024-03-22T07:08:55.771 INFO:tasks.workunit.client.1.smithi149.stdout:2/765: stat d1/d4d/d69/d75/db8/lc0 0 2024-03-22T07:08:55.771 INFO:tasks.workunit.client.1.smithi149.stdout:2/766: dread - d1/d3/d6/d3f/f10d zero size 2024-03-22T07:08:55.772 INFO:tasks.workunit.client.1.smithi149.stdout:7/680: link d1/d18/d1b/d1d/f8 d1/d18/d1b/d1d/d72/fdc 0 2024-03-22T07:08:55.774 INFO:tasks.workunit.client.1.smithi149.stdout:5/678: rename d7/da/d20/d30/d3a/d93/d94/dc2/fc9 to d7/da/d7d/ff3 0 2024-03-22T07:08:55.775 INFO:tasks.workunit.client.1.smithi149.stdout:1/685: symlink d4/d7/d94/le8 0 2024-03-22T07:08:55.775 INFO:tasks.workunit.client.1.smithi149.stdout:1/686: readlink d4/d10/dd8/d3c/d8e/db5/l4e 0 2024-03-22T07:08:55.776 INFO:tasks.workunit.client.1.smithi149.stdout:6/734: mknod d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/ced 0 2024-03-22T07:08:55.776 INFO:tasks.workunit.client.1.smithi149.stdout:6/735: stat d9/df/f56 0 2024-03-22T07:08:55.777 INFO:tasks.workunit.client.1.smithi149.stdout:7/681: symlink d1/d18/d1b/d51/d5a/d6a/ldd 0 2024-03-22T07:08:55.778 INFO:tasks.workunit.client.1.smithi149.stdout:8/710: rename d1/d2/d9/d11/d4d/dd6/f95 to d1/d2/d9/db2/fe4 0 2024-03-22T07:08:55.779 INFO:tasks.workunit.client.1.smithi149.stdout:8/711: write d1/d2/d9/d11/d8b/fdd [577469,70559] 0 2024-03-22T07:08:55.779 INFO:tasks.workunit.client.1.smithi149.stdout:5/679: mkdir d7/d35/de0/d92/df4 0 2024-03-22T07:08:55.780 INFO:tasks.workunit.client.1.smithi149.stdout:1/687: truncate d4/d7/d5a/f5e 3121564 0 2024-03-22T07:08:55.781 INFO:tasks.workunit.client.1.smithi149.stdout:7/682: creat d1/d18/d1b/d51/d5a/d6a/da6/d93/fde x:0 0 0 2024-03-22T07:08:55.783 INFO:tasks.workunit.client.1.smithi149.stdout:0/617: dwrite d1/d9/d14/d1f/f89 [0,4194304] 0 2024-03-22T07:08:55.784 INFO:tasks.workunit.client.1.smithi149.stdout:0/618: chown d1/d9/d59/fba 372 1 2024-03-22T07:08:55.784 INFO:tasks.workunit.client.1.smithi149.stdout:0/619: write d1/d31/d51/d64/d7e/fca [687866,32839] 0 2024-03-22T07:08:55.784 INFO:tasks.workunit.client.1.smithi149.stdout:4/716: sync 2024-03-22T07:08:55.785 INFO:tasks.workunit.client.1.smithi149.stdout:4/717: readlink d0/d8/d4c/lcf 0 2024-03-22T07:08:55.785 INFO:tasks.workunit.client.1.smithi149.stdout:4/718: write d0/d8/dc/d2e/d6f/f8e [240081,54582] 0 2024-03-22T07:08:55.786 INFO:tasks.workunit.client.1.smithi149.stdout:8/712: link d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f6e d1/d2/d9/d11/d4d/dd6/d35/d9d/fe5 0 2024-03-22T07:08:55.789 INFO:tasks.workunit.client.1.smithi149.stdout:7/683: unlink d1/d1c/d2f/d74/l8d 0 2024-03-22T07:08:55.793 INFO:tasks.workunit.client.1.smithi149.stdout:0/620: symlink d1/d27/le2 0 2024-03-22T07:08:55.794 INFO:tasks.workunit.client.1.smithi149.stdout:0/621: write d1/d1a/f2d [426728,79436] 0 2024-03-22T07:08:55.797 INFO:tasks.workunit.client.1.smithi149.stdout:8/713: unlink d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d84/f63 0 2024-03-22T07:08:55.797 INFO:tasks.workunit.client.1.smithi149.stdout:8/714: chown d1/d2/d9/d11/d4d/dd6/c7d 795 1 2024-03-22T07:08:55.800 INFO:tasks.workunit.client.1.smithi149.stdout:3/547: rename d0/d4/db5 to d0/d2/d30/dd1 0 2024-03-22T07:08:55.800 INFO:tasks.workunit.client.1.smithi149.stdout:3/548: getdents d0/d2/d5a/daf/dce 0 2024-03-22T07:08:55.800 INFO:tasks.workunit.client.1.smithi149.stdout:3/549: creat d0/d4/d8/fd2 x:0 0 0 2024-03-22T07:08:55.800 INFO:tasks.workunit.client.1.smithi149.stdout:3/550: truncate d0/d2/d30/d5d/d84/dc3/fba 969466 0 2024-03-22T07:08:55.800 INFO:tasks.workunit.client.1.smithi149.stdout:3/551: chown d0/d2/f8c 14 1 2024-03-22T07:08:55.801 INFO:tasks.workunit.client.1.smithi149.stdout:0/622: symlink d1/le3 0 2024-03-22T07:08:55.801 INFO:tasks.workunit.client.1.smithi149.stdout:0/623: chown d1/d31/d51/d64/fa2 292 1 2024-03-22T07:08:55.802 INFO:tasks.workunit.client.1.smithi149.stdout:4/719: getdents d0/d29 0 2024-03-22T07:08:55.802 INFO:tasks.workunit.client.1.smithi149.stdout:4/720: fdatasync d0/d8/f1f 0 2024-03-22T07:08:55.804 INFO:tasks.workunit.client.1.smithi149.stdout:3/552: mknod d0/d2/d30/d5d/d84/d8f/cd3 0 2024-03-22T07:08:55.806 INFO:tasks.workunit.client.1.smithi149.stdout:8/715: dread d1/f16 [0,4194304] 0 2024-03-22T07:08:55.807 INFO:tasks.workunit.client.1.smithi149.stdout:2/767: rename d1/d3/d6/d32/d6f/lf8 to d1/d3/d7d/l112 0 2024-03-22T07:08:55.807 INFO:tasks.workunit.client.1.smithi149.stdout:2/768: chown d1/d8/c43 0 1 2024-03-22T07:08:55.810 INFO:tasks.workunit.client.1.smithi149.stdout:0/624: creat d1/d9/d14/d74/fe4 x:0 0 0 2024-03-22T07:08:55.810 INFO:tasks.workunit.client.1.smithi149.stdout:0/625: dread - d1/d9/d59/dc0/dc9/fcd zero size 2024-03-22T07:08:55.810 INFO:tasks.workunit.client.1.smithi149.stdout:0/626: write d1/d31/d51/d64/d87/f9d [5136206,5282] 0 2024-03-22T07:08:55.810 INFO:tasks.workunit.client.1.smithi149.stdout:4/721: mknod d0/d8/d1d/d24/ce9 0 2024-03-22T07:08:55.810 INFO:tasks.workunit.client.1.smithi149.stdout:4/722: fsync d0/d8/dc/d2e/d6f/f7f 0 2024-03-22T07:08:55.810 INFO:tasks.workunit.client.1.smithi149.stdout:4/723: chown d0/d8/dc/d14/d85/d97 4467 1 2024-03-22T07:08:55.811 INFO:tasks.workunit.client.1.smithi149.stdout:8/716: mknod d1/d2/d9/d11/d8b/daf/ce6 0 2024-03-22T07:08:55.812 INFO:tasks.workunit.client.1.smithi149.stdout:2/769: mknod d1/d3/c113 0 2024-03-22T07:08:55.813 INFO:tasks.workunit.client.1.smithi149.stdout:0/627: mknod d1/d31/d51/d64/ce5 0 2024-03-22T07:08:55.814 INFO:tasks.workunit.client.1.smithi149.stdout:3/553: getdents d0/d2/d5a 0 2024-03-22T07:08:55.814 INFO:tasks.workunit.client.1.smithi149.stdout:3/554: fsync d0/d2/d30/d5d/d84/d8f/f20 0 2024-03-22T07:08:55.814 INFO:tasks.workunit.client.1.smithi149.stdout:3/555: chown d0 501 1 2024-03-22T07:08:55.816 INFO:tasks.workunit.client.1.smithi149.stdout:2/770: rename d1/d8/d34/f63 to d1/d3/d6/d3f/d74/f114 0 2024-03-22T07:08:55.816 INFO:tasks.workunit.client.1.smithi149.stdout:0/628: creat d1/d9/d59/dc0/dc9/dcc/fe6 x:0 0 0 2024-03-22T07:08:55.816 INFO:tasks.workunit.client.1.smithi149.stdout:0/629: stat d1/d31/d51/d64 0 2024-03-22T07:08:55.817 INFO:tasks.workunit.client.1.smithi149.stdout:3/556: creat d0/d4/d8/d24/d40/fd4 x:0 0 0 2024-03-22T07:08:55.818 INFO:tasks.workunit.client.1.smithi149.stdout:2/771: rename d1/d4d/d69/d75/db8/lc0 to d1/d3/d9a/l115 0 2024-03-22T07:08:55.818 INFO:tasks.workunit.client.1.smithi149.stdout:2/772: write d1/d3/d9a/fef [463216,88460] 0 2024-03-22T07:08:55.818 INFO:tasks.workunit.client.1.smithi149.stdout:2/773: stat d1/d4d/ld2 0 2024-03-22T07:08:55.819 INFO:tasks.workunit.client.1.smithi149.stdout:0/630: symlink d1/d31/d51/d64/d87/da6/le7 0 2024-03-22T07:08:55.819 INFO:tasks.workunit.client.1.smithi149.stdout:2/774: creat d1/d4d/d69/d75/d7e/d8c/f116 x:0 0 0 2024-03-22T07:08:55.820 INFO:tasks.workunit.client.1.smithi149.stdout:2/775: mknod d1/d4d/d90/dbc/c117 0 2024-03-22T07:08:55.821 INFO:tasks.workunit.client.1.smithi149.stdout:2/776: symlink d1/d72/l118 0 2024-03-22T07:08:55.822 INFO:tasks.workunit.client.1.smithi149.stdout:2/777: write d1/d4d/d90/ff5 [551817,46713] 0 2024-03-22T07:08:55.823 INFO:tasks.workunit.client.1.smithi149.stdout:2/778: rmdir d1/d3/d6/d32/d6f 39 2024-03-22T07:08:55.823 INFO:tasks.workunit.client.1.smithi149.stdout:2/779: read - d1/d3/d5/d1f/d68/d94/fc8 zero size 2024-03-22T07:08:55.824 INFO:tasks.workunit.client.1.smithi149.stdout:2/780: read d1/d8/fb0 [255243,25624] 0 2024-03-22T07:08:55.824 INFO:tasks.workunit.client.1.smithi149.stdout:2/781: truncate d1/d3/d5/d1f/d5e/f4a 234360 0 2024-03-22T07:08:55.825 INFO:tasks.workunit.client.1.smithi149.stdout:4/724: dread d0/d8/f34 [0,4194304] 0 2024-03-22T07:08:55.825 INFO:tasks.workunit.client.1.smithi149.stdout:8/717: dread d1/d2/d9/d11/d4d/dd6/d35/fb [0,4194304] 0 2024-03-22T07:08:55.828 INFO:tasks.workunit.client.1.smithi149.stdout:8/718: getdents d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/d9e 0 2024-03-22T07:08:55.832 INFO:tasks.workunit.client.1.smithi149.stdout:6/736: dwrite d9/df/d2e/f1c [0,4194304] 0 2024-03-22T07:08:55.832 INFO:tasks.workunit.client.1.smithi149.stdout:6/737: chown d9/df/d22/d6d/d7d/d96/d24/d34/d37/cd9 134 1 2024-03-22T07:08:55.832 INFO:tasks.workunit.client.1.smithi149.stdout:6/738: chown d9/df/d22/dc0/dc8 45942059 1 2024-03-22T07:08:55.832 INFO:tasks.workunit.client.1.smithi149.stdout:4/725: rename d0/d8/dc/d74/d90 to d0/d8/dc/d4b/db3/dea 0 2024-03-22T07:08:55.834 INFO:tasks.workunit.client.1.smithi149.stdout:5/680: dwrite d7/f58 [0,4194304] 0 2024-03-22T07:08:55.834 INFO:tasks.workunit.client.1.smithi149.stdout:5/681: chown d7/da/d20/d30/d3a/d93/d94/dc2 217 1 2024-03-22T07:08:55.837 INFO:tasks.workunit.client.1.smithi149.stdout:3/557: dread d0/d4/d8/d24/d40/d54/d85/dbc/f80 [0,4194304] 0 2024-03-22T07:08:55.838 INFO:tasks.workunit.client.1.smithi149.stdout:1/688: dwrite d4/d10/d58/d68/fd7 [0,4194304] 0 2024-03-22T07:08:55.838 INFO:tasks.workunit.client.1.smithi149.stdout:7/684: dwrite d1/d18/d1b/d51/d7a/d7d/d5d/fa9 [0,4194304] 0 2024-03-22T07:08:55.838 INFO:tasks.workunit.client.1.smithi149.stdout:1/689: chown d4/d20/da0/fb0 347732 1 2024-03-22T07:08:55.838 INFO:tasks.workunit.client.1.smithi149.stdout:1/690: truncate d4/d10/dd8/d3c/d8e/f8f 253533 0 2024-03-22T07:08:55.840 INFO:tasks.workunit.client.1.smithi149.stdout:7/685: dread d1/d1c/d49/d8e/d53/f67 [0,4194304] 0 2024-03-22T07:08:55.840 INFO:tasks.workunit.client.1.smithi149.stdout:7/686: truncate d1/d18/d1b/d51/f52 1062646 0 2024-03-22T07:08:55.840 INFO:tasks.workunit.client.1.smithi149.stdout:7/687: fdatasync d1/d1c/dbc/fcb 0 2024-03-22T07:08:55.842 INFO:tasks.workunit.client.1.smithi149.stdout:8/719: dread d1/d2/d3/fd [0,4194304] 0 2024-03-22T07:08:55.848 INFO:tasks.workunit.client.1.smithi149.stdout:0/631: dwrite d1/d9/d14/d74/f81 [4194304,4194304] 0 2024-03-22T07:08:55.848 INFO:tasks.workunit.client.1.smithi149.stdout:2/782: dwrite d1/d4d/d90/f99 [0,4194304] 0 2024-03-22T07:08:55.848 INFO:tasks.workunit.client.1.smithi149.stdout:6/739: unlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/fb9 0 2024-03-22T07:08:55.849 INFO:tasks.workunit.client.1.smithi149.stdout:6/740: readlink d9/l3a 0 2024-03-22T07:08:55.851 INFO:tasks.workunit.client.1.smithi149.stdout:5/682: mkdir d7/d35/de0/d92/df4/df5 0 2024-03-22T07:08:55.853 INFO:tasks.workunit.client.1.smithi149.stdout:3/558: creat d0/d4/d8/d24/d40/d54/d85/fd5 x:0 0 0 2024-03-22T07:08:55.859 INFO:tasks.workunit.client.1.smithi149.stdout:1/691: mknod d4/d20/d2d/ce9 0 2024-03-22T07:08:55.859 INFO:tasks.workunit.client.1.smithi149.stdout:1/692: write d4/d20/d2d/f91 [320342,1197] 0 2024-03-22T07:08:55.859 INFO:tasks.workunit.client.1.smithi149.stdout:1/693: fsync d4/d20/d2d/faf 0 2024-03-22T07:08:55.862 INFO:tasks.workunit.client.1.smithi149.stdout:8/720: rename d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d82/fa0 to d1/d2/d9/d11/d8b/daf/fe7 0 2024-03-22T07:08:55.862 INFO:tasks.workunit.client.1.smithi149.stdout:8/721: truncate d1/d2/d9/d11/d4d/dd6/d35/d22/fcc 1016026 0 2024-03-22T07:08:55.862 INFO:tasks.workunit.client.1.smithi149.stdout:7/688: dwrite d1/d18/d1b/d51/d7a/d7d/d5d/f6b [0,4194304] 0 2024-03-22T07:08:55.866 INFO:tasks.workunit.client.1.smithi149.stdout:0/632: symlink d1/d31/d51/d64/d7e/le8 0 2024-03-22T07:08:55.867 INFO:tasks.workunit.client.1.smithi149.stdout:4/726: write d0/d8/dc/f30 [2717944,108530] 0 2024-03-22T07:08:55.867 INFO:tasks.workunit.client.1.smithi149.stdout:4/727: getdents d0/d8/d5c/dc1/dd2 0 2024-03-22T07:08:55.867 INFO:tasks.workunit.client.1.smithi149.stdout:4/728: write d0/d6b/f7d [1018056,94322] 0 2024-03-22T07:08:55.867 INFO:tasks.workunit.client.1.smithi149.stdout:4/729: fdatasync d0/d8/f1c 0 2024-03-22T07:08:55.867 INFO:tasks.workunit.client.1.smithi149.stdout:3/559: rmdir d0/d2/d30/d5d 39 2024-03-22T07:08:55.868 INFO:tasks.workunit.client.1.smithi149.stdout:1/694: mknod d4/d10/cea 0 2024-03-22T07:08:55.870 INFO:tasks.workunit.client.1.smithi149.stdout:8/722: truncate d1/d2/d9/d11/d4d/dd6/d51/fa6 231389 0 2024-03-22T07:08:55.870 INFO:tasks.workunit.client.1.smithi149.stdout:8/723: write d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f7e [818114,130122] 0 2024-03-22T07:08:55.874 INFO:tasks.workunit.client.1.smithi149.stdout:0/633: truncate d1/d9/d14/d1f/f5f 2804288 0 2024-03-22T07:08:55.874 INFO:tasks.workunit.client.1.smithi149.stdout:0/634: truncate d1/d31/d51/d64/fa2 880823 0 2024-03-22T07:08:55.876 INFO:tasks.workunit.client.1.smithi149.stdout:4/730: mkdir d0/d8/dc/d74/d76/deb 0 2024-03-22T07:08:55.879 INFO:tasks.workunit.client.1.smithi149.stdout:3/560: symlink d0/d4/d8/d24/d40/ld6 0 2024-03-22T07:08:55.879 INFO:tasks.workunit.client.1.smithi149.stdout:3/561: fsync d0/d4/f58 0 2024-03-22T07:08:55.879 INFO:tasks.workunit.client.1.smithi149.stdout:3/562: truncate d0/d4/d6c/f9b 1303073 0 2024-03-22T07:08:55.879 INFO:tasks.workunit.client.1.smithi149.stdout:8/724: symlink d1/d2/d9/db2/le8 0 2024-03-22T07:08:55.880 INFO:tasks.workunit.client.1.smithi149.stdout:7/689: getdents d1/d18/d1b/d99 0 2024-03-22T07:08:55.881 INFO:tasks.workunit.client.1.smithi149.stdout:6/741: dwrite d9/df/d2e/f1b [0,4194304] 0 2024-03-22T07:08:55.882 INFO:tasks.workunit.client.1.smithi149.stdout:4/731: symlink d0/d8/d1d/d24/d9c/lec 0 2024-03-22T07:08:55.882 INFO:tasks.workunit.client.1.smithi149.stdout:9/691: sync 2024-03-22T07:08:55.884 INFO:tasks.workunit.client.1.smithi149.stdout:8/725: rename d1/d2/d9/df/l6b to d1/d2/d9/d11/d4d/dd6/d35/le9 0 2024-03-22T07:08:55.886 INFO:tasks.workunit.client.1.smithi149.stdout:7/690: rmdir d1/d1c/d49/d8e/dd1 0 2024-03-22T07:08:55.887 INFO:tasks.workunit.client.1.smithi149.stdout:4/732: mkdir d0/d8/dc/d4b/db3/ded 0 2024-03-22T07:08:55.887 INFO:tasks.workunit.client.1.smithi149.stdout:4/733: write d0/d8/dc/d14/d3e/fb1 [826041,117559] 0 2024-03-22T07:08:55.888 INFO:tasks.workunit.client.1.smithi149.stdout:3/563: read d0/d2/d30/f3f [2566704,13439] 0 2024-03-22T07:08:55.888 INFO:tasks.workunit.client.1.smithi149.stdout:3/564: fdatasync d0/d2/d42/f7c 0 2024-03-22T07:08:55.888 INFO:tasks.workunit.client.1.smithi149.stdout:9/692: rmdir db/d21/d37/d7c/da3 39 2024-03-22T07:08:55.888 INFO:tasks.workunit.client.1.smithi149.stdout:9/693: chown db/d42/d49/d4b/d4d 38291775 1 2024-03-22T07:08:55.890 INFO:tasks.workunit.client.1.smithi149.stdout:6/742: rename d9/df/c30 to d9/df/d22/d6d/d7d/cee 0 2024-03-22T07:08:55.892 INFO:tasks.workunit.client.1.smithi149.stdout:7/691: truncate d1/d18/d1b/d51/d7a/d7d/d5b/d97/fc5 3301701 0 2024-03-22T07:08:55.893 INFO:tasks.workunit.client.1.smithi149.stdout:2/783: dwrite d1/d8/d34/f57 [0,4194304] 0 2024-03-22T07:08:55.893 INFO:tasks.workunit.client.1.smithi149.stdout:3/565: mknod d0/d2/d5a/cd7 0 2024-03-22T07:08:55.894 INFO:tasks.workunit.client.1.smithi149.stdout:5/683: dwrite d7/d35/de0/dd7/da4/daf/fcb [0,4194304] 0 2024-03-22T07:08:55.894 INFO:tasks.workunit.client.1.smithi149.stdout:3/566: dread d0/d2/d30/d5d/d84/dc3/fba [0,4194304] 0 2024-03-22T07:08:55.895 INFO:tasks.workunit.client.1.smithi149.stdout:8/726: rename d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d84/fc4 to d1/d2/d9/fea 0 2024-03-22T07:08:55.897 INFO:tasks.workunit.client.1.smithi149.stdout:7/692: creat d1/d18/d1b/d1d/fdf x:0 0 0 2024-03-22T07:08:55.897 INFO:tasks.workunit.client.1.smithi149.stdout:1/695: dwrite d4/d10/d58/f9b [0,4194304] 0 2024-03-22T07:08:55.897 INFO:tasks.workunit.client.1.smithi149.stdout:4/734: getdents d0/d8/dc/d14/d85/d97 0 2024-03-22T07:08:55.897 INFO:tasks.workunit.client.1.smithi149.stdout:4/735: stat d0/d8/d4c/db4 0 2024-03-22T07:08:55.900 INFO:tasks.workunit.client.1.smithi149.stdout:2/784: creat d1/d3/d5/d41/d9f/da7/f119 x:0 0 0 2024-03-22T07:08:55.903 INFO:tasks.workunit.client.1.smithi149.stdout:5/684: symlink d7/da/d20/d89/da2/dda/lf6 0 2024-03-22T07:08:55.905 INFO:tasks.workunit.client.1.smithi149.stdout:5/685: write d7/d35/f55 [210658,72187] 0 2024-03-22T07:08:55.905 INFO:tasks.workunit.client.1.smithi149.stdout:5/686: fdatasync d7/da/d20/f5e 0 2024-03-22T07:08:55.905 INFO:tasks.workunit.client.1.smithi149.stdout:5/687: read d7/da/fd [84101,55109] 0 2024-03-22T07:08:55.906 INFO:tasks.workunit.client.1.smithi149.stdout:8/727: mkdir d1/d2/d9/d11/d4d/dd6/d51/deb 0 2024-03-22T07:08:55.906 INFO:tasks.workunit.client.1.smithi149.stdout:8/728: chown d1/d2/d9/d11/d4d/dd6/d35/l14 648588554 1 2024-03-22T07:08:55.906 INFO:tasks.workunit.client.1.smithi149.stdout:7/693: mkdir d1/d1c/d2f/d74/de0 0 2024-03-22T07:08:55.908 INFO:tasks.workunit.client.1.smithi149.stdout:6/743: dread d9/df/d22/d6d/d7d/d96/f6b [0,4194304] 0 2024-03-22T07:08:55.908 INFO:tasks.workunit.client.1.smithi149.stdout:1/696: mknod d4/d20/d2d/d64/d9f/ceb 0 2024-03-22T07:08:55.908 INFO:tasks.workunit.client.1.smithi149.stdout:1/697: fdatasync d4/d10/dd8/d3c/d85/d89/fe2 0 2024-03-22T07:08:55.910 INFO:tasks.workunit.client.1.smithi149.stdout:4/736: creat d0/d8/dc/d4b/db3/ded/fee x:0 0 0 2024-03-22T07:08:55.910 INFO:tasks.workunit.client.1.smithi149.stdout:2/785: symlink d1/d3/d6/d3f/d86/l11a 0 2024-03-22T07:08:55.910 INFO:tasks.workunit.client.1.smithi149.stdout:2/786: fsync d1/d3/f36 0 2024-03-22T07:08:55.912 INFO:tasks.workunit.client.1.smithi149.stdout:8/729: symlink d1/lec 0 2024-03-22T07:08:55.912 INFO:tasks.workunit.client.1.smithi149.stdout:8/730: readlink d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/l64 0 2024-03-22T07:08:55.914 INFO:tasks.workunit.client.1.smithi149.stdout:8/731: dread d1/d2/d9/d11/d8b/daf/fe7 [0,4194304] 0 2024-03-22T07:08:55.914 INFO:tasks.workunit.client.1.smithi149.stdout:8/732: chown d1/d2/d9/c53 32 1 2024-03-22T07:08:55.914 INFO:tasks.workunit.client.1.smithi149.stdout:5/688: dread d7/da/d20/d36/d84/fa6 [0,4194304] 0 2024-03-22T07:08:55.914 INFO:tasks.workunit.client.1.smithi149.stdout:5/689: truncate d7/da/d20/d36/fbd 924544 0 2024-03-22T07:08:55.914 INFO:tasks.workunit.client.1.smithi149.stdout:5/690: chown d7/da/d20/d45/cd4 413708 1 2024-03-22T07:08:55.914 INFO:tasks.workunit.client.1.smithi149.stdout:7/694: symlink d1/d18/d30/db5/le1 0 2024-03-22T07:08:55.915 INFO:tasks.workunit.client.1.smithi149.stdout:7/695: write d1/d1c/fcf [770088,22408] 0 2024-03-22T07:08:55.916 INFO:tasks.workunit.client.1.smithi149.stdout:6/744: symlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/lef 0 2024-03-22T07:08:55.919 INFO:tasks.workunit.client.1.smithi149.stdout:1/698: write d4/f1b [3651346,61768] 0 2024-03-22T07:08:55.919 INFO:tasks.workunit.client.1.smithi149.stdout:1/699: creat d4/d20/d2d/d64/dce/fec x:0 0 0 2024-03-22T07:08:55.925 INFO:tasks.workunit.client.1.smithi149.stdout:9/694: dwrite db/d21/fcd [0,4194304] 0 2024-03-22T07:08:55.925 INFO:tasks.workunit.client.1.smithi149.stdout:9/695: chown db/d19/l34 266314 1 2024-03-22T07:08:55.925 INFO:tasks.workunit.client.1.smithi149.stdout:3/567: dwrite d0/d4/d8/d24/d40/d54/d85/dbc/f80 [0,4194304] 0 2024-03-22T07:08:55.925 INFO:tasks.workunit.client.1.smithi149.stdout:3/568: write d0/d2/f83 [1391973,46577] 0 2024-03-22T07:08:55.927 INFO:tasks.workunit.client.1.smithi149.stdout:2/787: unlink d1/d8/d34/c67 0 2024-03-22T07:08:55.928 INFO:tasks.workunit.client.1.smithi149.stdout:8/733: creat d1/d2/fed x:0 0 0 2024-03-22T07:08:55.930 INFO:tasks.workunit.client.1.smithi149.stdout:5/691: link d7/d35/de0/dd7/d7b/c2d d7/d35/de0/dd7/d7b/d2b/cf7 0 2024-03-22T07:08:55.930 INFO:tasks.workunit.client.1.smithi149.stdout:5/692: write d7/d35/de0/dd7/d7b/fc [5783650,90633] 0 2024-03-22T07:08:55.931 INFO:tasks.workunit.client.1.smithi149.stdout:6/745: link d9/da2/fc2 d9/df/d22/d6d/deb/ff0 0 2024-03-22T07:08:55.934 INFO:tasks.workunit.client.1.smithi149.stdout:9/696: creat db/d42/dab/fd4 x:0 0 0 2024-03-22T07:08:55.936 INFO:tasks.workunit.client.1.smithi149.stdout:7/696: dwrite d1/f21 [0,4194304] 0 2024-03-22T07:08:55.936 INFO:tasks.workunit.client.1.smithi149.stdout:7/697: chown d1/d18/d1b/d99/d56/d92/d9e/dab 24 1 2024-03-22T07:08:55.936 INFO:tasks.workunit.client.1.smithi149.stdout:7/698: write d1/d18/d1b/d51/d7a/d7d/d5d/f6b [5117414,101466] 0 2024-03-22T07:08:55.937 INFO:tasks.workunit.client.1.smithi149.stdout:3/569: symlink d0/d2/d30/d5d/d84/d8f/ld8 0 2024-03-22T07:08:55.939 INFO:tasks.workunit.client.1.smithi149.stdout:2/788: link d1/d4d/d90/l91 d1/d4d/d69/d75/d7e/d8c/l11b 0 2024-03-22T07:08:55.940 INFO:tasks.workunit.client.1.smithi149.stdout:6/746: dread d9/df/fab [0,4194304] 0 2024-03-22T07:08:55.942 INFO:tasks.workunit.client.1.smithi149.stdout:5/693: unlink d7/d35/de0/dd7/da4/daf/fcb 0 2024-03-22T07:08:55.943 INFO:tasks.workunit.client.1.smithi149.stdout:5/694: truncate d7/da/d20/d30/f98 698364 0 2024-03-22T07:08:55.943 INFO:tasks.workunit.client.1.smithi149.stdout:5/695: chown d7/d35/de0/dd7/fae 31 1 2024-03-22T07:08:55.944 INFO:tasks.workunit.client.1.smithi149.stdout:9/697: truncate db/d12/f17 2269130 0 2024-03-22T07:08:55.944 INFO:tasks.workunit.client.1.smithi149.stdout:9/698: chown db/d19/d1c 122859 1 2024-03-22T07:08:55.944 INFO:tasks.workunit.client.1.smithi149.stdout:9/699: creat db/d19/d28/fd5 x:0 0 0 2024-03-22T07:08:55.947 INFO:tasks.workunit.client.1.smithi149.stdout:7/699: truncate d1/d18/d1b/d1d/f40 7307843 0 2024-03-22T07:08:55.948 INFO:tasks.workunit.client.1.smithi149.stdout:6/747: symlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/lf1 0 2024-03-22T07:08:55.949 INFO:tasks.workunit.client.1.smithi149.stdout:5/696: mknod d7/da/d20/d89/cf8 0 2024-03-22T07:08:55.949 INFO:tasks.workunit.client.1.smithi149.stdout:5/697: dread - d7/d35/de0/dd7/d7b/d81/fab zero size 2024-03-22T07:08:55.953 INFO:tasks.workunit.client.1.smithi149.stdout:7/700: rename d1/d18/d1b/d51/d7a/d7d/d5d/c7e to d1/d18/d1b/d1d/ce2 0 2024-03-22T07:08:55.956 INFO:tasks.workunit.client.1.smithi149.stdout:6/748: rename d9/da2/fe4 to d9/df/d22/ff2 0 2024-03-22T07:08:55.956 INFO:tasks.workunit.client.1.smithi149.stdout:6/749: fdatasync d9/df/d22/d6d/d7d/d96/d24/d42/fb7 0 2024-03-22T07:08:55.957 INFO:tasks.workunit.client.1.smithi149.stdout:6/750: dread - d9/fc9 zero size 2024-03-22T07:08:55.957 INFO:tasks.workunit.client.1.smithi149.stdout:7/701: unlink d1/d1c/d49/dcd/fd8 0 2024-03-22T07:08:55.957 INFO:tasks.workunit.client.1.smithi149.stdout:7/702: readlink d1/d18/d1b/d99/d56/l6c 0 2024-03-22T07:08:55.959 INFO:tasks.workunit.client.1.smithi149.stdout:5/698: rename d7/d35/de0/dd7/d7b/d81/l86 to d7/da/d20/d45/dd9/lf9 0 2024-03-22T07:08:55.959 INFO:tasks.workunit.client.1.smithi149.stdout:5/699: write d7/d35/d8e/fb5 [133648,2841] 0 2024-03-22T07:08:55.959 INFO:tasks.workunit.client.1.smithi149.stdout:5/700: fdatasync d7/da/d20/d30/d3a/d3d/d2c/f8b 0 2024-03-22T07:08:55.960 INFO:tasks.workunit.client.1.smithi149.stdout:6/751: mknod d9/df/d22/d6d/d7d/d96/d24/d34/d37/cf3 0 2024-03-22T07:08:55.960 INFO:tasks.workunit.client.1.smithi149.stdout:6/752: readlink d9/df/d22/l28 0 2024-03-22T07:08:55.962 INFO:tasks.workunit.client.1.smithi149.stdout:7/703: truncate d1/d18/fa4 25148 0 2024-03-22T07:08:55.962 INFO:tasks.workunit.client.1.smithi149.stdout:7/704: fdatasync d1/f21 0 2024-03-22T07:08:55.962 INFO:tasks.workunit.client.1.smithi149.stdout:7/705: write d1/d1c/d49/d8e/d53/fbd [721540,130231] 0 2024-03-22T07:08:55.962 INFO:tasks.workunit.client.1.smithi149.stdout:7/706: chown d1/c17 13070 1 2024-03-22T07:08:55.962 INFO:tasks.workunit.client.1.smithi149.stdout:7/707: write d1/d18/f37 [712955,37459] 0 2024-03-22T07:08:55.962 INFO:tasks.workunit.client.1.smithi149.stdout:7/708: write d1/d18/d1b/d1d/fd [4854335,100811] 0 2024-03-22T07:08:55.963 INFO:tasks.workunit.client.1.smithi149.stdout:7/709: chown d1/d18/d1b/d51/d5a/d6a/da6/d93 1625004690 1 2024-03-22T07:08:55.963 INFO:tasks.workunit.client.1.smithi149.stdout:7/710: chown d1/d18/d1b/d99/d56/d92/d9e/dab/fce 3 1 2024-03-22T07:08:55.968 INFO:tasks.workunit.client.1.smithi149.stdout:8/734: dwrite d1/d2/d3/f96 [0,4194304] 0 2024-03-22T07:08:55.968 INFO:tasks.workunit.client.1.smithi149.stdout:3/570: dwrite d0/d4/d8/d24/d40/d54/d85/fd5 [0,4194304] 0 2024-03-22T07:08:55.971 INFO:tasks.workunit.client.1.smithi149.stdout:9/700: dwrite db/d21/d37/d7c/da3/fc2 [0,4194304] 0 2024-03-22T07:08:55.972 INFO:tasks.workunit.client.1.smithi149.stdout:7/711: dread d1/f33 [0,4194304] 0 2024-03-22T07:08:55.972 INFO:tasks.workunit.client.1.smithi149.stdout:3/571: creat d0/d2/fd9 x:0 0 0 2024-03-22T07:08:55.972 INFO:tasks.workunit.client.1.smithi149.stdout:3/572: chown d0/d4/d8/d24/d40/d62/lca 51145 1 2024-03-22T07:08:55.973 INFO:tasks.workunit.client.1.smithi149.stdout:3/573: write d0/d2/d30/d5d/d84/fc7 [893477,8052] 0 2024-03-22T07:08:55.981 INFO:tasks.workunit.client.1.smithi149.stdout:8/735: write d1/d2/d9/d11/f58 [622003,120157] 0 2024-03-22T07:08:55.981 INFO:tasks.workunit.client.1.smithi149.stdout:9/701: rename db/d21/f23 to db/d21/fd6 0 2024-03-22T07:08:55.982 INFO:tasks.workunit.client.1.smithi149.stdout:9/702: chown db/d42/d49/f8e 218093 1 2024-03-22T07:08:55.982 INFO:tasks.workunit.client.1.smithi149.stdout:9/703: chown db/d21/d37/f64 12378355 1 2024-03-22T07:08:55.982 INFO:tasks.workunit.client.1.smithi149.stdout:3/574: creat d0/d2/d5a/daf/dce/fda x:0 0 0 2024-03-22T07:08:55.985 INFO:tasks.workunit.client.1.smithi149.stdout:9/704: truncate db/d21/d37/f64 958203 0 2024-03-22T07:08:55.987 INFO:tasks.workunit.client.1.smithi149.stdout:9/705: creat db/d19/fd7 x:0 0 0 2024-03-22T07:08:55.988 INFO:tasks.workunit.client.1.smithi149.stdout:7/712: read d1/d18/d1b/d1d/f16 [1737696,71650] 0 2024-03-22T07:08:55.989 INFO:tasks.workunit.client.1.smithi149.stdout:5/701: dwrite d7/da/f37 [4194304,4194304] 0 2024-03-22T07:08:55.989 INFO:tasks.workunit.client.1.smithi149.stdout:9/706: mkdir db/d19/d39/d3b/dd8 0 2024-03-22T07:08:55.991 INFO:tasks.workunit.client.1.smithi149.stdout:6/753: dwrite d9/d47/d52/fd7 [0,4194304] 0 2024-03-22T07:08:55.997 INFO:tasks.workunit.client.1.smithi149.stdout:7/713: fsync d1/d18/d1b/d99/d56/f81 0 2024-03-22T07:08:55.998 INFO:tasks.workunit.client.1.smithi149.stdout:7/714: truncate d1/d18/d1b/d51/d5a/d6a/fd6 3810363 0 2024-03-22T07:08:55.999 INFO:tasks.workunit.client.1.smithi149.stdout:7/715: read - d1/d1c/dbc/fcb zero size 2024-03-22T07:08:56.001 INFO:tasks.workunit.client.1.smithi149.stdout:0/635: sync 2024-03-22T07:08:56.003 INFO:tasks.workunit.client.1.smithi149.stdout:6/754: dread d9/df/d22/d6d/d7d/d96/d24/d42/fe5 [0,4194304] 0 2024-03-22T07:08:56.003 INFO:tasks.workunit.client.1.smithi149.stdout:0/636: mknod d1/d9/d14/d1f/ce9 0 2024-03-22T07:08:56.003 INFO:tasks.workunit.client.1.smithi149.stdout:0/637: truncate d1/d9/fd2 938647 0 2024-03-22T07:08:56.004 INFO:tasks.workunit.client.1.smithi149.stdout:6/755: unlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/caf 0 2024-03-22T07:08:56.004 INFO:tasks.workunit.client.1.smithi149.stdout:6/756: dread - d9/da2/fb0 zero size 2024-03-22T07:08:56.007 INFO:tasks.workunit.client.1.smithi149.stdout:6/757: symlink d9/d47/d52/lf4 0 2024-03-22T07:08:56.014 INFO:tasks.workunit.client.1.smithi149.stdout:8/736: dwrite d1/d2/d9/d11/d45/f70 [0,4194304] 0 2024-03-22T07:08:56.015 INFO:tasks.workunit.client.1.smithi149.stdout:8/737: truncate d1/d2/f8f 1710981 0 2024-03-22T07:08:56.017 INFO:tasks.workunit.client.1.smithi149.stdout:8/738: mkdir d1/d2/d9/d11/d8b/dee 0 2024-03-22T07:08:56.018 INFO:tasks.workunit.client.1.smithi149.stdout:0/638: write d1/d31/d51/d64/d87/f9d [3721721,11069] 0 2024-03-22T07:08:56.020 INFO:tasks.workunit.client.1.smithi149.stdout:8/739: mkdir d1/d2/d9/df/dd5/def 0 2024-03-22T07:08:56.020 INFO:tasks.workunit.client.1.smithi149.stdout:0/639: creat d1/d20/fea x:0 0 0 2024-03-22T07:08:56.020 INFO:tasks.workunit.client.1.smithi149.stdout:0/640: fdatasync d1/d9/d59/f6c 0 2024-03-22T07:08:56.020 INFO:tasks.workunit.client.1.smithi149.stdout:0/641: write d1/d31/d51/d64/d87/f9d [1448069,66675] 0 2024-03-22T07:08:56.022 INFO:tasks.workunit.client.1.smithi149.stdout:0/642: read d1/d6b/f92 [384489,71858] 0 2024-03-22T07:08:56.022 INFO:tasks.workunit.client.1.smithi149.stdout:0/643: write d1/d31/d51/d64/fbd [724707,122293] 0 2024-03-22T07:08:56.022 INFO:tasks.workunit.client.1.smithi149.stdout:7/716: dread d1/d1c/d49/d8e/f50 [0,4194304] 0 2024-03-22T07:08:56.022 INFO:tasks.workunit.client.1.smithi149.stdout:7/717: write d1/d18/d1b/d51/d7a/d7d/f77 [1306956,66386] 0 2024-03-22T07:08:56.028 INFO:tasks.workunit.client.1.smithi149.stdout:9/707: dwrite db/d19/d39/d3b/f48 [0,4194304] 0 2024-03-22T07:08:56.034 INFO:tasks.workunit.client.1.smithi149.stdout:6/758: dwrite d9/df/d22/d6d/d7d/fa8 [0,4194304] 0 2024-03-22T07:08:56.036 INFO:tasks.workunit.client.1.smithi149.stdout:6/759: creat d9/df/d22/ff5 x:0 0 0 2024-03-22T07:08:56.036 INFO:tasks.workunit.client.1.smithi149.stdout:6/760: read - d9/d47/fe3 zero size 2024-03-22T07:08:56.036 INFO:tasks.workunit.client.1.smithi149.stdout:6/761: chown d9/df/d22/d6d/deb/fdb 180 1 2024-03-22T07:08:56.036 INFO:tasks.workunit.client.1.smithi149.stdout:3/575: dwrite d0/d4/d8/fd2 [0,4194304] 0 2024-03-22T07:08:56.036 INFO:tasks.workunit.client.1.smithi149.stdout:8/740: dread d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d84/f48 [0,4194304] 0 2024-03-22T07:08:56.038 INFO:tasks.workunit.client.1.smithi149.stdout:6/762: mkdir d9/df/d22/d6d/d7d/d96/d24/d34/df6 0 2024-03-22T07:08:56.043 INFO:tasks.workunit.client.1.smithi149.stdout:9/708: dread db/d42/fad [0,4194304] 0 2024-03-22T07:08:56.044 INFO:tasks.workunit.client.1.smithi149.stdout:3/576: mknod d0/d2/d30/dd1/dc9/cdb 0 2024-03-22T07:08:56.045 INFO:tasks.workunit.client.1.smithi149.stdout:8/741: symlink d1/d2/d9/d11/d4d/dd6/d51/deb/lf0 0 2024-03-22T07:08:56.045 INFO:tasks.workunit.client.1.smithi149.stdout:3/577: truncate d0/d2/d30/f8a 1800172 0 2024-03-22T07:08:56.046 INFO:tasks.workunit.client.1.smithi149.stdout:8/742: creat d1/d2/d9/ff1 x:0 0 0 2024-03-22T07:08:56.046 INFO:tasks.workunit.client.1.smithi149.stdout:8/743: chown d1/d2/d9/df/dd5/def 5824 1 2024-03-22T07:08:56.047 INFO:tasks.workunit.client.1.smithi149.stdout:3/578: symlink d0/d4/d8/d24/ldc 0 2024-03-22T07:08:56.047 INFO:tasks.workunit.client.1.smithi149.stdout:3/579: getdents d0/d4/d8/d24/db6/dcd 0 2024-03-22T07:08:56.052 INFO:tasks.workunit.client.1.smithi149.stdout:0/644: dwrite d1/d9/d59/dc0/dc9/fcd [0,4194304] 0 2024-03-22T07:08:56.052 INFO:tasks.workunit.client.1.smithi149.stdout:0/645: write d1/d31/d51/d64/d87/da6/fd6 [725672,128687] 0 2024-03-22T07:08:56.053 INFO:tasks.workunit.client.1.smithi149.stdout:6/763: dread d9/df/d22/d6d/d7d/d96/d24/d68/fc3 [0,4194304] 0 2024-03-22T07:08:56.053 INFO:tasks.workunit.client.1.smithi149.stdout:7/718: dwrite d1/d1c/d49/d8e/d53/f67 [0,4194304] 0 2024-03-22T07:08:56.054 INFO:tasks.workunit.client.1.smithi149.stdout:7/719: chown d1/d18/d1b/l69 17034 1 2024-03-22T07:08:56.059 INFO:tasks.workunit.client.1.smithi149.stdout:9/709: dread db/d19/f7d [0,4194304] 0 2024-03-22T07:08:56.059 INFO:tasks.workunit.client.1.smithi149.stdout:9/710: chown db/d42/d49/d4b/d5f/cc3 13693190 1 2024-03-22T07:08:56.059 INFO:tasks.workunit.client.1.smithi149.stdout:6/764: mknod d9/df/d22/d6d/ddd/cf7 0 2024-03-22T07:08:56.059 INFO:tasks.workunit.client.1.smithi149.stdout:4/737: sync 2024-03-22T07:08:56.059 INFO:tasks.workunit.client.1.smithi149.stdout:4/738: chown d0/d8/d5c/dc1/dd2 46373531 1 2024-03-22T07:08:56.059 INFO:tasks.workunit.client.1.smithi149.stdout:3/580: dread d0/d4/d8/d24/f6e [0,4194304] 0 2024-03-22T07:08:56.059 INFO:tasks.workunit.client.1.smithi149.stdout:7/720: rename d1/d18/d1b/d51/d7a/d7d/d5b/d97 to d1/d18/d1b/d51/d7a/d7d/d5d/de3 0 2024-03-22T07:08:56.059 INFO:tasks.workunit.client.1.smithi149.stdout:7/721: chown d1/ce 13 1 2024-03-22T07:08:56.060 INFO:tasks.workunit.client.1.smithi149.stdout:6/765: mkdir d9/df/d22/d6d/d7d/d96/df8 0 2024-03-22T07:08:56.061 INFO:tasks.workunit.client.1.smithi149.stdout:4/739: mkdir d0/d8/dc/d2e/d6f/def 0 2024-03-22T07:08:56.062 INFO:tasks.workunit.client.1.smithi149.stdout:3/581: getdents d0/d2/d30/dd1 0 2024-03-22T07:08:56.062 INFO:tasks.workunit.client.1.smithi149.stdout:3/582: chown d0/d2/d42/c57 53 1 2024-03-22T07:08:56.062 INFO:tasks.workunit.client.1.smithi149.stdout:3/583: chown d0/d4/d8/f99 1 1 2024-03-22T07:08:56.062 INFO:tasks.workunit.client.1.smithi149.stdout:3/584: chown d0/d4/d8/d24/d39/fb1 3 1 2024-03-22T07:08:56.063 INFO:tasks.workunit.client.1.smithi149.stdout:6/766: creat d9/df/d22/d6d/d7d/ff9 x:0 0 0 2024-03-22T07:08:56.063 INFO:tasks.workunit.client.1.smithi149.stdout:8/744: dread d1/d2/d9/d11/d45/f70 [0,4194304] 0 2024-03-22T07:08:56.065 INFO:tasks.workunit.client.1.smithi149.stdout:4/740: mknod d0/d8/d1d/cf0 0 2024-03-22T07:08:56.066 INFO:tasks.workunit.client.1.smithi149.stdout:0/646: write d1/d9/d59/f61 [162404,71147] 0 2024-03-22T07:08:56.066 INFO:tasks.workunit.client.1.smithi149.stdout:0/647: write d1/d9/d14/fa1 [4990736,56657] 0 2024-03-22T07:08:56.066 INFO:tasks.workunit.client.1.smithi149.stdout:0/648: chown d1/d27/f6e 3009 1 2024-03-22T07:08:56.074 INFO:tasks.workunit.client.1.smithi149.stdout:1/700: sync 2024-03-22T07:08:56.074 INFO:tasks.workunit.client.1.smithi149.stdout:3/585: link d0/d4/d8/f8d d0/d2/d5a/daf/fdd 0 2024-03-22T07:08:56.074 INFO:tasks.workunit.client.1.smithi149.stdout:8/745: mkdir d1/d2/d9/d11/d8b/d8d/df2 0 2024-03-22T07:08:56.074 INFO:tasks.workunit.client.1.smithi149.stdout:0/649: getdents d1/d9/d14 0 2024-03-22T07:08:56.074 INFO:tasks.workunit.client.1.smithi149.stdout:0/650: truncate d1/d20/d9f/dc6/fcf 416862 0 2024-03-22T07:08:56.074 INFO:tasks.workunit.client.1.smithi149.stdout:1/701: rmdir d4/d10/dd8/d3c/d8e/db5/db2/dbf 39 2024-03-22T07:08:56.077 INFO:tasks.workunit.client.1.smithi149.stdout:8/746: unlink d1/d2/d9/d11/ce0 0 2024-03-22T07:08:56.079 INFO:tasks.workunit.client.1.smithi149.stdout:0/651: symlink d1/d9/d59/leb 0 2024-03-22T07:08:56.079 INFO:tasks.workunit.client.1.smithi149.stdout:0/652: creat d1/d31/d52/da9/fec x:0 0 0 2024-03-22T07:08:56.079 INFO:tasks.workunit.client.1.smithi149.stdout:8/747: mkdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/d9e/df3 0 2024-03-22T07:08:56.082 INFO:tasks.workunit.client.1.smithi149.stdout:7/722: write d1/d1c/d49/d8e/f87 [7262402,84862] 0 2024-03-22T07:08:56.082 INFO:tasks.workunit.client.1.smithi149.stdout:7/723: chown d1/d18/d1b/d1d/d72/l8f 7395 1 2024-03-22T07:08:56.082 INFO:tasks.workunit.client.1.smithi149.stdout:7/724: read d1/d18/d1b/d1d/f8 [329540,115899] 0 2024-03-22T07:08:56.085 INFO:tasks.workunit.client.1.smithi149.stdout:8/748: mkdir d1/d2/d9/d11/d8b/dee/df4 0 2024-03-22T07:08:56.086 INFO:tasks.workunit.client.1.smithi149.stdout:7/725: link d1/d18/d1b/d51/d7a/d7d/f86 d1/d18/d1b/d1d/fe4 0 2024-03-22T07:08:56.086 INFO:tasks.workunit.client.1.smithi149.stdout:7/726: chown d1/d18/d1b/d51/d5a/d6a/da6/d93 3259 1 2024-03-22T07:08:56.091 INFO:tasks.workunit.client.1.smithi149.stdout:7/727: rename d1/d18/d1b/d99/d56 to d1/d18/d1b/d99/de5 0 2024-03-22T07:08:56.102 INFO:tasks.workunit.client.1.smithi149.stdout:9/711: dwrite db/d33/f92 [0,4194304] 0 2024-03-22T07:08:56.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/702: dwrite d4/d10/dd8/d3c/d8e/db5/f53 [0,4194304] 0 2024-03-22T07:08:56.105 INFO:tasks.workunit.client.1.smithi149.stdout:1/703: chown d4/d10/dd8/d3c/d8e/db5/c45 959 1 2024-03-22T07:08:56.105 INFO:tasks.workunit.client.1.smithi149.stdout:8/749: dread d1/d2/d9/df/f15 [0,4194304] 0 2024-03-22T07:08:56.107 INFO:tasks.workunit.client.1.smithi149.stdout:1/704: mknod d4/d20/da0/dc0/ced 0 2024-03-22T07:08:56.108 INFO:tasks.workunit.client.1.smithi149.stdout:1/705: chown d4/d10/dd8/d3c/d8e/db5 36 1 2024-03-22T07:08:56.109 INFO:tasks.workunit.client.1.smithi149.stdout:8/750: creat d1/d2/d9/d11/d4d/dd6/d35/d22/ff5 x:0 0 0 2024-03-22T07:08:56.109 INFO:tasks.workunit.client.1.smithi149.stdout:8/751: write d1/d2/d18/f3d [731179,48240] 0 2024-03-22T07:08:56.109 INFO:tasks.workunit.client.1.smithi149.stdout:8/752: dread - d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/fc9 zero size 2024-03-22T07:08:56.111 INFO:tasks.workunit.client.1.smithi149.stdout:9/712: rmdir db/d42/d49/d4b 39 2024-03-22T07:08:56.112 INFO:tasks.workunit.client.1.smithi149.stdout:1/706: mknod d4/d10/dd8/d3c/d8e/cee 0 2024-03-22T07:08:56.114 INFO:tasks.workunit.client.1.smithi149.stdout:1/707: creat d4/d10/dd8/d2f/d8c/daa/fef x:0 0 0 2024-03-22T07:08:56.114 INFO:tasks.workunit.client.1.smithi149.stdout:1/708: write d4/d10/dd8/f6f [82710,98541] 0 2024-03-22T07:08:56.114 INFO:tasks.workunit.client.1.smithi149.stdout:1/709: creat d4/d7/d61/ff0 x:0 0 0 2024-03-22T07:08:56.116 INFO:tasks.workunit.client.1.smithi149.stdout:1/710: truncate d4/d10/f4c 186936 0 2024-03-22T07:08:56.120 INFO:tasks.workunit.client.1.smithi149.stdout:4/741: dwrite d0/d8/dc/d74/ddc/fe7 [0,4194304] 0 2024-03-22T07:08:56.120 INFO:tasks.workunit.client.1.smithi149.stdout:4/742: write d0/d29/f6e [4318794,126801] 0 2024-03-22T07:08:56.120 INFO:tasks.workunit.client.1.smithi149.stdout:4/743: read - d0/d8/d4c/fe4 zero size 2024-03-22T07:08:56.120 INFO:tasks.workunit.client.1.smithi149.stdout:4/744: chown d0/d8/d1d/fe6 1341 1 2024-03-22T07:08:56.120 INFO:tasks.workunit.client.1.smithi149.stdout:4/745: chown d0/d8/dc/d14 270013 1 2024-03-22T07:08:56.121 INFO:tasks.workunit.client.1.smithi149.stdout:4/746: mkdir d0/d8/d1d/df1 0 2024-03-22T07:08:56.125 INFO:tasks.workunit.client.1.smithi149.stdout:9/713: dread db/d42/f5a [0,4194304] 0 2024-03-22T07:08:56.130 INFO:tasks.workunit.client.1.smithi149.stdout:6/767: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 [0,4194304] 0 2024-03-22T07:08:56.130 INFO:tasks.workunit.client.1.smithi149.stdout:0/653: dwrite d1/d9/d14/d1f/f89 [0,4194304] 0 2024-03-22T07:08:56.130 INFO:tasks.workunit.client.1.smithi149.stdout:3/586: dwrite d0/d2/d30/f8a [0,4194304] 0 2024-03-22T07:08:56.131 INFO:tasks.workunit.client.1.smithi149.stdout:7/728: dwrite d1/f10 [0,4194304] 0 2024-03-22T07:08:56.132 INFO:tasks.workunit.client.1.smithi149.stdout:4/747: write d0/d8/dc/d14/d3e/fb1 [743520,2187] 0 2024-03-22T07:08:56.132 INFO:tasks.workunit.client.1.smithi149.stdout:4/748: truncate d0/d8/f21 798955 0 2024-03-22T07:08:56.134 INFO:tasks.workunit.client.1.smithi149.stdout:6/768: mknod d9/da2/cfa 0 2024-03-22T07:08:56.134 INFO:tasks.workunit.client.1.smithi149.stdout:6/769: chown d9/df/d22/d6d/d7d/d96/d24/d34/l6f 61321854 1 2024-03-22T07:08:56.138 INFO:tasks.workunit.client.1.smithi149.stdout:9/714: write db/d19/f3c [2985881,6908] 0 2024-03-22T07:08:56.141 INFO:tasks.workunit.client.1.smithi149.stdout:3/587: link d0/d2/d30/f73 d0/d2/d5a/fde 0 2024-03-22T07:08:56.141 INFO:tasks.workunit.client.1.smithi149.stdout:3/588: stat d0/d2/f83 0 2024-03-22T07:08:56.143 INFO:tasks.workunit.client.1.smithi149.stdout:0/654: symlink d1/d31/d52/da9/led 0 2024-03-22T07:08:56.143 INFO:tasks.workunit.client.1.smithi149.stdout:7/729: rename d1/d18/d1b/d51/d7a/d7d/f86 to d1/d18/d1b/d1d/fe6 0 2024-03-22T07:08:56.144 INFO:tasks.workunit.client.1.smithi149.stdout:0/655: write d1/d9/d14/d1f/f38 [99433,7500] 0 2024-03-22T07:08:56.144 INFO:tasks.workunit.client.1.smithi149.stdout:4/749: truncate d0/d8/dc/d14/d3e/f6a 1402594 0 2024-03-22T07:08:56.144 INFO:tasks.workunit.client.1.smithi149.stdout:4/750: chown d0/d29/l60 0 1 2024-03-22T07:08:56.147 INFO:tasks.workunit.client.1.smithi149.stdout:0/656: mkdir d1/d20/d9f/dc6/dee 0 2024-03-22T07:08:56.147 INFO:tasks.workunit.client.1.smithi149.stdout:0/657: truncate d1/d9/d14/fc4 796252 0 2024-03-22T07:08:56.147 INFO:tasks.workunit.client.1.smithi149.stdout:0/658: fsync d1/d9/f3a 0 2024-03-22T07:08:56.148 INFO:tasks.workunit.client.1.smithi149.stdout:4/751: mkdir d0/d8/dc/d4b/db3/ded/df2 0 2024-03-22T07:08:56.150 INFO:tasks.workunit.client.1.smithi149.stdout:7/730: rename d1/d1c/d2f to d1/d18/d1b/d51/d5a/de7 0 2024-03-22T07:08:56.151 INFO:tasks.workunit.client.1.smithi149.stdout:0/659: mknod d1/d9/cef 0 2024-03-22T07:08:56.152 INFO:tasks.workunit.client.1.smithi149.stdout:0/660: chown d1/d9/d14/dd3 1020 1 2024-03-22T07:08:56.152 INFO:tasks.workunit.client.1.smithi149.stdout:0/661: dread - d1/d9/d14/d74/d77/fb2 zero size 2024-03-22T07:08:56.155 INFO:tasks.workunit.client.1.smithi149.stdout:9/715: write db/d19/d28/fc1 [1315447,84209] 0 2024-03-22T07:08:56.159 INFO:tasks.workunit.client.1.smithi149.stdout:1/711: dwrite d4/d20/d2d/d64/fb3 [0,4194304] 0 2024-03-22T07:08:56.159 INFO:tasks.workunit.client.1.smithi149.stdout:8/753: dwrite d1/d2/d3/f6 [4194304,4194304] 0 2024-03-22T07:08:56.159 INFO:tasks.workunit.client.1.smithi149.stdout:6/770: dwrite d9/df/d22/d6d/d7d/d96/f6b [0,4194304] 0 2024-03-22T07:08:56.159 INFO:tasks.workunit.client.1.smithi149.stdout:3/589: dread d0/d2/d30/d5d/d84/d8f/f50 [0,4194304] 0 2024-03-22T07:08:56.159 INFO:tasks.workunit.client.1.smithi149.stdout:6/771: creat d9/da2/ffb x:0 0 0 2024-03-22T07:08:56.167 INFO:tasks.workunit.client.1.smithi149.stdout:1/712: unlink d4/ca8 0 2024-03-22T07:08:56.169 INFO:tasks.workunit.client.1.smithi149.stdout:3/590: link d0/d4/d8/l44 d0/d4/d8/d24/d40/d62/ldf 0 2024-03-22T07:08:56.172 INFO:tasks.workunit.client.1.smithi149.stdout:3/591: creat d0/d4/d8/d24/db6/dcd/fe0 x:0 0 0 2024-03-22T07:08:56.172 INFO:tasks.workunit.client.1.smithi149.stdout:3/592: chown d0/d4/d8/d24/d39/f71 10 1 2024-03-22T07:08:56.172 INFO:tasks.workunit.client.1.smithi149.stdout:2/789: sync 2024-03-22T07:08:56.173 INFO:tasks.workunit.client.1.smithi149.stdout:8/754: write d1/d2/d3/f86 [3049790,11378] 0 2024-03-22T07:08:56.174 INFO:tasks.workunit.client.1.smithi149.stdout:2/790: creat d1/d3/d5/d1f/d5e/d2f/f11c x:0 0 0 2024-03-22T07:08:56.174 INFO:tasks.workunit.client.1.smithi149.stdout:6/772: write d9/df/f1d [2561043,64355] 0 2024-03-22T07:08:56.175 INFO:tasks.workunit.client.1.smithi149.stdout:3/593: symlink d0/d4/d8/d24/d40/d54/d9e/le1 0 2024-03-22T07:08:56.175 INFO:tasks.workunit.client.1.smithi149.stdout:5/702: sync 2024-03-22T07:08:56.176 INFO:tasks.workunit.client.1.smithi149.stdout:8/755: mknod d1/d2/d9/df/dd5/cf6 0 2024-03-22T07:08:56.176 INFO:tasks.workunit.client.1.smithi149.stdout:8/756: chown d1/d2/d9/df/fdf 2823946 1 2024-03-22T07:08:56.179 INFO:tasks.workunit.client.1.smithi149.stdout:8/757: dread d1/d2/d9/d11/d8b/daf/fe7 [0,4194304] 0 2024-03-22T07:08:56.179 INFO:tasks.workunit.client.1.smithi149.stdout:8/758: readlink d1/d2/d18/ld1 0 2024-03-22T07:08:56.179 INFO:tasks.workunit.client.1.smithi149.stdout:6/773: mknod d9/df/d22/d6d/d7d/cfc 0 2024-03-22T07:08:56.180 INFO:tasks.workunit.client.1.smithi149.stdout:3/594: dread d0/d2/d30/d5d/d84/d8f/fc5 [4194304,4194304] 0 2024-03-22T07:08:56.180 INFO:tasks.workunit.client.1.smithi149.stdout:3/595: write d0/d4/d8/d24/db6/dcd/fe0 [392070,79724] 0 2024-03-22T07:08:56.180 INFO:tasks.workunit.client.1.smithi149.stdout:8/759: truncate d1/d2/d9/d11/d4d/dd6/d35/d22/f38 2505316 0 2024-03-22T07:08:56.180 INFO:tasks.workunit.client.1.smithi149.stdout:3/596: chown d0/d4/d6c/f9b 298749442 1 2024-03-22T07:08:56.183 INFO:tasks.workunit.client.1.smithi149.stdout:3/597: rename d0/d4/d8/f21 to d0/d4/d8/d24/db6/dcd/fe2 0 2024-03-22T07:08:56.184 INFO:tasks.workunit.client.1.smithi149.stdout:8/760: creat d1/d2/d9/d11/d4d/dd6/db4/ff7 x:0 0 0 2024-03-22T07:08:56.184 INFO:tasks.workunit.client.1.smithi149.stdout:8/761: truncate d1/d2/d9/db2/fc1 88566 0 2024-03-22T07:08:56.185 INFO:tasks.workunit.client.1.smithi149.stdout:3/598: truncate d0/d4/d8/f25 3379517 0 2024-03-22T07:08:56.185 INFO:tasks.workunit.client.1.smithi149.stdout:1/713: read d4/d7/f57 [64591,31810] 0 2024-03-22T07:08:56.186 INFO:tasks.workunit.client.1.smithi149.stdout:8/762: creat d1/d2/d9/df/dd5/ff8 x:0 0 0 2024-03-22T07:08:56.186 INFO:tasks.workunit.client.1.smithi149.stdout:8/763: readlink d1/d2/d9/d11/l80 0 2024-03-22T07:08:56.189 INFO:tasks.workunit.client.1.smithi149.stdout:1/714: creat d4/d10/dd8/d3c/d8e/db5/db2/dbf/ff1 x:0 0 0 2024-03-22T07:08:56.189 INFO:tasks.workunit.client.1.smithi149.stdout:1/715: chown d4/d10/dd8/d3c/cb6 14 1 2024-03-22T07:08:56.189 INFO:tasks.workunit.client.1.smithi149.stdout:1/716: write d4/d10/dd8/f6f [172502,21461] 0 2024-03-22T07:08:56.190 INFO:tasks.workunit.client.1.smithi149.stdout:8/764: symlink d1/d2/d9/d11/d4d/dd6/d51/lf9 0 2024-03-22T07:08:56.190 INFO:tasks.workunit.client.1.smithi149.stdout:0/662: dwrite d1/d9/d14/f55 [0,4194304] 0 2024-03-22T07:08:56.190 INFO:tasks.workunit.client.1.smithi149.stdout:7/731: dwrite d1/d18/d1b/d1d/f13 [0,4194304] 0 2024-03-22T07:08:56.192 INFO:tasks.workunit.client.1.smithi149.stdout:1/717: symlink d4/d7/d5a/dca/lf2 0 2024-03-22T07:08:56.194 INFO:tasks.workunit.client.1.smithi149.stdout:2/791: dread d1/d4d/d6a/de1/fa8 [0,4194304] 0 2024-03-22T07:08:56.199 INFO:tasks.workunit.client.1.smithi149.stdout:7/732: mkdir d1/d18/de8 0 2024-03-22T07:08:56.200 INFO:tasks.workunit.client.1.smithi149.stdout:0/663: mknod d1/d31/d51/cf0 0 2024-03-22T07:08:56.200 INFO:tasks.workunit.client.1.smithi149.stdout:0/664: chown d1/d20/d9f/cd1 1433465174 1 2024-03-22T07:08:56.201 INFO:tasks.workunit.client.1.smithi149.stdout:8/765: getdents d1/d2/d9/d11/d8b/d8d/d91 0 2024-03-22T07:08:56.202 INFO:tasks.workunit.client.1.smithi149.stdout:1/718: creat d4/d20/da0/dc0/ff3 x:0 0 0 2024-03-22T07:08:56.202 INFO:tasks.workunit.client.1.smithi149.stdout:1/719: write d4/d7/f50 [749413,35598] 0 2024-03-22T07:08:56.202 INFO:tasks.workunit.client.1.smithi149.stdout:1/720: fdatasync d4/d10/d7c/f98 0 2024-03-22T07:08:56.202 INFO:tasks.workunit.client.1.smithi149.stdout:1/721: readlink l3 0 2024-03-22T07:08:56.202 INFO:tasks.workunit.client.1.smithi149.stdout:2/792: creat d1/d4d/d69/d75/db6/de3/f11d x:0 0 0 2024-03-22T07:08:56.203 INFO:tasks.workunit.client.1.smithi149.stdout:7/733: creat d1/d18/d1b/d51/d5a/fe9 x:0 0 0 2024-03-22T07:08:56.204 INFO:tasks.workunit.client.1.smithi149.stdout:2/793: creat d1/d4d/d8b/f11e x:0 0 0 2024-03-22T07:08:56.204 INFO:tasks.workunit.client.1.smithi149.stdout:2/794: write d1/d3/f3e [1152264,35468] 0 2024-03-22T07:08:56.204 INFO:tasks.workunit.client.1.smithi149.stdout:2/795: truncate d1/d4d/d90/dbc/fe2 344148 0 2024-03-22T07:08:56.204 INFO:tasks.workunit.client.1.smithi149.stdout:3/599: write d0/d4/f2f [3670561,94633] 0 2024-03-22T07:08:56.205 INFO:tasks.workunit.client.1.smithi149.stdout:7/734: rename d1/d1c/d49/f80 to d1/d1c/d49/dc0/fea 0 2024-03-22T07:08:56.207 INFO:tasks.workunit.client.1.smithi149.stdout:3/600: symlink d0/d4/d8/d24/d40/d62/le3 0 2024-03-22T07:08:56.209 INFO:tasks.workunit.client.1.smithi149.stdout:3/601: fsync d0/d4/d8/d24/d39/f89 0 2024-03-22T07:08:56.209 INFO:tasks.workunit.client.1.smithi149.stdout:7/735: link d1/f33 d1/d18/d1b/d99/de5/d92/d9e/feb 0 2024-03-22T07:08:56.210 INFO:tasks.workunit.client.1.smithi149.stdout:4/752: dwrite d0/d8/d4c/fbb [0,4194304] 0 2024-03-22T07:08:56.211 INFO:tasks.workunit.client.1.smithi149.stdout:7/736: rename d1/f33 to d1/d18/d1b/d51/d5a/fec 0 2024-03-22T07:08:56.211 INFO:tasks.workunit.client.1.smithi149.stdout:6/774: dwrite d9/df/f1e [0,4194304] 0 2024-03-22T07:08:56.214 INFO:tasks.workunit.client.1.smithi149.stdout:1/722: read d4/d10/dd8/d3c/d8e/db5/fe1 [2420319,26925] 0 2024-03-22T07:08:56.216 INFO:tasks.workunit.client.1.smithi149.stdout:4/753: getdents d0/d8/d1d/d24/d82 0 2024-03-22T07:08:56.216 INFO:tasks.workunit.client.1.smithi149.stdout:5/703: dwrite d7/da/d20/d30/f98 [0,4194304] 0 2024-03-22T07:08:56.216 INFO:tasks.workunit.client.1.smithi149.stdout:1/723: dread d4/d10/dd8/d3c/d8e/f8f [0,4194304] 0 2024-03-22T07:08:56.218 INFO:tasks.workunit.client.1.smithi149.stdout:7/737: creat d1/d18/d1b/d99/fed x:0 0 0 2024-03-22T07:08:56.220 INFO:tasks.workunit.client.1.smithi149.stdout:6/775: unlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/ce8 0 2024-03-22T07:08:56.222 INFO:tasks.workunit.client.1.smithi149.stdout:3/602: getdents d0/d4/d8/d24/d40/d54/d85/dbc 0 2024-03-22T07:08:56.222 INFO:tasks.workunit.client.1.smithi149.stdout:3/603: chown d0/d2/d30/d5d/d84/dc3/l68 13247 1 2024-03-22T07:08:56.222 INFO:tasks.workunit.client.1.smithi149.stdout:3/604: chown d0/d4/d8/d24/d40/d54/d85/dbc/f80 1653 1 2024-03-22T07:08:56.223 INFO:tasks.workunit.client.1.smithi149.stdout:1/724: dread d4/d10/d58/f5b [4194304,4194304] 0 2024-03-22T07:08:56.226 INFO:tasks.workunit.client.1.smithi149.stdout:8/766: read d1/d2/d9/d11/d4d/dd6/fc7 [1536489,121177] 0 2024-03-22T07:08:56.227 INFO:tasks.workunit.client.1.smithi149.stdout:7/738: mkdir d1/d18/d1b/d51/d7a/d7d/d5d/de3/dee 0 2024-03-22T07:08:56.228 INFO:tasks.workunit.client.1.smithi149.stdout:6/776: mknod d9/da2/cfd 0 2024-03-22T07:08:56.229 INFO:tasks.workunit.client.1.smithi149.stdout:2/796: dwrite d1/d8/d34/f57 [0,4194304] 0 2024-03-22T07:08:56.230 INFO:tasks.workunit.client.1.smithi149.stdout:3/605: mknod d0/d2/d30/dd1/ce4 0 2024-03-22T07:08:56.232 INFO:tasks.workunit.client.1.smithi149.stdout:7/739: mknod d1/db8/cef 0 2024-03-22T07:08:56.232 INFO:tasks.workunit.client.1.smithi149.stdout:7/740: dread - d1/d18/d1b/d1d/fdf zero size 2024-03-22T07:08:56.234 INFO:tasks.workunit.client.1.smithi149.stdout:4/754: write d0/d6b/f94 [3341869,125001] 0 2024-03-22T07:08:56.234 INFO:tasks.workunit.client.1.smithi149.stdout:6/777: rename d9/df/d22/d6d/d7d/d96/d24/d34/df6 to d9/df/d2e/d63/dfe 0 2024-03-22T07:08:56.235 INFO:tasks.workunit.client.1.smithi149.stdout:0/665: dwrite d1/d9/d59/f61 [0,4194304] 0 2024-03-22T07:08:56.238 INFO:tasks.workunit.client.1.smithi149.stdout:8/767: link d1/d2/d9/d11/d4d/dd6/d51/fa6 d1/d2/d9/d11/d4d/ffa 0 2024-03-22T07:08:56.239 INFO:tasks.workunit.client.1.smithi149.stdout:7/741: mkdir d1/d18/d1b/d51/d5a/df0 0 2024-03-22T07:08:56.243 INFO:tasks.workunit.client.1.smithi149.stdout:4/755: mknod d0/d8/d4c/cf3 0 2024-03-22T07:08:56.244 INFO:tasks.workunit.client.1.smithi149.stdout:6/778: rename d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/fe2 to d9/da2/fff 0 2024-03-22T07:08:56.246 INFO:tasks.workunit.client.1.smithi149.stdout:1/725: dwrite d4/f3e [0,4194304] 0 2024-03-22T07:08:56.246 INFO:tasks.workunit.client.1.smithi149.stdout:1/726: creat d4/d20/ff4 x:0 0 0 2024-03-22T07:08:56.246 INFO:tasks.workunit.client.1.smithi149.stdout:8/768: mknod d1/d2/d9/df/cfb 0 2024-03-22T07:08:56.248 INFO:tasks.workunit.client.1.smithi149.stdout:4/756: link d0/d8/dc/d4b/db3/ld8 d0/d8/dc/d14/d3e/lf4 0 2024-03-22T07:08:56.249 INFO:tasks.workunit.client.1.smithi149.stdout:0/666: rename d1/db4 to d1/d27/d66/df1 0 2024-03-22T07:08:56.252 INFO:tasks.workunit.client.1.smithi149.stdout:1/727: symlink d4/d10/d58/d68/lf5 0 2024-03-22T07:08:56.252 INFO:tasks.workunit.client.1.smithi149.stdout:1/728: stat d4/d10/d7c/d6a/d6b/f84 0 2024-03-22T07:08:56.253 INFO:tasks.workunit.client.1.smithi149.stdout:4/757: chown d0/d8/dc/ca9 12777041 1 2024-03-22T07:08:56.253 INFO:tasks.workunit.client.1.smithi149.stdout:4/758: fdatasync d0/d8/d1d/d24/d82/fa1 0 2024-03-22T07:08:56.254 INFO:tasks.workunit.client.1.smithi149.stdout:4/759: dread - d0/d8/d1d/fba zero size 2024-03-22T07:08:56.254 INFO:tasks.workunit.client.1.smithi149.stdout:4/760: truncate d0/d8/dc/d14/d3e/fab 4403673 0 2024-03-22T07:08:56.254 INFO:tasks.workunit.client.1.smithi149.stdout:4/761: chown d0/d8/dc/d74 485 1 2024-03-22T07:08:56.254 INFO:tasks.workunit.client.1.smithi149.stdout:4/762: fsync d0/d8/d4c/faf 0 2024-03-22T07:08:56.255 INFO:tasks.workunit.client.1.smithi149.stdout:5/704: dwrite d7/d35/de0/dd7/da4/fd2 [0,4194304] 0 2024-03-22T07:08:56.255 INFO:tasks.workunit.client.1.smithi149.stdout:8/769: rename d1/d2/d9/d11/d4d/dd6/d51/deb/lf0 to d1/d2/d9/d11/d4d/dd6/d35/d22/lfc 0 2024-03-22T07:08:56.256 INFO:tasks.workunit.client.1.smithi149.stdout:6/779: dread d9/df/d2e/f23 [0,4194304] 0 2024-03-22T07:08:56.256 INFO:tasks.workunit.client.1.smithi149.stdout:6/780: chown d9/df/d22/d6d/d7d/d96/d24/d68/f9f 897166239 1 2024-03-22T07:08:56.258 INFO:tasks.workunit.client.1.smithi149.stdout:4/763: creat d0/d8/d1d/d24/d82/ff5 x:0 0 0 2024-03-22T07:08:56.261 INFO:tasks.workunit.client.1.smithi149.stdout:4/764: write d0/d8/f21 [347543,81590] 0 2024-03-22T07:08:56.261 INFO:tasks.workunit.client.1.smithi149.stdout:5/705: rename d7/d35/de0/dd7/d7b/d2b/laa to d7/da/d20/d30/d3a/d3d/db0/lfa 0 2024-03-22T07:08:56.261 INFO:tasks.workunit.client.1.smithi149.stdout:5/706: write d7/da/d20/d36/d44/f6f [447731,13179] 0 2024-03-22T07:08:56.263 INFO:tasks.workunit.client.1.smithi149.stdout:4/765: link d0/d8/d4c/db4/ld7 d0/d8/dc/d2e/d6f/lf6 0 2024-03-22T07:08:56.263 INFO:tasks.workunit.client.1.smithi149.stdout:4/766: write d0/d8/d5c/f5e [1270437,42910] 0 2024-03-22T07:08:56.265 INFO:tasks.workunit.client.1.smithi149.stdout:7/742: dwrite d1/d18/d1b/d1d/fd [0,4194304] 0 2024-03-22T07:08:56.267 INFO:tasks.workunit.client.1.smithi149.stdout:5/707: rename d7/da/d20/d30/d3a/d93/d94/ce3 to d7/da/d20/d30/d3a/d93/cfb 0 2024-03-22T07:08:56.267 INFO:tasks.workunit.client.1.smithi149.stdout:5/708: chown d7/d35/de0/d92/df4/df5 2348 1 2024-03-22T07:08:56.267 INFO:tasks.workunit.client.1.smithi149.stdout:5/709: read - d7/da/d20/d30/d3a/fc0 zero size 2024-03-22T07:08:56.267 INFO:tasks.workunit.client.1.smithi149.stdout:4/767: creat d0/d8/dc/d14/d85/ff7 x:0 0 0 2024-03-22T07:08:56.268 INFO:tasks.workunit.client.1.smithi149.stdout:4/768: creat d0/d8/dc/d74/ddc/ff8 x:0 0 0 2024-03-22T07:08:56.269 INFO:tasks.workunit.client.1.smithi149.stdout:7/743: creat d1/d18/d1b/ff1 x:0 0 0 2024-03-22T07:08:56.270 INFO:tasks.workunit.client.1.smithi149.stdout:7/744: truncate d1/d18/d1b/d1d/fe6 987821 0 2024-03-22T07:08:56.270 INFO:tasks.workunit.client.1.smithi149.stdout:8/770: write d1/d2/d3/f96 [1504671,79276] 0 2024-03-22T07:08:56.270 INFO:tasks.workunit.client.1.smithi149.stdout:8/771: write d1/d2/d9/d11/d4d/dd6/d35/f6f [422255,54198] 0 2024-03-22T07:08:56.270 INFO:tasks.workunit.client.1.smithi149.stdout:5/710: creat d7/da/d20/d45/ffc x:0 0 0 2024-03-22T07:08:56.271 INFO:tasks.workunit.client.1.smithi149.stdout:6/781: dread d9/f19 [0,4194304] 0 2024-03-22T07:08:56.272 INFO:tasks.workunit.client.1.smithi149.stdout:4/769: symlink d0/d8/dc/d74/lf9 0 2024-03-22T07:08:56.272 INFO:tasks.workunit.client.1.smithi149.stdout:4/770: chown d0/d8/dc/d2e/c53 0 1 2024-03-22T07:08:56.275 INFO:tasks.workunit.client.1.smithi149.stdout:8/772: rmdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f 39 2024-03-22T07:08:56.275 INFO:tasks.workunit.client.1.smithi149.stdout:8/773: write d1/d2/d3/f76 [298076,101782] 0 2024-03-22T07:08:56.275 INFO:tasks.workunit.client.1.smithi149.stdout:8/774: write d1/d2/d9/ff1 [335989,85781] 0 2024-03-22T07:08:56.275 INFO:tasks.workunit.client.1.smithi149.stdout:8/775: readlink d1/d2/d9/d11/d4d/dd6/l8a 0 2024-03-22T07:08:56.275 INFO:tasks.workunit.client.1.smithi149.stdout:8/776: stat d1/d2/d9/d11/f77 0 2024-03-22T07:08:56.275 INFO:tasks.workunit.client.1.smithi149.stdout:7/745: truncate d1/f5 570872 0 2024-03-22T07:08:56.275 INFO:tasks.workunit.client.1.smithi149.stdout:8/777: write d1/d2/fed [747633,25876] 0 2024-03-22T07:08:56.276 INFO:tasks.workunit.client.1.smithi149.stdout:5/711: creat d7/da/d20/d30/d3a/d3d/d2c/db2/ffd x:0 0 0 2024-03-22T07:08:56.276 INFO:tasks.workunit.client.1.smithi149.stdout:6/782: dread d9/df/d22/d6d/d7d/d96/d24/d42/fe5 [0,4194304] 0 2024-03-22T07:08:56.276 INFO:tasks.workunit.client.1.smithi149.stdout:6/783: fdatasync d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/fe0 0 2024-03-22T07:08:56.277 INFO:tasks.workunit.client.1.smithi149.stdout:7/746: mkdir d1/d18/d1b/d99/df2 0 2024-03-22T07:08:56.277 INFO:tasks.workunit.client.1.smithi149.stdout:5/712: rmdir d7/da/d20/d89/da2/de7 0 2024-03-22T07:08:56.278 INFO:tasks.workunit.client.1.smithi149.stdout:6/784: truncate d9/df/d2e/d63/f69 804380 0 2024-03-22T07:08:56.278 INFO:tasks.workunit.client.1.smithi149.stdout:6/785: getdents d9/df/d2e/d63/dfe 0 2024-03-22T07:08:56.279 INFO:tasks.workunit.client.1.smithi149.stdout:6/786: mkdir d9/df/d22/d6d/d7d/d96/d24/d100 0 2024-03-22T07:08:56.282 INFO:tasks.workunit.client.1.smithi149.stdout:5/713: dread d7/da/d20/d45/f4f [0,4194304] 0 2024-03-22T07:08:56.282 INFO:tasks.workunit.client.1.smithi149.stdout:0/667: dwrite d1/d31/d51/d64/d87/f9d [4194304,4194304] 0 2024-03-22T07:08:56.283 INFO:tasks.workunit.client.1.smithi149.stdout:0/668: stat d1/d9/d14/lb7 0 2024-03-22T07:08:56.284 INFO:tasks.workunit.client.1.smithi149.stdout:1/729: dwrite d4/d10/dd8/d2f/d93/f9c [0,4194304] 0 2024-03-22T07:08:56.284 INFO:tasks.workunit.client.1.smithi149.stdout:5/714: symlink d7/da/d20/d45/dd9/lfe 0 2024-03-22T07:08:56.291 INFO:tasks.workunit.client.1.smithi149.stdout:1/730: mknod d4/d7/d5d/cf6 0 2024-03-22T07:08:56.292 INFO:tasks.workunit.client.1.smithi149.stdout:1/731: write d4/d10/dd8/d3c/d8e/f8f [1175539,127763] 0 2024-03-22T07:08:56.292 INFO:tasks.workunit.client.1.smithi149.stdout:1/732: write d4/d10/dd8/d3c/d8e/db5/db2/dbf/ff1 [428290,94469] 0 2024-03-22T07:08:56.292 INFO:tasks.workunit.client.1.smithi149.stdout:1/733: creat d4/d20/da0/dc0/ff7 x:0 0 0 2024-03-22T07:08:56.294 INFO:tasks.workunit.client.1.smithi149.stdout:8/778: dread d1/f24 [0,4194304] 0 2024-03-22T07:08:56.298 INFO:tasks.workunit.client.1.smithi149.stdout:8/779: dread d1/d2/d9/d11/d4d/dd6/d35/d22/dd7/fd9 [0,4194304] 0 2024-03-22T07:08:56.298 INFO:tasks.workunit.client.1.smithi149.stdout:8/780: write d1/d2/d9/d11/d4d/dd6/d35/d22/d44/f52 [120539,46187] 0 2024-03-22T07:08:56.300 INFO:tasks.workunit.client.1.smithi149.stdout:8/781: creat d1/d2/d3/ffd x:0 0 0 2024-03-22T07:08:56.300 INFO:tasks.workunit.client.1.smithi149.stdout:8/782: creat d1/d2/ffe x:0 0 0 2024-03-22T07:08:56.300 INFO:tasks.workunit.client.1.smithi149.stdout:8/783: write d1/d2/d3/fa9 [878286,58460] 0 2024-03-22T07:08:56.300 INFO:tasks.workunit.client.1.smithi149.stdout:8/784: truncate d1/d2/d9/df/dd5/ff8 619053 0 2024-03-22T07:08:56.302 INFO:tasks.workunit.client.1.smithi149.stdout:5/715: getdents d7/da/d20/d30/d3a/d3d/db0 0 2024-03-22T07:08:56.305 INFO:tasks.workunit.client.1.smithi149.stdout:8/785: mkdir d1/d2/d9/d11/d8b/dee/dff 0 2024-03-22T07:08:56.305 INFO:tasks.workunit.client.1.smithi149.stdout:1/734: dread d4/d10/f55 [0,4194304] 0 2024-03-22T07:08:56.305 INFO:tasks.workunit.client.1.smithi149.stdout:1/735: chown d4/d7/d5d/l9e 107 1 2024-03-22T07:08:56.306 INFO:tasks.workunit.client.1.smithi149.stdout:5/716: rename d7/da/d20/d30/d3a/d93/ddc to d7/da/dff 0 2024-03-22T07:08:56.308 INFO:tasks.workunit.client.1.smithi149.stdout:7/747: truncate d1/d18/d1b/d1d/fd 464433 0 2024-03-22T07:08:56.310 INFO:tasks.workunit.client.1.smithi149.stdout:1/736: getdents d4/d7 0 2024-03-22T07:08:56.310 INFO:tasks.workunit.client.1.smithi149.stdout:1/737: fdatasync d4/d10/d58/f5b 0 2024-03-22T07:08:56.312 INFO:tasks.workunit.client.1.smithi149.stdout:7/748: symlink d1/d18/d1b/d99/de5/d92/lf3 0 2024-03-22T07:08:56.312 INFO:tasks.workunit.client.1.smithi149.stdout:7/749: truncate d1/d18/d1b/d1d/d72/fd0 298555 0 2024-03-22T07:08:56.314 INFO:tasks.workunit.client.1.smithi149.stdout:5/717: dread d7/da/d20/f99 [0,4194304] 0 2024-03-22T07:08:56.314 INFO:tasks.workunit.client.1.smithi149.stdout:5/718: chown d7/d35/de0/dd7/d7b/l42 13615 1 2024-03-22T07:08:56.314 INFO:tasks.workunit.client.1.smithi149.stdout:5/719: write d7/da/d20/d30/d3a/d3d/d2c/db2/ffd [875358,93120] 0 2024-03-22T07:08:56.315 INFO:tasks.workunit.client.1.smithi149.stdout:1/738: unlink d4/d7/d5a/f6d 0 2024-03-22T07:08:56.316 INFO:tasks.workunit.client.1.smithi149.stdout:7/750: symlink d1/d1c/d49/d8e/lf4 0 2024-03-22T07:08:56.319 INFO:tasks.workunit.client.1.smithi149.stdout:5/720: symlink d7/d35/de0/d92/df4/l100 0 2024-03-22T07:08:56.320 INFO:tasks.workunit.client.1.smithi149.stdout:1/739: truncate d4/d10/dd8/d3c/d8e/db5/fe1 3888395 0 2024-03-22T07:08:56.322 INFO:tasks.workunit.client.1.smithi149.stdout:7/751: creat d1/d1c/ff5 x:0 0 0 2024-03-22T07:08:56.323 INFO:tasks.workunit.client.1.smithi149.stdout:1/740: creat d4/d10/dd8/d2f/d8c/ff8 x:0 0 0 2024-03-22T07:08:56.325 INFO:tasks.workunit.client.1.smithi149.stdout:1/741: creat d4/d10/d7c/d6a/ff9 x:0 0 0 2024-03-22T07:08:56.326 INFO:tasks.workunit.client.1.smithi149.stdout:1/742: truncate d4/d7/d5d/f72 1485289 0 2024-03-22T07:08:56.326 INFO:tasks.workunit.client.1.smithi149.stdout:1/743: write d4/d20/d2d/fa1 [59724,76749] 0 2024-03-22T07:08:56.326 INFO:tasks.workunit.client.1.smithi149.stdout:0/669: dwrite d1/d31/d52/da9/fce [0,4194304] 0 2024-03-22T07:08:56.326 INFO:tasks.workunit.client.1.smithi149.stdout:0/670: creat d1/d9/d14/d74/d77/ff2 x:0 0 0 2024-03-22T07:08:56.340 INFO:tasks.workunit.client.1.smithi149.stdout:6/787: dwrite d9/d47/d52/fe6 [0,4194304] 0 2024-03-22T07:08:56.341 INFO:tasks.workunit.client.1.smithi149.stdout:4/771: dwrite d0/d8/dc/f13 [0,4194304] 0 2024-03-22T07:08:56.353 INFO:tasks.workunit.client.1.smithi149.stdout:0/671: read d1/d31/d52/da9/fce [1165326,20187] 0 2024-03-22T07:08:56.354 INFO:tasks.workunit.client.1.smithi149.stdout:0/672: dread d1/d9/fd2 [0,4194304] 0 2024-03-22T07:08:56.360 INFO:tasks.workunit.client.1.smithi149.stdout:8/786: dwrite d1/d2/d9/d11/d8b/daf/fb9 [0,4194304] 0 2024-03-22T07:08:56.361 INFO:tasks.workunit.client.1.smithi149.stdout:8/787: symlink d1/l100 0 2024-03-22T07:08:56.362 INFO:tasks.workunit.client.1.smithi149.stdout:8/788: chown d1/d2/d18/ld1 1219 1 2024-03-22T07:08:56.362 INFO:tasks.workunit.client.1.smithi149.stdout:8/789: chown d1/d2/d9/d11/f58 10650176 1 2024-03-22T07:08:56.364 INFO:tasks.workunit.client.1.smithi149.stdout:0/673: read d1/d9/d14/d74/f81 [7689158,121769] 0 2024-03-22T07:08:56.366 INFO:tasks.workunit.client.1.smithi149.stdout:0/674: link d1/d31/d52/da9/fec d1/d31/d51/d64/d8d/d9c/d9e/ff3 0 2024-03-22T07:08:56.368 INFO:tasks.workunit.client.1.smithi149.stdout:0/675: rename d1/d31/d52/da9/fec to d1/d9/d14/d1f/ff4 0 2024-03-22T07:08:56.368 INFO:tasks.workunit.client.1.smithi149.stdout:0/676: creat d1/d20/da3/ff5 x:0 0 0 2024-03-22T07:08:56.370 INFO:tasks.workunit.client.1.smithi149.stdout:5/721: dwrite d7/da/d20/d45/fb7 [0,4194304] 0 2024-03-22T07:08:56.370 INFO:tasks.workunit.client.1.smithi149.stdout:7/752: dwrite d1/f41 [4194304,4194304] 0 2024-03-22T07:08:56.372 INFO:tasks.workunit.client.1.smithi149.stdout:6/788: dwrite d9/fc6 [0,4194304] 0 2024-03-22T07:08:56.372 INFO:tasks.workunit.client.1.smithi149.stdout:6/789: fsync d9/df/d22/dc0/fe1 0 2024-03-22T07:08:56.375 INFO:tasks.workunit.client.1.smithi149.stdout:1/744: dwrite d4/d10/f63 [0,4194304] 0 2024-03-22T07:08:56.379 INFO:tasks.workunit.client.1.smithi149.stdout:1/745: symlink d4/d10/dd8/d59/lfa 0 2024-03-22T07:08:56.379 INFO:tasks.workunit.client.1.smithi149.stdout:1/746: write d4/d7/f50 [1288944,103845] 0 2024-03-22T07:08:56.380 INFO:tasks.workunit.client.1.smithi149.stdout:9/716: sync 2024-03-22T07:08:56.382 INFO:tasks.workunit.client.1.smithi149.stdout:9/717: mknod db/d19/d39/dd3/cd9 0 2024-03-22T07:08:56.382 INFO:tasks.workunit.client.1.smithi149.stdout:9/718: write db/d12/f17 [2660889,23493] 0 2024-03-22T07:08:56.389 INFO:tasks.workunit.client.1.smithi149.stdout:5/722: dread d7/da/d20/f49 [0,4194304] 0 2024-03-22T07:08:56.389 INFO:tasks.workunit.client.1.smithi149.stdout:5/723: truncate d7/d35/de0/dd7/da4/fd2 4224079 0 2024-03-22T07:08:56.389 INFO:tasks.workunit.client.1.smithi149.stdout:5/724: stat d7/da/d20/d45/f9c 0 2024-03-22T07:08:56.389 INFO:tasks.workunit.client.1.smithi149.stdout:5/725: fsync d7/d35/de0/dd7/d7b/fe2 0 2024-03-22T07:08:56.390 INFO:tasks.workunit.client.1.smithi149.stdout:5/726: truncate d7/da/d20/d36/f41 2399352 0 2024-03-22T07:08:56.391 INFO:tasks.workunit.client.1.smithi149.stdout:5/727: stat d7/d35/f40 0 2024-03-22T07:08:56.392 INFO:tasks.workunit.client.1.smithi149.stdout:1/747: dread d4/d20/f52 [4194304,4194304] 0 2024-03-22T07:08:56.392 INFO:tasks.workunit.client.1.smithi149.stdout:5/728: truncate d7/f8 156075 0 2024-03-22T07:08:56.393 INFO:tasks.workunit.client.1.smithi149.stdout:9/719: write db/d22/f2b [1025887,93688] 0 2024-03-22T07:08:56.393 INFO:tasks.workunit.client.1.smithi149.stdout:1/748: mknod d4/d10/d7c/d6a/d6b/cfb 0 2024-03-22T07:08:56.395 INFO:tasks.workunit.client.1.smithi149.stdout:5/729: creat d7/da/d20/d30/d3a/f101 x:0 0 0 2024-03-22T07:08:56.395 INFO:tasks.workunit.client.1.smithi149.stdout:5/730: stat d7/da/d20/d89/da2/fa8 0 2024-03-22T07:08:56.395 INFO:tasks.workunit.client.1.smithi149.stdout:5/731: truncate d7/da/d20/d36/fba 1046446 0 2024-03-22T07:08:56.395 INFO:tasks.workunit.client.1.smithi149.stdout:9/720: rmdir db/d21/d9e 39 2024-03-22T07:08:56.405 INFO:tasks.workunit.client.1.smithi149.stdout:6/790: dwrite d9/df/d22/d6d/d7d/d96/d24/f99 [0,4194304] 0 2024-03-22T07:08:56.406 INFO:tasks.workunit.client.1.smithi149.stdout:6/791: fsync d9/df/d2e/f23 0 2024-03-22T07:08:56.406 INFO:tasks.workunit.client.1.smithi149.stdout:1/749: read d4/d10/dd8/d3c/d8e/db5/f83 [3598403,35970] 0 2024-03-22T07:08:56.406 INFO:tasks.workunit.client.1.smithi149.stdout:1/750: creat d4/d10/dd8/d3c/d85/d89/ffc x:0 0 0 2024-03-22T07:08:56.406 INFO:tasks.workunit.client.1.smithi149.stdout:1/751: dread - d4/d10/dd8/d3c/d8e/fac zero size 2024-03-22T07:08:56.406 INFO:tasks.workunit.client.1.smithi149.stdout:7/753: dwrite d1/d18/f8c [0,4194304] 0 2024-03-22T07:08:56.412 INFO:tasks.workunit.client.1.smithi149.stdout:1/752: creat d4/d10/dd8/d3c/d8e/db5/db2/ffd x:0 0 0 2024-03-22T07:08:56.412 INFO:tasks.workunit.client.1.smithi149.stdout:1/753: write d4/d7/fb4 [567490,127555] 0 2024-03-22T07:08:56.414 INFO:tasks.workunit.client.1.smithi149.stdout:7/754: mkdir d1/d18/d1b/d1d/df6 0 2024-03-22T07:08:56.414 INFO:tasks.workunit.client.1.smithi149.stdout:7/755: chown d1/d18/d1b/d51/d7a/d7d/d5d/f6b 5425444 1 2024-03-22T07:08:56.414 INFO:tasks.workunit.client.1.smithi149.stdout:6/792: getdents d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38 0 2024-03-22T07:08:56.418 INFO:tasks.workunit.client.1.smithi149.stdout:6/793: unlink d9/d47/d52/lf4 0 2024-03-22T07:08:56.418 INFO:tasks.workunit.client.1.smithi149.stdout:5/732: dwrite d7/d35/f40 [4194304,4194304] 0 2024-03-22T07:08:56.420 INFO:tasks.workunit.client.1.smithi149.stdout:7/756: rename d1/d1c/d49/dcd to d1/d1c/d49/d8e/d9c/df7 0 2024-03-22T07:08:56.420 INFO:tasks.workunit.client.1.smithi149.stdout:7/757: fsync d1/d18/d1b/d51/d7a/d7d/d5b/f79 0 2024-03-22T07:08:56.420 INFO:tasks.workunit.client.1.smithi149.stdout:7/758: write d1/d18/d1b/ff1 [900508,99378] 0 2024-03-22T07:08:56.422 INFO:tasks.workunit.client.1.smithi149.stdout:7/759: mkdir d1/d18/d1b/d99/de5/db4/df8 0 2024-03-22T07:08:56.422 INFO:tasks.workunit.client.1.smithi149.stdout:7/760: write d1/fdb [777453,73877] 0 2024-03-22T07:08:56.424 INFO:tasks.workunit.client.1.smithi149.stdout:7/761: rmdir d1/d18/d1b/d51/d5a/de7 39 2024-03-22T07:08:56.425 INFO:tasks.workunit.client.1.smithi149.stdout:5/733: rename d7/d35/de0/dd7/da4/daf to d7/d35/de0/dd7/dea/d102 0 2024-03-22T07:08:56.426 INFO:tasks.workunit.client.1.smithi149.stdout:5/734: read d7/d35/de0/dd7/f52 [55616,53516] 0 2024-03-22T07:08:56.426 INFO:tasks.workunit.client.1.smithi149.stdout:5/735: write d7/da/d20/d45/f9c [56495,70911] 0 2024-03-22T07:08:56.428 INFO:tasks.workunit.client.1.smithi149.stdout:7/762: creat d1/d18/d1b/d51/d7a/d7d/d5b/ff9 x:0 0 0 2024-03-22T07:08:56.428 INFO:tasks.workunit.client.1.smithi149.stdout:5/736: symlink d7/da/d20/d30/d3a/d3d/l103 0 2024-03-22T07:08:56.428 INFO:tasks.workunit.client.1.smithi149.stdout:6/794: write d9/df/d22/d6d/d7d/fb6 [2581272,77028] 0 2024-03-22T07:08:56.428 INFO:tasks.workunit.client.1.smithi149.stdout:1/754: dwrite d4/d10/f55 [0,4194304] 0 2024-03-22T07:08:56.428 INFO:tasks.workunit.client.1.smithi149.stdout:6/795: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/f101 x:0 0 0 2024-03-22T07:08:56.431 INFO:tasks.workunit.client.1.smithi149.stdout:1/755: dread d4/d20/f22 [0,4194304] 0 2024-03-22T07:08:56.432 INFO:tasks.workunit.client.1.smithi149.stdout:5/737: mkdir d7/da/d20/d30/d3a/d3d/d104 0 2024-03-22T07:08:56.433 INFO:tasks.workunit.client.1.smithi149.stdout:5/738: truncate d7/da/d20/d30/fef 370845 0 2024-03-22T07:08:56.433 INFO:tasks.workunit.client.1.smithi149.stdout:5/739: dread - d7/d35/de0/dd7/fa9 zero size 2024-03-22T07:08:56.437 INFO:tasks.workunit.client.1.smithi149.stdout:5/740: mknod d7/d35/de0/dd7/c105 0 2024-03-22T07:08:56.437 INFO:tasks.workunit.client.1.smithi149.stdout:5/741: readlink d7/da/d20/d89/da2/dda/le1 0 2024-03-22T07:08:56.437 INFO:tasks.workunit.client.1.smithi149.stdout:5/742: read d7/da/d20/d30/d3a/f79 [164231,96856] 0 2024-03-22T07:08:56.440 INFO:tasks.workunit.client.1.smithi149.stdout:1/756: read d4/d20/fe7 [1298416,16518] 0 2024-03-22T07:08:56.440 INFO:tasks.workunit.client.1.smithi149.stdout:5/743: getdents d7/da/d20/d30/d3a/d93/d94 0 2024-03-22T07:08:56.441 INFO:tasks.workunit.client.1.smithi149.stdout:1/757: symlink d4/d7/d5d/lfe 0 2024-03-22T07:08:56.444 INFO:tasks.workunit.client.1.smithi149.stdout:1/758: creat d4/d10/dd8/d2f/fff x:0 0 0 2024-03-22T07:08:56.444 INFO:tasks.workunit.client.1.smithi149.stdout:1/759: write d4/d7/f79 [1143766,64920] 0 2024-03-22T07:08:56.453 INFO:tasks.workunit.client.1.smithi149.stdout:7/763: dwrite d1/d18/d1b/d51/fc4 [0,4194304] 0 2024-03-22T07:08:56.455 INFO:tasks.workunit.client.1.smithi149.stdout:7/764: symlink d1/d18/d1b/d51/d5a/de7/d74/de0/lfa 0 2024-03-22T07:08:56.456 INFO:tasks.workunit.client.1.smithi149.stdout:7/765: link d1/d1c/d49/d8e/d53/l9d d1/d18/d1b/d99/de5/db4/lfb 0 2024-03-22T07:08:56.456 INFO:tasks.workunit.client.1.smithi149.stdout:7/766: write d1/d18/d1b/f2c [4893841,47900] 0 2024-03-22T07:08:56.466 INFO:tasks.workunit.client.1.smithi149.stdout:5/744: dwrite d7/f8 [0,4194304] 0 2024-03-22T07:08:56.467 INFO:tasks.workunit.client.1.smithi149.stdout:6/796: dwrite d9/df/d22/d6d/deb/fcc [0,4194304] 0 2024-03-22T07:08:56.467 INFO:tasks.workunit.client.1.smithi149.stdout:6/797: chown d9/df/d22/dc0/fe1 5 1 2024-03-22T07:08:56.469 INFO:tasks.workunit.client.1.smithi149.stdout:5/745: mkdir d7/da/d20/d30/d3a/d3d/d2c/db2/d106 0 2024-03-22T07:08:56.469 INFO:tasks.workunit.client.1.smithi149.stdout:5/746: write d7/da/d7d/f9f [850494,5202] 0 2024-03-22T07:08:56.471 INFO:tasks.workunit.client.1.smithi149.stdout:7/767: read d1/d18/d1b/d51/d7a/faa [1827022,108894] 0 2024-03-22T07:08:56.471 INFO:tasks.workunit.client.1.smithi149.stdout:7/768: write d1/d18/d1b/d51/d7a/faa [386774,14388] 0 2024-03-22T07:08:56.471 INFO:tasks.workunit.client.1.smithi149.stdout:7/769: write d1/d18/d1b/f42 [3414518,128538] 0 2024-03-22T07:08:56.471 INFO:tasks.workunit.client.1.smithi149.stdout:1/760: dwrite d4/d10/f12 [0,4194304] 0 2024-03-22T07:08:56.473 INFO:tasks.workunit.client.1.smithi149.stdout:5/747: symlink d7/da/d20/d89/l107 0 2024-03-22T07:08:56.479 INFO:tasks.workunit.client.1.smithi149.stdout:7/770: dread d1/d1c/f9f [0,4194304] 0 2024-03-22T07:08:56.479 INFO:tasks.workunit.client.1.smithi149.stdout:7/771: truncate d1/d18/d1b/d51/d5a/de7/f76 5000501 0 2024-03-22T07:08:56.483 INFO:tasks.workunit.client.1.smithi149.stdout:1/761: rename d4/c16 to d4/d10/dd8/d3c/d8e/db5/db2/dbf/c100 0 2024-03-22T07:08:56.484 INFO:tasks.workunit.client.1.smithi149.stdout:6/798: dwrite d9/df/d22/d6d/d7d/fa8 [0,4194304] 0 2024-03-22T07:08:56.484 INFO:tasks.workunit.client.1.smithi149.stdout:5/748: creat d7/da/d20/d30/d3a/d3d/d2c/db2/d106/f108 x:0 0 0 2024-03-22T07:08:56.485 INFO:tasks.workunit.client.1.smithi149.stdout:7/772: mkdir d1/d1c/d49/dc0/dfc 0 2024-03-22T07:08:56.486 INFO:tasks.workunit.client.1.smithi149.stdout:7/773: write d1/d18/d1b/d51/d5a/d6a/da6/d93/fde [437327,1685] 0 2024-03-22T07:08:56.489 INFO:tasks.workunit.client.1.smithi149.stdout:5/749: rename d7/da/d20/d30/fef to d7/da/d20/d30/d3a/d3d/d2c/db2/d106/f109 0 2024-03-22T07:08:56.489 INFO:tasks.workunit.client.1.smithi149.stdout:5/750: dread - d7/da/d20/d30/d3a/d93/fa0 zero size 2024-03-22T07:08:56.490 INFO:tasks.workunit.client.1.smithi149.stdout:7/774: creat d1/d18/d1b/d99/de5/ffd x:0 0 0 2024-03-22T07:08:56.491 INFO:tasks.workunit.client.1.smithi149.stdout:5/751: symlink d7/da/d20/d36/d44/d4a/l10a 0 2024-03-22T07:08:56.491 INFO:tasks.workunit.client.1.smithi149.stdout:7/775: write d1/d18/d1b/f2c [4522376,102818] 0 2024-03-22T07:08:56.493 INFO:tasks.workunit.client.1.smithi149.stdout:1/762: write d4/d20/d2d/d64/d9f/dbe/fd5 [3331298,88895] 0 2024-03-22T07:08:56.494 INFO:tasks.workunit.client.1.smithi149.stdout:7/776: mkdir d1/d18/d1b/d99/de5/d92/dfe 0 2024-03-22T07:08:56.494 INFO:tasks.workunit.client.1.smithi149.stdout:7/777: chown d1/d1c/c91 7947 1 2024-03-22T07:08:56.495 INFO:tasks.workunit.client.1.smithi149.stdout:6/799: dread d9/df/f56 [4194304,4194304] 0 2024-03-22T07:08:56.496 INFO:tasks.workunit.client.1.smithi149.stdout:7/778: unlink d1/d18/d1b/d51/d5a/f7c 0 2024-03-22T07:08:56.497 INFO:tasks.workunit.client.1.smithi149.stdout:1/763: rmdir d4/d10/d58/d68 39 2024-03-22T07:08:56.499 INFO:tasks.workunit.client.1.smithi149.stdout:1/764: symlink d4/d20/d2d/d64/dce/l101 0 2024-03-22T07:08:56.500 INFO:tasks.workunit.client.1.smithi149.stdout:1/765: write d4/d20/d2d/f91 [533311,14883] 0 2024-03-22T07:08:56.500 INFO:tasks.workunit.client.1.smithi149.stdout:1/766: read d4/d10/dd8/d3c/d8e/db5/fe1 [2027127,80678] 0 2024-03-22T07:08:56.500 INFO:tasks.workunit.client.1.smithi149.stdout:2/797: sync 2024-03-22T07:08:56.500 INFO:tasks.workunit.client.1.smithi149.stdout:6/800: rmdir d9/df/d22/dc0 39 2024-03-22T07:08:56.500 INFO:tasks.workunit.client.1.smithi149.stdout:6/801: dread - d9/df/d22/d6d/deb/fdb zero size 2024-03-22T07:08:56.501 INFO:tasks.workunit.client.1.smithi149.stdout:1/767: rename d4/d10/dd8/d3c/d8e/db5/fe1 to d4/d7/d5d/d82/f102 0 2024-03-22T07:08:56.501 INFO:tasks.workunit.client.1.smithi149.stdout:5/752: dread d7/f58 [4194304,4194304] 0 2024-03-22T07:08:56.502 INFO:tasks.workunit.client.1.smithi149.stdout:2/798: mknod d1/c11f 0 2024-03-22T07:08:56.503 INFO:tasks.workunit.client.1.smithi149.stdout:1/768: rename d4/d20/d2d/l38 to d4/d10/dd8/d3c/d8e/db5/l103 0 2024-03-22T07:08:56.504 INFO:tasks.workunit.client.1.smithi149.stdout:5/753: mkdir d7/d35/de0/d92/df4/df5/d10b 0 2024-03-22T07:08:56.507 INFO:tasks.workunit.client.1.smithi149.stdout:2/799: mkdir d1/d3/d5/d1f/d5e/d120 0 2024-03-22T07:08:56.508 INFO:tasks.workunit.client.1.smithi149.stdout:1/769: mkdir d4/d20/d2d/d64/d104 0 2024-03-22T07:08:56.508 INFO:tasks.workunit.client.1.smithi149.stdout:7/779: dread d1/d18/d1b/d51/d5a/de7/f76 [0,4194304] 0 2024-03-22T07:08:56.508 INFO:tasks.workunit.client.1.smithi149.stdout:7/780: write d1/d18/d1b/f42 [646295,21840] 0 2024-03-22T07:08:56.509 INFO:tasks.workunit.client.1.smithi149.stdout:5/754: mknod d7/d35/de0/c10c 0 2024-03-22T07:08:56.516 INFO:tasks.workunit.client.1.smithi149.stdout:7/781: creat d1/d18/d1b/d99/de5/d92/d9e/fff x:0 0 0 2024-03-22T07:08:56.516 INFO:tasks.workunit.client.1.smithi149.stdout:7/782: read d1/d1c/f73 [4171611,81491] 0 2024-03-22T07:08:56.516 INFO:tasks.workunit.client.1.smithi149.stdout:5/755: creat d7/d35/de0/dd7/f10d x:0 0 0 2024-03-22T07:08:56.544 INFO:tasks.workunit.client.1.smithi149.stdout:6/802: dwrite d9/df/d22/f58 [0,4194304] 0 2024-03-22T07:08:56.544 INFO:tasks.workunit.client.1.smithi149.stdout:2/800: dwrite d1/d3/d5/d1f/d68/fb4 [0,4194304] 0 2024-03-22T07:08:56.546 INFO:tasks.workunit.client.1.smithi149.stdout:6/803: mknod d9/df/d22/d6d/d7d/d96/d24/d42/c102 0 2024-03-22T07:08:56.547 INFO:tasks.workunit.client.1.smithi149.stdout:2/801: truncate d1/d8/fb5 88368 0 2024-03-22T07:08:56.547 INFO:tasks.workunit.client.1.smithi149.stdout:2/802: write d1/d3/d6/d32/d6f/fcd [1665765,16674] 0 2024-03-22T07:08:56.548 INFO:tasks.workunit.client.1.smithi149.stdout:2/803: creat d1/d72/f121 x:0 0 0 2024-03-22T07:08:56.548 INFO:tasks.workunit.client.1.smithi149.stdout:7/783: dwrite d1/d18/d1b/d1d/fd [0,4194304] 0 2024-03-22T07:08:56.548 INFO:tasks.workunit.client.1.smithi149.stdout:5/756: dwrite d7/da/d20/d36/fad [0,4194304] 0 2024-03-22T07:08:56.548 INFO:tasks.workunit.client.1.smithi149.stdout:5/757: write d7/da/d20/d30/d3a/d3d/d2c/db2/ffd [1096115,42115] 0 2024-03-22T07:08:56.559 INFO:tasks.workunit.client.1.smithi149.stdout:2/804: read d1/d3/d5/d1f/f93 [1257177,73041] 0 2024-03-22T07:08:56.560 INFO:tasks.workunit.client.1.smithi149.stdout:2/805: dread - d1/ffe zero size 2024-03-22T07:08:56.560 INFO:tasks.workunit.client.1.smithi149.stdout:5/758: getdents d7/da/d7d 0 2024-03-22T07:08:56.566 INFO:tasks.workunit.client.1.smithi149.stdout:6/804: dwrite d9/df/f1e [0,4194304] 0 2024-03-22T07:08:56.571 INFO:tasks.workunit.client.1.smithi149.stdout:2/806: mkdir d1/d3/d9a/dee/d122 0 2024-03-22T07:08:56.573 INFO:tasks.workunit.client.1.smithi149.stdout:6/805: link d9/df/d2e/c13 d9/d47/d52/c103 0 2024-03-22T07:08:56.573 INFO:tasks.workunit.client.1.smithi149.stdout:6/806: fdatasync d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f43 0 2024-03-22T07:08:56.574 INFO:tasks.workunit.client.1.smithi149.stdout:7/784: rename d1/d18 to d1/d1c/d49/d8e/d100 0 2024-03-22T07:08:56.575 INFO:tasks.workunit.client.1.smithi149.stdout:2/807: creat d1/d3/de0/f123 x:0 0 0 2024-03-22T07:08:56.575 INFO:tasks.workunit.client.1.smithi149.stdout:2/808: creat d1/d72/f124 x:0 0 0 2024-03-22T07:08:56.575 INFO:tasks.workunit.client.1.smithi149.stdout:3/606: sync 2024-03-22T07:08:56.575 INFO:tasks.workunit.client.1.smithi149.stdout:4/772: sync 2024-03-22T07:08:56.575 INFO:tasks.workunit.client.1.smithi149.stdout:3/607: stat d0/d2/d30/d5d/d84/fc7 0 2024-03-22T07:08:56.575 INFO:tasks.workunit.client.1.smithi149.stdout:3/608: write d0/d4/d6c/f9b [1486470,71731] 0 2024-03-22T07:08:56.576 INFO:tasks.workunit.client.1.smithi149.stdout:7/785: rename d1/d1c/d49/dc0/fea to d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/f101 0 2024-03-22T07:08:56.576 INFO:tasks.workunit.client.1.smithi149.stdout:7/786: readlink d1/d1c/d49/d8e/d100/d1b/d1d/l14 0 2024-03-22T07:08:56.578 INFO:tasks.workunit.client.1.smithi149.stdout:3/609: rename d0/f38 to d0/fe5 0 2024-03-22T07:08:56.578 INFO:tasks.workunit.client.1.smithi149.stdout:5/759: dwrite d7/da/d20/d36/d44/f6f [0,4194304] 0 2024-03-22T07:08:56.578 INFO:tasks.workunit.client.1.smithi149.stdout:4/773: rename d0/d8/dc/d2e/d6f/f7f to d0/d8/d5c/dc1/ffa 0 2024-03-22T07:08:56.579 INFO:tasks.workunit.client.1.smithi149.stdout:7/787: symlink d1/d1c/d49/d8e/d100/d1b/d99/de5/db4/df8/l102 0 2024-03-22T07:08:56.579 INFO:tasks.workunit.client.1.smithi149.stdout:7/788: write d1/d1c/d49/d8e/d100/d1b/d1d/fd [4236393,37133] 0 2024-03-22T07:08:56.580 INFO:tasks.workunit.client.1.smithi149.stdout:7/789: readlink d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/d5d/de3/l9b 0 2024-03-22T07:08:56.580 INFO:tasks.workunit.client.1.smithi149.stdout:2/809: getdents d1/d4d/d69/d75/db6/de3 0 2024-03-22T07:08:56.580 INFO:tasks.workunit.client.1.smithi149.stdout:2/810: dread - d1/d3/d6/db2/d71/fe8 zero size 2024-03-22T07:08:56.583 INFO:tasks.workunit.client.1.smithi149.stdout:3/610: creat d0/d2/d5a/daf/fe6 x:0 0 0 2024-03-22T07:08:56.586 INFO:tasks.workunit.client.1.smithi149.stdout:5/760: rmdir d7/d35/de0/d92/df4 39 2024-03-22T07:08:56.586 INFO:tasks.workunit.client.1.smithi149.stdout:5/761: chown d7/d35/de0/d92/c96 974381632 1 2024-03-22T07:08:56.596 INFO:tasks.workunit.client.1.smithi149.stdout:6/807: dwrite d9/d47/d52/fd7 [0,4194304] 0 2024-03-22T07:08:56.596 INFO:tasks.workunit.client.1.smithi149.stdout:6/808: readlink d9/df/d22/d6d/d7d/d96/d24/l7b 0 2024-03-22T07:08:56.596 INFO:tasks.workunit.client.1.smithi149.stdout:4/774: mknod d0/d8/dc/d4b/db3/dea/cfb 0 2024-03-22T07:08:56.596 INFO:tasks.workunit.client.1.smithi149.stdout:4/775: fdatasync d0/d8/dc/d4b/f4d 0 2024-03-22T07:08:56.596 INFO:tasks.workunit.client.1.smithi149.stdout:4/776: write d0/d6b/f94 [2397299,42682] 0 2024-03-22T07:08:56.596 INFO:tasks.workunit.client.1.smithi149.stdout:2/811: getdents d1/d3/d5/d1f 0 2024-03-22T07:08:56.597 INFO:tasks.workunit.client.1.smithi149.stdout:3/611: mknod d0/d2/d30/d7e/ce7 0 2024-03-22T07:08:56.598 INFO:tasks.workunit.client.1.smithi149.stdout:2/812: creat d1/d8/d59/df6/f125 x:0 0 0 2024-03-22T07:08:56.598 INFO:tasks.workunit.client.1.smithi149.stdout:3/612: creat d0/d2/d30/d7e/fe8 x:0 0 0 2024-03-22T07:08:56.599 INFO:tasks.workunit.client.1.smithi149.stdout:2/813: symlink d1/d3/d5/d41/d9f/da7/l126 0 2024-03-22T07:08:56.602 INFO:tasks.workunit.client.1.smithi149.stdout:2/814: link d1/d3/d5/d1f/f1b d1/d8/d59/f127 0 2024-03-22T07:08:56.603 INFO:tasks.workunit.client.1.smithi149.stdout:7/790: dwrite d1/d1c/d49/d8e/d100/d1b/d1d/fdf [0,4194304] 0 2024-03-22T07:08:56.607 INFO:tasks.workunit.client.1.smithi149.stdout:5/762: dwrite d7/da/d20/d30/d3a/fc0 [0,4194304] 0 2024-03-22T07:08:56.607 INFO:tasks.workunit.client.1.smithi149.stdout:5/763: dread - d7/d35/de0/dd7/f9a zero size 2024-03-22T07:08:56.607 INFO:tasks.workunit.client.1.smithi149.stdout:7/791: link d1/d1c/d49/d8e/d100/f37 d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/f103 0 2024-03-22T07:08:56.611 INFO:tasks.workunit.client.1.smithi149.stdout:7/792: symlink d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/l104 0 2024-03-22T07:08:56.614 INFO:tasks.workunit.client.1.smithi149.stdout:7/793: rename d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/fbf to d1/d1c/d49/d8e/d100/dda/f105 0 2024-03-22T07:08:56.616 INFO:tasks.workunit.client.1.smithi149.stdout:7/794: creat d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/d5d/de3/db6/f106 x:0 0 0 2024-03-22T07:08:56.617 INFO:tasks.workunit.client.1.smithi149.stdout:7/795: mknod d1/d1c/d49/d8e/d100/d1b/d1d/df6/c107 0 2024-03-22T07:08:56.618 INFO:tasks.workunit.client.1.smithi149.stdout:7/796: creat d1/d1c/d49/d8e/d100/d1b/d51/d5a/df0/f108 x:0 0 0 2024-03-22T07:08:56.628 INFO:tasks.workunit.client.1.smithi149.stdout:5/764: dread d7/f33 [0,4194304] 0 2024-03-22T07:08:56.631 INFO:tasks.workunit.client.1.smithi149.stdout:7/797: dread d1/d1c/d49/d8e/d53/f67 [0,4194304] 0 2024-03-22T07:08:56.631 INFO:tasks.workunit.client.1.smithi149.stdout:7/798: write d1/d1c/d49/d8e/d100/d1b/d51/d7a/faa [4366126,51315] 0 2024-03-22T07:08:56.632 INFO:tasks.workunit.client.1.smithi149.stdout:7/799: rename d1/d1c/d49/d8e/d100/d1b/d51/d5a/fec to d1/d1c/d49/dc0/f109 0 2024-03-22T07:08:56.633 INFO:tasks.workunit.client.1.smithi149.stdout:7/800: write d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/f101 [4826954,15147] 0 2024-03-22T07:08:56.635 INFO:tasks.workunit.client.1.smithi149.stdout:6/809: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 [4194304,4194304] 0 2024-03-22T07:08:56.636 INFO:tasks.workunit.client.1.smithi149.stdout:2/815: dwrite d1/d3/d5/f5a [4194304,4194304] 0 2024-03-22T07:08:56.636 INFO:tasks.workunit.client.1.smithi149.stdout:2/816: read - d1/d3/ff3 zero size 2024-03-22T07:08:56.637 INFO:tasks.workunit.client.1.smithi149.stdout:6/810: truncate f5 3022604 0 2024-03-22T07:08:56.638 INFO:tasks.workunit.client.1.smithi149.stdout:5/765: write d7/da/d20/d30/d3a/d3d/f34 [3139232,28354] 0 2024-03-22T07:08:56.638 INFO:tasks.workunit.client.1.smithi149.stdout:4/777: dwrite d0/d8/dc/d14/fc7 [0,4194304] 0 2024-03-22T07:08:56.639 INFO:tasks.workunit.client.1.smithi149.stdout:2/817: mknod d1/d3/d9a/dee/d122/c128 0 2024-03-22T07:08:56.639 INFO:tasks.workunit.client.1.smithi149.stdout:2/818: chown d1/d3/d6/db2/f21 105866100 1 2024-03-22T07:08:56.639 INFO:tasks.workunit.client.1.smithi149.stdout:2/819: write d1/d3/d5/f5a [3408450,5841] 0 2024-03-22T07:08:56.641 INFO:tasks.workunit.client.1.smithi149.stdout:3/613: dread d0/d4/d6c/f9b [0,4194304] 0 2024-03-22T07:08:56.641 INFO:tasks.workunit.client.1.smithi149.stdout:3/614: chown d0/d2/d5a/daf/dce/fda 824444 1 2024-03-22T07:08:56.648 INFO:tasks.workunit.client.1.smithi149.stdout:6/811: write d9/df/d22/d6d/fc4 [1708566,91146] 0 2024-03-22T07:08:56.649 INFO:tasks.workunit.client.1.smithi149.stdout:9/721: sync 2024-03-22T07:08:56.649 INFO:tasks.workunit.client.1.smithi149.stdout:8/790: sync 2024-03-22T07:08:56.651 INFO:tasks.workunit.client.1.smithi149.stdout:0/677: sync 2024-03-22T07:08:56.652 INFO:tasks.workunit.client.1.smithi149.stdout:2/820: rename d1/d3/d6/d3f to d1/d4d/d8b/d129 0 2024-03-22T07:08:56.654 INFO:tasks.workunit.client.1.smithi149.stdout:0/678: creat d1/d27/ff6 x:0 0 0 2024-03-22T07:08:56.657 INFO:tasks.workunit.client.1.smithi149.stdout:9/722: rename db/d19/c2c to db/d42/d49/cda 0 2024-03-22T07:08:56.657 INFO:tasks.workunit.client.1.smithi149.stdout:9/723: read db/d12/f17 [1975340,100702] 0 2024-03-22T07:08:56.657 INFO:tasks.workunit.client.1.smithi149.stdout:2/821: creat d1/d3/de0/f12a x:0 0 0 2024-03-22T07:08:56.657 INFO:tasks.workunit.client.1.smithi149.stdout:0/679: truncate d1/d27/d66/df1/fb9 3213692 0 2024-03-22T07:08:56.659 INFO:tasks.workunit.client.1.smithi149.stdout:3/615: dread d0/d2/d30/f8a [0,4194304] 0 2024-03-22T07:08:56.659 INFO:tasks.workunit.client.1.smithi149.stdout:9/724: mkdir db/d42/d49/d4b/d5f/ddb 0 2024-03-22T07:08:56.660 INFO:tasks.workunit.client.1.smithi149.stdout:0/680: mknod d1/d31/d51/d64/d8d/d9c/d9e/cf7 0 2024-03-22T07:08:56.660 INFO:tasks.workunit.client.1.smithi149.stdout:0/681: creat d1/d9/d59/dc0/dc9/ff8 x:0 0 0 2024-03-22T07:08:56.662 INFO:tasks.workunit.client.1.smithi149.stdout:9/725: unlink db/d21/fcd 0 2024-03-22T07:08:56.663 INFO:tasks.workunit.client.1.smithi149.stdout:2/822: rename d1/d3/de0/f123 to d1/d3/de0/f12b 0 2024-03-22T07:08:56.663 INFO:tasks.workunit.client.1.smithi149.stdout:2/823: read - d1/d3/d6/db2/f5f zero size 2024-03-22T07:08:56.663 INFO:tasks.workunit.client.1.smithi149.stdout:2/824: chown d1/d8/fb0 95014 1 2024-03-22T07:08:56.664 INFO:tasks.workunit.client.1.smithi149.stdout:9/726: getdents db/d19/d93 0 2024-03-22T07:08:56.664 INFO:tasks.workunit.client.1.smithi149.stdout:9/727: creat db/d42/d49/fdc x:0 0 0 2024-03-22T07:08:56.664 INFO:tasks.workunit.client.1.smithi149.stdout:9/728: rename db/d19/d93 to db/d19/d93/ddd 22 2024-03-22T07:08:56.664 INFO:tasks.workunit.client.1.smithi149.stdout:0/682: dread d1/f39 [0,4194304] 0 2024-03-22T07:08:56.665 INFO:tasks.workunit.client.1.smithi149.stdout:0/683: fdatasync d1/d31/d52/da9/fce 0 2024-03-22T07:08:56.665 INFO:tasks.workunit.client.1.smithi149.stdout:0/684: fdatasync d1/d31/f3f 0 2024-03-22T07:08:56.667 INFO:tasks.workunit.client.1.smithi149.stdout:5/766: dwrite d7/da/d20/d36/fd5 [0,4194304] 0 2024-03-22T07:08:56.667 INFO:tasks.workunit.client.1.smithi149.stdout:5/767: write d7/d35/de0/dd7/f6e [4832794,3753] 0 2024-03-22T07:08:56.668 INFO:tasks.workunit.client.1.smithi149.stdout:5/768: chown d7/da/d20/d36/fd5 61672 1 2024-03-22T07:08:56.668 INFO:tasks.workunit.client.1.smithi149.stdout:5/769: read d7/d35/de0/dd7/d7b/fca [2152269,99645] 0 2024-03-22T07:08:56.668 INFO:tasks.workunit.client.1.smithi149.stdout:5/770: chown d7/da/d7d/f9f 27143 1 2024-03-22T07:08:56.668 INFO:tasks.workunit.client.1.smithi149.stdout:2/825: mknod d1/d4d/d8b/c12c 0 2024-03-22T07:08:56.668 INFO:tasks.workunit.client.1.smithi149.stdout:0/685: rename d1/fb to d1/d9/d59/ff9 0 2024-03-22T07:08:56.673 INFO:tasks.workunit.client.1.smithi149.stdout:3/616: read d0/d4/d8/fd2 [4046150,83485] 0 2024-03-22T07:08:56.673 INFO:tasks.workunit.client.1.smithi149.stdout:1/770: sync 2024-03-22T07:08:56.681 INFO:tasks.workunit.client.1.smithi149.stdout:6/812: dwrite d9/df/d22/f95 [0,4194304] 0 2024-03-22T07:08:56.683 INFO:tasks.workunit.client.1.smithi149.stdout:2/826: dread d1/d3/d9a/fad [0,4194304] 0 2024-03-22T07:08:56.683 INFO:tasks.workunit.client.1.smithi149.stdout:2/827: creat d1/d72/f12d x:0 0 0 2024-03-22T07:08:56.683 INFO:tasks.workunit.client.1.smithi149.stdout:2/828: readlink d1/d8/d34/lf7 0 2024-03-22T07:08:56.685 INFO:tasks.workunit.client.1.smithi149.stdout:3/617: write d0/d4/d8/d24/f6e [2240218,4389] 0 2024-03-22T07:08:56.685 INFO:tasks.workunit.client.1.smithi149.stdout:0/686: write d1/d9/d14/d1f/f5f [1405344,9541] 0 2024-03-22T07:08:56.686 INFO:tasks.workunit.client.1.smithi149.stdout:2/829: mkdir d1/d4d/d8b/d104/d12e 0 2024-03-22T07:08:56.690 INFO:tasks.workunit.client.1.smithi149.stdout:1/771: dread d4/d10/dd8/d3c/f47 [0,4194304] 0 2024-03-22T07:08:56.690 INFO:tasks.workunit.client.1.smithi149.stdout:3/618: creat d0/d2/d30/d5d/d84/dc3/fe9 x:0 0 0 2024-03-22T07:08:56.690 INFO:tasks.workunit.client.1.smithi149.stdout:3/619: dread - d0/d4/d8/d24/d39/f71 zero size 2024-03-22T07:08:56.693 INFO:tasks.workunit.client.1.smithi149.stdout:6/813: dread d9/df/d2e/f12 [0,4194304] 0 2024-03-22T07:08:56.694 INFO:tasks.workunit.client.1.smithi149.stdout:0/687: link d1/d31/d51/f47 d1/d9/d59/ffa 0 2024-03-22T07:08:56.695 INFO:tasks.workunit.client.1.smithi149.stdout:2/830: mknod d1/d4d/d69/c12f 0 2024-03-22T07:08:56.695 INFO:tasks.workunit.client.1.smithi149.stdout:2/831: fsync d1/d3/d9a/fef 0 2024-03-22T07:08:56.696 INFO:tasks.workunit.client.1.smithi149.stdout:8/791: dwrite d1/d2/d9/d11/fa3 [0,4194304] 0 2024-03-22T07:08:56.696 INFO:tasks.workunit.client.1.smithi149.stdout:9/729: dwrite db/d19/f3c [0,4194304] 0 2024-03-22T07:08:56.696 INFO:tasks.workunit.client.1.smithi149.stdout:9/730: write db/d19/d1c/f51 [1031367,49424] 0 2024-03-22T07:08:56.698 INFO:tasks.workunit.client.1.smithi149.stdout:0/688: rmdir d1/d20/d9f/dc6/dee 0 2024-03-22T07:08:56.701 INFO:tasks.workunit.client.1.smithi149.stdout:2/832: rmdir d1/d3/d5/d41/da5 0 2024-03-22T07:08:56.704 INFO:tasks.workunit.client.1.smithi149.stdout:1/772: dread d4/d7/d61/fd1 [0,4194304] 0 2024-03-22T07:08:56.704 INFO:tasks.workunit.client.1.smithi149.stdout:1/773: fsync d4/d20/d2d/fa1 0 2024-03-22T07:08:56.705 INFO:tasks.workunit.client.1.smithi149.stdout:9/731: mknod db/d19/cde 0 2024-03-22T07:08:56.708 INFO:tasks.workunit.client.1.smithi149.stdout:1/774: unlink d4/d10/d58/f9b 0 2024-03-22T07:08:56.708 INFO:tasks.workunit.client.1.smithi149.stdout:1/775: chown d4/f69 0 1 2024-03-22T07:08:56.708 INFO:tasks.workunit.client.1.smithi149.stdout:1/776: write d4/d20/d2d/d64/d9f/fbb [4610618,47014] 0 2024-03-22T07:08:56.708 INFO:tasks.workunit.client.1.smithi149.stdout:1/777: chown d4/d10/dd8 228 1 2024-03-22T07:08:56.708 INFO:tasks.workunit.client.1.smithi149.stdout:1/778: fdatasync d4/f3e 0 2024-03-22T07:08:56.708 INFO:tasks.workunit.client.1.smithi149.stdout:1/779: chown d4/d7/d5d/d82/f102 399044 1 2024-03-22T07:08:56.709 INFO:tasks.workunit.client.1.smithi149.stdout:9/732: creat db/d21/d74/fdf x:0 0 0 2024-03-22T07:08:56.709 INFO:tasks.workunit.client.1.smithi149.stdout:6/814: write d9/df/d22/d6d/deb/ff0 [3645100,77749] 0 2024-03-22T07:08:56.709 INFO:tasks.workunit.client.1.smithi149.stdout:6/815: write d9/f19 [1568816,61481] 0 2024-03-22T07:08:56.710 INFO:tasks.workunit.client.1.smithi149.stdout:6/816: dread - d9/da2/fff zero size 2024-03-22T07:08:56.710 INFO:tasks.workunit.client.1.smithi149.stdout:6/817: fsync d9/df/d22/d6d/d7d/fa8 0 2024-03-22T07:08:56.710 INFO:tasks.workunit.client.1.smithi149.stdout:6/818: chown d9/df/d22/f58 2 1 2024-03-22T07:08:56.710 INFO:tasks.workunit.client.1.smithi149.stdout:6/819: truncate d9/df/d22/d6d/fc4 4654630 0 2024-03-22T07:08:56.710 INFO:tasks.workunit.client.1.smithi149.stdout:6/820: chown d9/df/f91 103305 1 2024-03-22T07:08:56.710 INFO:tasks.workunit.client.1.smithi149.stdout:6/821: chown d9/df/d22/d6d/fc4 33797 1 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:1/780: creat d4/d7/d5a/dca/f105 x:0 0 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:1/781: write d4/d10/f4d [5006510,31400] 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:1/782: chown d4/d20/l25 169 1 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:8/792: write d1/d2/d9/f4a [3532780,101596] 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:9/733: creat db/d33/fe0 x:0 0 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:9/734: truncate db/d19/d28/f9a 136859 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:3/620: dwrite d0/d2/d5a/fbf [0,4194304] 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:9/735: write db/d19/d1c/f20 [3797554,30371] 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:9/736: fsync db/d19/f1f 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:9/737: stat db/d42/d49/d4b/d4d/f55 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:3/621: creat d0/d2/d5a/daf/dce/fea x:0 0 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:0/689: dread d1/d9/d59/ff9 [0,4194304] 0 2024-03-22T07:08:56.712 INFO:tasks.workunit.client.1.smithi149.stdout:0/690: chown d1/d9/f3a 118554340 1 2024-03-22T07:08:56.713 INFO:tasks.workunit.client.1.smithi149.stdout:0/691: readlink d1/d20/l7d 0 2024-03-22T07:08:56.713 INFO:tasks.workunit.client.1.smithi149.stdout:1/783: write d4/d20/f73 [380503,91503] 0 2024-03-22T07:08:56.713 INFO:tasks.workunit.client.1.smithi149.stdout:6/822: link d9/df/d22/d6d/d7d/d96/f41 d9/d47/f104 0 2024-03-22T07:08:56.714 INFO:tasks.workunit.client.1.smithi149.stdout:8/793: symlink d1/d2/d9/d11/d8b/dee/df4/l101 0 2024-03-22T07:08:56.721 INFO:tasks.workunit.client.1.smithi149.stdout:3/622: creat d0/d2/d30/d5d/d84/d8f/feb x:0 0 0 2024-03-22T07:08:56.721 INFO:tasks.workunit.client.1.smithi149.stdout:3/623: read d0/d2/d42/d66/f7a [488415,17249] 0 2024-03-22T07:08:56.722 INFO:tasks.workunit.client.1.smithi149.stdout:1/784: symlink d4/d7/d5d/l106 0 2024-03-22T07:08:56.722 INFO:tasks.workunit.client.1.smithi149.stdout:1/785: chown d4/d10/dd8 633 1 2024-03-22T07:08:56.723 INFO:tasks.workunit.client.1.smithi149.stdout:6/823: symlink d9/df/d22/d6d/d7d/d96/db1/d55/l105 0 2024-03-22T07:08:56.724 INFO:tasks.workunit.client.1.smithi149.stdout:8/794: truncate d1/d2/d3/fb0 398718 0 2024-03-22T07:08:56.726 INFO:tasks.workunit.client.1.smithi149.stdout:3/624: mknod d0/d4/d8/d24/d40/d62/cec 0 2024-03-22T07:08:56.728 INFO:tasks.workunit.client.1.smithi149.stdout:0/692: mknod d1/d1a/cfb 0 2024-03-22T07:08:56.732 INFO:tasks.workunit.client.1.smithi149.stdout:6/824: rename d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/l9b to d9/df/d22/d6d/d7d/d96/d24/d34/d37/l106 0 2024-03-22T07:08:56.733 INFO:tasks.workunit.client.1.smithi149.stdout:8/795: mkdir d1/d2/d9/d11/d8b/dee/dff/d102 0 2024-03-22T07:08:56.735 INFO:tasks.workunit.client.1.smithi149.stdout:3/625: creat d0/d2/d30/d7e/da3/fed x:0 0 0 2024-03-22T07:08:56.737 INFO:tasks.workunit.client.1.smithi149.stdout:0/693: symlink d1/d31/d51/d64/d7e/lfc 0 2024-03-22T07:08:56.738 INFO:tasks.workunit.client.1.smithi149.stdout:9/738: dwrite db/d19/d1c/f20 [0,4194304] 0 2024-03-22T07:08:56.738 INFO:tasks.workunit.client.1.smithi149.stdout:9/739: chown db/d21/d74 30396 1 2024-03-22T07:08:56.739 INFO:tasks.workunit.client.1.smithi149.stdout:6/825: unlink d9/df/d22/f74 0 2024-03-22T07:08:56.740 INFO:tasks.workunit.client.1.smithi149.stdout:8/796: rename d1/d2/d3/f76 to d1/d2/d9/d11/d8b/d8d/f103 0 2024-03-22T07:08:56.740 INFO:tasks.workunit.client.1.smithi149.stdout:8/797: fdatasync d1/d2/d9/d11/d4d/dd6/db4/ff7 0 2024-03-22T07:08:56.744 INFO:tasks.workunit.client.1.smithi149.stdout:9/740: dread db/d19/d39/d68/d6f/fa7 [0,4194304] 0 2024-03-22T07:08:56.744 INFO:tasks.workunit.client.1.smithi149.stdout:9/741: chown db/d21/d37/cce 509 1 2024-03-22T07:08:56.744 INFO:tasks.workunit.client.1.smithi149.stdout:0/694: unlink d1/d31/d51/l9b 0 2024-03-22T07:08:56.744 INFO:tasks.workunit.client.1.smithi149.stdout:0/695: chown d1/d27/le2 91653045 1 2024-03-22T07:08:56.746 INFO:tasks.workunit.client.1.smithi149.stdout:9/742: stat db/d42/d49/cda 0 2024-03-22T07:08:56.747 INFO:tasks.workunit.client.1.smithi149.stdout:6/826: getdents d9/df/d22/d6d/d7d/d96/d24/d42 0 2024-03-22T07:08:56.747 INFO:tasks.workunit.client.1.smithi149.stdout:6/827: truncate d9/df/d22/d6d/d7d/d96/d24/d68/d73/fa6 1530007 0 2024-03-22T07:08:56.747 INFO:tasks.workunit.client.1.smithi149.stdout:6/828: readlink d9/df/d22/d6d/d7d/d96/d24/d68/l77 0 2024-03-22T07:08:56.747 INFO:tasks.workunit.client.1.smithi149.stdout:6/829: chown d9/df/d22/d6d/d7d/d96/lbd 5322602 1 2024-03-22T07:08:56.749 INFO:tasks.workunit.client.1.smithi149.stdout:6/830: mkdir d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/d107 0 2024-03-22T07:08:56.754 INFO:tasks.workunit.client.1.smithi149.stdout:1/786: dwrite d4/d7/f33 [0,4194304] 0 2024-03-22T07:08:56.763 INFO:tasks.workunit.client.1.smithi149.stdout:9/743: write db/d19/f3c [3048803,32143] 0 2024-03-22T07:08:56.764 INFO:tasks.workunit.client.1.smithi149.stdout:9/744: symlink db/d42/dab/le1 0 2024-03-22T07:08:56.766 INFO:tasks.workunit.client.1.smithi149.stdout:6/831: dread d9/df/d2e/f1b [0,4194304] 0 2024-03-22T07:08:56.767 INFO:tasks.workunit.client.1.smithi149.stdout:6/832: mknod d9/df/d22/c108 0 2024-03-22T07:08:56.768 INFO:tasks.workunit.client.1.smithi149.stdout:6/833: rename d9/df/c35 to d9/df/d22/d6d/d7d/d96/d24/d68/c109 0 2024-03-22T07:08:56.769 INFO:tasks.workunit.client.1.smithi149.stdout:6/834: truncate d9/df/d2e/f20 2637263 0 2024-03-22T07:08:56.770 INFO:tasks.workunit.client.1.smithi149.stdout:6/835: rename d9/da2/fb0 to d9/df/d22/d6d/d7d/d96/d24/d34/dac/f10a 0 2024-03-22T07:08:56.771 INFO:tasks.workunit.client.1.smithi149.stdout:6/836: truncate d9/df/fab 681994 0 2024-03-22T07:08:56.772 INFO:tasks.workunit.client.1.smithi149.stdout:3/626: dwrite d0/d4/d8/d24/d40/d54/d9e/fab [0,4194304] 0 2024-03-22T07:08:56.773 INFO:tasks.workunit.client.1.smithi149.stdout:6/837: rename d9/da2/cec to d9/df/d22/d6d/d7d/d96/d24/d34/d37/c10b 0 2024-03-22T07:08:56.774 INFO:tasks.workunit.client.1.smithi149.stdout:3/627: mkdir d0/d2/d30/dee 0 2024-03-22T07:08:56.775 INFO:tasks.workunit.client.1.smithi149.stdout:0/696: dwrite d1/d9/fd2 [0,4194304] 0 2024-03-22T07:08:56.775 INFO:tasks.workunit.client.1.smithi149.stdout:6/838: symlink d9/df/l10c 0 2024-03-22T07:08:56.776 INFO:tasks.workunit.client.1.smithi149.stdout:3/628: symlink d0/d2/d30/d5d/d84/lef 0 2024-03-22T07:08:56.776 INFO:tasks.workunit.client.1.smithi149.stdout:8/798: dwrite d1/d2/d3/fd [0,4194304] 0 2024-03-22T07:08:56.778 INFO:tasks.workunit.client.1.smithi149.stdout:9/745: dread db/d21/d37/d7c/da3/fc2 [0,4194304] 0 2024-03-22T07:08:56.778 INFO:tasks.workunit.client.1.smithi149.stdout:9/746: fsync db/d21/d74/f76 0 2024-03-22T07:08:56.778 INFO:tasks.workunit.client.1.smithi149.stdout:9/747: stat db/d19 0 2024-03-22T07:08:56.778 INFO:tasks.workunit.client.1.smithi149.stdout:9/748: chown db/d33/fe0 71283 1 2024-03-22T07:08:56.778 INFO:tasks.workunit.client.1.smithi149.stdout:3/629: unlink d0/l9 0 2024-03-22T07:08:56.781 INFO:tasks.workunit.client.1.smithi149.stdout:0/697: symlink d1/d1a/lfd 0 2024-03-22T07:08:56.781 INFO:tasks.workunit.client.1.smithi149.stdout:0/698: dread - d1/d1a/f90 zero size 2024-03-22T07:08:56.782 INFO:tasks.workunit.client.1.smithi149.stdout:9/749: truncate db/d42/fad 3839408 0 2024-03-22T07:08:56.784 INFO:tasks.workunit.client.1.smithi149.stdout:3/630: symlink d0/d2/lf0 0 2024-03-22T07:08:56.784 INFO:tasks.workunit.client.1.smithi149.stdout:3/631: readlink d0/d4/d8/d24/d40/d62/lca 0 2024-03-22T07:08:56.784 INFO:tasks.workunit.client.1.smithi149.stdout:3/632: write d0/f1a [2746873,91414] 0 2024-03-22T07:08:56.787 INFO:tasks.workunit.client.1.smithi149.stdout:9/750: truncate db/d21/d37/d7c/fbc 977685 0 2024-03-22T07:08:56.789 INFO:tasks.workunit.client.1.smithi149.stdout:3/633: unlink d0/d2/f55 0 2024-03-22T07:08:56.789 INFO:tasks.workunit.client.1.smithi149.stdout:3/634: chown d0/d2/f4d 5000782 1 2024-03-22T07:08:56.790 INFO:tasks.workunit.client.1.smithi149.stdout:8/799: rmdir d1/d2/d9/d11/d4d/dd6/d51/deb 0 2024-03-22T07:08:56.790 INFO:tasks.workunit.client.1.smithi149.stdout:8/800: write d1/d2/d9/d11/d8b/daf/fb9 [4267947,68660] 0 2024-03-22T07:08:56.791 INFO:tasks.workunit.client.1.smithi149.stdout:8/801: dread - d1/d2/fd8 zero size 2024-03-22T07:08:56.792 INFO:tasks.workunit.client.1.smithi149.stdout:8/802: mknod d1/d2/d9/d11/d4d/c104 0 2024-03-22T07:08:56.792 INFO:tasks.workunit.client.1.smithi149.stdout:8/803: dread - d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/fc9 zero size 2024-03-22T07:08:56.792 INFO:tasks.workunit.client.1.smithi149.stdout:8/804: fsync d1/d2/d9/d11/d4d/dd6/d35/d22/d44/f52 0 2024-03-22T07:08:56.793 INFO:tasks.workunit.client.1.smithi149.stdout:8/805: truncate d1/d2/d9/d11/d4d/dd6/d35/d22/fcc 834993 0 2024-03-22T07:08:56.793 INFO:tasks.workunit.client.1.smithi149.stdout:9/751: dread db/d19/f3c [0,4194304] 0 2024-03-22T07:08:56.794 INFO:tasks.workunit.client.1.smithi149.stdout:8/806: mkdir d1/d2/d9/d11/d8b/d8d/d105 0 2024-03-22T07:08:56.795 INFO:tasks.workunit.client.1.smithi149.stdout:5/771: sync 2024-03-22T07:08:56.795 INFO:tasks.workunit.client.1.smithi149.stdout:7/801: sync 2024-03-22T07:08:56.795 INFO:tasks.workunit.client.1.smithi149.stdout:4/778: sync 2024-03-22T07:08:56.796 INFO:tasks.workunit.client.1.smithi149.stdout:5/772: rename d7/da/d20/d30/d3a/d93/cfb to d7/da/d20/c10e 0 2024-03-22T07:08:56.797 INFO:tasks.workunit.client.1.smithi149.stdout:5/773: chown d7/d35/de0/dd7/d7b/d2b 43327067 1 2024-03-22T07:08:56.798 INFO:tasks.workunit.client.1.smithi149.stdout:4/779: symlink d0/d8/lfc 0 2024-03-22T07:08:56.798 INFO:tasks.workunit.client.1.smithi149.stdout:4/780: write d0/d8/f56 [1509410,121521] 0 2024-03-22T07:08:56.800 INFO:tasks.workunit.client.1.smithi149.stdout:7/802: rename d1/d1c/d49/d8e/d100/d1b/d99/f68 to d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/f10a 0 2024-03-22T07:08:56.801 INFO:tasks.workunit.client.1.smithi149.stdout:7/803: mkdir d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/d10b 0 2024-03-22T07:08:56.802 INFO:tasks.workunit.client.1.smithi149.stdout:0/699: dwrite d1/d31/d51/d64/d87/da6/fd6 [0,4194304] 0 2024-03-22T07:08:56.803 INFO:tasks.workunit.client.1.smithi149.stdout:4/781: rename d0/d8/d4c/faf to d0/d8/dc/d74/d76/deb/ffd 0 2024-03-22T07:08:56.808 INFO:tasks.workunit.client.1.smithi149.stdout:7/804: creat d1/d1c/d49/d8e/d100/d1b/d51/d5a/f10c x:0 0 0 2024-03-22T07:08:56.808 INFO:tasks.workunit.client.1.smithi149.stdout:5/774: write d7/da/d20/d36/fd5 [3820979,30734] 0 2024-03-22T07:08:56.808 INFO:tasks.workunit.client.1.smithi149.stdout:5/775: fdatasync d7/da/fee 0 2024-03-22T07:08:56.812 INFO:tasks.workunit.client.1.smithi149.stdout:4/782: rmdir d0/d8/dc/d14 39 2024-03-22T07:08:56.812 INFO:tasks.workunit.client.1.smithi149.stdout:4/783: chown d0 1474684 1 2024-03-22T07:08:56.812 INFO:tasks.workunit.client.1.smithi149.stdout:4/784: stat d0/d8/lfc 0 2024-03-22T07:08:56.813 INFO:tasks.workunit.client.1.smithi149.stdout:9/752: dread db/d19/d28/fc1 [0,4194304] 0 2024-03-22T07:08:56.814 INFO:tasks.workunit.client.1.smithi149.stdout:9/753: creat db/d21/fe2 x:0 0 0 2024-03-22T07:08:56.815 INFO:tasks.workunit.client.1.smithi149.stdout:5/776: unlink d7/d35/de0/dd7/d7b/fe2 0 2024-03-22T07:08:56.817 INFO:tasks.workunit.client.1.smithi149.stdout:4/785: truncate d0/d8/dc/d14/fc7 815588 0 2024-03-22T07:08:56.817 INFO:tasks.workunit.client.1.smithi149.stdout:4/786: dread - d0/d8/d4c/fe4 zero size 2024-03-22T07:08:56.819 INFO:tasks.workunit.client.1.smithi149.stdout:3/635: write d0/d2/d30/f8a [188478,59680] 0 2024-03-22T07:08:56.819 INFO:tasks.workunit.client.1.smithi149.stdout:3/636: write d0/d2/d30/dd1/fbb [123258,88471] 0 2024-03-22T07:08:56.819 INFO:tasks.workunit.client.1.smithi149.stdout:8/807: dwrite d1/d2/fd8 [0,4194304] 0 2024-03-22T07:08:56.820 INFO:tasks.workunit.client.1.smithi149.stdout:7/805: dread d1/d1c/d49/d8e/d100/d1b/d51/fc4 [0,4194304] 0 2024-03-22T07:08:56.820 INFO:tasks.workunit.client.1.smithi149.stdout:9/754: write db/d33/f3d [116944,71149] 0 2024-03-22T07:08:56.821 INFO:tasks.workunit.client.1.smithi149.stdout:5/777: rename d7/c78 to d7/da/d20/d30/d3a/d3d/db0/c10f 0 2024-03-22T07:08:56.823 INFO:tasks.workunit.client.1.smithi149.stdout:4/787: mknod d0/d8/d5c/dc1/cfe 0 2024-03-22T07:08:56.827 INFO:tasks.workunit.client.1.smithi149.stdout:7/806: getdents d1/d1c/d49/d8e/d100/d1b/d99 0 2024-03-22T07:08:56.829 INFO:tasks.workunit.client.1.smithi149.stdout:0/700: dwrite d1/d1a/f90 [0,4194304] 0 2024-03-22T07:08:56.831 INFO:tasks.workunit.client.1.smithi149.stdout:9/755: symlink db/d19/d39/d68/d6f/le3 0 2024-03-22T07:08:56.832 INFO:tasks.workunit.client.1.smithi149.stdout:8/808: write d1/d2/d9/d11/d4d/dd6/f66 [3507022,104526] 0 2024-03-22T07:08:56.832 INFO:tasks.workunit.client.1.smithi149.stdout:8/809: read d1/d2/d9/d11/d4d/dd6/d35/f6f [37646,79353] 0 2024-03-22T07:08:56.832 INFO:tasks.workunit.client.1.smithi149.stdout:8/810: read - d1/d2/d9/d11/d4d/dd6/d35/d9d/fb7 zero size 2024-03-22T07:08:56.832 INFO:tasks.workunit.client.1.smithi149.stdout:8/811: write d1/d2/d9/d11/d4d/f5e [2322642,57171] 0 2024-03-22T07:08:56.832 INFO:tasks.workunit.client.1.smithi149.stdout:7/807: symlink d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/d5d/de3/l10d 0 2024-03-22T07:08:56.836 INFO:tasks.workunit.client.1.smithi149.stdout:4/788: dread d0/d8/dc/d14/d3e/f6a [0,4194304] 0 2024-03-22T07:08:56.836 INFO:tasks.workunit.client.1.smithi149.stdout:9/756: stat db/d42/d49/d4b/c88 0 2024-03-22T07:08:56.836 INFO:tasks.workunit.client.1.smithi149.stdout:9/757: chown db/d19/fd7 35 1 2024-03-22T07:08:56.836 INFO:tasks.workunit.client.1.smithi149.stdout:7/808: rmdir d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d 39 2024-03-22T07:08:56.838 INFO:tasks.workunit.client.1.smithi149.stdout:4/789: rename d0/d8/dc/d4b/f71 to d0/d29/fff 0 2024-03-22T07:08:56.839 INFO:tasks.workunit.client.1.smithi149.stdout:9/758: creat db/d19/d93/fe4 x:0 0 0 2024-03-22T07:08:56.839 INFO:tasks.workunit.client.1.smithi149.stdout:9/759: fdatasync db/d22/f45 0 2024-03-22T07:08:56.840 INFO:tasks.workunit.client.1.smithi149.stdout:7/809: symlink d1/d1c/d49/d8e/d100/d1b/d51/d5a/d6a/da6/l10e 0 2024-03-22T07:08:56.841 INFO:tasks.workunit.client.1.smithi149.stdout:9/760: symlink db/d21/d37/d7c/da3/le5 0 2024-03-22T07:08:56.841 INFO:tasks.workunit.client.1.smithi149.stdout:9/761: creat db/d19/d1c/fe6 x:0 0 0 2024-03-22T07:08:56.841 INFO:tasks.workunit.client.1.smithi149.stdout:9/762: fsync db/d42/d49/fdc 0 2024-03-22T07:08:56.842 INFO:tasks.workunit.client.1.smithi149.stdout:0/701: read d1/d9/d14/d74/dc1/fd0 [1676351,80356] 0 2024-03-22T07:08:56.844 INFO:tasks.workunit.client.1.smithi149.stdout:9/763: rmdir db/d21/d8c/dc9 0 2024-03-22T07:08:56.846 INFO:tasks.workunit.client.1.smithi149.stdout:9/764: link db/d19/d39/d68/cba db/d19/d39/dd3/ce7 0 2024-03-22T07:08:56.846 INFO:tasks.workunit.client.1.smithi149.stdout:9/765: write db/d21/d9e/fd0 [420402,37974] 0 2024-03-22T07:08:56.846 INFO:tasks.workunit.client.1.smithi149.stdout:5/778: dwrite d7/da/d20/d45/fb7 [0,4194304] 0 2024-03-22T07:08:56.847 INFO:tasks.workunit.client.1.smithi149.stdout:9/766: unlink db/d21/d37/d3a/l77 0 2024-03-22T07:08:56.847 INFO:tasks.workunit.client.1.smithi149.stdout:9/767: readlink l6 0 2024-03-22T07:08:56.850 INFO:tasks.workunit.client.1.smithi149.stdout:5/779: truncate d7/da/d20/d30/d3a/d3d/f34 101036 0 2024-03-22T07:08:56.851 INFO:tasks.workunit.client.1.smithi149.stdout:9/768: dread db/d42/f5a [0,4194304] 0 2024-03-22T07:08:56.851 INFO:tasks.workunit.client.1.smithi149.stdout:5/780: mkdir d7/da/d7d/d110 0 2024-03-22T07:08:56.852 INFO:tasks.workunit.client.1.smithi149.stdout:5/781: truncate d7/d35/de0/dd7/f9a 246824 0 2024-03-22T07:08:56.853 INFO:tasks.workunit.client.1.smithi149.stdout:7/810: dread d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/f10a [0,4194304] 0 2024-03-22T07:08:56.853 INFO:tasks.workunit.client.1.smithi149.stdout:9/769: mknod db/d12/ce8 0 2024-03-22T07:08:56.853 INFO:tasks.workunit.client.1.smithi149.stdout:9/770: write db/d19/d1c/f20 [4806684,53866] 0 2024-03-22T07:08:56.853 INFO:tasks.workunit.client.1.smithi149.stdout:9/771: readlink db/d21/d2e/l3f 0 2024-03-22T07:08:56.854 INFO:tasks.workunit.client.1.smithi149.stdout:5/782: write d7/da/d20/f5e [210595,43672] 0 2024-03-22T07:08:56.855 INFO:tasks.workunit.client.1.smithi149.stdout:4/790: dread d0/d8/d4c/f57 [0,4194304] 0 2024-03-22T07:08:56.855 INFO:tasks.workunit.client.1.smithi149.stdout:4/791: write d0/d29/f58 [1464646,49578] 0 2024-03-22T07:08:56.861 INFO:tasks.workunit.client.1.smithi149.stdout:9/772: mkdir db/d19/d28/de9 0 2024-03-22T07:08:56.861 INFO:tasks.workunit.client.1.smithi149.stdout:2/833: sync 2024-03-22T07:08:56.861 INFO:tasks.workunit.client.1.smithi149.stdout:5/783: truncate d7/d35/f40 5067765 0 2024-03-22T07:08:56.861 INFO:tasks.workunit.client.1.smithi149.stdout:5/784: readlink d7/da/d20/d30/d3a/d3d/d2c/l3e 0 2024-03-22T07:08:56.861 INFO:tasks.workunit.client.1.smithi149.stdout:5/785: write d7/d35/f55 [809565,62637] 0 2024-03-22T07:08:56.861 INFO:tasks.workunit.client.1.smithi149.stdout:7/811: rename d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/d5d to d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f 0 2024-03-22T07:08:56.861 INFO:tasks.workunit.client.1.smithi149.stdout:7/812: write d1/d1c/d49/d8e/d100/d1b/d1d/f8 [868907,125531] 0 2024-03-22T07:08:56.861 INFO:tasks.workunit.client.1.smithi149.stdout:7/813: stat d1/d1c/d49/d8e/d100/d1b/d1d/d72/c9a 0 2024-03-22T07:08:56.864 INFO:tasks.workunit.client.1.smithi149.stdout:5/786: truncate d7/da/d7d/f9e 189777 0 2024-03-22T07:08:56.869 INFO:tasks.workunit.client.1.smithi149.stdout:0/702: dwrite d1/d9/d59/ffa [0,4194304] 0 2024-03-22T07:08:56.870 INFO:tasks.workunit.client.1.smithi149.stdout:5/787: symlink d7/da/d20/d36/d44/d4a/l111 0 2024-03-22T07:08:56.870 INFO:tasks.workunit.client.1.smithi149.stdout:5/788: read - d7/d35/d8e/ff1 zero size 2024-03-22T07:08:56.874 INFO:tasks.workunit.client.1.smithi149.stdout:0/703: mkdir d1/d31/d51/d64/d87/da6/dfe 0 2024-03-22T07:08:56.874 INFO:tasks.workunit.client.1.smithi149.stdout:0/704: stat d1/d20/c97 0 2024-03-22T07:08:56.876 INFO:tasks.workunit.client.1.smithi149.stdout:5/789: link d7/d35/de0/dd7/d7b/d2b/c61 d7/da/d20/d89/da2/c112 0 2024-03-22T07:08:56.876 INFO:tasks.workunit.client.1.smithi149.stdout:5/790: chown d7/da/d20/d36/d44/l5a 8852506 1 2024-03-22T07:08:56.879 INFO:tasks.workunit.client.1.smithi149.stdout:5/791: symlink d7/da/d20/d30/l113 0 2024-03-22T07:08:56.879 INFO:tasks.workunit.client.1.smithi149.stdout:0/705: dread d1/d6b/f92 [0,4194304] 0 2024-03-22T07:08:56.880 INFO:tasks.workunit.client.1.smithi149.stdout:0/706: read d1/d31/d51/d64/d8d/d9c/fb5 [143437,111321] 0 2024-03-22T07:08:56.883 INFO:tasks.workunit.client.1.smithi149.stdout:0/707: mknod d1/d20/d9f/dc6/dd9/cff 0 2024-03-22T07:08:56.884 INFO:tasks.workunit.client.1.smithi149.stdout:0/708: creat d1/d31/d52/dd4/f100 x:0 0 0 2024-03-22T07:08:56.884 INFO:tasks.workunit.client.1.smithi149.stdout:0/709: truncate d1/d9/d14/d74/f81 9427745 0 2024-03-22T07:08:56.884 INFO:tasks.workunit.client.1.smithi149.stdout:0/710: creat d1/d31/d51/d64/d8d/d9c/f101 x:0 0 0 2024-03-22T07:08:56.884 INFO:tasks.workunit.client.1.smithi149.stdout:7/814: dread d1/d1c/d49/d8e/d100/d1b/d1d/f1f [0,4194304] 0 2024-03-22T07:08:56.885 INFO:tasks.workunit.client.1.smithi149.stdout:7/815: creat d1/d1c/d49/d8e/d100/d1b/d51/d5a/df0/f110 x:0 0 0 2024-03-22T07:08:56.887 INFO:tasks.workunit.client.1.smithi149.stdout:0/711: rename d1/d31/d51/d64/fa2 to d1/d27/f102 0 2024-03-22T07:08:56.887 INFO:tasks.workunit.client.1.smithi149.stdout:0/712: write d1/d9/d14/f76 [179167,17797] 0 2024-03-22T07:08:56.887 INFO:tasks.workunit.client.1.smithi149.stdout:0/713: write d1/d9/d59/f5d [414887,85283] 0 2024-03-22T07:08:56.888 INFO:tasks.workunit.client.1.smithi149.stdout:7/816: rename d1/f5 to d1/d1c/d49/d8e/d100/d1b/d51/d7a/f111 0 2024-03-22T07:08:56.890 INFO:tasks.workunit.client.1.smithi149.stdout:0/714: rename d1/d1a/l4c to d1/d9/d59/l103 0 2024-03-22T07:08:56.891 INFO:tasks.workunit.client.1.smithi149.stdout:0/715: dread - d1/d9/d14/d74/fe4 zero size 2024-03-22T07:08:56.891 INFO:tasks.workunit.client.1.smithi149.stdout:5/792: dread d7/da/d20/f49 [0,4194304] 0 2024-03-22T07:08:56.891 INFO:tasks.workunit.client.1.smithi149.stdout:5/793: readlink d7/l16 0 2024-03-22T07:08:56.891 INFO:tasks.workunit.client.1.smithi149.stdout:4/792: dwrite d0/d6b/f7d [0,4194304] 0 2024-03-22T07:08:56.891 INFO:tasks.workunit.client.1.smithi149.stdout:4/793: truncate d0/d6b/fb9 760672 0 2024-03-22T07:08:56.891 INFO:tasks.workunit.client.1.smithi149.stdout:4/794: truncate d0/d8/dc/d14/f43 633818 0 2024-03-22T07:08:56.892 INFO:tasks.workunit.client.1.smithi149.stdout:5/794: symlink d7/da/d20/d36/d44/l114 0 2024-03-22T07:08:56.895 INFO:tasks.workunit.client.1.smithi149.stdout:5/795: mknod d7/da/d20/d30/d3a/d3d/d104/c115 0 2024-03-22T07:08:56.896 INFO:tasks.workunit.client.1.smithi149.stdout:5/796: creat d7/da/d7d/f116 x:0 0 0 2024-03-22T07:08:56.896 INFO:tasks.workunit.client.1.smithi149.stdout:5/797: stat d7/da/d20/d89/da2/fec 0 2024-03-22T07:08:56.898 INFO:tasks.workunit.client.1.smithi149.stdout:5/798: mknod d7/da/d20/d30/d3a/d3d/d2c/db2/c117 0 2024-03-22T07:08:56.898 INFO:tasks.workunit.client.1.smithi149.stdout:9/773: dwrite db/d21/d37/d7c/f9b [0,4194304] 0 2024-03-22T07:08:56.902 INFO:tasks.workunit.client.1.smithi149.stdout:2/834: dwrite d1/d4d/d69/d75/d7e/fd6 [0,4194304] 0 2024-03-22T07:08:56.902 INFO:tasks.workunit.client.1.smithi149.stdout:2/835: read d1/f4 [5954943,114123] 0 2024-03-22T07:08:56.902 INFO:tasks.workunit.client.1.smithi149.stdout:2/836: dread d1/d8/fb5 [0,4194304] 0 2024-03-22T07:08:56.902 INFO:tasks.workunit.client.1.smithi149.stdout:2/837: chown d1/d4d/d69/d75/db6/de3 97745750 1 2024-03-22T07:08:56.903 INFO:tasks.workunit.client.1.smithi149.stdout:2/838: write d1/d3/d6/db2/f21 [3751223,38303] 0 2024-03-22T07:08:56.904 INFO:tasks.workunit.client.1.smithi149.stdout:7/817: dread d1/d1c/d49/d8e/d100/f8c [0,4194304] 0 2024-03-22T07:08:56.907 INFO:tasks.workunit.client.1.smithi149.stdout:2/839: creat d1/d3/de0/f130 x:0 0 0 2024-03-22T07:08:56.910 INFO:tasks.workunit.client.1.smithi149.stdout:2/840: mknod d1/d3/d5/d41/c131 0 2024-03-22T07:08:56.910 INFO:tasks.workunit.client.1.smithi149.stdout:7/818: read d1/d1c/d49/d8e/d100/d1b/d1d/f1f [76609,78204] 0 2024-03-22T07:08:56.910 INFO:tasks.workunit.client.1.smithi149.stdout:7/819: write d1/d1c/d49/d8e/d100/d1b/d51/d5a/fad [531169,65792] 0 2024-03-22T07:08:56.910 INFO:tasks.workunit.client.1.smithi149.stdout:0/716: dread d1/d9/d14/fa1 [0,4194304] 0 2024-03-22T07:08:56.911 INFO:tasks.workunit.client.1.smithi149.stdout:2/841: mknod d1/d4d/d69/d75/db8/c132 0 2024-03-22T07:08:56.911 INFO:tasks.workunit.client.1.smithi149.stdout:2/842: creat d1/d3/f133 x:0 0 0 2024-03-22T07:08:56.911 INFO:tasks.workunit.client.1.smithi149.stdout:4/795: dwrite d0/d8/dc/d2e/d6f/ddf/fe2 [0,4194304] 0 2024-03-22T07:08:56.912 INFO:tasks.workunit.client.1.smithi149.stdout:7/820: truncate d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/f3b 8157271 0 2024-03-22T07:08:56.916 INFO:tasks.workunit.client.1.smithi149.stdout:7/821: mknod d1/d1c/d49/d8e/d100/d1b/d1d/c112 0 2024-03-22T07:08:56.916 INFO:tasks.workunit.client.1.smithi149.stdout:7/822: write d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/d5b/f79 [225505,2619] 0 2024-03-22T07:08:56.918 INFO:tasks.workunit.client.1.smithi149.stdout:7/823: symlink d1/d1c/d49/d8e/d100/d1b/d1d/df6/l113 0 2024-03-22T07:08:56.918 INFO:tasks.workunit.client.1.smithi149.stdout:7/824: fdatasync d1/f21 0 2024-03-22T07:08:56.920 INFO:tasks.workunit.client.1.smithi149.stdout:7/825: unlink d1/d1c/d49/d8e/d53/l9d 0 2024-03-22T07:08:56.923 INFO:tasks.workunit.client.1.smithi149.stdout:2/843: dread d1/d3/d5/d1f/f39 [0,4194304] 0 2024-03-22T07:08:56.924 INFO:tasks.workunit.client.1.smithi149.stdout:2/844: mkdir d1/d4d/d8b/d104/d12e/d134 0 2024-03-22T07:08:56.924 INFO:tasks.workunit.client.1.smithi149.stdout:2/845: fdatasync d1/d4d/d69/d75/fa1 0 2024-03-22T07:08:56.925 INFO:tasks.workunit.client.1.smithi149.stdout:2/846: write d1/d4d/d6a/de1/faf [4783332,64883] 0 2024-03-22T07:08:56.927 INFO:tasks.workunit.client.1.smithi149.stdout:1/787: sync 2024-03-22T07:08:56.927 INFO:tasks.workunit.client.1.smithi149.stdout:1/788: truncate d4/d10/dd8/d3c/d8e/db5/db2/ffd 389416 0 2024-03-22T07:08:56.929 INFO:tasks.workunit.client.1.smithi149.stdout:1/789: symlink d4/d20/d2d/d64/d104/l107 0 2024-03-22T07:08:56.929 INFO:tasks.workunit.client.1.smithi149.stdout:4/796: dwrite d0/d8/dc/d2e/fbf [0,4194304] 0 2024-03-22T07:08:56.929 INFO:tasks.workunit.client.1.smithi149.stdout:4/797: truncate d0/d8/dc/d14/d85/ff7 733106 0 2024-03-22T07:08:56.933 INFO:tasks.workunit.client.1.smithi149.stdout:4/798: link d0/d8/dc/d74/fe1 d0/d8/dc/d4b/dbc/f100 0 2024-03-22T07:08:56.934 INFO:tasks.workunit.client.1.smithi149.stdout:4/799: mknod d0/d8/dc/d74/ddc/c101 0 2024-03-22T07:08:56.935 INFO:tasks.workunit.client.1.smithi149.stdout:2/847: write d1/d8/d34/f57 [2894627,97391] 0 2024-03-22T07:08:56.935 INFO:tasks.workunit.client.1.smithi149.stdout:2/848: fdatasync d1/d3/d5/d1f/d68/d94/fd4 0 2024-03-22T07:08:56.935 INFO:tasks.workunit.client.1.smithi149.stdout:2/849: read - d1/d3/d5/d1f/d68/d94/fc8 zero size 2024-03-22T07:08:56.936 INFO:tasks.workunit.client.1.smithi149.stdout:4/800: creat d0/d8/dc/d2e/d6f/f102 x:0 0 0 2024-03-22T07:08:56.936 INFO:tasks.workunit.client.1.smithi149.stdout:4/801: creat d0/d8/d1d/f103 x:0 0 0 2024-03-22T07:08:56.942 INFO:tasks.workunit.client.1.smithi149.stdout:2/850: unlink d1/d3/d5/d1f/dfc/f111 0 2024-03-22T07:08:56.942 INFO:tasks.workunit.client.1.smithi149.stdout:2/851: write d1/d3/d5/d41/d9f/da7/f119 [176263,82691] 0 2024-03-22T07:08:56.942 INFO:tasks.workunit.client.1.smithi149.stdout:2/852: creat d1/d3/f135 x:0 0 0 2024-03-22T07:08:56.945 INFO:tasks.workunit.client.1.smithi149.stdout:4/802: creat d0/d8/dc/d2e/d6f/d96/f104 x:0 0 0 2024-03-22T07:08:56.945 INFO:tasks.workunit.client.1.smithi149.stdout:4/803: truncate d0/d8/d1d/d24/f54 4251492 0 2024-03-22T07:08:56.945 INFO:tasks.workunit.client.1.smithi149.stdout:4/804: write d0/d8/d4c/f6d [1777750,15544] 0 2024-03-22T07:08:56.947 INFO:tasks.workunit.client.1.smithi149.stdout:2/853: rmdir d1/d3/d6/db2/d71 39 2024-03-22T07:08:56.947 INFO:tasks.workunit.client.1.smithi149.stdout:2/854: chown d1/d3/d7d 109 1 2024-03-22T07:08:56.947 INFO:tasks.workunit.client.1.smithi149.stdout:2/855: read d1/d3/d5/d41/d9f/da7/f119 [221847,127132] 0 2024-03-22T07:08:56.951 INFO:tasks.workunit.client.1.smithi149.stdout:4/805: mknod d0/d8/dc/d4b/dbc/c105 0 2024-03-22T07:08:56.959 INFO:tasks.workunit.client.1.smithi149.stdout:4/806: dread d0/d8/dc/d14/d3e/f51 [0,4194304] 0 2024-03-22T07:08:56.960 INFO:tasks.workunit.client.1.smithi149.stdout:7/826: dwrite d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/d5b/f79 [0,4194304] 0 2024-03-22T07:08:56.960 INFO:tasks.workunit.client.1.smithi149.stdout:7/827: dread - d1/d1c/d49/d8e/d100/d1b/d99/fed zero size 2024-03-22T07:08:56.960 INFO:tasks.workunit.client.1.smithi149.stdout:1/790: dwrite d4/d10/dd8/f6f [0,4194304] 0 2024-03-22T07:08:56.964 INFO:tasks.workunit.client.1.smithi149.stdout:2/856: write d1/d4d/d6a/de1/fa8 [565816,103166] 0 2024-03-22T07:08:56.965 INFO:tasks.workunit.client.1.smithi149.stdout:2/857: read d1/d3/d5/d1f/d68/d94/fd4 [3575880,55373] 0 2024-03-22T07:08:56.965 INFO:tasks.workunit.client.1.smithi149.stdout:2/858: write d1/d3/d6/db2/f65 [1141383,13366] 0 2024-03-22T07:08:56.965 INFO:tasks.workunit.client.1.smithi149.stdout:2/859: chown d1/d3/d5/d1f/d5e/l31 7 1 2024-03-22T07:08:56.965 INFO:tasks.workunit.client.1.smithi149.stdout:2/860: truncate d1/d3/fcc 332736 0 2024-03-22T07:08:56.968 INFO:tasks.workunit.client.1.smithi149.stdout:1/791: mknod d4/d10/dd8/d3c/d8e/db5/db2/dbf/de5/c108 0 2024-03-22T07:08:56.968 INFO:tasks.workunit.client.1.smithi149.stdout:1/792: stat d4/d10/dd8/d3c/d8e/db5/db2/dbf 0 2024-03-22T07:08:56.969 INFO:tasks.workunit.client.1.smithi149.stdout:2/861: rmdir d1/d4d/d8b/d129 39 2024-03-22T07:08:56.969 INFO:tasks.workunit.client.1.smithi149.stdout:2/862: read d1/d3/f4b [488717,98849] 0 2024-03-22T07:08:56.969 INFO:tasks.workunit.client.1.smithi149.stdout:2/863: chown d1/d3/l1d 171093 1 2024-03-22T07:08:56.970 INFO:tasks.workunit.client.1.smithi149.stdout:2/864: creat d1/d4d/d69/d75/f136 x:0 0 0 2024-03-22T07:08:56.971 INFO:tasks.workunit.client.1.smithi149.stdout:2/865: chown d1/d3/de0/l102 1487878923 1 2024-03-22T07:08:56.973 INFO:tasks.workunit.client.1.smithi149.stdout:3/637: sync 2024-03-22T07:08:56.974 INFO:tasks.workunit.client.1.smithi149.stdout:4/807: dread d0/d8/f46 [0,4194304] 0 2024-03-22T07:08:56.974 INFO:tasks.workunit.client.1.smithi149.stdout:3/638: symlink d0/d4/d8/lf1 0 2024-03-22T07:08:56.975 INFO:tasks.workunit.client.1.smithi149.stdout:4/808: mkdir d0/d8/dc/d14/d85/d97/d106 0 2024-03-22T07:08:56.976 INFO:tasks.workunit.client.1.smithi149.stdout:3/639: mkdir d0/d4/d8/d24/d40/d54/d85/dbc/df2 0 2024-03-22T07:08:56.978 INFO:tasks.workunit.client.1.smithi149.stdout:6/839: sync 2024-03-22T07:08:56.980 INFO:tasks.workunit.client.1.smithi149.stdout:6/840: unlink d9/d47/d52/f60 0 2024-03-22T07:08:56.984 INFO:tasks.workunit.client.1.smithi149.stdout:7/828: dwrite d1/d1c/d49/d8e/d100/d1b/d51/d7a/f7f [0,4194304] 0 2024-03-22T07:08:56.984 INFO:tasks.workunit.client.1.smithi149.stdout:7/829: chown d1/d1c/d49/d8e/d100/d1b/d1d/df6/l113 1 1 2024-03-22T07:08:56.986 INFO:tasks.workunit.client.1.smithi149.stdout:6/841: rmdir d9/df/d22/d6d/d7d/d96 39 2024-03-22T07:08:56.988 INFO:tasks.workunit.client.1.smithi149.stdout:6/842: symlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/d107/l10d 0 2024-03-22T07:08:56.988 INFO:tasks.workunit.client.1.smithi149.stdout:6/843: chown d9/df/d22/d6d/d7d/d96/d24/d42/l64 602717214 1 2024-03-22T07:08:56.990 INFO:tasks.workunit.client.1.smithi149.stdout:6/844: rename d9/df/d22/d6d/d7d/d96/c7c to d9/df/d22/d6d/d7d/d96/d24/d68/d73/c10e 0 2024-03-22T07:08:56.991 INFO:tasks.workunit.client.1.smithi149.stdout:6/845: symlink d9/df/d22/d6d/d7d/d96/l10f 0 2024-03-22T07:08:56.993 INFO:tasks.workunit.client.1.smithi149.stdout:1/793: dread d4/f69 [0,4194304] 0 2024-03-22T07:08:56.993 INFO:tasks.workunit.client.1.smithi149.stdout:6/846: symlink d9/df/d22/d6d/l110 0 2024-03-22T07:08:56.994 INFO:tasks.workunit.client.1.smithi149.stdout:1/794: dread d4/d20/d2d/faf [0,4194304] 0 2024-03-22T07:08:56.995 INFO:tasks.workunit.client.1.smithi149.stdout:1/795: fdatasync d4/d10/dd8/f74 0 2024-03-22T07:08:56.995 INFO:tasks.workunit.client.1.smithi149.stdout:6/847: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/fc7 [314194,94440] 0 2024-03-22T07:08:56.996 INFO:tasks.workunit.client.1.smithi149.stdout:2/866: dwrite d1/d3/d5/f2b [0,4194304] 0 2024-03-22T07:08:56.996 INFO:tasks.workunit.client.1.smithi149.stdout:2/867: creat d1/d3/d5/d1f/d5e/f137 x:0 0 0 2024-03-22T07:08:56.996 INFO:tasks.workunit.client.1.smithi149.stdout:1/796: mkdir d4/d10/dd8/d3c/d109 0 2024-03-22T07:08:56.996 INFO:tasks.workunit.client.1.smithi149.stdout:1/797: write d4/d20/d2d/d64/d9f/dbe/fd5 [4306562,129350] 0 2024-03-22T07:08:56.997 INFO:tasks.workunit.client.1.smithi149.stdout:6/848: creat d9/df/d22/dc0/dc8/de9/f111 x:0 0 0 2024-03-22T07:08:56.999 INFO:tasks.workunit.client.1.smithi149.stdout:2/868: symlink d1/d4d/d69/d75/db8/l138 0 2024-03-22T07:08:57.001 INFO:tasks.workunit.client.1.smithi149.stdout:6/849: symlink d9/df/d22/l112 0 2024-03-22T07:08:57.003 INFO:tasks.workunit.client.1.smithi149.stdout:1/798: rename d4/d7/d5d/lfe to d4/d7/d5a/l10a 0 2024-03-22T07:08:57.005 INFO:tasks.workunit.client.1.smithi149.stdout:6/850: mkdir d9/df/d22/d6d/d7d/d96/d24/d68/d113 0 2024-03-22T07:08:57.007 INFO:tasks.workunit.client.1.smithi149.stdout:1/799: rename d4/f15 to d4/d20/da0/f10b 0 2024-03-22T07:08:57.007 INFO:tasks.workunit.client.1.smithi149.stdout:1/800: fsync d4/d20/da0/dc0/ff3 0 2024-03-22T07:08:57.007 INFO:tasks.workunit.client.1.smithi149.stdout:1/801: readlink d4/d7/d94/lc4 0 2024-03-22T07:08:57.007 INFO:tasks.workunit.client.1.smithi149.stdout:1/802: fsync d4/d10/d58/f5b 0 2024-03-22T07:08:57.007 INFO:tasks.workunit.client.1.smithi149.stdout:1/803: truncate d4/d10/f4c 219479 0 2024-03-22T07:08:57.009 INFO:tasks.workunit.client.1.smithi149.stdout:6/851: unlink d9/df/d22/dc0/dc8/de9/f111 0 2024-03-22T07:08:57.009 INFO:tasks.workunit.client.1.smithi149.stdout:6/852: write f1 [1471892,45464] 0 2024-03-22T07:08:57.021 INFO:tasks.workunit.client.1.smithi149.stdout:3/640: dwrite d0/d4/d8/fd2 [4194304,4194304] 0 2024-03-22T07:08:57.021 INFO:tasks.workunit.client.1.smithi149.stdout:3/641: write d0/d2/d42/d66/f7d [1176131,66621] 0 2024-03-22T07:08:57.021 INFO:tasks.workunit.client.1.smithi149.stdout:3/642: stat d0/d2/d30/d5d/d84/d8f/f50 0 2024-03-22T07:08:57.021 INFO:tasks.workunit.client.1.smithi149.stdout:3/643: readlink d0/d4/d8/d24/d40/l88 0 2024-03-22T07:08:57.023 INFO:tasks.workunit.client.1.smithi149.stdout:3/644: rename d0/d2/fd9 to d0/d2/d30/d5d/d84/d8f/ff3 0 2024-03-22T07:08:57.025 INFO:tasks.workunit.client.1.smithi149.stdout:4/809: dwrite d0/d8/dc/f5b [0,4194304] 0 2024-03-22T07:08:57.025 INFO:tasks.workunit.client.1.smithi149.stdout:1/804: dwrite d4/d10/dd8/f6f [0,4194304] 0 2024-03-22T07:08:57.025 INFO:tasks.workunit.client.1.smithi149.stdout:4/810: chown d0/d8/dc/d74/f95 31 1 2024-03-22T07:08:57.025 INFO:tasks.workunit.client.1.smithi149.stdout:3/645: rename d0/d2/d42/d66/d82/f94 to d0/d2/d5a/daf/ff4 0 2024-03-22T07:08:57.025 INFO:tasks.workunit.client.1.smithi149.stdout:3/646: stat d0/d4/d8/d24/d40/d54/d85/f86 0 2024-03-22T07:08:57.025 INFO:tasks.workunit.client.1.smithi149.stdout:2/869: dwrite d1/d3/d6/f92 [0,4194304] 0 2024-03-22T07:08:57.029 INFO:tasks.workunit.client.1.smithi149.stdout:4/811: unlink d0/d8/f1c 0 2024-03-22T07:08:57.031 INFO:tasks.workunit.client.1.smithi149.stdout:3/647: creat d0/d4/d6c/ff5 x:0 0 0 2024-03-22T07:08:57.039 INFO:tasks.workunit.client.1.smithi149.stdout:2/870: symlink d1/d4d/d69/d75/d7e/d8c/l139 0 2024-03-22T07:08:57.040 INFO:tasks.workunit.client.1.smithi149.stdout:4/812: mkdir d0/d8/d1d/d24/d9c/d107 0 2024-03-22T07:08:57.040 INFO:tasks.workunit.client.1.smithi149.stdout:1/805: dwrite d4/d20/da0/f10b [4194304,4194304] 0 2024-03-22T07:08:57.040 INFO:tasks.workunit.client.1.smithi149.stdout:4/813: write d0/d8/dc/d4b/dbc/f100 [611400,57386] 0 2024-03-22T07:08:57.041 INFO:tasks.workunit.client.1.smithi149.stdout:3/648: link d0/d2/d30/d5d/d84/d8f/cd3 d0/d2/cf6 0 2024-03-22T07:08:57.043 INFO:tasks.workunit.client.1.smithi149.stdout:2/871: mkdir d1/d8/d59/d13a 0 2024-03-22T07:08:57.043 INFO:tasks.workunit.client.1.smithi149.stdout:2/872: getdents d1/d3/d6/db2/d71 0 2024-03-22T07:08:57.044 INFO:tasks.workunit.client.1.smithi149.stdout:1/806: getdents d4/d10/dd8/d3c/d8e 0 2024-03-22T07:08:57.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/814: creat d0/d8/d5c/dc1/dd2/f108 x:0 0 0 2024-03-22T07:08:57.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/815: truncate d0/d8/dc/d74/d76/deb/ffd 86929 0 2024-03-22T07:08:57.046 INFO:tasks.workunit.client.1.smithi149.stdout:3/649: unlink d0/d4/d8/d24/fae 0 2024-03-22T07:08:57.047 INFO:tasks.workunit.client.1.smithi149.stdout:3/650: write d0/fe5 [253436,97889] 0 2024-03-22T07:08:57.047 INFO:tasks.workunit.client.1.smithi149.stdout:3/651: truncate d0/d4/d8/d24/d40/d54/d85/dbc/fc2 125796 0 2024-03-22T07:08:57.049 INFO:tasks.workunit.client.1.smithi149.stdout:8/812: sync 2024-03-22T07:08:57.049 INFO:tasks.workunit.client.1.smithi149.stdout:8/813: dread - d1/d2/ffe zero size 2024-03-22T07:08:57.052 INFO:tasks.workunit.client.1.smithi149.stdout:8/814: creat d1/d2/d9/d11/d8b/daf/f106 x:0 0 0 2024-03-22T07:08:57.052 INFO:tasks.workunit.client.1.smithi149.stdout:1/807: dread d4/d10/f63 [0,4194304] 0 2024-03-22T07:08:57.056 INFO:tasks.workunit.client.1.smithi149.stdout:4/816: dread d0/d8/d1d/d24/d82/fa1 [0,4194304] 0 2024-03-22T07:08:57.058 INFO:tasks.workunit.client.1.smithi149.stdout:4/817: symlink d0/d8/d4c/l109 0 2024-03-22T07:08:57.060 INFO:tasks.workunit.client.1.smithi149.stdout:4/818: creat d0/d8/dc/d14/f10a x:0 0 0 2024-03-22T07:08:57.060 INFO:tasks.workunit.client.1.smithi149.stdout:4/819: readlink d0/lca 0 2024-03-22T07:08:57.060 INFO:tasks.workunit.client.1.smithi149.stdout:4/820: fdatasync d0/f67 0 2024-03-22T07:08:57.063 INFO:tasks.workunit.client.1.smithi149.stdout:2/873: write d1/d3/d5/d1f/fde [4158600,87120] 0 2024-03-22T07:08:57.063 INFO:tasks.workunit.client.1.smithi149.stdout:2/874: chown d1/d3/d5/d1f/d5e/d2f/fab 26 1 2024-03-22T07:08:57.077 INFO:tasks.workunit.client.1.smithi149.stdout:8/815: dwrite d1/d2/d9/d11/d4d/dd6/d35/d22/f40 [0,4194304] 0 2024-03-22T07:08:57.077 INFO:tasks.workunit.client.1.smithi149.stdout:8/816: creat d1/d2/d9/d11/d4d/dd6/db4/f107 x:0 0 0 2024-03-22T07:08:57.080 INFO:tasks.workunit.client.1.smithi149.stdout:8/817: link d1/fc3 d1/d2/d9/d11/d8b/dee/df4/f108 0 2024-03-22T07:08:57.081 INFO:tasks.workunit.client.1.smithi149.stdout:3/652: dwrite d0/d4/d8/d24/f47 [4194304,4194304] 0 2024-03-22T07:08:57.102 INFO:tasks.workunit.client.1.smithi149.stdout:2/875: dwrite d1/d3/d6/db2/f40 [0,4194304] 0 2024-03-22T07:08:57.104 INFO:tasks.workunit.client.1.smithi149.stdout:2/876: creat d1/d8/d59/d13a/f13b x:0 0 0 2024-03-22T07:08:57.104 INFO:tasks.workunit.client.1.smithi149.stdout:2/877: fsync d1/d8/d34/f7a 0 2024-03-22T07:08:57.105 INFO:tasks.workunit.client.1.smithi149.stdout:2/878: truncate d1/d3/d5/d1f/f55 3597581 0 2024-03-22T07:08:57.105 INFO:tasks.workunit.client.1.smithi149.stdout:2/879: write d1/d3/d6/db2/f65 [3785962,51037] 0 2024-03-22T07:08:57.106 INFO:tasks.workunit.client.1.smithi149.stdout:4/821: dwrite d0/d8/dc/f13 [0,4194304] 0 2024-03-22T07:08:57.107 INFO:tasks.workunit.client.1.smithi149.stdout:4/822: creat d0/d8/dc/d2e/f10b x:0 0 0 2024-03-22T07:08:57.113 INFO:tasks.workunit.client.1.smithi149.stdout:8/818: dwrite d1/d2/d9/d11/d4d/ffa [0,4194304] 0 2024-03-22T07:08:57.119 INFO:tasks.workunit.client.1.smithi149.stdout:9/774: sync 2024-03-22T07:08:57.119 INFO:tasks.workunit.client.1.smithi149.stdout:0/717: sync 2024-03-22T07:08:57.119 INFO:tasks.workunit.client.1.smithi149.stdout:0/718: creat d1/d20/d9f/dc6/f104 x:0 0 0 2024-03-22T07:08:57.120 INFO:tasks.workunit.client.1.smithi149.stdout:0/719: read - d1/d9/d14/fa8 zero size 2024-03-22T07:08:57.120 INFO:tasks.workunit.client.1.smithi149.stdout:5/799: sync 2024-03-22T07:08:57.121 INFO:tasks.workunit.client.1.smithi149.stdout:0/720: creat d1/d20/d9f/dc6/dd9/f105 x:0 0 0 2024-03-22T07:08:57.122 INFO:tasks.workunit.client.1.smithi149.stdout:5/800: rename d7/da/d20/d30/d3a/d3d/db0/c10f to d7/da/ded/c118 0 2024-03-22T07:08:57.123 INFO:tasks.workunit.client.1.smithi149.stdout:3/653: dwrite d0/d2/d30/f3f [0,4194304] 0 2024-03-22T07:08:57.123 INFO:tasks.workunit.client.1.smithi149.stdout:5/801: truncate d7/da/d20/d30/f98 2063578 0 2024-03-22T07:08:57.125 INFO:tasks.workunit.client.1.smithi149.stdout:0/721: rmdir d1/d9/d14/d1f 39 2024-03-22T07:08:57.125 INFO:tasks.workunit.client.1.smithi149.stdout:3/654: getdents d0/d2/d5a/d93 0 2024-03-22T07:08:57.128 INFO:tasks.workunit.client.1.smithi149.stdout:5/802: mkdir d7/da/d20/d89/d119 0 2024-03-22T07:08:57.128 INFO:tasks.workunit.client.1.smithi149.stdout:0/722: symlink d1/d31/d51/d64/d87/da6/dfe/l106 0 2024-03-22T07:08:57.128 INFO:tasks.workunit.client.1.smithi149.stdout:0/723: readlink d1/l2 0 2024-03-22T07:08:57.128 INFO:tasks.workunit.client.1.smithi149.stdout:3/655: unlink d0/d2/d30/d5d/d84/d8f/f20 0 2024-03-22T07:08:57.128 INFO:tasks.workunit.client.1.smithi149.stdout:3/656: write d0/d2/f4d [854316,15413] 0 2024-03-22T07:08:57.130 INFO:tasks.workunit.client.1.smithi149.stdout:0/724: mknod d1/d20/d9f/dc6/ddc/c107 0 2024-03-22T07:08:57.131 INFO:tasks.workunit.client.1.smithi149.stdout:3/657: link d0/d2/d30/d5d/d84/dc3/c4b d0/d2/d30/d7e/db8/cf7 0 2024-03-22T07:08:57.131 INFO:tasks.workunit.client.1.smithi149.stdout:2/880: dwrite d1/d8/f24 [0,4194304] 0 2024-03-22T07:08:57.133 INFO:tasks.workunit.client.1.smithi149.stdout:8/819: write d1/d2/d9/d11/d45/f70 [941928,69960] 0 2024-03-22T07:08:57.134 INFO:tasks.workunit.client.1.smithi149.stdout:0/725: mkdir d1/d9/d14/d74/dc1/d108 0 2024-03-22T07:08:57.137 INFO:tasks.workunit.client.1.smithi149.stdout:4/823: dwrite d0/d8/d1d/f8f [0,4194304] 0 2024-03-22T07:08:57.137 INFO:tasks.workunit.client.1.smithi149.stdout:0/726: mkdir d1/d31/d51/d64/d87/da6/dfe/d109 0 2024-03-22T07:08:57.138 INFO:tasks.workunit.client.1.smithi149.stdout:2/881: getdents d1/d4d 0 2024-03-22T07:08:57.144 INFO:tasks.workunit.client.1.smithi149.stdout:4/824: rename d0/d29/fff to d0/d8/f10c 0 2024-03-22T07:08:57.145 INFO:tasks.workunit.client.1.smithi149.stdout:2/882: mknod d1/d8/d59/c13c 0 2024-03-22T07:08:57.145 INFO:tasks.workunit.client.1.smithi149.stdout:2/883: creat d1/f13d x:0 0 0 2024-03-22T07:08:57.146 INFO:tasks.workunit.client.1.smithi149.stdout:4/825: creat d0/d8/dc/d74/d76/deb/f10d x:0 0 0 2024-03-22T07:08:57.146 INFO:tasks.workunit.client.1.smithi149.stdout:4/826: chown d0/d8/dc/d74/d76/cc4 500879 1 2024-03-22T07:08:57.147 INFO:tasks.workunit.client.1.smithi149.stdout:0/727: getdents d1/d6b 0 2024-03-22T07:08:57.147 INFO:tasks.workunit.client.1.smithi149.stdout:0/728: write d1/d20/d9f/dc6/dd9/f105 [60893,46864] 0 2024-03-22T07:08:57.147 INFO:tasks.workunit.client.1.smithi149.stdout:0/729: creat d1/d31/d51/d64/d87/f10a x:0 0 0 2024-03-22T07:08:57.148 INFO:tasks.workunit.client.1.smithi149.stdout:2/884: mknod d1/d8/d59/df6/c13e 0 2024-03-22T07:08:57.149 INFO:tasks.workunit.client.1.smithi149.stdout:4/827: mkdir d0/d8/dc/d14/d10e 0 2024-03-22T07:08:57.149 INFO:tasks.workunit.client.1.smithi149.stdout:0/730: mknod d1/d27/c10b 0 2024-03-22T07:08:57.150 INFO:tasks.workunit.client.1.smithi149.stdout:0/731: fdatasync d1/d9/d14/d1f/f24 0 2024-03-22T07:08:57.150 INFO:tasks.workunit.client.1.smithi149.stdout:0/732: readlink d1/d9/d59/leb 0 2024-03-22T07:08:57.150 INFO:tasks.workunit.client.1.smithi149.stdout:0/733: stat d1/d9/f71 0 2024-03-22T07:08:57.152 INFO:tasks.workunit.client.1.smithi149.stdout:2/885: mknod d1/d3/d9a/dee/c13f 0 2024-03-22T07:08:57.152 INFO:tasks.workunit.client.1.smithi149.stdout:4/828: rmdir d0/d8/dc/d14/d85/d97 39 2024-03-22T07:08:57.152 INFO:tasks.workunit.client.1.smithi149.stdout:7/830: sync 2024-03-22T07:08:57.152 INFO:tasks.workunit.client.1.smithi149.stdout:6/853: sync 2024-03-22T07:08:57.152 INFO:tasks.workunit.client.1.smithi149.stdout:7/831: chown d1/d1c/d49/d8e/d100/d1b/d1d/f8 0 1 2024-03-22T07:08:57.152 INFO:tasks.workunit.client.1.smithi149.stdout:6/854: write d9/df/d22/d6d/d7d/d96/f41 [2462952,909] 0 2024-03-22T07:08:57.154 INFO:tasks.workunit.client.1.smithi149.stdout:7/832: unlink d1/d1c/d49/d8e/d100/d1b/d1d/c82 0 2024-03-22T07:08:57.154 INFO:tasks.workunit.client.1.smithi149.stdout:6/855: creat d9/df/d22/d6d/d7d/d96/d24/d68/d113/f114 x:0 0 0 2024-03-22T07:08:57.155 INFO:tasks.workunit.client.1.smithi149.stdout:7/833: creat d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/f114 x:0 0 0 2024-03-22T07:08:57.156 INFO:tasks.workunit.client.1.smithi149.stdout:6/856: symlink d9/df/d2e/d63/dfe/l115 0 2024-03-22T07:08:57.157 INFO:tasks.workunit.client.1.smithi149.stdout:7/834: creat d1/db8/f115 x:0 0 0 2024-03-22T07:08:57.157 INFO:tasks.workunit.client.1.smithi149.stdout:6/857: rmdir d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38 39 2024-03-22T07:08:57.157 INFO:tasks.workunit.client.1.smithi149.stdout:9/775: dwrite db/d21/d74/fdf [0,4194304] 0 2024-03-22T07:08:57.159 INFO:tasks.workunit.client.1.smithi149.stdout:7/835: symlink d1/d1c/d49/d8e/d100/d1b/d1d/l116 0 2024-03-22T07:08:57.159 INFO:tasks.workunit.client.1.smithi149.stdout:7/836: chown d1/d1c/d49/d8e/d100/d30/c84 820979 1 2024-03-22T07:08:57.159 INFO:tasks.workunit.client.1.smithi149.stdout:7/837: write d1/d1c/dbc/fcb [334709,12246] 0 2024-03-22T07:08:57.159 INFO:tasks.workunit.client.1.smithi149.stdout:7/838: stat d1/d1c/d49/d8e/d100/d1b/d1d/f8 0 2024-03-22T07:08:57.159 INFO:tasks.workunit.client.1.smithi149.stdout:7/839: chown d1/d1c/d49/d8e/d100/d1b/ff1 14549 1 2024-03-22T07:08:57.159 INFO:tasks.workunit.client.1.smithi149.stdout:7/840: readlink d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/l10d 0 2024-03-22T07:08:57.162 INFO:tasks.workunit.client.1.smithi149.stdout:6/858: rename d9/df/d22/d6d/d7d/d96/d24/d68/d73/f8b to d9/df/f116 0 2024-03-22T07:08:57.163 INFO:tasks.workunit.client.1.smithi149.stdout:7/841: getdents d1/d1c/d49/d8e/d100/d1b/d1d/df6 0 2024-03-22T07:08:57.164 INFO:tasks.workunit.client.1.smithi149.stdout:6/859: mkdir d9/df/d22/d6d/d117 0 2024-03-22T07:08:57.165 INFO:tasks.workunit.client.1.smithi149.stdout:6/860: creat d9/da2/f118 x:0 0 0 2024-03-22T07:08:57.165 INFO:tasks.workunit.client.1.smithi149.stdout:6/861: chown d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/fe0 123611 1 2024-03-22T07:08:57.165 INFO:tasks.workunit.client.1.smithi149.stdout:4/829: write d0/d8/d4c/f57 [6772855,32078] 0 2024-03-22T07:08:57.165 INFO:tasks.workunit.client.1.smithi149.stdout:9/776: rename db/d19/d39/d3b/f53 to db/d12/fea 0 2024-03-22T07:08:57.167 INFO:tasks.workunit.client.1.smithi149.stdout:9/777: unlink db/d19/d39/d68/d6f/fa7 0 2024-03-22T07:08:57.170 INFO:tasks.workunit.client.1.smithi149.stdout:9/778: creat db/d21/d37/d7c/dc7/feb x:0 0 0 2024-03-22T07:08:57.171 INFO:tasks.workunit.client.1.smithi149.stdout:9/779: symlink db/d22/lec 0 2024-03-22T07:08:57.171 INFO:tasks.workunit.client.1.smithi149.stdout:9/780: stat db/d19/d93/cca 0 2024-03-22T07:08:57.172 INFO:tasks.workunit.client.1.smithi149.stdout:9/781: chown db/d19/d39/d3b/f48 92234 1 2024-03-22T07:08:57.174 INFO:tasks.workunit.client.1.smithi149.stdout:9/782: getdents db/d21/d37/d7c 0 2024-03-22T07:08:57.174 INFO:tasks.workunit.client.1.smithi149.stdout:9/783: creat db/d42/fed x:0 0 0 2024-03-22T07:08:57.175 INFO:tasks.workunit.client.1.smithi149.stdout:4/830: write d0/d8/d1d/fe6 [3044183,81845] 0 2024-03-22T07:08:57.175 INFO:tasks.workunit.client.1.smithi149.stdout:4/831: chown d0/d29/l8d 14475097 1 2024-03-22T07:08:57.175 INFO:tasks.workunit.client.1.smithi149.stdout:9/784: mknod db/d42/cee 0 2024-03-22T07:08:57.185 INFO:tasks.workunit.client.1.smithi149.stdout:1/808: sync 2024-03-22T07:08:57.185 INFO:tasks.workunit.client.1.smithi149.stdout:1/809: stat d4/d10/dd8/d3c/d8e/db5/f83 0 2024-03-22T07:08:57.185 INFO:tasks.workunit.client.1.smithi149.stdout:9/785: read db/d19/f7d [481722,1235] 0 2024-03-22T07:08:57.187 INFO:tasks.workunit.client.1.smithi149.stdout:9/786: rmdir db/d42/d49/d4b/d5f 39 2024-03-22T07:08:57.188 INFO:tasks.workunit.client.1.smithi149.stdout:9/787: mknod db/d19/d1c/cef 0 2024-03-22T07:08:57.188 INFO:tasks.workunit.client.1.smithi149.stdout:5/803: dwrite d7/da/d20/d36/f66 [0,4194304] 0 2024-03-22T07:08:57.189 INFO:tasks.workunit.client.1.smithi149.stdout:3/658: dwrite d0/d2/d30/d7e/fe8 [0,4194304] 0 2024-03-22T07:08:57.195 INFO:tasks.workunit.client.1.smithi149.stdout:3/659: mknod d0/d2/d30/d7e/cf8 0 2024-03-22T07:08:57.200 INFO:tasks.workunit.client.1.smithi149.stdout:6/862: dwrite d9/da2/fbc [0,4194304] 0 2024-03-22T07:08:57.201 INFO:tasks.workunit.client.1.smithi149.stdout:8/820: dwrite d1/d2/d9/d11/d8b/daf/fb9 [0,4194304] 0 2024-03-22T07:08:57.205 INFO:tasks.workunit.client.1.smithi149.stdout:7/842: rmdir d1/d1c/d49/d8e/d100/d1b/d51/d5a 39 2024-03-22T07:08:57.209 INFO:tasks.workunit.client.1.smithi149.stdout:6/863: rename d9/df/l10c to d9/d47/d52/l119 0 2024-03-22T07:08:57.209 INFO:tasks.workunit.client.1.smithi149.stdout:4/832: dwrite d0/d8/dc/d2e/fbf [0,4194304] 0 2024-03-22T07:08:57.209 INFO:tasks.workunit.client.1.smithi149.stdout:4/833: chown d0/d8/dc/d2e/d6f/ddf 159865797 1 2024-03-22T07:08:57.210 INFO:tasks.workunit.client.1.smithi149.stdout:2/886: dread d1/d3/d6/db2/f65 [0,4194304] 0 2024-03-22T07:08:57.213 INFO:tasks.workunit.client.1.smithi149.stdout:4/834: rename d0/d8/lde to d0/d8/dc/d4b/db3/ded/df2/l10f 0 2024-03-22T07:08:57.214 INFO:tasks.workunit.client.1.smithi149.stdout:4/835: dread - d0/d8/dc/d2e/d6f/f102 zero size 2024-03-22T07:08:57.214 INFO:tasks.workunit.client.1.smithi149.stdout:9/788: rmdir db/d42 39 2024-03-22T07:08:57.215 INFO:tasks.workunit.client.1.smithi149.stdout:2/887: mkdir d1/d3/d5/d41/d9f/da7/d140 0 2024-03-22T07:08:57.216 INFO:tasks.workunit.client.1.smithi149.stdout:4/836: mkdir d0/d6b/db0/d110 0 2024-03-22T07:08:57.217 INFO:tasks.workunit.client.1.smithi149.stdout:4/837: creat d0/d8/d4c/f111 x:0 0 0 2024-03-22T07:08:57.217 INFO:tasks.workunit.client.1.smithi149.stdout:9/789: mkdir db/d42/d49/df0 0 2024-03-22T07:08:57.218 INFO:tasks.workunit.client.1.smithi149.stdout:9/790: stat ca 0 2024-03-22T07:08:57.219 INFO:tasks.workunit.client.1.smithi149.stdout:9/791: fsync db/d21/d37/f5c 0 2024-03-22T07:08:57.219 INFO:tasks.workunit.client.1.smithi149.stdout:9/792: creat db/d21/ff1 x:0 0 0 2024-03-22T07:08:57.219 INFO:tasks.workunit.client.1.smithi149.stdout:9/793: write db/d19/d1c/d85/fb6 [101263,20751] 0 2024-03-22T07:08:57.219 INFO:tasks.workunit.client.1.smithi149.stdout:9/794: write db/d19/d93/fe4 [57274,108759] 0 2024-03-22T07:08:57.219 INFO:tasks.workunit.client.1.smithi149.stdout:2/888: symlink d1/d3/d6/db2/l141 0 2024-03-22T07:08:57.221 INFO:tasks.workunit.client.1.smithi149.stdout:2/889: creat d1/d3/d9a/f142 x:0 0 0 2024-03-22T07:08:57.221 INFO:tasks.workunit.client.1.smithi149.stdout:2/890: chown d1/d3/d6/d32/d6f/ffd 372 1 2024-03-22T07:08:57.221 INFO:tasks.workunit.client.1.smithi149.stdout:2/891: creat d1/d3/d5/d1f/d5e/f143 x:0 0 0 2024-03-22T07:08:57.222 INFO:tasks.workunit.client.1.smithi149.stdout:2/892: chown d1/d3/d6/db2/d71/fda 2605 1 2024-03-22T07:08:57.224 INFO:tasks.workunit.client.1.smithi149.stdout:2/893: symlink d1/d4d/d69/d75/d7e/d8c/l144 0 2024-03-22T07:08:57.225 INFO:tasks.workunit.client.1.smithi149.stdout:9/795: creat db/d33/ff2 x:0 0 0 2024-03-22T07:08:57.226 INFO:tasks.workunit.client.1.smithi149.stdout:2/894: truncate d1/d3/d5/d1f/d68/d94/fd4 1362722 0 2024-03-22T07:08:57.232 INFO:tasks.workunit.client.1.smithi149.stdout:3/660: dwrite d0/d2/d5a/daf/ff4 [0,4194304] 0 2024-03-22T07:08:57.233 INFO:tasks.workunit.client.1.smithi149.stdout:3/661: write d0/f72 [168250,45573] 0 2024-03-22T07:08:57.241 INFO:tasks.workunit.client.1.smithi149.stdout:1/810: dwrite d4/d10/dd8/d3c/d8e/f8f [0,4194304] 0 2024-03-22T07:08:57.241 INFO:tasks.workunit.client.1.smithi149.stdout:1/811: chown d4/d10/dd8/d3c/lb1 8884375 1 2024-03-22T07:08:57.244 INFO:tasks.workunit.client.1.smithi149.stdout:4/838: rename d0/d8/dc/d4b/db3/ded/df2/l10f to d0/d8/dc/dbd/l112 0 2024-03-22T07:08:57.244 INFO:tasks.workunit.client.1.smithi149.stdout:4/839: chown d0/d8/dc/d4b/db3/dea/fad 394 1 2024-03-22T07:08:57.244 INFO:tasks.workunit.client.1.smithi149.stdout:4/840: fsync d0/d8/dc/dbd/fbe 0 2024-03-22T07:08:57.244 INFO:tasks.workunit.client.1.smithi149.stdout:4/841: readlink d0/d8/dc/d14/l40 0 2024-03-22T07:08:57.250 INFO:tasks.workunit.client.1.smithi149.stdout:3/662: read d0/d4/d8/d24/d40/d54/d85/fd5 [1465057,101888] 0 2024-03-22T07:08:57.250 INFO:tasks.workunit.client.1.smithi149.stdout:3/663: write d0/d2/d30/d5d/d84/d8e/d9d/fb9 [1902393,7986] 0 2024-03-22T07:08:57.255 INFO:tasks.workunit.client.1.smithi149.stdout:6/864: dwrite d9/df/d22/ff2 [0,4194304] 0 2024-03-22T07:08:57.256 INFO:tasks.workunit.client.1.smithi149.stdout:6/865: read - d9/df/d22/d6d/d7d/d96/d24/d68/d73/fea zero size 2024-03-22T07:08:57.256 INFO:tasks.workunit.client.1.smithi149.stdout:6/866: write d9/df/d22/d6d/d7d/fb6 [4929441,121891] 0 2024-03-22T07:08:57.257 INFO:tasks.workunit.client.1.smithi149.stdout:8/821: dwrite d1/d2/ffe [0,4194304] 0 2024-03-22T07:08:57.259 INFO:tasks.workunit.client.1.smithi149.stdout:6/867: symlink d9/df/d22/d6d/d7d/d96/d24/d42/l11a 0 2024-03-22T07:08:57.259 INFO:tasks.workunit.client.1.smithi149.stdout:6/868: write d9/df/f1d [187931,33933] 0 2024-03-22T07:08:57.262 INFO:tasks.workunit.client.1.smithi149.stdout:1/812: dread d4/d20/da0/f10b [0,4194304] 0 2024-03-22T07:08:57.262 INFO:tasks.workunit.client.1.smithi149.stdout:1/813: chown l3 0 1 2024-03-22T07:08:57.262 INFO:tasks.workunit.client.1.smithi149.stdout:8/822: unlink d1/d2/d9/d11/d8b/cae 0 2024-03-22T07:08:57.265 INFO:tasks.workunit.client.1.smithi149.stdout:1/814: mkdir d4/d10c 0 2024-03-22T07:08:57.265 INFO:tasks.workunit.client.1.smithi149.stdout:0/734: dwrite d1/d20/d9f/dc6/dd9/f105 [0,4194304] 0 2024-03-22T07:08:57.265 INFO:tasks.workunit.client.1.smithi149.stdout:0/735: chown d1/d9/d59/ff9 54001 1 2024-03-22T07:08:57.269 INFO:tasks.workunit.client.1.smithi149.stdout:0/736: rmdir d1/d9/d14/dd3 0 2024-03-22T07:08:57.269 INFO:tasks.workunit.client.1.smithi149.stdout:0/737: fdatasync d1/d31/d51/d64/fbd 0 2024-03-22T07:08:57.269 INFO:tasks.workunit.client.1.smithi149.stdout:0/738: truncate d1/d9/d14/f94 606765 0 2024-03-22T07:08:57.269 INFO:tasks.workunit.client.1.smithi149.stdout:1/815: rmdir d4/d10/d58/d96 39 2024-03-22T07:08:57.269 INFO:tasks.workunit.client.1.smithi149.stdout:1/816: write d4/d20/ff4 [167732,92773] 0 2024-03-22T07:08:57.271 INFO:tasks.workunit.client.1.smithi149.stdout:0/739: rename d1/d9/d14/d74/dc1 to d1/d31/d51/d64/d8d/d9c/d9e/d10c 0 2024-03-22T07:08:57.271 INFO:tasks.workunit.client.1.smithi149.stdout:0/740: dread - d1/fd7 zero size 2024-03-22T07:08:57.271 INFO:tasks.workunit.client.1.smithi149.stdout:0/741: stat d1/d9/d59/f6c 0 2024-03-22T07:08:57.272 INFO:tasks.workunit.client.1.smithi149.stdout:9/796: dwrite db/d19/d28/fc1 [0,4194304] 0 2024-03-22T07:08:57.272 INFO:tasks.workunit.client.1.smithi149.stdout:9/797: write db/fc [2602460,8225] 0 2024-03-22T07:08:57.272 INFO:tasks.workunit.client.1.smithi149.stdout:1/817: mkdir d4/d10/dd8/d3c/d8e/db5/db2/dbf/d10d 0 2024-03-22T07:08:57.272 INFO:tasks.workunit.client.1.smithi149.stdout:6/869: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/f7a [2841720,86998] 0 2024-03-22T07:08:57.273 INFO:tasks.workunit.client.1.smithi149.stdout:7/843: dwrite d1/d1c/d49/d8e/d100/d1b/d51/d5a/f78 [0,4194304] 0 2024-03-22T07:08:57.273 INFO:tasks.workunit.client.1.smithi149.stdout:7/844: truncate d1/d1c/d49/d8e/d100/d1b/d51/d5a/df0/f110 611750 0 2024-03-22T07:08:57.273 INFO:tasks.workunit.client.1.smithi149.stdout:2/895: dwrite d1/d4d/d69/d75/d7e/d8c/f116 [0,4194304] 0 2024-03-22T07:08:57.273 INFO:tasks.workunit.client.1.smithi149.stdout:2/896: chown d1/d3/d6/db2/l5c 321 1 2024-03-22T07:08:57.274 INFO:tasks.workunit.client.1.smithi149.stdout:4/842: write d0/d8/dc/d14/f43 [1023817,4989] 0 2024-03-22T07:08:57.280 INFO:tasks.workunit.client.1.smithi149.stdout:7/845: symlink d1/d1c/d49/d8e/d100/d1b/d51/d5a/d6a/l117 0 2024-03-22T07:08:57.281 INFO:tasks.workunit.client.1.smithi149.stdout:3/664: dwrite d0/d2/f76 [0,4194304] 0 2024-03-22T07:08:57.281 INFO:tasks.workunit.client.1.smithi149.stdout:3/665: write d0/d2/d42/d66/f7d [4594582,118432] 0 2024-03-22T07:08:57.283 INFO:tasks.workunit.client.1.smithi149.stdout:2/897: mkdir d1/d3/d5/d41/d9f/da7/d145 0 2024-03-22T07:08:57.283 INFO:tasks.workunit.client.1.smithi149.stdout:2/898: chown d1/d3/d5/d1f/f93 13412 1 2024-03-22T07:08:57.285 INFO:tasks.workunit.client.1.smithi149.stdout:4/843: unlink d0/d8/d5c/f5e 0 2024-03-22T07:08:57.285 INFO:tasks.workunit.client.1.smithi149.stdout:9/798: getdents db/d21/d8c 0 2024-03-22T07:08:57.287 INFO:tasks.workunit.client.1.smithi149.stdout:4/844: read d0/d8/dc/d74/fe1 [305324,41088] 0 2024-03-22T07:08:57.287 INFO:tasks.workunit.client.1.smithi149.stdout:4/845: truncate d0/d8/dc/d14/f43 1337222 0 2024-03-22T07:08:57.288 INFO:tasks.workunit.client.1.smithi149.stdout:6/870: link d9/df/d22/ff2 d9/df/d22/d6d/d117/f11b 0 2024-03-22T07:08:57.289 INFO:tasks.workunit.client.1.smithi149.stdout:7/846: creat d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/f118 x:0 0 0 2024-03-22T07:08:57.289 INFO:tasks.workunit.client.1.smithi149.stdout:1/818: write d4/d10/d58/f5b [226130,53642] 0 2024-03-22T07:08:57.294 INFO:tasks.workunit.client.1.smithi149.stdout:4/846: mknod d0/d8/c113 0 2024-03-22T07:08:57.297 INFO:tasks.workunit.client.1.smithi149.stdout:0/742: dwrite d1/d31/d51/d64/d8d/d9c/fb5 [0,4194304] 0 2024-03-22T07:08:57.298 INFO:tasks.workunit.client.1.smithi149.stdout:0/743: fsync d1/d31/f5c 0 2024-03-22T07:08:57.298 INFO:tasks.workunit.client.1.smithi149.stdout:0/744: creat d1/d31/d51/d64/d87/f10d x:0 0 0 2024-03-22T07:08:57.299 INFO:tasks.workunit.client.1.smithi149.stdout:0/745: creat d1/d9/f10e x:0 0 0 2024-03-22T07:08:57.299 INFO:tasks.workunit.client.1.smithi149.stdout:6/871: truncate d9/df/d22/d6d/d7d/d96/d24/d68/fc3 2297035 0 2024-03-22T07:08:57.299 INFO:tasks.workunit.client.1.smithi149.stdout:6/872: write d9/df/d22/ff5 [712384,72802] 0 2024-03-22T07:08:57.299 INFO:tasks.workunit.client.1.smithi149.stdout:6/873: fdatasync d9/fc9 0 2024-03-22T07:08:57.300 INFO:tasks.workunit.client.1.smithi149.stdout:7/847: symlink d1/d1c/d49/d8e/d100/d1b/d51/d5a/l119 0 2024-03-22T07:08:57.301 INFO:tasks.workunit.client.1.smithi149.stdout:1/819: truncate d4/d20/fe7 2177519 0 2024-03-22T07:08:57.302 INFO:tasks.workunit.client.1.smithi149.stdout:9/799: dread db/d42/f4c [0,4194304] 0 2024-03-22T07:08:57.302 INFO:tasks.workunit.client.1.smithi149.stdout:9/800: chown db/d19/d39/d3b/c95 686519 1 2024-03-22T07:08:57.302 INFO:tasks.workunit.client.1.smithi149.stdout:4/847: mknod d0/d6b/db0/c114 0 2024-03-22T07:08:57.305 INFO:tasks.workunit.client.1.smithi149.stdout:7/848: mkdir d1/d1c/d49/d8e/d100/d1b/d11a 0 2024-03-22T07:08:57.306 INFO:tasks.workunit.client.1.smithi149.stdout:1/820: symlink d4/d10/d58/d96/l10e 0 2024-03-22T07:08:57.307 INFO:tasks.workunit.client.1.smithi149.stdout:1/821: stat d4/d20/f73 0 2024-03-22T07:08:57.307 INFO:tasks.workunit.client.1.smithi149.stdout:9/801: link db/d42/f4c db/d42/d49/df0/ff3 0 2024-03-22T07:08:57.309 INFO:tasks.workunit.client.1.smithi149.stdout:1/822: symlink d4/d10/dd8/d3c/d8e/db5/db2/dbf/d10d/l10f 0 2024-03-22T07:08:57.309 INFO:tasks.workunit.client.1.smithi149.stdout:2/899: dwrite d1/d3/de0/f130 [0,4194304] 0 2024-03-22T07:08:57.309 INFO:tasks.workunit.client.1.smithi149.stdout:2/900: fsync d1/d8/d34/f57 0 2024-03-22T07:08:57.310 INFO:tasks.workunit.client.1.smithi149.stdout:2/901: fdatasync d1/d3/d9a/fd5 0 2024-03-22T07:08:57.310 INFO:tasks.workunit.client.1.smithi149.stdout:2/902: creat d1/d4d/d8b/f146 x:0 0 0 2024-03-22T07:08:57.311 INFO:tasks.workunit.client.1.smithi149.stdout:1/823: symlink d4/d7/d61/l110 0 2024-03-22T07:08:57.313 INFO:tasks.workunit.client.1.smithi149.stdout:3/666: dwrite d0/d4/d8/f8d [0,4194304] 0 2024-03-22T07:08:57.313 INFO:tasks.workunit.client.1.smithi149.stdout:6/874: write d9/d47/d52/f8a [92992,103417] 0 2024-03-22T07:08:57.313 INFO:tasks.workunit.client.1.smithi149.stdout:6/875: creat d9/df/d22/dc0/dc8/de9/f11c x:0 0 0 2024-03-22T07:08:57.314 INFO:tasks.workunit.client.1.smithi149.stdout:1/824: dread d4/d20/d2d/fa1 [0,4194304] 0 2024-03-22T07:08:57.314 INFO:tasks.workunit.client.1.smithi149.stdout:9/802: getdents db/d21/d37/d3a 0 2024-03-22T07:08:57.314 INFO:tasks.workunit.client.1.smithi149.stdout:9/803: readlink db/lcc 0 2024-03-22T07:08:57.318 INFO:tasks.workunit.client.1.smithi149.stdout:7/849: dread d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/f8a [0,4194304] 0 2024-03-22T07:08:57.321 INFO:tasks.workunit.client.1.smithi149.stdout:1/825: dread d4/d10/d7c/f98 [0,4194304] 0 2024-03-22T07:08:57.324 INFO:tasks.workunit.client.1.smithi149.stdout:3/667: creat d0/d4/d8/d24/d40/d54/d85/dbc/df2/ff9 x:0 0 0 2024-03-22T07:08:57.325 INFO:tasks.workunit.client.1.smithi149.stdout:4/848: dwrite d0/d8/dc/d4b/db3/dea/fad [4194304,4194304] 0 2024-03-22T07:08:57.327 INFO:tasks.workunit.client.1.smithi149.stdout:6/876: mkdir d9/df/d22/d6d/d7d/d96/d24/d68/d113/d11d 0 2024-03-22T07:08:57.327 INFO:tasks.workunit.client.1.smithi149.stdout:9/804: truncate db/d19/d93/fe4 47676 0 2024-03-22T07:08:57.329 INFO:tasks.workunit.client.1.smithi149.stdout:2/903: mkdir d1/d3/de0/d147 0 2024-03-22T07:08:57.329 INFO:tasks.workunit.client.1.smithi149.stdout:2/904: write d1/d3/f36 [2110025,20444] 0 2024-03-22T07:08:57.329 INFO:tasks.workunit.client.1.smithi149.stdout:1/826: getdents d4/d10/dd8/d3c 0 2024-03-22T07:08:57.330 INFO:tasks.workunit.client.1.smithi149.stdout:3/668: mkdir d0/d2/d30/dd1/dfa 0 2024-03-22T07:08:57.335 INFO:tasks.workunit.client.1.smithi149.stdout:9/805: dread db/d42/d49/df0/ff3 [0,4194304] 0 2024-03-22T07:08:57.335 INFO:tasks.workunit.client.1.smithi149.stdout:9/806: chown db/d21/l38 184874583 1 2024-03-22T07:08:57.337 INFO:tasks.workunit.client.1.smithi149.stdout:7/850: mknod d1/d1c/d49/d8e/d100/d1b/d99/de5/db4/df8/c11b 0 2024-03-22T07:08:57.337 INFO:tasks.workunit.client.1.smithi149.stdout:1/827: mknod d4/d20/c111 0 2024-03-22T07:08:57.337 INFO:tasks.workunit.client.1.smithi149.stdout:1/828: write d4/d10/dd8/f44 [1060873,81692] 0 2024-03-22T07:08:57.337 INFO:tasks.workunit.client.1.smithi149.stdout:6/877: dread d9/df/d22/f87 [0,4194304] 0 2024-03-22T07:08:57.337 INFO:tasks.workunit.client.1.smithi149.stdout:6/878: creat d9/df/d2e/d63/f11e x:0 0 0 2024-03-22T07:08:57.338 INFO:tasks.workunit.client.1.smithi149.stdout:2/905: dread d1/d3/d6/f92 [0,4194304] 0 2024-03-22T07:08:57.339 INFO:tasks.workunit.client.1.smithi149.stdout:7/851: creat d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/dee/f11c x:0 0 0 2024-03-22T07:08:57.340 INFO:tasks.workunit.client.1.smithi149.stdout:1/829: getdents d4/d10/dd8/d3c/d8e/db5/db2/dbf/d10d 0 2024-03-22T07:08:57.343 INFO:tasks.workunit.client.1.smithi149.stdout:9/807: rmdir db/d19/d39/d68/d6f 39 2024-03-22T07:08:57.345 INFO:tasks.workunit.client.1.smithi149.stdout:2/906: creat d1/d3/d5/f148 x:0 0 0 2024-03-22T07:08:57.345 INFO:tasks.workunit.client.1.smithi149.stdout:2/907: chown d1/d4d/d8b/cb1 2799 1 2024-03-22T07:08:57.345 INFO:tasks.workunit.client.1.smithi149.stdout:2/908: chown d1/d3/d5/d1f/d68/d94 35480924 1 2024-03-22T07:08:57.345 INFO:tasks.workunit.client.1.smithi149.stdout:2/909: readlink d1/d3/l1d 0 2024-03-22T07:08:57.345 INFO:tasks.workunit.client.1.smithi149.stdout:9/808: mkdir db/d21/d9e/df4 0 2024-03-22T07:08:57.348 INFO:tasks.workunit.client.1.smithi149.stdout:2/910: rename d1/d4d/d69/d75/db8 to d1/d4d/d8b/d129/d149 0 2024-03-22T07:08:57.348 INFO:tasks.workunit.client.1.smithi149.stdout:2/911: chown d1/d3/f133 191 1 2024-03-22T07:08:57.349 INFO:tasks.workunit.client.1.smithi149.stdout:9/809: symlink db/d42/dab/lf5 0 2024-03-22T07:08:57.351 INFO:tasks.workunit.client.1.smithi149.stdout:2/912: mknod d1/d4d/d8b/d129/d74/c14a 0 2024-03-22T07:08:57.351 INFO:tasks.workunit.client.1.smithi149.stdout:9/810: truncate f3 3042310 0 2024-03-22T07:08:57.351 INFO:tasks.workunit.client.1.smithi149.stdout:6/879: dread d9/df/d22/d6d/d7d/d96/f6b [0,4194304] 0 2024-03-22T07:08:57.352 INFO:tasks.workunit.client.1.smithi149.stdout:2/913: mknod d1/d4d/d69/d75/db6/c14b 0 2024-03-22T07:08:57.354 INFO:tasks.workunit.client.1.smithi149.stdout:9/811: creat db/d21/d37/ff6 x:0 0 0 2024-03-22T07:08:57.355 INFO:tasks.workunit.client.1.smithi149.stdout:6/880: symlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/l11f 0 2024-03-22T07:08:57.356 INFO:tasks.workunit.client.1.smithi149.stdout:2/914: mkdir d1/d8/d34/d14c 0 2024-03-22T07:08:57.357 INFO:tasks.workunit.client.1.smithi149.stdout:9/812: creat db/d21/d74/ff7 x:0 0 0 2024-03-22T07:08:57.359 INFO:tasks.workunit.client.1.smithi149.stdout:6/881: getdents d9/df/d22/d6d/d7d/d96/d24/d34 0 2024-03-22T07:08:57.359 INFO:tasks.workunit.client.1.smithi149.stdout:6/882: write d9/f19 [2179962,84044] 0 2024-03-22T07:08:57.360 INFO:tasks.workunit.client.1.smithi149.stdout:2/915: creat d1/d4d/f14d x:0 0 0 2024-03-22T07:08:57.362 INFO:tasks.workunit.client.1.smithi149.stdout:2/916: creat d1/d4d/d8b/d104/d12e/f14e x:0 0 0 2024-03-22T07:08:57.362 INFO:tasks.workunit.client.1.smithi149.stdout:2/917: readlink d1/d3/d5/d1f/d5e/l2c 0 2024-03-22T07:08:57.363 INFO:tasks.workunit.client.1.smithi149.stdout:1/830: dwrite d4/d7/f50 [0,4194304] 0 2024-03-22T07:08:57.363 INFO:tasks.workunit.client.1.smithi149.stdout:1/831: fdatasync f1 0 2024-03-22T07:08:57.367 INFO:tasks.workunit.client.1.smithi149.stdout:7/852: dwrite d1/d1c/d49/d8e/d100/d1b/d99/de5/ffd [0,4194304] 0 2024-03-22T07:08:57.367 INFO:tasks.workunit.client.1.smithi149.stdout:9/813: dread db/d19/f94 [0,4194304] 0 2024-03-22T07:08:57.370 INFO:tasks.workunit.client.1.smithi149.stdout:7/853: creat d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/f11d x:0 0 0 2024-03-22T07:08:57.370 INFO:tasks.workunit.client.1.smithi149.stdout:7/854: chown d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/db6/cb9 43405841 1 2024-03-22T07:08:57.370 INFO:tasks.workunit.client.1.smithi149.stdout:7/855: dread - d1/d1c/d49/d8e/d100/d1b/d99/faf zero size 2024-03-22T07:08:57.370 INFO:tasks.workunit.client.1.smithi149.stdout:7/856: write d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/f114 [630228,102054] 0 2024-03-22T07:08:57.370 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:57 smithi081 ceph-mon[29245]: pgmap v11: 65 pgs: 65 active+clean; 1.3 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 54 MiB/s rd, 72 MiB/s wr, 702 op/s 2024-03-22T07:08:57.370 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:57 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:57.370 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:57 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:57.371 INFO:tasks.workunit.client.1.smithi149.stdout:9/814: unlink db/d42/cee 0 2024-03-22T07:08:57.373 INFO:tasks.workunit.client.1.smithi149.stdout:7/857: rename d1/d1c/d49/d8e/d100/d1b/d99/faf to d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/db6/f11e 0 2024-03-22T07:08:57.374 INFO:tasks.workunit.client.1.smithi149.stdout:9/815: unlink db/d19/d28/ld1 0 2024-03-22T07:08:57.376 INFO:tasks.workunit.client.1.smithi149.stdout:7/858: link d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/f11d d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/f11f 0 2024-03-22T07:08:57.376 INFO:tasks.workunit.client.1.smithi149.stdout:9/816: symlink db/d42/dab/lf8 0 2024-03-22T07:08:57.378 INFO:tasks.workunit.client.1.smithi149.stdout:9/817: symlink db/d21/d9e/lf9 0 2024-03-22T07:08:57.378 INFO:tasks.workunit.client.1.smithi149.stdout:9/818: creat db/d19/ffa x:0 0 0 2024-03-22T07:08:57.380 INFO:tasks.workunit.client.1.smithi149.stdout:9/819: creat db/d12/ffb x:0 0 0 2024-03-22T07:08:57.381 INFO:tasks.workunit.client.1.smithi149.stdout:1/832: dread d4/d10/f63 [0,4194304] 0 2024-03-22T07:08:57.381 INFO:tasks.workunit.client.1.smithi149.stdout:1/833: write d4/d7/d5d/f72 [2038557,112010] 0 2024-03-22T07:08:57.383 INFO:tasks.workunit.client.1.smithi149.stdout:1/834: mknod d4/d7/d5a/dca/c112 0 2024-03-22T07:08:57.383 INFO:tasks.workunit.client.1.smithi149.stdout:1/835: readlink d4/d20/d2d/l66 0 2024-03-22T07:08:57.383 INFO:tasks.workunit.client.1.smithi149.stdout:1/836: creat d4/d10/d7c/d6a/f113 x:0 0 0 2024-03-22T07:08:57.383 INFO:tasks.workunit.client.1.smithi149.stdout:1/837: chown d4/d7/dd3 23439326 1 2024-03-22T07:08:57.384 INFO:tasks.workunit.client.1.smithi149.stdout:1/838: mkdir d4/d20/da0/d114 0 2024-03-22T07:08:57.384 INFO:tasks.workunit.client.1.smithi149.stdout:1/839: chown d4/d10/d58/d96/l10e 180 1 2024-03-22T07:08:57.385 INFO:tasks.workunit.client.1.smithi149.stdout:6/883: dwrite d9/df/d22/f95 [0,4194304] 0 2024-03-22T07:08:57.385 INFO:tasks.workunit.client.1.smithi149.stdout:6/884: write d9/df/d22/dc0/dc8/de9/f11c [289955,106164] 0 2024-03-22T07:08:57.385 INFO:tasks.workunit.client.1.smithi149.stdout:6/885: dread - d9/da2/f118 zero size 2024-03-22T07:08:57.385 INFO:tasks.workunit.client.1.smithi149.stdout:1/840: symlink d4/d10/dd8/d2f/l115 0 2024-03-22T07:08:57.387 INFO:tasks.workunit.client.1.smithi149.stdout:6/886: mknod d9/df/d22/d6d/d7d/d96/db1/c120 0 2024-03-22T07:08:57.388 INFO:tasks.workunit.client.1.smithi149.stdout:1/841: unlink d4/d10/d7c/d6a/d6b/f81 0 2024-03-22T07:08:57.389 INFO:tasks.workunit.client.1.smithi149.stdout:6/887: link d9/df/d22/d6d/d7d/d96/d24/l70 d9/df/d22/d6d/d7d/d96/d24/d34/d37/l121 0 2024-03-22T07:08:57.390 INFO:tasks.workunit.client.1.smithi149.stdout:6/888: symlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/l122 0 2024-03-22T07:08:57.399 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:57 smithi149 ceph-mon[36068]: pgmap v11: 65 pgs: 65 active+clean; 1.3 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 54 MiB/s rd, 72 MiB/s wr, 702 op/s 2024-03-22T07:08:57.399 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:57 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:57.399 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:57 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:08:57.401 INFO:tasks.workunit.client.1.smithi149.stdout:9/820: dread f3 [0,4194304] 0 2024-03-22T07:08:57.401 INFO:tasks.workunit.client.1.smithi149.stdout:2/918: dwrite d1/d4d/d8b/d104/d12e/f14e [0,4194304] 0 2024-03-22T07:08:57.403 INFO:tasks.workunit.client.1.smithi149.stdout:7/859: dwrite d1/d1c/d49/d8e/d53/fbd [0,4194304] 0 2024-03-22T07:08:57.403 INFO:tasks.workunit.client.1.smithi149.stdout:7/860: fdatasync d1/d1c/d49/d8e/d100/d1b/d99/de5/ffd 0 2024-03-22T07:08:57.403 INFO:tasks.workunit.client.1.smithi149.stdout:9/821: symlink db/d21/d37/d7c/da3/lfc 0 2024-03-22T07:08:57.403 INFO:tasks.workunit.client.1.smithi149.stdout:9/822: write db/d19/d39/d68/d6f/fb7 [501513,126127] 0 2024-03-22T07:08:57.406 INFO:tasks.workunit.client.1.smithi149.stdout:7/861: mkdir d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/dfe/d120 0 2024-03-22T07:08:57.406 INFO:tasks.workunit.client.1.smithi149.stdout:9/823: symlink db/d21/lfd 0 2024-03-22T07:08:57.407 INFO:tasks.workunit.client.1.smithi149.stdout:9/824: fsync db/d19/f7d 0 2024-03-22T07:08:57.408 INFO:tasks.workunit.client.1.smithi149.stdout:9/825: creat db/d42/ffe x:0 0 0 2024-03-22T07:08:57.409 INFO:tasks.workunit.client.1.smithi149.stdout:7/862: rename d1/d1c/d49/d8e/d100/d1b/d1d/d72/c9a to d1/d1c/c121 0 2024-03-22T07:08:57.409 INFO:tasks.workunit.client.1.smithi149.stdout:9/826: mkdir db/d42/d49/d4b/d5f/ddb/dff 0 2024-03-22T07:08:57.410 INFO:tasks.workunit.client.1.smithi149.stdout:7/863: symlink d1/d1c/d49/d8e/d100/l122 0 2024-03-22T07:08:57.410 INFO:tasks.workunit.client.1.smithi149.stdout:7/864: fdatasync d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/f103 0 2024-03-22T07:08:57.411 INFO:tasks.workunit.client.1.smithi149.stdout:9/827: dread db/d19/f94 [0,4194304] 0 2024-03-22T07:08:57.411 INFO:tasks.workunit.client.1.smithi149.stdout:9/828: fsync db/d22/f69 0 2024-03-22T07:08:57.411 INFO:tasks.workunit.client.1.smithi149.stdout:9/829: fsync db/d21/d37/d3a/f6c 0 2024-03-22T07:08:57.411 INFO:tasks.workunit.client.1.smithi149.stdout:7/865: creat d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/d10b/f123 x:0 0 0 2024-03-22T07:08:57.411 INFO:tasks.workunit.client.1.smithi149.stdout:7/866: dread - d1/d1c/d49/d8e/d100/d1b/d1d/d72/fd2 zero size 2024-03-22T07:08:57.416 INFO:tasks.workunit.client.1.smithi149.stdout:1/842: dwrite d4/d10/dd8/d2f/d93/f9c [0,4194304] 0 2024-03-22T07:08:57.416 INFO:tasks.workunit.client.1.smithi149.stdout:1/843: creat d4/d20/d2d/f116 x:0 0 0 2024-03-22T07:08:57.416 INFO:tasks.workunit.client.1.smithi149.stdout:7/867: dread d1/d1c/d49/d8e/d100/d1b/d1d/f1f [0,4194304] 0 2024-03-22T07:08:57.416 INFO:tasks.workunit.client.1.smithi149.stdout:6/889: dwrite d9/df/d22/d6d/d7d/d96/d24/d68/d113/f114 [0,4194304] 0 2024-03-22T07:08:57.419 INFO:tasks.workunit.client.1.smithi149.stdout:1/844: symlink d4/d20/da0/dc0/l117 0 2024-03-22T07:08:57.421 INFO:tasks.workunit.client.1.smithi149.stdout:9/830: dread db/d19/d39/d68/d6f/fb7 [0,4194304] 0 2024-03-22T07:08:57.421 INFO:tasks.workunit.client.1.smithi149.stdout:9/831: read - db/d42/fed zero size 2024-03-22T07:08:57.424 INFO:tasks.workunit.client.1.smithi149.stdout:9/832: symlink db/d19/d93/l100 0 2024-03-22T07:08:57.426 INFO:tasks.workunit.client.1.smithi149.stdout:9/833: creat db/d19/d28/de9/f101 x:0 0 0 2024-03-22T07:08:57.428 INFO:tasks.workunit.client.1.smithi149.stdout:7/868: write d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/fc5 [21820,117327] 0 2024-03-22T07:08:57.428 INFO:tasks.workunit.client.1.smithi149.stdout:7/869: chown d1/d1c/d49/d8e/d100/d1b/d11a 0 1 2024-03-22T07:08:57.428 INFO:tasks.workunit.client.1.smithi149.stdout:7/870: chown d1/d1c/d49/d8e/d100/d1b 9668685 1 2024-03-22T07:08:57.432 INFO:tasks.workunit.client.1.smithi149.stdout:6/890: write d9/df/d22/ff2 [4018761,95604] 0 2024-03-22T07:08:57.435 INFO:tasks.workunit.client.1.smithi149.stdout:6/891: mknod d9/df/d2e/d63/dfe/c123 0 2024-03-22T07:08:57.435 INFO:tasks.workunit.client.1.smithi149.stdout:1/845: dwrite d4/d10/dd8/d2f/fff [0,4194304] 0 2024-03-22T07:08:57.436 INFO:tasks.workunit.client.1.smithi149.stdout:1/846: dread - d4/d10/dd8/d2f/d93/fc9 zero size 2024-03-22T07:08:57.436 INFO:tasks.workunit.client.1.smithi149.stdout:1/847: readlink d4/d20/l25 0 2024-03-22T07:08:57.436 INFO:tasks.workunit.client.1.smithi149.stdout:6/892: dread d9/df/d22/dc0/dc8/de9/f11c [0,4194304] 0 2024-03-22T07:08:57.441 INFO:tasks.workunit.client.1.smithi149.stdout:1/848: rename d4/d20/da0/dc0/ff3 to d4/d7/d94/dd4/f118 0 2024-03-22T07:08:57.441 INFO:tasks.workunit.client.1.smithi149.stdout:1/849: fsync d4/d10/dd8/d3c/d8e/db5/f34 0 2024-03-22T07:08:57.443 INFO:tasks.workunit.client.1.smithi149.stdout:1/850: mknod d4/d10/dd8/d3c/d109/c119 0 2024-03-22T07:08:57.443 INFO:tasks.workunit.client.1.smithi149.stdout:1/851: truncate d4/d7/d61/ff0 408253 0 2024-03-22T07:08:57.446 INFO:tasks.workunit.client.1.smithi149.stdout:1/852: rename d4/l46 to d4/d20/d2d/d64/d9f/l11a 0 2024-03-22T07:08:57.446 INFO:tasks.workunit.client.1.smithi149.stdout:1/853: truncate d4/d10/dd8/d2f/f35 1393318 0 2024-03-22T07:08:57.447 INFO:tasks.workunit.client.1.smithi149.stdout:1/854: mkdir d4/d7/d61/d11b 0 2024-03-22T07:08:57.462 INFO:tasks.workunit.client.1.smithi149.stdout:9/834: dwrite db/d19/d39/d68/fac [0,4194304] 0 2024-03-22T07:08:57.465 INFO:tasks.workunit.client.1.smithi149.stdout:9/835: creat db/d19/d39/dd3/f102 x:0 0 0 2024-03-22T07:08:57.465 INFO:tasks.workunit.client.1.smithi149.stdout:9/836: chown db/d42/dab/le1 24202 1 2024-03-22T07:08:57.465 INFO:tasks.workunit.client.1.smithi149.stdout:9/837: readlink db/d19/d1c/l56 0 2024-03-22T07:08:57.468 INFO:tasks.workunit.client.1.smithi149.stdout:7/871: dwrite d1/d1c/d49/d8e/d100/d1b/f42 [4194304,4194304] 0 2024-03-22T07:08:57.468 INFO:tasks.workunit.client.1.smithi149.stdout:6/893: dwrite d9/d47/d52/fd7 [0,4194304] 0 2024-03-22T07:08:57.469 INFO:tasks.workunit.client.1.smithi149.stdout:6/894: chown d9/d47/d52/f8a 21 1 2024-03-22T07:08:57.470 INFO:tasks.workunit.client.1.smithi149.stdout:7/872: creat d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/dfe/f124 x:0 0 0 2024-03-22T07:08:57.474 INFO:tasks.workunit.client.1.smithi149.stdout:9/838: write db/d42/d49/df0/ff3 [3700931,125004] 0 2024-03-22T07:08:57.480 INFO:tasks.workunit.client.1.smithi149.stdout:6/895: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 [4885932,35066] 0 2024-03-22T07:08:57.481 INFO:tasks.workunit.client.1.smithi149.stdout:1/855: dwrite d4/f1b [0,4194304] 0 2024-03-22T07:08:57.481 INFO:tasks.workunit.client.1.smithi149.stdout:6/896: chown d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f43 3092160 1 2024-03-22T07:08:57.481 INFO:tasks.workunit.client.1.smithi149.stdout:6/897: read d9/df/d22/ff5 [421122,61154] 0 2024-03-22T07:08:57.481 INFO:tasks.workunit.client.1.smithi149.stdout:6/898: creat d9/df/d22/f124 x:0 0 0 2024-03-22T07:08:57.482 INFO:tasks.workunit.client.1.smithi149.stdout:9/839: rename db/d21/d2e/l3f to db/d21/d8c/l103 0 2024-03-22T07:08:57.483 INFO:tasks.workunit.client.1.smithi149.stdout:7/873: getdents d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7 0 2024-03-22T07:08:57.485 INFO:tasks.workunit.client.1.smithi149.stdout:1/856: truncate d4/d7/f57 1378878 0 2024-03-22T07:08:57.491 INFO:tasks.workunit.client.1.smithi149.stdout:6/899: rename d9/df/d22/d6d/d7d/d96/d24/d42/c102 to d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/c125 0 2024-03-22T07:08:57.492 INFO:tasks.workunit.client.1.smithi149.stdout:9/840: mkdir db/d21/d37/d3a/d104 0 2024-03-22T07:08:57.494 INFO:tasks.workunit.client.1.smithi149.stdout:1/857: creat d4/d7/f11c x:0 0 0 2024-03-22T07:08:57.495 INFO:tasks.workunit.client.1.smithi149.stdout:6/900: mknod d9/df/d22/d6d/d7d/d96/d24/d68/d113/d11d/c126 0 2024-03-22T07:08:57.497 INFO:tasks.workunit.client.1.smithi149.stdout:1/858: dread d4/d20/d2d/fa1 [0,4194304] 0 2024-03-22T07:08:57.501 INFO:tasks.workunit.client.1.smithi149.stdout:6/901: symlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/l127 0 2024-03-22T07:08:57.501 INFO:tasks.workunit.client.1.smithi149.stdout:1/859: dread d4/d20/d2d/faf [0,4194304] 0 2024-03-22T07:08:57.501 INFO:tasks.workunit.client.1.smithi149.stdout:8/823: sync 2024-03-22T07:08:57.502 INFO:tasks.workunit.client.1.smithi149.stdout:1/860: mknod d4/c11d 0 2024-03-22T07:08:57.502 INFO:tasks.workunit.client.1.smithi149.stdout:8/824: creat d1/d2/d9/db2/f109 x:0 0 0 2024-03-22T07:08:57.503 INFO:tasks.workunit.client.1.smithi149.stdout:1/861: link d4/d10/d7c/d6a/d6b/cfb d4/d7/d5d/d82/c11e 0 2024-03-22T07:08:57.503 INFO:tasks.workunit.client.1.smithi149.stdout:8/825: read d1/d2/d9/d11/d4d/dd6/d35/f6f [87128,69449] 0 2024-03-22T07:08:57.504 INFO:tasks.workunit.client.1.smithi149.stdout:5/804: sync 2024-03-22T07:08:57.514 INFO:tasks.workunit.client.1.smithi149.stdout:6/902: dread d9/da2/fc2 [0,4194304] 0 2024-03-22T07:08:57.514 INFO:tasks.workunit.client.1.smithi149.stdout:6/903: write d9/df/d22/d6d/d7d/d96/d24/d42/f75 [931169,94525] 0 2024-03-22T07:08:57.514 INFO:tasks.workunit.client.1.smithi149.stdout:6/904: write d9/df/fda [1857344,126235] 0 2024-03-22T07:08:57.518 INFO:tasks.workunit.client.1.smithi149.stdout:9/841: dwrite db/d42/d49/d4b/d4d/f55 [0,4194304] 0 2024-03-22T07:08:57.518 INFO:tasks.workunit.client.1.smithi149.stdout:9/842: creat db/d19/d1c/d85/f105 x:0 0 0 2024-03-22T07:08:57.527 INFO:tasks.workunit.client.1.smithi149.stdout:8/826: mkdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/d10a 0 2024-03-22T07:08:57.528 INFO:tasks.workunit.client.1.smithi149.stdout:8/827: creat d1/d2/d9/df/f10b x:0 0 0 2024-03-22T07:08:57.528 INFO:tasks.workunit.client.1.smithi149.stdout:8/828: truncate d1/d2/d9/d11/d4d/dd6/f66 4452868 0 2024-03-22T07:08:57.539 INFO:tasks.workunit.client.1.smithi149.stdout:5/805: dwrite d7/da/d20/d36/f91 [0,4194304] 0 2024-03-22T07:08:57.542 INFO:tasks.workunit.client.1.smithi149.stdout:1/862: dwrite d4/d7/d5d/f92 [0,4194304] 0 2024-03-22T07:08:57.544 INFO:tasks.workunit.client.1.smithi149.stdout:8/829: dwrite d1/d2/d9/d11/d4d/dd6/d51/fa6 [4194304,4194304] 0 2024-03-22T07:08:57.575 INFO:tasks.workunit.client.1.smithi149.stdout:6/905: mkdir d9/df/d22/dc0/dc8/d128 0 2024-03-22T07:08:57.596 INFO:tasks.workunit.client.1.smithi149.stdout:9/843: mknod db/d42/d49/d4b/d5f/c106 0 2024-03-22T07:08:57.602 INFO:tasks.workunit.client.1.smithi149.stdout:9/844: dread db/d19/d39/d3b/f48 [0,4194304] 0 2024-03-22T07:08:57.670 INFO:tasks.workunit.client.1.smithi149.stdout:8/830: creat d1/d2/d9/d11/d4d/dd6/db4/dce/f10c x:0 0 0 2024-03-22T07:08:57.671 INFO:tasks.workunit.client.1.smithi149.stdout:6/906: symlink d9/df/d22/dc0/dc8/l129 0 2024-03-22T07:08:57.672 INFO:tasks.workunit.client.1.smithi149.stdout:9/845: mknod db/d21/d8c/c107 0 2024-03-22T07:08:57.678 INFO:tasks.workunit.client.1.smithi149.stdout:1/863: mkdir d4/d7/d5a/d11f 0 2024-03-22T07:08:57.689 INFO:tasks.workunit.client.1.smithi149.stdout:8/831: dwrite d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f7e [0,4194304] 0 2024-03-22T07:08:57.707 INFO:tasks.workunit.client.1.smithi149.stdout:8/832: dwrite d1/d2/d9/df/dd5/ff8 [0,4194304] 0 2024-03-22T07:08:57.712 INFO:tasks.workunit.client.1.smithi149.stdout:0/746: sync 2024-03-22T07:08:57.727 INFO:tasks.workunit.client.1.smithi149.stdout:0/747: dwrite d1/d20/d9f/dc6/dd9/f105 [0,4194304] 0 2024-03-22T07:08:57.738 INFO:tasks.workunit.client.1.smithi149.stdout:4/849: sync 2024-03-22T07:08:57.738 INFO:tasks.workunit.client.1.smithi149.stdout:2/919: sync 2024-03-22T07:08:57.738 INFO:tasks.workunit.client.1.smithi149.stdout:3/669: sync 2024-03-22T07:08:57.738 INFO:tasks.workunit.client.1.smithi149.stdout:2/920: dread - d1/d4d/d8b/d129/d86/f107 zero size 2024-03-22T07:08:57.760 INFO:tasks.workunit.client.1.smithi149.stdout:4/850: dwrite d0/d8/dc/d4b/db3/dea/fad [4194304,4194304] 0 2024-03-22T07:08:57.766 INFO:tasks.workunit.client.1.smithi149.stdout:4/851: write d0/f32 [3536591,98996] 0 2024-03-22T07:08:57.881 INFO:tasks.workunit.client.1.smithi149.stdout:6/907: rmdir d9/d47 39 2024-03-22T07:08:57.882 INFO:tasks.workunit.client.1.smithi149.stdout:6/908: truncate d9/df/fda 3460983 0 2024-03-22T07:08:57.882 INFO:tasks.workunit.client.1.smithi149.stdout:9/846: creat db/f108 x:0 0 0 2024-03-22T07:08:57.882 INFO:tasks.workunit.client.1.smithi149.stdout:9/847: readlink l6 0 2024-03-22T07:08:57.883 INFO:tasks.workunit.client.1.smithi149.stdout:9/848: truncate db/d33/ff2 399496 0 2024-03-22T07:08:57.883 INFO:tasks.workunit.client.1.smithi149.stdout:9/849: creat db/d21/d37/d7c/dc7/f109 x:0 0 0 2024-03-22T07:08:57.883 INFO:tasks.workunit.client.1.smithi149.stdout:1/864: mknod d4/d10c/c120 0 2024-03-22T07:08:57.883 INFO:tasks.workunit.client.1.smithi149.stdout:9/850: dread db/d19/d1c/d85/fb6 [0,4194304] 0 2024-03-22T07:08:57.883 INFO:tasks.workunit.client.1.smithi149.stdout:9/851: readlink l6 0 2024-03-22T07:08:57.884 INFO:tasks.workunit.client.1.smithi149.stdout:9/852: dread - db/d21/d37/d7c/dc7/feb zero size 2024-03-22T07:08:57.884 INFO:tasks.workunit.client.1.smithi149.stdout:1/865: dread d4/d20/d2d/faf [0,4194304] 0 2024-03-22T07:08:57.884 INFO:tasks.workunit.client.1.smithi149.stdout:1/866: write d4/d10/dd8/d2f/f35 [263266,81950] 0 2024-03-22T07:08:57.884 INFO:tasks.workunit.client.1.smithi149.stdout:1/867: fdatasync d4/d20/d2d/d64/d9f/fbb 0 2024-03-22T07:08:57.885 INFO:tasks.workunit.client.1.smithi149.stdout:8/833: symlink d1/d2/d9/d11/d8b/dee/dff/l10d 0 2024-03-22T07:08:57.885 INFO:tasks.workunit.client.1.smithi149.stdout:8/834: write d1/d2/d9/f1d [1902019,44540] 0 2024-03-22T07:08:57.888 INFO:tasks.workunit.client.1.smithi149.stdout:0/748: rename d1/d31/d51/d64/d87/da6/dfe to d1/d31/d10f 0 2024-03-22T07:08:57.889 INFO:tasks.workunit.client.1.smithi149.stdout:0/749: write d1/d20/d9f/dc6/fcf [593506,19127] 0 2024-03-22T07:08:57.890 INFO:tasks.workunit.client.1.smithi149.stdout:3/670: link d0/d4/d8/f19 d0/d2/d30/dee/ffb 0 2024-03-22T07:08:57.890 INFO:tasks.workunit.client.1.smithi149.stdout:3/671: chown d0/d2/f83 66223 1 2024-03-22T07:08:57.892 INFO:tasks.workunit.client.1.smithi149.stdout:2/921: link d1/d72/l118 d1/d4d/l14f 0 2024-03-22T07:08:57.892 INFO:tasks.workunit.client.1.smithi149.stdout:2/922: truncate d1/d3/d6/db2/f5f 256500 0 2024-03-22T07:08:57.892 INFO:tasks.workunit.client.1.smithi149.stdout:2/923: fsync d1/d4d/d8b/f11e 0 2024-03-22T07:08:57.895 INFO:tasks.workunit.client.1.smithi149.stdout:4/852: truncate d0/d8/dc/d74/d76/fb8 2503068 0 2024-03-22T07:08:57.896 INFO:tasks.workunit.client.1.smithi149.stdout:4/853: read d0/d8/dc/d14/d3e/f51 [4049934,24070] 0 2024-03-22T07:08:57.904 INFO:tasks.workunit.client.1.smithi149.stdout:6/909: rename d9/df/d22/d6d/d117 to d9/df/d22/d6d/d7d/d96/db1/d12a 0 2024-03-22T07:08:57.907 INFO:tasks.workunit.client.1.smithi149.stdout:0/750: symlink d1/d20/d9f/dc6/ddc/l110 0 2024-03-22T07:08:57.909 INFO:tasks.workunit.client.1.smithi149.stdout:2/924: mkdir d1/d4d/d8b/d104/d12e/d134/d150 0 2024-03-22T07:08:57.909 INFO:tasks.workunit.client.1.smithi149.stdout:0/751: dread d1/d9/d59/f61 [4194304,4194304] 0 2024-03-22T07:08:57.910 INFO:tasks.workunit.client.1.smithi149.stdout:0/752: write d1/d9/d59/dc0/dc9/dcc/fe6 [358478,30216] 0 2024-03-22T07:08:57.910 INFO:tasks.workunit.client.1.smithi149.stdout:2/925: mkdir d1/d3/d5/d41/d9f/da7/d140/d151 0 2024-03-22T07:08:57.912 INFO:tasks.workunit.client.1.smithi149.stdout:0/753: mknod d1/d31/d10f/c111 0 2024-03-22T07:08:57.915 INFO:tasks.workunit.client.1.smithi149.stdout:9/853: rename db/d21/d37/d7c/lbe to db/l10a 0 2024-03-22T07:08:57.915 INFO:tasks.workunit.client.1.smithi149.stdout:9/854: write db/d19/d1c/d85/fb3 [663167,64551] 0 2024-03-22T07:08:57.917 INFO:tasks.workunit.client.1.smithi149.stdout:8/835: dwrite d1/fc3 [4194304,4194304] 0 2024-03-22T07:08:57.917 INFO:tasks.workunit.client.1.smithi149.stdout:8/836: creat d1/d2/d9/df/dd5/f10e x:0 0 0 2024-03-22T07:08:57.917 INFO:tasks.workunit.client.1.smithi149.stdout:8/837: stat d1/d2/d9/d11/d4d/dd6/c7d 0 2024-03-22T07:08:57.917 INFO:tasks.workunit.client.1.smithi149.stdout:8/838: creat d1/d2/d9/db2/f10f x:0 0 0 2024-03-22T07:08:57.917 INFO:tasks.workunit.client.1.smithi149.stdout:1/868: rename d4/d7/d5a/dca/lde to d4/d10/dd8/d59/l121 0 2024-03-22T07:08:57.918 INFO:tasks.workunit.client.1.smithi149.stdout:5/806: sync 2024-03-22T07:08:57.918 INFO:tasks.workunit.client.1.smithi149.stdout:7/874: sync 2024-03-22T07:08:57.918 INFO:tasks.workunit.client.1.smithi149.stdout:7/875: fdatasync d1/d1c/f9f 0 2024-03-22T07:08:57.919 INFO:tasks.workunit.client.1.smithi149.stdout:3/672: dwrite d0/d2/d30/f73 [0,4194304] 0 2024-03-22T07:08:57.919 INFO:tasks.workunit.client.1.smithi149.stdout:9/855: mknod db/d21/d37/d3a/d104/c10b 0 2024-03-22T07:08:57.920 INFO:tasks.workunit.client.1.smithi149.stdout:6/910: dread d9/d47/d52/f8a [0,4194304] 0 2024-03-22T07:08:57.922 INFO:tasks.workunit.client.1.smithi149.stdout:7/876: write d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/d9e/dab/fb2 [101084,91207] 0 2024-03-22T07:08:57.922 INFO:tasks.workunit.client.1.smithi149.stdout:7/877: readlink d1/d1c/d49/d8e/d100/d1b/l69 0 2024-03-22T07:08:57.930 INFO:tasks.workunit.client.1.smithi149.stdout:2/926: dread d1/d3/de0/f130 [0,4194304] 0 2024-03-22T07:08:57.930 INFO:tasks.workunit.client.1.smithi149.stdout:2/927: write d1/d3/d5/d1f/f55 [3991230,78007] 0 2024-03-22T07:08:57.930 INFO:tasks.workunit.client.1.smithi149.stdout:4/854: dwrite d0/d8/dc/d14/d85/ff7 [0,4194304] 0 2024-03-22T07:08:57.935 INFO:tasks.workunit.client.1.smithi149.stdout:8/839: dread d1/d2/d3/f96 [0,4194304] 0 2024-03-22T07:08:57.935 INFO:tasks.workunit.client.1.smithi149.stdout:8/840: chown d1/d2/d18/ld1 27278 1 2024-03-22T07:08:57.935 INFO:tasks.workunit.client.1.smithi149.stdout:8/841: truncate d1/d2/d9/d11/d4d/dd6/db4/dce/f10c 117976 0 2024-03-22T07:08:57.937 INFO:tasks.workunit.client.1.smithi149.stdout:7/878: dread d1/d1c/d49/d8e/d100/d1b/d1d/fd [0,4194304] 0 2024-03-22T07:08:57.939 INFO:tasks.workunit.client.1.smithi149.stdout:0/754: dwrite d1/d9/d59/dc0/dc9/ff8 [0,4194304] 0 2024-03-22T07:08:58.067 INFO:tasks.workunit.client.1.smithi149.stdout:1/869: rename d4/f42 to d4/d7/d94/dd4/f122 0 2024-03-22T07:08:58.070 INFO:tasks.workunit.client.1.smithi149.stdout:5/807: mknod d7/da/d20/d36/c11a 0 2024-03-22T07:08:58.070 INFO:tasks.workunit.client.1.smithi149.stdout:5/808: truncate d7/da/d20/d45/ffc 88948 0 2024-03-22T07:08:58.070 INFO:tasks.workunit.client.1.smithi149.stdout:5/809: write d7/da/d20/d45/f9c [381401,42206] 0 2024-03-22T07:08:58.071 INFO:tasks.workunit.client.1.smithi149.stdout:3/673: creat d0/d2/d30/d7e/db8/ffc x:0 0 0 2024-03-22T07:08:58.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/911: truncate d9/df/d22/f58 4531431 0 2024-03-22T07:08:58.074 INFO:tasks.workunit.client.1.smithi149.stdout:6/912: chown d9/df/d22/d6d/d7d/d96/d24/fa9 35 1 2024-03-22T07:08:58.076 INFO:tasks.workunit.client.1.smithi149.stdout:2/928: symlink d1/d4d/d8b/d104/d12e/d134/d150/l152 0 2024-03-22T07:08:58.077 INFO:tasks.workunit.client.1.smithi149.stdout:1/870: dread d4/d10/dd8/d3c/d8e/db5/fb8 [0,4194304] 0 2024-03-22T07:08:58.080 INFO:tasks.workunit.client.1.smithi149.stdout:4/855: link d0/d8/c4e d0/d8/dc/d14/c115 0 2024-03-22T07:08:58.081 INFO:tasks.workunit.client.1.smithi149.stdout:7/879: creat d1/d1c/d49/d8e/d53/f125 x:0 0 0 2024-03-22T07:08:58.085 INFO:tasks.workunit.client.1.smithi149.stdout:0/755: symlink d1/d31/d51/d64/d7e/l112 0 2024-03-22T07:08:58.085 INFO:tasks.workunit.client.1.smithi149.stdout:0/756: creat d1/d31/d52/da9/f113 x:0 0 0 2024-03-22T07:08:58.087 INFO:tasks.workunit.client.1.smithi149.stdout:8/842: rename d1/d2/d9/d11/d4d/cb8 to d1/d2/d9/d11/d8b/d8d/d91/c110 0 2024-03-22T07:08:58.087 INFO:tasks.workunit.client.1.smithi149.stdout:8/843: creat d1/d2/d9/d11/d4d/dd6/d51/f111 x:0 0 0 2024-03-22T07:08:58.087 INFO:tasks.workunit.client.1.smithi149.stdout:8/844: fdatasync d1/d2/d3/f86 0 2024-03-22T07:08:58.088 INFO:tasks.workunit.client.1.smithi149.stdout:4/856: dread d0/d8/dc/d4b/db3/dea/fad [0,4194304] 0 2024-03-22T07:08:58.088 INFO:tasks.workunit.client.1.smithi149.stdout:5/810: creat d7/da/d20/d30/d3a/f11b x:0 0 0 2024-03-22T07:08:58.089 INFO:tasks.workunit.client.1.smithi149.stdout:7/880: dread d1/d1c/d49/d8e/d100/d1b/d51/d5a/d6a/f6e [0,4194304] 0 2024-03-22T07:08:58.089 INFO:tasks.workunit.client.1.smithi149.stdout:3/674: symlink d0/lfd 0 2024-03-22T07:08:58.091 INFO:tasks.workunit.client.1.smithi149.stdout:9/856: rmdir db/d19/d28 39 2024-03-22T07:08:58.091 INFO:tasks.workunit.client.1.smithi149.stdout:9/857: chown db/d21/d9e/cb8 111235 1 2024-03-22T07:08:58.091 INFO:tasks.workunit.client.1.smithi149.stdout:9/858: dread - db/d21/d37/d7c/dc7/f109 zero size 2024-03-22T07:08:58.091 INFO:tasks.workunit.client.1.smithi149.stdout:9/859: dread - db/d42/ffe zero size 2024-03-22T07:08:58.093 INFO:tasks.workunit.client.1.smithi149.stdout:6/913: symlink d9/df/d22/d6d/d7d/d96/d24/d34/dac/l12b 0 2024-03-22T07:08:58.094 INFO:tasks.workunit.client.1.smithi149.stdout:2/929: mkdir d1/d8/d59/df6/d153 0 2024-03-22T07:08:58.094 INFO:tasks.workunit.client.1.smithi149.stdout:2/930: truncate d1/d3/f133 109982 0 2024-03-22T07:08:58.096 INFO:tasks.workunit.client.1.smithi149.stdout:1/871: creat d4/d10/d7c/d6a/d6b/f123 x:0 0 0 2024-03-22T07:08:58.101 INFO:tasks.workunit.client.1.smithi149.stdout:0/757: symlink d1/d27/d66/df1/l114 0 2024-03-22T07:08:58.101 INFO:tasks.workunit.client.1.smithi149.stdout:8/845: mknod d1/d2/d9/d11/d4d/dd6/db4/c112 0 2024-03-22T07:08:58.101 INFO:tasks.workunit.client.1.smithi149.stdout:8/846: fdatasync d1/d2/d9/d11/d4d/dd6/d51/fa2 0 2024-03-22T07:08:58.104 INFO:tasks.workunit.client.1.smithi149.stdout:4/857: mknod d0/d8/dc/dbd/c116 0 2024-03-22T07:08:58.106 INFO:tasks.workunit.client.1.smithi149.stdout:7/881: creat d1/d1c/d49/d8e/d100/d1b/d11a/f126 x:0 0 0 2024-03-22T07:08:58.106 INFO:tasks.workunit.client.1.smithi149.stdout:3/675: unlink d0/d2/cf6 0 2024-03-22T07:08:58.109 INFO:tasks.workunit.client.1.smithi149.stdout:9/860: rename db/d21/d37/d7c/da3/lfc to db/d21/d37/d3a/d104/l10c 0 2024-03-22T07:08:58.109 INFO:tasks.workunit.client.1.smithi149.stdout:1/872: mknod d4/d7/d5d/d82/c124 0 2024-03-22T07:08:58.109 INFO:tasks.workunit.client.1.smithi149.stdout:9/861: chown db/d21/fd6 13111 1 2024-03-22T07:08:58.110 INFO:tasks.workunit.client.1.smithi149.stdout:1/873: fsync d4/d20/d2d/f116 0 2024-03-22T07:08:58.113 INFO:tasks.workunit.client.1.smithi149.stdout:0/758: dread d1/d27/f62 [0,4194304] 0 2024-03-22T07:08:58.114 INFO:tasks.workunit.client.1.smithi149.stdout:7/882: creat d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/f127 x:0 0 0 2024-03-22T07:08:58.116 INFO:tasks.workunit.client.1.smithi149.stdout:9/862: rename db/d42/d49/d4b/d5f/lcb to db/d21/d9e/l10d 0 2024-03-22T07:08:58.116 INFO:tasks.workunit.client.1.smithi149.stdout:9/863: chown db/d21/d8c 19045807 1 2024-03-22T07:08:58.116 INFO:tasks.workunit.client.1.smithi149.stdout:9/864: write db/d19/d39/d3b/f48 [3348196,27491] 0 2024-03-22T07:08:58.117 INFO:tasks.workunit.client.1.smithi149.stdout:0/759: getdents d1/d31/d51/d64/d8d/d9c/d9e 0 2024-03-22T07:08:58.117 INFO:tasks.workunit.client.1.smithi149.stdout:0/760: dread - d1/d31/d51/d64/d87/f10a zero size 2024-03-22T07:08:58.117 INFO:tasks.workunit.client.1.smithi149.stdout:0/761: fsync d1/d31/f8a 0 2024-03-22T07:08:58.117 INFO:tasks.workunit.client.1.smithi149.stdout:0/762: fsync d1/d9/d59/fba 0 2024-03-22T07:08:58.118 INFO:tasks.workunit.client.1.smithi149.stdout:0/763: chown d1/d6b/c79 1 1 2024-03-22T07:08:58.118 INFO:tasks.workunit.client.1.smithi149.stdout:3/676: write d0/d4/f58 [94430,1982] 0 2024-03-22T07:08:58.119 INFO:tasks.workunit.client.1.smithi149.stdout:7/883: link d1/d1c/d49/d8e/d100/d30/c84 d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/c128 0 2024-03-22T07:08:58.119 INFO:tasks.workunit.client.1.smithi149.stdout:7/884: write d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/f77 [2012783,128777] 0 2024-03-22T07:08:58.120 INFO:tasks.workunit.client.1.smithi149.stdout:6/914: rename f5 to d9/df/d22/d6d/d7d/d96/d24/d100/f12c 0 2024-03-22T07:08:58.120 INFO:tasks.workunit.client.1.smithi149.stdout:6/915: creat d9/df/d22/dc0/dc8/de9/f12d x:0 0 0 2024-03-22T07:08:58.125 INFO:tasks.workunit.client.1.smithi149.stdout:0/764: link d1/d31/d51/d64/d87/f9d d1/d20/da3/f115 0 2024-03-22T07:08:58.127 INFO:tasks.workunit.client.1.smithi149.stdout:1/874: dread d4/d10/dd8/d3c/d8e/db5/f53 [0,4194304] 0 2024-03-22T07:08:58.127 INFO:tasks.workunit.client.1.smithi149.stdout:1/875: write d4/f69 [4692038,37989] 0 2024-03-22T07:08:58.127 INFO:tasks.workunit.client.1.smithi149.stdout:1/876: stat d4/d20/d2d/d64/d9f/cab 0 2024-03-22T07:08:58.127 INFO:tasks.workunit.client.1.smithi149.stdout:3/677: mkdir d0/d2/dfe 0 2024-03-22T07:08:58.127 INFO:tasks.workunit.client.1.smithi149.stdout:3/678: stat d0/d4/d8/d24/la8 0 2024-03-22T07:08:58.127 INFO:tasks.workunit.client.1.smithi149.stdout:7/885: write d1/f21 [249948,51102] 0 2024-03-22T07:08:58.128 INFO:tasks.workunit.client.1.smithi149.stdout:2/931: dwrite d1/d4d/d8b/d104/d12e/f14e [0,4194304] 0 2024-03-22T07:08:58.128 INFO:tasks.workunit.client.1.smithi149.stdout:5/811: rmdir d7/da/d20/d30/d3a/d93/d94/dc2 0 2024-03-22T07:08:58.128 INFO:tasks.workunit.client.1.smithi149.stdout:5/812: readlink d7/d35/l50 0 2024-03-22T07:08:58.128 INFO:tasks.workunit.client.1.smithi149.stdout:5/813: fsync d7/da/d7d/f116 0 2024-03-22T07:08:58.129 INFO:tasks.workunit.client.1.smithi149.stdout:8/847: dwrite d1/d2/d9/d11/d4d/dd6/d51/fe2 [0,4194304] 0 2024-03-22T07:08:58.131 INFO:tasks.workunit.client.1.smithi149.stdout:4/858: rename d0/d8/dc/d14/d3e/ccd to d0/d8/d1d/df1/c117 0 2024-03-22T07:08:58.133 INFO:tasks.workunit.client.1.smithi149.stdout:6/916: unlink d9/df/d22/d6d/d7d/d96/d24/d68/c109 0 2024-03-22T07:08:58.134 INFO:tasks.workunit.client.1.smithi149.stdout:9/865: write db/d33/f67 [1134678,70013] 0 2024-03-22T07:08:58.134 INFO:tasks.workunit.client.1.smithi149.stdout:9/866: chown db/d21/d37/d3a/d104/c10b 8347597 1 2024-03-22T07:08:58.136 INFO:tasks.workunit.client.1.smithi149.stdout:1/877: creat d4/d10/dd8/d3c/d109/f125 x:0 0 0 2024-03-22T07:08:58.136 INFO:tasks.workunit.client.1.smithi149.stdout:1/878: fsync d4/d10/d7c/f7d 0 2024-03-22T07:08:58.136 INFO:tasks.workunit.client.1.smithi149.stdout:3/679: write d0/d4/d8/f8d [2047902,13684] 0 2024-03-22T07:08:58.141 INFO:tasks.workunit.client.1.smithi149.stdout:2/932: creat d1/d3/d5/d41/d9f/f154 x:0 0 0 2024-03-22T07:08:58.142 INFO:tasks.workunit.client.1.smithi149.stdout:5/814: readlink d7/d35/l7a 0 2024-03-22T07:08:58.142 INFO:tasks.workunit.client.1.smithi149.stdout:5/815: dread - d7/da/d20/d30/d3a/f101 zero size 2024-03-22T07:08:58.145 INFO:tasks.workunit.client.1.smithi149.stdout:4/859: write d0/d8/dc/f13 [691233,58604] 0 2024-03-22T07:08:58.147 INFO:tasks.workunit.client.1.smithi149.stdout:6/917: creat d9/df/d22/d6d/d7d/d96/d24/d34/dac/f12e x:0 0 0 2024-03-22T07:08:58.147 INFO:tasks.workunit.client.1.smithi149.stdout:0/765: rmdir d1/d1a 39 2024-03-22T07:08:58.150 INFO:tasks.workunit.client.1.smithi149.stdout:9/867: creat db/d19/d1c/f10e x:0 0 0 2024-03-22T07:08:58.150 INFO:tasks.workunit.client.1.smithi149.stdout:9/868: chown db/d21/d9e/df4 23273445 1 2024-03-22T07:08:58.151 INFO:tasks.workunit.client.1.smithi149.stdout:3/680: mknod d0/d2/d30/dee/cff 0 2024-03-22T07:08:58.151 INFO:tasks.workunit.client.1.smithi149.stdout:0/766: dread d1/d9/d14/d74/d77/f9a [0,4194304] 0 2024-03-22T07:08:58.156 INFO:tasks.workunit.client.1.smithi149.stdout:5/816: rename d7/da/fee to d7/da/ded/f11c 0 2024-03-22T07:08:58.156 INFO:tasks.workunit.client.1.smithi149.stdout:5/817: readlink d7/da/d20/d30/d3a/d3d/d2c/l3e 0 2024-03-22T07:08:58.156 INFO:tasks.workunit.client.1.smithi149.stdout:9/869: mkdir db/d21/d10f 0 2024-03-22T07:08:58.158 INFO:tasks.workunit.client.1.smithi149.stdout:5/818: rmdir d7/da/d20/d30/d3a/d3d/d104 39 2024-03-22T07:08:58.162 INFO:tasks.workunit.client.1.smithi149.stdout:4/860: rename d0/d8/dc/d14/d3e/f6a to d0/d8/dc/d74/f118 0 2024-03-22T07:08:58.162 INFO:tasks.workunit.client.1.smithi149.stdout:4/861: write d0/d8/d1d/d24/d82/fa1 [5034766,94077] 0 2024-03-22T07:08:58.163 INFO:tasks.workunit.client.1.smithi149.stdout:9/870: creat db/d21/d9e/df4/f110 x:0 0 0 2024-03-22T07:08:58.164 INFO:tasks.workunit.client.1.smithi149.stdout:6/918: write d9/df/d2e/f1c [1733500,24408] 0 2024-03-22T07:08:58.164 INFO:tasks.workunit.client.1.smithi149.stdout:5/819: creat d7/da/d20/d89/da2/dda/f11d x:0 0 0 2024-03-22T07:08:58.165 INFO:tasks.workunit.client.1.smithi149.stdout:1/879: dwrite d4/d10/dd8/f44 [0,4194304] 0 2024-03-22T07:08:58.165 INFO:tasks.workunit.client.1.smithi149.stdout:1/880: fdatasync d4/d10/dd8/f74 0 2024-03-22T07:08:58.166 INFO:tasks.workunit.client.1.smithi149.stdout:4/862: mkdir d0/d8/dc/d4b/d119 0 2024-03-22T07:08:58.166 INFO:tasks.workunit.client.1.smithi149.stdout:9/871: symlink db/d19/d28/l111 0 2024-03-22T07:08:58.168 INFO:tasks.workunit.client.1.smithi149.stdout:6/919: symlink d9/df/d22/d6d/d7d/d96/l12f 0 2024-03-22T07:08:58.168 INFO:tasks.workunit.client.1.smithi149.stdout:0/767: write d1/d31/f8a [3242357,2233] 0 2024-03-22T07:08:58.169 INFO:tasks.workunit.client.1.smithi149.stdout:5/820: mkdir d7/da/dff/d11e 0 2024-03-22T07:08:58.169 INFO:tasks.workunit.client.1.smithi149.stdout:5/821: fsync d7/d35/d8e/ff1 0 2024-03-22T07:08:58.169 INFO:tasks.workunit.client.1.smithi149.stdout:5/822: dread - d7/d35/de0/dd7/d7b/d81/f90 zero size 2024-03-22T07:08:58.172 INFO:tasks.workunit.client.1.smithi149.stdout:1/881: link d4/d20/d2d/d64/d9f/l11a d4/d7/dd3/l126 0 2024-03-22T07:08:58.172 INFO:tasks.workunit.client.1.smithi149.stdout:1/882: fdatasync d4/d10/dd8/d2f/d8c/ff8 0 2024-03-22T07:08:58.172 INFO:tasks.workunit.client.1.smithi149.stdout:1/883: dread - d4/d7/d94/dd4/f118 zero size 2024-03-22T07:08:58.173 INFO:tasks.workunit.client.1.smithi149.stdout:2/933: dwrite d1/d3/d5/d1f/d68/d94/fd4 [0,4194304] 0 2024-03-22T07:08:58.174 INFO:tasks.workunit.client.1.smithi149.stdout:9/872: mkdir db/d19/d112 0 2024-03-22T07:08:58.175 INFO:tasks.workunit.client.1.smithi149.stdout:6/920: rmdir d9/df/d22/d6d/d7d/d96/db1/d12a 39 2024-03-22T07:08:58.178 INFO:tasks.workunit.client.1.smithi149.stdout:1/884: creat d4/d7/d5a/f127 x:0 0 0 2024-03-22T07:08:58.178 INFO:tasks.workunit.client.1.smithi149.stdout:1/885: chown d4/d10/dd8/d3c/d8e/db5/fb8 36742 1 2024-03-22T07:08:58.183 INFO:tasks.workunit.client.1.smithi149.stdout:4/863: dread d0/d8/d4c/f57 [0,4194304] 0 2024-03-22T07:08:58.183 INFO:tasks.workunit.client.1.smithi149.stdout:4/864: write d0/d8/dc/d74/fdd [1048524,103366] 0 2024-03-22T07:08:58.183 INFO:tasks.workunit.client.1.smithi149.stdout:4/865: dread - d0/d8/dc/d74/fe3 zero size 2024-03-22T07:08:58.187 INFO:tasks.workunit.client.1.smithi149.stdout:7/886: truncate d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/d9e/dab/fb2 20053 0 2024-03-22T07:08:58.188 INFO:tasks.workunit.client.1.smithi149.stdout:7/887: truncate d1/d1c/fbb 491227 0 2024-03-22T07:08:58.188 INFO:tasks.workunit.client.1.smithi149.stdout:1/886: creat d4/d10/d58/d68/f128 x:0 0 0 2024-03-22T07:08:58.195 INFO:tasks.workunit.client.1.smithi149.stdout:4/866: mknod d0/d8/d1d/d24/d9c/c11a 0 2024-03-22T07:08:58.195 INFO:tasks.workunit.client.1.smithi149.stdout:7/888: creat d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/f129 x:0 0 0 2024-03-22T07:08:58.197 INFO:tasks.workunit.client.1.smithi149.stdout:8/848: dwrite d1/d2/d9/d11/d4d/dd6/d35/d9d/fe5 [0,4194304] 0 2024-03-22T07:08:58.197 INFO:tasks.workunit.client.1.smithi149.stdout:8/849: rename d1 to d1/d2/d9/d11/d8b/d113 22 2024-03-22T07:08:58.197 INFO:tasks.workunit.client.1.smithi149.stdout:8/850: chown d1/d2/d9/df/dd5/def 510 1 2024-03-22T07:08:58.197 INFO:tasks.workunit.client.1.smithi149.stdout:8/851: stat d1/d2/d9/d11/d4d/dd6/db4/ccb 0 2024-03-22T07:08:58.197 INFO:tasks.workunit.client.1.smithi149.stdout:8/852: creat d1/d2/d9/d11/d4d/dd6/db4/dce/f114 x:0 0 0 2024-03-22T07:08:58.203 INFO:tasks.workunit.client.1.smithi149.stdout:5/823: dread d7/f58 [0,4194304] 0 2024-03-22T07:08:58.205 INFO:tasks.workunit.client.1.smithi149.stdout:5/824: unlink d7/da/d7d/ff3 0 2024-03-22T07:08:58.207 INFO:tasks.workunit.client.1.smithi149.stdout:5/825: link d7/da/d20/d36/d44/f5d d7/da/d20/d30/d3a/d3d/d2c/db2/f11f 0 2024-03-22T07:08:58.210 INFO:tasks.workunit.client.1.smithi149.stdout:5/826: unlink d7/da/d20/d45/ffc 0 2024-03-22T07:08:58.214 INFO:tasks.workunit.client.1.smithi149.stdout:5/827: symlink d7/da/d20/d30/d3a/d3d/l120 0 2024-03-22T07:08:58.224 INFO:tasks.workunit.client.1.smithi149.stdout:1/887: dread d4/d10/dd8/d3c/d8e/db5/f83 [0,4194304] 0 2024-03-22T07:08:58.224 INFO:tasks.workunit.client.1.smithi149.stdout:0/768: dwrite d1/d20/d9f/dc6/dd9/fda [0,4194304] 0 2024-03-22T07:08:58.226 INFO:tasks.workunit.client.1.smithi149.stdout:2/934: dwrite d1/d3/d5/d1f/f1e [0,4194304] 0 2024-03-22T07:08:58.226 INFO:tasks.workunit.client.1.smithi149.stdout:0/769: mknod d1/d27/c116 0 2024-03-22T07:08:58.231 INFO:tasks.workunit.client.1.smithi149.stdout:9/873: dwrite db/d19/d1c/f10e [0,4194304] 0 2024-03-22T07:08:58.231 INFO:tasks.workunit.client.1.smithi149.stdout:0/770: truncate d1/f4a 1765199 0 2024-03-22T07:08:58.232 INFO:tasks.workunit.client.1.smithi149.stdout:1/888: rename d4/d10/dd8/d2f to d4/d7/dd3/d129 0 2024-03-22T07:08:58.232 INFO:tasks.workunit.client.1.smithi149.stdout:1/889: fsync d4/d7/d5a/f5e 0 2024-03-22T07:08:58.235 INFO:tasks.workunit.client.1.smithi149.stdout:0/771: rmdir d1/d27/d66/df1 39 2024-03-22T07:08:58.237 INFO:tasks.workunit.client.1.smithi149.stdout:1/890: rename d4/d20/d2d/d64/d9f/cc8 to d4/d7/d94/dd4/c12a 0 2024-03-22T07:08:58.237 INFO:tasks.workunit.client.1.smithi149.stdout:1/891: stat d4/d20/d2d/l66 0 2024-03-22T07:08:58.238 INFO:tasks.workunit.client.1.smithi149.stdout:2/935: write d1/d3/d5/d1f/f84 [883734,44115] 0 2024-03-22T07:08:58.238 INFO:tasks.workunit.client.1.smithi149.stdout:0/772: unlink d1/d9/d14/d74/d77/fb2 0 2024-03-22T07:08:58.238 INFO:tasks.workunit.client.1.smithi149.stdout:0/773: readlink d1/d20/ldf 0 2024-03-22T07:08:58.239 INFO:tasks.workunit.client.1.smithi149.stdout:1/892: rename d4/d10/dd8/l29 to d4/d20/d2d/d64/d104/l12b 0 2024-03-22T07:08:58.240 INFO:tasks.workunit.client.1.smithi149.stdout:1/893: write d4/d20/d2d/d64/d9f/fbb [2700231,122380] 0 2024-03-22T07:08:58.241 INFO:tasks.workunit.client.1.smithi149.stdout:0/774: symlink d1/d31/d51/d64/d8d/l117 0 2024-03-22T07:08:58.244 INFO:tasks.workunit.client.1.smithi149.stdout:0/775: creat d1/d1a/f118 x:0 0 0 2024-03-22T07:08:58.252 INFO:tasks.workunit.client.1.smithi149.stdout:5/828: dread d7/d35/de0/dd7/f52 [0,4194304] 0 2024-03-22T07:08:58.259 INFO:tasks.workunit.client.1.smithi149.stdout:2/936: dread d1/d72/f9e [0,4194304] 0 2024-03-22T07:08:58.259 INFO:tasks.workunit.client.1.smithi149.stdout:6/921: dwrite d9/df/d22/d6d/deb/fdb [0,4194304] 0 2024-03-22T07:08:58.261 INFO:tasks.workunit.client.1.smithi149.stdout:6/922: rmdir d9/df/d22/dc0 39 2024-03-22T07:08:58.261 INFO:tasks.workunit.client.1.smithi149.stdout:6/923: creat d9/df/d2e/d63/f130 x:0 0 0 2024-03-22T07:08:58.262 INFO:tasks.workunit.client.1.smithi149.stdout:2/937: mkdir d1/d3/de0/d155 0 2024-03-22T07:08:58.263 INFO:tasks.workunit.client.1.smithi149.stdout:6/924: creat d9/d47/f131 x:0 0 0 2024-03-22T07:08:58.264 INFO:tasks.workunit.client.1.smithi149.stdout:2/938: truncate d1/d3/d5/d1f/d68/fb4 834864 0 2024-03-22T07:08:58.264 INFO:tasks.workunit.client.1.smithi149.stdout:6/925: symlink d9/df/d22/d6d/d7d/d96/db1/d12a/l132 0 2024-03-22T07:08:58.266 INFO:tasks.workunit.client.1.smithi149.stdout:0/776: write d1/d9/d59/ff9 [1680912,68520] 0 2024-03-22T07:08:58.266 INFO:tasks.workunit.client.1.smithi149.stdout:0/777: write d1/d20/da3/f115 [8990690,119864] 0 2024-03-22T07:08:58.266 INFO:tasks.workunit.client.1.smithi149.stdout:2/939: mkdir d1/d3/d5/d1f/dfc/d156 0 2024-03-22T07:08:58.266 INFO:tasks.workunit.client.1.smithi149.stdout:2/940: chown d1/d4d/d69/d75/d7e/fd6 21 1 2024-03-22T07:08:58.266 INFO:tasks.workunit.client.1.smithi149.stdout:2/941: chown d1/d4d/d8b/d129/l85 318425 1 2024-03-22T07:08:58.266 INFO:tasks.workunit.client.1.smithi149.stdout:2/942: fsync d1/d72/f9e 0 2024-03-22T07:08:58.267 INFO:tasks.workunit.client.1.smithi149.stdout:9/874: dwrite db/d33/f67 [0,4194304] 0 2024-03-22T07:08:58.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/926: link d9/df/d22/d6d/d7d/d96/d24/d42/l64 d9/df/d22/d6d/d7d/d96/d24/d34/l133 0 2024-03-22T07:08:58.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/927: chown d9/d47/d52/fd7 6 1 2024-03-22T07:08:58.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/928: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 [6315260,20100] 0 2024-03-22T07:08:58.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/929: write d9/da2/fcb [790034,71164] 0 2024-03-22T07:08:58.267 INFO:tasks.workunit.client.1.smithi149.stdout:6/930: chown d9/df/d22/d6d/d7d/d96/d24/fa9 1476 1 2024-03-22T07:08:58.268 INFO:tasks.workunit.client.1.smithi149.stdout:6/931: read d9/df/d2e/d63/fb3 [745885,125816] 0 2024-03-22T07:08:58.269 INFO:tasks.workunit.client.1.smithi149.stdout:4/867: dwrite d0/d8/dc/d14/d3e/fcc [0,4194304] 0 2024-03-22T07:08:58.270 INFO:tasks.workunit.client.1.smithi149.stdout:8/853: dwrite d1/d2/d9/d11/d4d/dd6/f72 [0,4194304] 0 2024-03-22T07:08:58.270 INFO:tasks.workunit.client.1.smithi149.stdout:9/875: creat db/d42/f113 x:0 0 0 2024-03-22T07:08:58.271 INFO:tasks.workunit.client.1.smithi149.stdout:0/778: rmdir d1/d20/da3 39 2024-03-22T07:08:58.271 INFO:tasks.workunit.client.1.smithi149.stdout:0/779: truncate d1/d9/f10e 367757 0 2024-03-22T07:08:58.271 INFO:tasks.workunit.client.1.smithi149.stdout:0/780: write d1/d9/d59/f5d [584975,28661] 0 2024-03-22T07:08:58.271 INFO:tasks.workunit.client.1.smithi149.stdout:0/781: chown d1/d6b/la4 290637 1 2024-03-22T07:08:58.272 INFO:tasks.workunit.client.1.smithi149.stdout:0/782: readlink d1/d31/d51/d64/d7e/l112 0 2024-03-22T07:08:58.275 INFO:tasks.workunit.client.1.smithi149.stdout:6/932: mkdir d9/df/d22/d6d/d7d/d96/d24/d34/d37/d134 0 2024-03-22T07:08:58.276 INFO:tasks.workunit.client.1.smithi149.stdout:7/889: dwrite d1/d1c/f73 [0,4194304] 0 2024-03-22T07:08:58.276 INFO:tasks.workunit.client.1.smithi149.stdout:7/890: chown d1/d1c/d49/d8e/d100/d1b/d1d/df6 7438448 1 2024-03-22T07:08:58.277 INFO:tasks.workunit.client.1.smithi149.stdout:9/876: mkdir db/d42/d114 0 2024-03-22T07:08:58.277 INFO:tasks.workunit.client.1.smithi149.stdout:1/894: dwrite d4/d20/f52 [0,4194304] 0 2024-03-22T07:08:58.278 INFO:tasks.workunit.client.1.smithi149.stdout:5/829: dwrite d7/d35/de0/dd7/f6e [0,4194304] 0 2024-03-22T07:08:58.289 INFO:tasks.workunit.client.1.smithi149.stdout:7/891: write d1/d1c/d49/d8e/d100/d1b/f2c [105101,49177] 0 2024-03-22T07:08:58.289 INFO:tasks.workunit.client.1.smithi149.stdout:4/868: dwrite d0/d8/dc/d2e/f2f [0,4194304] 0 2024-03-22T07:08:58.301 INFO:tasks.workunit.client.1.smithi149.stdout:0/783: unlink d1/d31/f3f 0 2024-03-22T07:08:58.302 INFO:tasks.workunit.client.1.smithi149.stdout:2/943: rename d1/d4d/d8b to d1/d3/d5/d41/d9f/da7/d157 0 2024-03-22T07:08:58.304 INFO:tasks.workunit.client.1.smithi149.stdout:6/933: creat d9/df/d22/d6d/d7d/d96/d24/d68/f135 x:0 0 0 2024-03-22T07:08:58.304 INFO:tasks.workunit.client.1.smithi149.stdout:6/934: chown d9/df/d22/d6d/d7d/fb6 1520 1 2024-03-22T07:08:58.304 INFO:tasks.workunit.client.1.smithi149.stdout:0/784: dread d1/d20/da3/fab [0,4194304] 0 2024-03-22T07:08:58.305 INFO:tasks.workunit.client.1.smithi149.stdout:9/877: symlink db/d21/d10f/l115 0 2024-03-22T07:08:58.305 INFO:tasks.workunit.client.1.smithi149.stdout:7/892: dread d1/d1c/d49/d8e/d100/d1b/d51/d5a/f78 [0,4194304] 0 2024-03-22T07:08:58.307 INFO:tasks.workunit.client.1.smithi149.stdout:0/785: dread d1/d20/da3/fab [0,4194304] 0 2024-03-22T07:08:58.311 INFO:tasks.workunit.client.1.smithi149.stdout:8/854: rename d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/c8e to d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/d9e/c115 0 2024-03-22T07:08:58.311 INFO:tasks.workunit.client.1.smithi149.stdout:0/786: dread d1/d31/d51/d64/fbd [0,4194304] 0 2024-03-22T07:08:58.311 INFO:tasks.workunit.client.1.smithi149.stdout:0/787: fsync d1/fb1 0 2024-03-22T07:08:58.312 INFO:tasks.workunit.client.1.smithi149.stdout:2/944: mknod d1/d3/d5/d1f/dfc/d156/c158 0 2024-03-22T07:08:58.312 INFO:tasks.workunit.client.1.smithi149.stdout:2/945: read - d1/d4d/d69/d75/f136 zero size 2024-03-22T07:08:58.314 INFO:tasks.workunit.client.1.smithi149.stdout:9/878: unlink db/f108 0 2024-03-22T07:08:58.314 INFO:tasks.workunit.client.1.smithi149.stdout:5/830: dwrite d7/f33 [0,4194304] 0 2024-03-22T07:08:58.317 INFO:tasks.workunit.client.1.smithi149.stdout:1/895: rmdir d4/d10/dd8/d3c/d8e/db5/db2/dbf 39 2024-03-22T07:08:58.317 INFO:tasks.workunit.client.1.smithi149.stdout:8/855: symlink d1/d2/d9/d11/d4d/dd6/d51/l116 0 2024-03-22T07:08:58.318 INFO:tasks.workunit.client.1.smithi149.stdout:8/856: chown d1/d2/d3/c9b 3 1 2024-03-22T07:08:58.319 INFO:tasks.workunit.client.1.smithi149.stdout:2/946: rmdir d1/d3/d7d 39 2024-03-22T07:08:58.322 INFO:tasks.workunit.client.1.smithi149.stdout:9/879: rmdir db/d42/d49/d4b/d5f/ddb 39 2024-03-22T07:08:58.325 INFO:tasks.workunit.client.1.smithi149.stdout:4/869: dwrite d0/d8/dc/dbd/fbe [4194304,4194304] 0 2024-03-22T07:08:58.325 INFO:tasks.workunit.client.1.smithi149.stdout:4/870: dread - d0/d8/dc/d2e/d6f/d96/f104 zero size 2024-03-22T07:08:58.327 INFO:tasks.workunit.client.1.smithi149.stdout:7/893: read d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/f3b [1754416,25108] 0 2024-03-22T07:08:58.327 INFO:tasks.workunit.client.1.smithi149.stdout:1/896: dread d4/d7/f57 [0,4194304] 0 2024-03-22T07:08:58.328 INFO:tasks.workunit.client.1.smithi149.stdout:4/871: unlink d0/d8/dc/d2e/c53 0 2024-03-22T07:08:58.329 INFO:tasks.workunit.client.1.smithi149.stdout:6/935: dwrite d9/d47/fe3 [0,4194304] 0 2024-03-22T07:08:58.329 INFO:tasks.workunit.client.1.smithi149.stdout:7/894: read d1/d1c/d49/d8e/d100/d1b/d1d/fe6 [850257,79107] 0 2024-03-22T07:08:58.329 INFO:tasks.workunit.client.1.smithi149.stdout:7/895: truncate d1/d1c/d49/d8e/d100/d1b/d51/d5a/fad 1383250 0 2024-03-22T07:08:58.338 INFO:tasks.workunit.client.1.smithi149.stdout:6/936: truncate d9/df/d22/f89 118305 0 2024-03-22T07:08:58.339 INFO:tasks.workunit.client.1.smithi149.stdout:6/937: write f1 [1948576,126154] 0 2024-03-22T07:08:58.347 INFO:tasks.workunit.client.1.smithi149.stdout:4/872: getdents d0/d8/d4c/db4 0 2024-03-22T07:08:58.347 INFO:tasks.workunit.client.1.smithi149.stdout:4/873: readlink d0/d29/l60 0 2024-03-22T07:08:58.347 INFO:tasks.workunit.client.1.smithi149.stdout:4/874: readlink d0/d8/dc/d14/d85/d97/lac 0 2024-03-22T07:08:58.348 INFO:tasks.workunit.client.1.smithi149.stdout:5/831: dwrite d7/da/d20/d30/d3a/f101 [0,4194304] 0 2024-03-22T07:08:58.348 INFO:tasks.workunit.client.1.smithi149.stdout:5/832: fsync d7/da/d20/d36/f9d 0 2024-03-22T07:08:58.348 INFO:tasks.workunit.client.1.smithi149.stdout:9/880: write db/d33/f92 [2800239,82771] 0 2024-03-22T07:08:58.348 INFO:tasks.workunit.client.1.smithi149.stdout:9/881: chown db/d42/f5a 89778 1 2024-03-22T07:08:58.348 INFO:tasks.workunit.client.1.smithi149.stdout:9/882: chown db/d19/d1c/f20 387 1 2024-03-22T07:08:58.348 INFO:tasks.workunit.client.1.smithi149.stdout:9/883: dread - db/d42/f113 zero size 2024-03-22T07:08:58.349 INFO:tasks.workunit.client.1.smithi149.stdout:5/833: dread d7/da/d20/d36/fbd [0,4194304] 0 2024-03-22T07:08:58.350 INFO:tasks.workunit.client.1.smithi149.stdout:4/875: mknod d0/d8/dc/d2e/d6f/def/c11b 0 2024-03-22T07:08:58.350 INFO:tasks.workunit.client.1.smithi149.stdout:4/876: creat d0/d8/dc/d4b/db3/ded/f11c x:0 0 0 2024-03-22T07:08:58.352 INFO:tasks.workunit.client.1.smithi149.stdout:9/884: creat db/d21/d37/d7c/da3/f116 x:0 0 0 2024-03-22T07:08:58.352 INFO:tasks.workunit.client.1.smithi149.stdout:9/885: write db/d21/d37/ff6 [237552,74820] 0 2024-03-22T07:08:58.353 INFO:tasks.workunit.client.1.smithi149.stdout:9/886: creat db/f117 x:0 0 0 2024-03-22T07:08:58.353 INFO:tasks.workunit.client.1.smithi149.stdout:9/887: fsync db/d21/ff1 0 2024-03-22T07:08:58.355 INFO:tasks.workunit.client.1.smithi149.stdout:9/888: link db/d21/d2e/la1 db/d42/d49/d4b/d5f/l118 0 2024-03-22T07:08:58.356 INFO:tasks.workunit.client.1.smithi149.stdout:9/889: creat db/d19/d93/db9/f119 x:0 0 0 2024-03-22T07:08:58.358 INFO:tasks.workunit.client.1.smithi149.stdout:9/890: mkdir db/d19/d39/dd3/d11a 0 2024-03-22T07:08:58.358 INFO:tasks.workunit.client.1.smithi149.stdout:9/891: stat db/d12/l81 0 2024-03-22T07:08:58.358 INFO:tasks.workunit.client.1.smithi149.stdout:9/892: write db/f2f [3505350,76976] 0 2024-03-22T07:08:58.359 INFO:tasks.workunit.client.1.smithi149.stdout:5/834: write d7/da/d20/d45/f7c [824432,119625] 0 2024-03-22T07:08:58.362 INFO:tasks.workunit.client.1.smithi149.stdout:5/835: rename d7/d35/de0/dd7/d7b/l42 to d7/da/d7d/l121 0 2024-03-22T07:08:58.362 INFO:tasks.workunit.client.1.smithi149.stdout:5/836: fdatasync d7/da/d20/d89/da2/dda/f11d 0 2024-03-22T07:08:58.364 INFO:tasks.workunit.client.1.smithi149.stdout:5/837: mkdir d7/d35/de0/d92/df4/df5/d10b/d122 0 2024-03-22T07:08:58.365 INFO:tasks.workunit.client.1.smithi149.stdout:5/838: getdents d7/da/d20 0 2024-03-22T07:08:58.366 INFO:tasks.workunit.client.1.smithi149.stdout:5/839: creat d7/da/d20/d36/d44/d4a/f123 x:0 0 0 2024-03-22T07:08:58.366 INFO:tasks.workunit.client.1.smithi149.stdout:5/840: write d7/da/d7d/f9f [1622507,108473] 0 2024-03-22T07:08:58.367 INFO:tasks.workunit.client.1.smithi149.stdout:5/841: symlink d7/da/d20/d36/deb/l124 0 2024-03-22T07:08:58.368 INFO:tasks.workunit.client.1.smithi149.stdout:5/842: unlink d7/da/d20/d89/da2/dda/f11d 0 2024-03-22T07:08:58.370 INFO:tasks.workunit.client.1.smithi149.stdout:9/893: write db/d19/d39/d68/fac [1733737,119959] 0 2024-03-22T07:08:58.370 INFO:tasks.workunit.client.1.smithi149.stdout:9/894: fsync db/d19/f7d 0 2024-03-22T07:08:58.372 INFO:tasks.workunit.client.1.smithi149.stdout:9/895: mkdir db/d21/d37/d3a/d104/d11b 0 2024-03-22T07:08:58.378 INFO:tasks.workunit.client.1.smithi149.stdout:4/877: dread d0/d29/f3c [4194304,4194304] 0 2024-03-22T07:08:58.380 INFO:tasks.workunit.client.1.smithi149.stdout:4/878: symlink d0/d8/dc/d2e/d6f/ddf/l11d 0 2024-03-22T07:08:58.381 INFO:tasks.workunit.client.1.smithi149.stdout:4/879: mknod d0/d8/d4c/db4/c11e 0 2024-03-22T07:08:58.383 INFO:tasks.workunit.client.1.smithi149.stdout:4/880: symlink d0/d6b/db0/l11f 0 2024-03-22T07:08:58.385 INFO:tasks.workunit.client.1.smithi149.stdout:8/857: dwrite d1/d2/d3/f86 [4194304,4194304] 0 2024-03-22T07:08:58.386 INFO:tasks.workunit.client.1.smithi149.stdout:4/881: rename d0/f67 to d0/d8/dc/d4b/dbc/f120 0 2024-03-22T07:08:58.386 INFO:tasks.workunit.client.1.smithi149.stdout:4/882: read d0/f5a [1591780,71722] 0 2024-03-22T07:08:58.386 INFO:tasks.workunit.client.1.smithi149.stdout:4/883: readlink d0/d8/d1d/lb5 0 2024-03-22T07:08:58.388 INFO:tasks.workunit.client.1.smithi149.stdout:8/858: getdents d1/d2/d9/d11/d45 0 2024-03-22T07:08:58.388 INFO:tasks.workunit.client.1.smithi149.stdout:4/884: stat d0/d8/dc/c25 0 2024-03-22T07:08:58.389 INFO:tasks.workunit.client.1.smithi149.stdout:4/885: chown d0/d8/dc/d2e/d6f/ddf/fe2 46627005 1 2024-03-22T07:08:58.389 INFO:tasks.workunit.client.1.smithi149.stdout:4/886: readlink d0/d6b/db0/lc2 0 2024-03-22T07:08:58.389 INFO:tasks.workunit.client.1.smithi149.stdout:4/887: write d0/d8/d4c/f111 [848139,91490] 0 2024-03-22T07:08:58.389 INFO:tasks.workunit.client.1.smithi149.stdout:8/859: link d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/fc9 d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f117 0 2024-03-22T07:08:58.389 INFO:tasks.workunit.client.1.smithi149.stdout:8/860: dread - d1/d2/d9/d11/d4d/dd6/db4/f107 zero size 2024-03-22T07:08:58.390 INFO:tasks.workunit.client.1.smithi149.stdout:8/861: mknod d1/d2/d18/c118 0 2024-03-22T07:08:58.391 INFO:tasks.workunit.client.1.smithi149.stdout:8/862: truncate d1/d2/d9/ff1 1268201 0 2024-03-22T07:08:58.391 INFO:tasks.workunit.client.1.smithi149.stdout:8/863: creat d1/d2/d9/d11/d4d/dd6/d35/d22/f119 x:0 0 0 2024-03-22T07:08:58.397 INFO:tasks.workunit.client.1.smithi149.stdout:2/947: dwrite d1/d3/d5/d1f/d5e/f4a [0,4194304] 0 2024-03-22T07:08:58.397 INFO:tasks.workunit.client.1.smithi149.stdout:1/897: dwrite d4/d20/d2d/d64/dce/fec [0,4194304] 0 2024-03-22T07:08:58.397 INFO:tasks.workunit.client.1.smithi149.stdout:2/948: chown d1/d3/d5/d1f/d68/fb4 3038 1 2024-03-22T07:08:58.397 INFO:tasks.workunit.client.1.smithi149.stdout:1/898: write d4/d10/dd8/d3c/d8e/db5/f83 [4388275,19972] 0 2024-03-22T07:08:58.399 INFO:tasks.workunit.client.1.smithi149.stdout:7/896: dwrite d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/fc5 [0,4194304] 0 2024-03-22T07:08:58.399 INFO:tasks.workunit.client.1.smithi149.stdout:5/843: dwrite d7/da/d20/d45/f51 [0,4194304] 0 2024-03-22T07:08:58.399 INFO:tasks.workunit.client.1.smithi149.stdout:6/938: dwrite d9/df/d22/d6d/d7d/d96/f6b [4194304,4194304] 0 2024-03-22T07:08:58.404 INFO:tasks.workunit.client.1.smithi149.stdout:8/864: dread d1/d2/d9/d11/f77 [4194304,4194304] 0 2024-03-22T07:08:58.413 INFO:tasks.workunit.client.1.smithi149.stdout:9/896: dwrite db/d42/f113 [0,4194304] 0 2024-03-22T07:08:58.416 INFO:tasks.workunit.client.1.smithi149.stdout:1/899: dread d4/d10/f12 [0,4194304] 0 2024-03-22T07:08:58.416 INFO:tasks.workunit.client.1.smithi149.stdout:4/888: dwrite d0/f2d [0,4194304] 0 2024-03-22T07:08:58.417 INFO:tasks.workunit.client.1.smithi149.stdout:1/900: write d4/d7/dd3/d129/f35 [830219,49054] 0 2024-03-22T07:08:58.420 INFO:tasks.workunit.client.1.smithi149.stdout:2/949: symlink d1/d3/d5/d41/d9f/l159 0 2024-03-22T07:08:58.420 INFO:tasks.workunit.client.1.smithi149.stdout:7/897: dwrite d1/d1c/d49/d8e/d100/d1b/d1d/fe4 [0,4194304] 0 2024-03-22T07:08:58.422 INFO:tasks.workunit.client.1.smithi149.stdout:5/844: link d7/f58 d7/d35/de0/dd7/d7b/f125 0 2024-03-22T07:08:58.424 INFO:tasks.workunit.client.1.smithi149.stdout:5/845: dread d7/da/d20/d45/f4f [0,4194304] 0 2024-03-22T07:08:58.424 INFO:tasks.workunit.client.1.smithi149.stdout:5/846: write d7/da/d20/d30/d3a/d3d/d2c/db2/d106/f108 [588386,56802] 0 2024-03-22T07:08:58.424 INFO:tasks.workunit.client.1.smithi149.stdout:6/939: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/d107/f136 x:0 0 0 2024-03-22T07:08:58.425 INFO:tasks.workunit.client.1.smithi149.stdout:6/940: stat d9/df/d22/d6d/d7d/d96/d24/d42/f75 0 2024-03-22T07:08:58.425 INFO:tasks.workunit.client.1.smithi149.stdout:6/941: write d9/df/d22/ff2 [179059,52370] 0 2024-03-22T07:08:58.425 INFO:tasks.workunit.client.1.smithi149.stdout:6/942: readlink d9/df/d22/d6d/d7d/d96/l33 0 2024-03-22T07:08:58.425 INFO:tasks.workunit.client.1.smithi149.stdout:6/943: stat d9/d47/f5a 0 2024-03-22T07:08:58.425 INFO:tasks.workunit.client.1.smithi149.stdout:8/865: truncate d1/d2/d9/d11/d4d/dd6/d35/d22/dd7/fd9 383383 0 2024-03-22T07:08:58.431 INFO:tasks.workunit.client.1.smithi149.stdout:4/889: mkdir d0/d8/dc/d4b/db3/ded/d121 0 2024-03-22T07:08:58.433 INFO:tasks.workunit.client.1.smithi149.stdout:1/901: creat d4/d10/dd8/d3c/d8e/db5/ddb/f12c x:0 0 0 2024-03-22T07:08:58.434 INFO:tasks.workunit.client.1.smithi149.stdout:7/898: write d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/d5b/f79 [2526481,56074] 0 2024-03-22T07:08:58.434 INFO:tasks.workunit.client.1.smithi149.stdout:7/899: fsync d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/d10b/f123 0 2024-03-22T07:08:58.435 INFO:tasks.workunit.client.1.smithi149.stdout:7/900: creat d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/dee/f12a x:0 0 0 2024-03-22T07:08:58.438 INFO:tasks.workunit.client.1.smithi149.stdout:5/847: creat d7/da/d20/d30/d3a/d3d/d104/f126 x:0 0 0 2024-03-22T07:08:58.438 INFO:tasks.workunit.client.1.smithi149.stdout:5/848: write d7/da/d20/d45/f9c [657245,15187] 0 2024-03-22T07:08:58.438 INFO:tasks.workunit.client.1.smithi149.stdout:5/849: write d7/da/d20/d30/d3a/f101 [4656063,43277] 0 2024-03-22T07:08:58.438 INFO:tasks.workunit.client.1.smithi149.stdout:9/897: getdents db/d19/d1c 0 2024-03-22T07:08:58.440 INFO:tasks.workunit.client.1.smithi149.stdout:6/944: write d9/da2/fb5 [546857,95148] 0 2024-03-22T07:08:58.440 INFO:tasks.workunit.client.1.smithi149.stdout:4/890: symlink d0/l122 0 2024-03-22T07:08:58.441 INFO:tasks.workunit.client.1.smithi149.stdout:2/950: symlink d1/d3/d6/l15a 0 2024-03-22T07:08:58.441 INFO:tasks.workunit.client.1.smithi149.stdout:1/902: dread d4/d10/d7c/d6a/d6b/f84 [0,4194304] 0 2024-03-22T07:08:58.443 INFO:tasks.workunit.client.1.smithi149.stdout:7/901: unlink d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/lac 0 2024-03-22T07:08:58.445 INFO:tasks.workunit.client.1.smithi149.stdout:9/898: symlink db/d42/d49/d4b/l11c 0 2024-03-22T07:08:58.446 INFO:tasks.workunit.client.1.smithi149.stdout:6/945: mkdir d9/d47/d137 0 2024-03-22T07:08:58.448 INFO:tasks.workunit.client.1.smithi149.stdout:4/891: mknod d0/d29/c123 0 2024-03-22T07:08:58.448 INFO:tasks.workunit.client.1.smithi149.stdout:4/892: getdents d0/d6b 0 2024-03-22T07:08:58.449 INFO:tasks.workunit.client.1.smithi149.stdout:2/951: symlink d1/d3/d5/d41/d9f/da7/l15b 0 2024-03-22T07:08:58.450 INFO:tasks.workunit.client.1.smithi149.stdout:1/903: truncate d4/d7/d61/ff0 279655 0 2024-03-22T07:08:58.452 INFO:tasks.workunit.client.1.smithi149.stdout:7/902: symlink d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/l12b 0 2024-03-22T07:08:58.458 INFO:tasks.workunit.client.1.smithi149.stdout:2/952: write d1/d8/f24 [4500200,67103] 0 2024-03-22T07:08:58.458 INFO:tasks.workunit.client.1.smithi149.stdout:2/953: fdatasync d1/d3/d6/d32/d6f/fcd 0 2024-03-22T07:08:58.458 INFO:tasks.workunit.client.1.smithi149.stdout:4/893: rename d0/d8/dc/d4b/dbc/f100 to d0/d8/dc/d14/d85/d97/d106/f124 0 2024-03-22T07:08:58.458 INFO:tasks.workunit.client.1.smithi149.stdout:3/681: sync 2024-03-22T07:08:58.458 INFO:tasks.workunit.client.1.smithi149.stdout:4/894: creat d0/d8/dc/d4b/db3/ded/d121/f125 x:0 0 0 2024-03-22T07:08:58.458 INFO:tasks.workunit.client.1.smithi149.stdout:4/895: truncate d0/d8/dc/d14/f43 2364867 0 2024-03-22T07:08:58.459 INFO:tasks.workunit.client.1.smithi149.stdout:5/850: dread d7/da/d20/d45/fb7 [0,4194304] 0 2024-03-22T07:08:58.459 INFO:tasks.workunit.client.1.smithi149.stdout:5/851: truncate d7/d35/de0/dd7/f10d 721375 0 2024-03-22T07:08:58.459 INFO:tasks.workunit.client.1.smithi149.stdout:5/852: truncate d7/da/d20/d30/d3a/d3d/fb4 1307715 0 2024-03-22T07:08:58.459 INFO:tasks.workunit.client.1.smithi149.stdout:3/682: mknod d0/d2/d30/dee/c100 0 2024-03-22T07:08:58.459 INFO:tasks.workunit.client.1.smithi149.stdout:6/946: dread d9/df/d2e/d63/f69 [0,4194304] 0 2024-03-22T07:08:58.461 INFO:tasks.workunit.client.1.smithi149.stdout:3/683: link d0/d4/d8/d24/d40/d54/d85/dbc/fcf d0/d2/d30/dd1/dc9/f101 0 2024-03-22T07:08:58.461 INFO:tasks.workunit.client.1.smithi149.stdout:3/684: write d0/f72 [1039768,28640] 0 2024-03-22T07:08:58.466 INFO:tasks.workunit.client.1.smithi149.stdout:6/947: mknod d9/df/d22/d6d/d7d/d96/d24/d34/d37/d134/c138 0 2024-03-22T07:08:58.467 INFO:tasks.workunit.client.1.smithi149.stdout:7/903: dread d1/d1c/d49/d8e/d100/d1b/d51/d5a/d6a/fb0 [0,4194304] 0 2024-03-22T07:08:58.467 INFO:tasks.workunit.client.1.smithi149.stdout:1/904: dread d4/d10/f12 [0,4194304] 0 2024-03-22T07:08:58.468 INFO:tasks.workunit.client.1.smithi149.stdout:8/866: dwrite d1/d2/d9/d11/d4d/dd6/d51/fa2 [0,4194304] 0 2024-03-22T07:08:58.468 INFO:tasks.workunit.client.1.smithi149.stdout:8/867: truncate d1/d2/d9/fea 966677 0 2024-03-22T07:08:58.469 INFO:tasks.workunit.client.1.smithi149.stdout:6/948: rename d9/df/d22/d6d/d7d/d96/d24/d68/l77 to d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/d107/l139 0 2024-03-22T07:08:58.469 INFO:tasks.workunit.client.1.smithi149.stdout:6/949: write d9/f3c [303051,20243] 0 2024-03-22T07:08:58.471 INFO:tasks.workunit.client.1.smithi149.stdout:7/904: rmdir d1/d1c/d49/d8e/d100/d1b/d51/d5a 39 2024-03-22T07:08:58.473 INFO:tasks.workunit.client.1.smithi149.stdout:1/905: mknod d4/d10c/c12d 0 2024-03-22T07:08:58.475 INFO:tasks.workunit.client.1.smithi149.stdout:8/868: mknod d1/d2/d9/db2/c11a 0 2024-03-22T07:08:58.475 INFO:tasks.workunit.client.1.smithi149.stdout:8/869: readlink d1/d2/d9/d11/d4d/lda 0 2024-03-22T07:08:58.479 INFO:tasks.workunit.client.1.smithi149.stdout:8/870: symlink d1/d2/d9/d11/d4d/l11b 0 2024-03-22T07:08:58.488 INFO:tasks.workunit.client.1.smithi149.stdout:7/905: dread d1/d1c/d49/d8e/d100/d1b/d1d/fe6 [0,4194304] 0 2024-03-22T07:08:58.489 INFO:tasks.workunit.client.1.smithi149.stdout:2/954: dwrite d1/d3/d6/db2/d71/fe8 [0,4194304] 0 2024-03-22T07:08:58.489 INFO:tasks.workunit.client.1.smithi149.stdout:2/955: fdatasync d1/d3/d5/d1f/f1e 0 2024-03-22T07:08:58.489 INFO:tasks.workunit.client.1.smithi149.stdout:7/906: rmdir d1/d1c/d49/d8e/d100/d1b/d99/de5/db4 39 2024-03-22T07:08:58.489 INFO:tasks.workunit.client.1.smithi149.stdout:7/907: fdatasync d1/d1c/d49/d8e/d100/d1b/d51/d7a/d7d/f77 0 2024-03-22T07:08:58.491 INFO:tasks.workunit.client.1.smithi149.stdout:5/853: dread d7/da/d20/d89/da2/fec [0,4194304] 0 2024-03-22T07:08:58.491 INFO:tasks.workunit.client.1.smithi149.stdout:9/899: dwrite db/d21/d37/d7c/dc7/f109 [0,4194304] 0 2024-03-22T07:08:58.491 INFO:tasks.workunit.client.1.smithi149.stdout:5/854: write d7/da/d20/d30/d3a/d3d/d2c/fb1 [367929,87028] 0 2024-03-22T07:08:58.492 INFO:tasks.workunit.client.1.smithi149.stdout:2/956: symlink d1/d3/d5/d1f/d68/d97/l15c 0 2024-03-22T07:08:58.497 INFO:tasks.workunit.client.1.smithi149.stdout:2/957: creat d1/d3/d5/d1f/d5e/d120/f15d x:0 0 0 2024-03-22T07:08:58.498 INFO:tasks.workunit.client.1.smithi149.stdout:7/908: mkdir d1/d1c/d12c 0 2024-03-22T07:08:58.499 INFO:tasks.workunit.client.1.smithi149.stdout:4/896: dwrite d0/d8/dc/d2e/d6f/f102 [0,4194304] 0 2024-03-22T07:08:58.499 INFO:tasks.workunit.client.1.smithi149.stdout:4/897: write d0/d8/dc/d74/d76/f7b [1274364,28346] 0 2024-03-22T07:08:58.501 INFO:tasks.workunit.client.1.smithi149.stdout:2/958: rename d1/d4d/d69/c12f to d1/d3/d5/d41/d9f/c15e 0 2024-03-22T07:08:58.502 INFO:tasks.workunit.client.1.smithi149.stdout:7/909: truncate d1/d1c/d49/d8e/d100/d1b/d1d/d72/fdc 975585 0 2024-03-22T07:08:58.502 INFO:tasks.workunit.client.1.smithi149.stdout:7/910: fsync d1/fa7 0 2024-03-22T07:08:58.504 INFO:tasks.workunit.client.1.smithi149.stdout:4/898: rename d0/d8/fd5 to d0/d8/d1d/d24/d82/f126 0 2024-03-22T07:08:58.505 INFO:tasks.workunit.client.1.smithi149.stdout:9/900: write db/d12/f14 [7686808,95353] 0 2024-03-22T07:08:58.505 INFO:tasks.workunit.client.1.smithi149.stdout:2/959: creat d1/d8/d34/d14c/f15f x:0 0 0 2024-03-22T07:08:58.506 INFO:tasks.workunit.client.1.smithi149.stdout:2/960: fsync d1/d3/d5/d1f/d5e/f137 0 2024-03-22T07:08:58.507 INFO:tasks.workunit.client.1.smithi149.stdout:7/911: rmdir d1/d1c/d49/d8e/d100/d1b/d99/de5/db4 39 2024-03-22T07:08:58.508 INFO:tasks.workunit.client.1.smithi149.stdout:4/899: creat d0/d8/dc/dbd/f127 x:0 0 0 2024-03-22T07:08:58.508 INFO:tasks.workunit.client.1.smithi149.stdout:4/900: write d0/d8/dc/d14/d3e/fb1 [124617,27648] 0 2024-03-22T07:08:58.508 INFO:tasks.workunit.client.1.smithi149.stdout:4/901: write d0/d8/dc/d74/d76/deb/ffd [791038,38921] 0 2024-03-22T07:08:58.508 INFO:tasks.workunit.client.1.smithi149.stdout:4/902: chown d0/l6 0 1 2024-03-22T07:08:58.509 INFO:tasks.workunit.client.1.smithi149.stdout:2/961: creat d1/f160 x:0 0 0 2024-03-22T07:08:58.509 INFO:tasks.workunit.client.1.smithi149.stdout:2/962: chown d1/d3/d5/d1f/f1e 2 1 2024-03-22T07:08:58.511 INFO:tasks.workunit.client.1.smithi149.stdout:4/903: rename d0/d29/f4a to d0/d6b/db0/d110/f128 0 2024-03-22T07:08:58.513 INFO:tasks.workunit.client.1.smithi149.stdout:4/904: mknod d0/c129 0 2024-03-22T07:08:58.513 INFO:tasks.workunit.client.1.smithi149.stdout:4/905: write d0/d29/f6e [4392000,118391] 0 2024-03-22T07:08:58.518 INFO:tasks.workunit.client.1.smithi149.stdout:4/906: mknod d0/d8/dc/d2e/d6f/c12a 0 2024-03-22T07:08:58.519 INFO:tasks.workunit.client.1.smithi149.stdout:3/685: dwrite d0/d4/d8/d24/d40/d54/d85/fb0 [4194304,4194304] 0 2024-03-22T07:08:58.523 INFO:tasks.workunit.client.1.smithi149.stdout:3/686: mknod d0/d2/d30/d5d/d84/d8f/c102 0 2024-03-22T07:08:58.523 INFO:tasks.workunit.client.1.smithi149.stdout:3/687: write d0/f72 [1909536,13298] 0 2024-03-22T07:08:58.523 INFO:tasks.workunit.client.1.smithi149.stdout:8/871: dwrite d1/d2/d9/d11/d4d/dd6/d51/fe2 [0,4194304] 0 2024-03-22T07:08:58.524 INFO:tasks.workunit.client.1.smithi149.stdout:7/912: dwrite d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/dee/f11c [0,4194304] 0 2024-03-22T07:08:58.524 INFO:tasks.workunit.client.1.smithi149.stdout:6/950: dwrite d9/da2/f118 [0,4194304] 0 2024-03-22T07:08:58.525 INFO:tasks.workunit.client.1.smithi149.stdout:5/855: dwrite d7/da/d20/d30/d3a/d3d/f34 [0,4194304] 0 2024-03-22T07:08:58.525 INFO:tasks.workunit.client.1.smithi149.stdout:5/856: stat d7/d35/de0/dd7/d7b/d81/f6a 0 2024-03-22T07:08:58.527 INFO:tasks.workunit.client.1.smithi149.stdout:3/688: mkdir d0/d4/d8/d24/db6/dcd/d103 0 2024-03-22T07:08:58.529 INFO:tasks.workunit.client.1.smithi149.stdout:8/872: mknod d1/d2/d9/df/c11c 0 2024-03-22T07:08:58.537 INFO:tasks.workunit.client.1.smithi149.stdout:4/907: dwrite d0/d8/dc/d2e/f10b [0,4194304] 0 2024-03-22T07:08:58.540 INFO:tasks.workunit.client.1.smithi149.stdout:5/857: dread d7/d35/de0/f71 [0,4194304] 0 2024-03-22T07:08:58.540 INFO:tasks.workunit.client.1.smithi149.stdout:5/858: write d7/d35/de0/dd7/d7b/d81/f90 [493976,100045] 0 2024-03-22T07:08:58.540 INFO:tasks.workunit.client.1.smithi149.stdout:5/859: fsync d7/d35/de0/dd7/fae 0 2024-03-22T07:08:58.542 INFO:tasks.workunit.client.1.smithi149.stdout:3/689: write d0/d2/f76 [3120237,39094] 0 2024-03-22T07:08:58.542 INFO:tasks.workunit.client.1.smithi149.stdout:3/690: write d0/d2/d30/d5d/d84/d8f/feb [564014,70931] 0 2024-03-22T07:08:58.552 INFO:tasks.workunit.client.1.smithi149.stdout:7/913: mkdir d1/d1c/d49/d8e/d100/d1b/d99/de5/db4/d12d 0 2024-03-22T07:08:58.554 INFO:tasks.workunit.client.1.smithi149.stdout:4/908: mknod d0/d8/d1d/d24/d9c/d107/c12b 0 2024-03-22T07:08:58.555 INFO:tasks.workunit.client.1.smithi149.stdout:6/951: rmdir d9/df/d22/d6d/d7d/d96/d24/d34/dac 39 2024-03-22T07:08:58.556 INFO:tasks.workunit.client.1.smithi149.stdout:6/952: write d9/df/d22/d6d/d7d/d96/f21 [914532,59413] 0 2024-03-22T07:08:58.556 INFO:tasks.workunit.client.1.smithi149.stdout:6/953: dread - d9/df/d22/dc0/dc8/de9/f12d zero size 2024-03-22T07:08:58.557 INFO:tasks.workunit.client.1.smithi149.stdout:5/860: mkdir d7/da/d20/d127 0 2024-03-22T07:08:58.559 INFO:tasks.workunit.client.1.smithi149.stdout:3/691: rename d0/d4/d8/d24/d39/c56 to d0/d4/d8/d24/d39/c104 0 2024-03-22T07:08:58.561 INFO:tasks.workunit.client.1.smithi149.stdout:8/873: link d1/d2/d3/c60 d1/d2/d9/d11/d4d/c11d 0 2024-03-22T07:08:58.561 INFO:tasks.workunit.client.1.smithi149.stdout:8/874: stat d1/d2/d3/f86 0 2024-03-22T07:08:58.563 INFO:tasks.workunit.client.1.smithi149.stdout:6/954: creat d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/f13a x:0 0 0 2024-03-22T07:08:58.565 INFO:tasks.workunit.client.1.smithi149.stdout:5/861: link d7/da/d20/d30/d3a/d93/fa0 d7/da/d20/d30/d3a/d3d/db0/f128 0 2024-03-22T07:08:58.569 INFO:tasks.workunit.client.1.smithi149.stdout:7/914: rmdir d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f 39 2024-03-22T07:08:58.570 INFO:tasks.workunit.client.1.smithi149.stdout:8/875: read d1/fc3 [4516949,67003] 0 2024-03-22T07:08:58.571 INFO:tasks.workunit.client.1.smithi149.stdout:6/955: getdents d9/df/d22/d6d/deb 0 2024-03-22T07:08:58.574 INFO:tasks.workunit.client.1.smithi149.stdout:7/915: creat d1/d1c/d49/d8e/d9c/df7/f12e x:0 0 0 2024-03-22T07:08:58.574 INFO:tasks.workunit.client.1.smithi149.stdout:7/916: truncate d1/d1c/ff5 146296 0 2024-03-22T07:08:58.574 INFO:tasks.workunit.client.1.smithi149.stdout:7/917: fsync d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/fa8 0 2024-03-22T07:08:58.575 INFO:tasks.workunit.client.1.smithi149.stdout:8/876: mkdir d1/d2/d9/df/dd5/def/d11e 0 2024-03-22T07:08:58.576 INFO:tasks.workunit.client.1.smithi149.stdout:6/956: symlink d9/df/d22/d6d/ddd/l13b 0 2024-03-22T07:08:58.576 INFO:tasks.workunit.client.1.smithi149.stdout:6/957: chown d9/df/d22/d6d/d7d/d96/l12f 4031624 1 2024-03-22T07:08:58.577 INFO:tasks.workunit.client.1.smithi149.stdout:3/692: getdents d0/d4/d8/d24/db6/dcd 0 2024-03-22T07:08:58.577 INFO:tasks.workunit.client.1.smithi149.stdout:3/693: dread - d0/d2/d5a/daf/dce/fda zero size 2024-03-22T07:08:58.577 INFO:tasks.workunit.client.1.smithi149.stdout:3/694: getdents d0/d4/d8/d24/db6/dcd 0 2024-03-22T07:08:58.578 INFO:tasks.workunit.client.1.smithi149.stdout:5/862: rmdir d7/da/d20/d45 39 2024-03-22T07:08:58.578 INFO:tasks.workunit.client.1.smithi149.stdout:5/863: write d7/da/d20/d30/d3a/d93/fa0 [703979,58686] 0 2024-03-22T07:08:58.579 INFO:tasks.workunit.client.1.smithi149.stdout:3/695: rmdir d0/d4/d8/d24/db6 39 2024-03-22T07:08:58.613 INFO:tasks.workunit.client.1.smithi149.stdout:7/918: read d1/d1c/d49/d8e/d100/d1b/ff1 [49764,89018] 0 2024-03-22T07:08:58.624 INFO:tasks.workunit.client.1.smithi149.stdout:7/919: write d1/d1c/d49/d8e/d100/d1b/d51/d5a/d6a/fb0 [7318827,85191] 0 2024-03-22T07:08:58.625 INFO:tasks.workunit.client.1.smithi149.stdout:7/920: getdents d1/d1c/d49/d8e/d100/d1b/d11a 0 2024-03-22T07:08:58.627 INFO:tasks.workunit.client.1.smithi149.stdout:7/921: truncate d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/f114 9680 0 2024-03-22T07:08:58.628 INFO:tasks.workunit.client.1.smithi149.stdout:7/922: mknod d1/d1c/d49/d8e/d100/d30/db5/c12f 0 2024-03-22T07:08:58.629 INFO:tasks.workunit.client.1.smithi149.stdout:7/923: truncate d1/d1c/d49/d8e/d100/dda/f105 99624 0 2024-03-22T07:08:58.651 INFO:tasks.workunit.client.1.smithi149.stdout:4/909: dwrite d0/d8/dc/d4b/f4d [4194304,4194304] 0 2024-03-22T07:08:58.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/910: unlink d0/d6b/f81 0 2024-03-22T07:08:58.653 INFO:tasks.workunit.client.1.smithi149.stdout:4/911: chown d0/d8/dc/d2e/d6f 17785000 1 2024-03-22T07:08:58.655 INFO:tasks.workunit.client.1.smithi149.stdout:4/912: mkdir d0/d8/dc/d4b/db3/dea/d12c 0 2024-03-22T07:08:58.655 INFO:tasks.workunit.client.1.smithi149.stdout:7/924: dwrite d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/d10b/f123 [0,4194304] 0 2024-03-22T07:08:58.655 INFO:tasks.workunit.client.1.smithi149.stdout:6/958: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/da5/f101 [0,4194304] 0 2024-03-22T07:08:58.655 INFO:tasks.workunit.client.1.smithi149.stdout:6/959: dread - d9/df/d22/d6d/d7d/d96/d24/d68/d73/fea zero size 2024-03-22T07:08:58.655 INFO:tasks.workunit.client.1.smithi149.stdout:8/877: dwrite d1/d2/d9/d11/f17 [0,4194304] 0 2024-03-22T07:08:58.655 INFO:tasks.workunit.client.1.smithi149.stdout:5/864: dwrite d7/d35/de0/dd7/d7b/f125 [0,4194304] 0 2024-03-22T07:08:58.661 INFO:tasks.workunit.client.1.smithi149.stdout:7/925: unlink d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/d74/de0/lfa 0 2024-03-22T07:08:58.661 INFO:tasks.workunit.client.1.smithi149.stdout:7/926: write d1/d1c/fbb [134844,12564] 0 2024-03-22T07:08:58.662 INFO:tasks.workunit.client.1.smithi149.stdout:7/927: creat d1/d1c/d49/d8e/d53/f130 x:0 0 0 2024-03-22T07:08:58.662 INFO:tasks.workunit.client.1.smithi149.stdout:7/928: readlink d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/l32 0 2024-03-22T07:08:58.662 INFO:tasks.workunit.client.1.smithi149.stdout:6/960: unlink d9/df/d22/l28 0 2024-03-22T07:08:58.665 INFO:tasks.workunit.client.1.smithi149.stdout:8/878: rmdir d1/d2/d9/d11/d8b/dee/dff/d102 0 2024-03-22T07:08:58.665 INFO:tasks.workunit.client.1.smithi149.stdout:4/913: dread d0/d8/dc/d2e/d6f/f102 [0,4194304] 0 2024-03-22T07:08:58.667 INFO:tasks.workunit.client.1.smithi149.stdout:4/914: write d0/d8/d4c/f111 [317249,54008] 0 2024-03-22T07:08:58.667 INFO:tasks.workunit.client.1.smithi149.stdout:8/879: dread d1/d2/d9/d11/d4d/faa [0,4194304] 0 2024-03-22T07:08:58.667 INFO:tasks.workunit.client.1.smithi149.stdout:8/880: stat d1/d2/d9/d11/d45/l46 0 2024-03-22T07:08:58.669 INFO:tasks.workunit.client.1.smithi149.stdout:6/961: creat d9/df/d22/d6d/d7d/d96/d24/d34/f13c x:0 0 0 2024-03-22T07:08:58.669 INFO:tasks.workunit.client.1.smithi149.stdout:6/962: write d9/df/d22/d6d/d7d/d96/d24/d68/d73/fd8 [943586,94715] 0 2024-03-22T07:08:58.670 INFO:tasks.workunit.client.1.smithi149.stdout:5/865: getdents d7/d35/de0/dd7 0 2024-03-22T07:08:58.676 INFO:tasks.workunit.client.1.smithi149.stdout:8/881: mkdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d11f 0 2024-03-22T07:08:58.676 INFO:tasks.workunit.client.1.smithi149.stdout:4/915: dread d0/d8/dc/d2e/d6f/ddf/fe2 [0,4194304] 0 2024-03-22T07:08:58.678 INFO:tasks.workunit.client.1.smithi149.stdout:6/963: unlink d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/f85 0 2024-03-22T07:08:58.678 INFO:tasks.workunit.client.1.smithi149.stdout:5/866: mkdir d7/d35/de0/dd7/d7b/d2b/d129 0 2024-03-22T07:08:58.678 INFO:tasks.workunit.client.1.smithi149.stdout:5/867: chown d7/da/c76 2 1 2024-03-22T07:08:58.680 INFO:tasks.workunit.client.1.smithi149.stdout:7/929: dwrite d1/d1c/d49/d8e/d100/d1b/d1d/f13 [0,4194304] 0 2024-03-22T07:08:58.680 INFO:tasks.workunit.client.1.smithi149.stdout:7/930: write d1/d1c/d49/d8e/d100/d1b/d51/d5a/d6a/da6/d93/fde [1242968,92677] 0 2024-03-22T07:08:58.680 INFO:tasks.workunit.client.1.smithi149.stdout:8/882: mknod d1/d2/d9/d11/d4d/dd6/d35/d22/c120 0 2024-03-22T07:08:58.683 INFO:tasks.workunit.client.1.smithi149.stdout:4/916: unlink d0/d8/dc/d2e/d6f/ddf/fe2 0 2024-03-22T07:08:58.684 INFO:tasks.workunit.client.1.smithi149.stdout:6/964: mknod d9/df/d22/d6d/d7d/d96/c13d 0 2024-03-22T07:08:58.687 INFO:tasks.workunit.client.1.smithi149.stdout:7/931: mkdir d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/dee/d131 0 2024-03-22T07:08:58.690 INFO:tasks.workunit.client.1.smithi149.stdout:8/883: rename d1/d2/fed to d1/d2/d9/d11/d8b/d8d/df2/f121 0 2024-03-22T07:08:58.691 INFO:tasks.workunit.client.1.smithi149.stdout:4/917: truncate d0/d8/d1d/fe6 3709267 0 2024-03-22T07:08:58.694 INFO:tasks.workunit.client.1.smithi149.stdout:6/965: link d9/df/d2e/fce d9/df/d22/d6d/d7d/d96/d24/d68/d73/f13e 0 2024-03-22T07:08:58.695 INFO:tasks.workunit.client.1.smithi149.stdout:5/868: dwrite d7/da/d20/d30/d3a/d3d/d2c/db2/ffd [0,4194304] 0 2024-03-22T07:08:58.695 INFO:tasks.workunit.client.1.smithi149.stdout:7/932: symlink d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/db6/l132 0 2024-03-22T07:08:58.700 INFO:tasks.workunit.client.1.smithi149.stdout:8/884: symlink d1/d2/d18/l122 0 2024-03-22T07:08:58.700 INFO:tasks.workunit.client.1.smithi149.stdout:8/885: creat d1/d2/d9/d11/d8b/daf/f123 x:0 0 0 2024-03-22T07:08:58.703 INFO:tasks.workunit.client.1.smithi149.stdout:5/869: unlink d7/da/d20/f99 0 2024-03-22T07:08:58.703 INFO:tasks.workunit.client.1.smithi149.stdout:5/870: readlink d7/da/d20/d30/d3a/d3d/db0/lfa 0 2024-03-22T07:08:58.704 INFO:tasks.workunit.client.1.smithi149.stdout:5/871: fsync d7/da/d20/d89/da2/fec 0 2024-03-22T07:08:58.705 INFO:tasks.workunit.client.1.smithi149.stdout:6/966: write d9/df/d22/d6d/d7d/fa8 [776205,104720] 0 2024-03-22T07:08:58.705 INFO:tasks.workunit.client.1.smithi149.stdout:6/967: truncate d9/df/d22/d6d/d7d/d96/d24/d34/f13c 754481 0 2024-03-22T07:08:58.713 INFO:tasks.workunit.client.1.smithi149.stdout:8/886: mkdir d1/d2/d9/df/d124 0 2024-03-22T07:08:58.714 INFO:tasks.workunit.client.1.smithi149.stdout:8/887: chown d1/d2/d3/c60 90733 1 2024-03-22T07:08:58.715 INFO:tasks.workunit.client.1.smithi149.stdout:5/872: symlink d7/da/d20/d30/d3a/d3d/d2c/db2/d106/l12a 0 2024-03-22T07:08:58.716 INFO:tasks.workunit.client.1.smithi149.stdout:8/888: mkdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d125 0 2024-03-22T07:08:58.718 INFO:tasks.workunit.client.1.smithi149.stdout:5/873: mkdir d7/da/d20/d30/d3a/d3d/d2c/d12b 0 2024-03-22T07:08:58.718 INFO:tasks.workunit.client.1.smithi149.stdout:5/874: dread - d7/da/d20/d30/d3a/d93/d94/fb3 zero size 2024-03-22T07:08:58.718 INFO:tasks.workunit.client.1.smithi149.stdout:5/875: rename d7/da/d20/d30 to d7/da/d20/d30/d3a/d3d/d2c/d12b/d12c 22 2024-03-22T07:08:58.729 INFO:tasks.workunit.client.1.smithi149.stdout:6/968: dwrite d9/d47/d52/fe6 [0,4194304] 0 2024-03-22T07:08:58.733 INFO:tasks.workunit.client.1.smithi149.stdout:8/889: dread d1/d2/d3/f86 [4194304,4194304] 0 2024-03-22T07:08:58.733 INFO:tasks.workunit.client.1.smithi149.stdout:8/890: write d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f117 [305797,47293] 0 2024-03-22T07:08:58.734 INFO:tasks.workunit.client.1.smithi149.stdout:5/876: read d7/da/d20/d30/d3a/d93/f9b [449432,34568] 0 2024-03-22T07:08:58.734 INFO:tasks.workunit.client.1.smithi149.stdout:8/891: rmdir d1/d2/d9/d11/d8b/dee/df4 39 2024-03-22T07:08:58.734 INFO:tasks.workunit.client.1.smithi149.stdout:8/892: write d1/d2/f8f [1262375,43713] 0 2024-03-22T07:08:58.737 INFO:tasks.workunit.client.1.smithi149.stdout:8/893: mknod d1/d2/d9/d11/d8b/d8d/df2/c126 0 2024-03-22T07:08:58.754 INFO:tasks.workunit.client.1.smithi149.stdout:8/894: read d1/d2/d9/d11/d4d/dd6/d35/d22/f40 [523072,118757] 0 2024-03-22T07:08:58.764 INFO:tasks.workunit.client.1.smithi149.stdout:8/895: write d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f56 [3566461,118943] 0 2024-03-22T07:08:58.764 INFO:tasks.workunit.client.1.smithi149.stdout:8/896: dread - d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d82/fc0 zero size 2024-03-22T07:08:58.769 INFO:tasks.workunit.client.1.smithi149.stdout:6/969: dwrite d9/df/d22/d6d/d7d/d96/d24/d68/d73/fd8 [0,4194304] 0 2024-03-22T07:08:58.769 INFO:tasks.workunit.client.1.smithi149.stdout:5/877: dwrite d7/d35/de0/dd7/fa9 [0,4194304] 0 2024-03-22T07:08:58.780 INFO:tasks.workunit.client.1.smithi149.stdout:5/878: mknod d7/da/d20/d30/d3a/d3d/d104/c12d 0 2024-03-22T07:08:58.780 INFO:tasks.workunit.client.1.smithi149.stdout:5/879: dread - d7/da/d20/d36/d44/d4a/f123 zero size 2024-03-22T07:08:58.795 INFO:tasks.workunit.client.1.smithi149.stdout:5/880: dread d7/d35/de0/dd7/d7b/d81/f6a [0,4194304] 0 2024-03-22T07:08:58.798 INFO:tasks.workunit.client.1.smithi149.stdout:8/897: dwrite d1/d2/d3/f96 [0,4194304] 0 2024-03-22T07:08:58.799 INFO:tasks.workunit.client.1.smithi149.stdout:6/970: dwrite d9/df/d22/d6d/d7d/d96/d24/d42/fb7 [0,4194304] 0 2024-03-22T07:08:58.823 INFO:tasks.workunit.client.1.smithi149.stdout:6/971: dwrite d9/df/d22/ff2 [0,4194304] 0 2024-03-22T07:08:58.843 INFO:tasks.workunit.client.1.smithi149.stdout:6/972: dwrite d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/fa0 [0,4194304] 0 2024-03-22T07:08:58.843 INFO:tasks.workunit.client.1.smithi149.stdout:6/973: readlink d9/df/d22/d6d/d7d/d96/d24/d42/l59 0 2024-03-22T07:08:59.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:08:59 smithi081 ceph-mon[29245]: pgmap v12: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 163 MiB/s rd, 210 MiB/s wr, 885 op/s 2024-03-22T07:08:59.430 INFO:tasks.workunit.client.1.smithi149.stdout:5/881: mkdir d7/da/d20/d30/d3a/d3d/d2c/d12e 0 2024-03-22T07:08:59.430 INFO:tasks.workunit.client.1.smithi149.stdout:5/882: write d7/da/d7d/f116 [703570,83952] 0 2024-03-22T07:08:59.443 INFO:tasks.workunit.client.1.smithi149.stdout:8/898: creat d1/d2/d9/d11/d8b/d8d/f127 x:0 0 0 2024-03-22T07:08:59.449 INFO:tasks.workunit.client.1.smithi149.stdout:6/974: getdents d9/df/d22/d6d/d7d/d96/d24/d34/dac 0 2024-03-22T07:08:59.453 INFO:tasks.workunit.client.1.smithi149.stdout:8/899: mkdir d1/d2/d9/df/dd5/def/d128 0 2024-03-22T07:08:59.461 INFO:tasks.workunit.client.1.smithi149.stdout:5/883: rmdir d7/da/d20/d36/d44/d4a 39 2024-03-22T07:08:59.462 INFO:tasks.workunit.client.1.smithi149.stdout:5/884: write d7/da/d20/d36/fba [968912,116281] 0 2024-03-22T07:08:59.462 INFO:tasks.workunit.client.1.smithi149.stdout:5/885: chown d7/d35/de0/dd7/c105 19775 1 2024-03-22T07:08:59.462 INFO:tasks.workunit.client.1.smithi149.stdout:5/886: getdents d7/da/d20/d89/d119 0 2024-03-22T07:08:59.463 INFO:tasks.workunit.client.1.smithi149.stdout:5/887: unlink d7/da/d20/d30/d3a/d3d/d2c/l3e 0 2024-03-22T07:08:59.464 INFO:tasks.workunit.client.1.smithi149.stdout:5/888: chown d7/d35/c5b 826204 1 2024-03-22T07:08:59.465 INFO:tasks.workunit.client.1.smithi149.stdout:5/889: rename d7/da/d20/d30/d3a/d3d/d104/c115 to d7/da/d20/d30/d3a/d3d/d2c/d12b/c12f 0 2024-03-22T07:08:59.465 INFO:tasks.workunit.client.1.smithi149.stdout:5/890: write d7/da/d20/d30/d3a/d93/d94/fb3 [36572,79138] 0 2024-03-22T07:08:59.472 INFO:tasks.workunit.client.1.smithi149.stdout:8/900: dwrite d1/d2/f8f [0,4194304] 0 2024-03-22T07:08:59.475 INFO:tasks.workunit.client.1.smithi149.stdout:5/891: dread d7/da/d20/d45/f51 [0,4194304] 0 2024-03-22T07:08:59.477 INFO:tasks.workunit.client.1.smithi149.stdout:5/892: symlink d7/da/d20/d30/d3a/l130 0 2024-03-22T07:08:59.477 INFO:tasks.workunit.client.1.smithi149.stdout:5/893: readlink d7/d35/l7a 0 2024-03-22T07:08:59.480 INFO:tasks.workunit.client.1.smithi149.stdout:8/901: dread d1/d2/d9/d11/f55 [0,4194304] 0 2024-03-22T07:08:59.484 INFO:tasks.workunit.client.1.smithi149.stdout:5/894: dread d7/da/d20/d30/d3a/d3d/d2c/f4d [0,4194304] 0 2024-03-22T07:08:59.487 INFO:tasks.workunit.client.1.smithi149.stdout:5/895: dread d7/da/d20/f5e [0,4194304] 0 2024-03-22T07:08:59.488 INFO:tasks.workunit.client.1.smithi149.stdout:5/896: link d7/d35/de0/c8a d7/d35/de0/d92/df4/df5/c131 0 2024-03-22T07:08:59.488 INFO:tasks.workunit.client.1.smithi149.stdout:5/897: creat d7/da/d7d/f132 x:0 0 0 2024-03-22T07:08:59.489 INFO:tasks.workunit.client.1.smithi149.stdout:8/902: write d1/d2/d9/d11/f17 [3105525,36524] 0 2024-03-22T07:08:59.489 INFO:tasks.workunit.client.1.smithi149.stdout:5/898: symlink d7/d35/de0/dd7/l133 0 2024-03-22T07:08:59.490 INFO:tasks.workunit.client.1.smithi149.stdout:8/903: stat d1/d2/d9/d11/d4d/dd6/d35/le9 0 2024-03-22T07:08:59.491 INFO:tasks.workunit.client.1.smithi149.stdout:5/899: creat d7/d35/de0/dd7/da4/f134 x:0 0 0 2024-03-22T07:08:59.493 INFO:tasks.workunit.client.1.smithi149.stdout:8/904: rename d1/d2/d9/f3b to d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d125/f129 0 2024-03-22T07:08:59.493 INFO:tasks.workunit.client.1.smithi149.stdout:8/905: write d1/d2/d9/d11/d8b/daf/fe7 [1268903,78775] 0 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:2/963: sync 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:3/696: sync 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:7/933: sync 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:4/918: sync 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:9/901: sync 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:0/788: sync 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:4/919: fsync d0/d8/dc/f5b 0 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:9/902: fsync db/d21/fe2 0 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:0/789: fdatasync d1/d27/f62 0 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:4/920: write d0/d8/f1f [3268506,83605] 0 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:2/964: write d1/d4d/f14d [881779,48989] 0 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:1/906: sync 2024-03-22T07:08:59.525 INFO:tasks.workunit.client.1.smithi149.stdout:1/907: chown d4/d7/dd3/d129/l115 1 1 2024-03-22T07:08:59.526 INFO:tasks.workunit.client.1.smithi149.stdout:1/908: read d4/d10/d58/f5b [3012710,63396] 0 2024-03-22T07:08:59.526 INFO:tasks.workunit.client.1.smithi149.stdout:7/934: symlink d1/d1c/d49/d8e/d100/d1b/d1d/df6/l133 0 2024-03-22T07:08:59.527 INFO:tasks.workunit.client.1.smithi149.stdout:9/903: rename db/d42/d49/d4b/d5f to db/d19/d11d 0 2024-03-22T07:08:59.528 INFO:tasks.workunit.client.1.smithi149.stdout:0/790: symlink d1/d9/d59/dc0/l119 0 2024-03-22T07:08:59.532 INFO:tasks.workunit.client.1.smithi149.stdout:1/909: rename d4/d10/d58/d68/ce0 to d4/d10/dd8/d3c/d109/c12e 0 2024-03-22T07:08:59.534 INFO:tasks.workunit.client.1.smithi149.stdout:9/904: creat db/d19/d39/d3b/dd8/f11e x:0 0 0 2024-03-22T07:08:59.535 INFO:tasks.workunit.client.1.smithi149.stdout:3/697: write d0/d2/d5a/fbf [2729715,68827] 0 2024-03-22T07:08:59.535 INFO:tasks.workunit.client.1.smithi149.stdout:8/906: dwrite d1/d2/d9/d11/d4d/dd6/db4/dce/f10c [0,4194304] 0 2024-03-22T07:08:59.535 INFO:tasks.workunit.client.1.smithi149.stdout:8/907: chown d1/d2/d9/df/c11c 1596 1 2024-03-22T07:08:59.535 INFO:tasks.workunit.client.1.smithi149.stdout:8/908: chown d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f7e 440 1 2024-03-22T07:08:59.535 INFO:tasks.workunit.client.1.smithi149.stdout:8/909: write d1/d2/d9/d11/d8b/daf/f123 [309121,112145] 0 2024-03-22T07:08:59.535 INFO:tasks.workunit.client.1.smithi149.stdout:4/921: getdents d0/d8/d4c 0 2024-03-22T07:08:59.535 INFO:tasks.workunit.client.1.smithi149.stdout:4/922: readlink d0/d8/dc/d14/d85/la8 0 2024-03-22T07:08:59.540 INFO:tasks.workunit.client.1.smithi149.stdout:7/935: rename d1/d1c/d49/d8e/d100/d1b/d51/d5a/df0/f108 to d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/f134 0 2024-03-22T07:08:59.540 INFO:tasks.workunit.client.1.smithi149.stdout:7/936: chown d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/d9e/fff 3843798 1 2024-03-22T07:08:59.541 INFO:tasks.workunit.client.1.smithi149.stdout:1/910: mkdir d4/d7/d5d/d12f 0 2024-03-22T07:08:59.546 INFO:tasks.workunit.client.1.smithi149.stdout:8/910: rmdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/d9e/df3 0 2024-03-22T07:08:59.547 INFO:tasks.workunit.client.1.smithi149.stdout:9/905: rename db/d42/dab to db/d22/d11f 0 2024-03-22T07:08:59.548 INFO:tasks.workunit.client.1.smithi149.stdout:7/937: link d1/d1c/d49/d8e/d100/d1b/d51/d5a/fe9 d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/f135 0 2024-03-22T07:08:59.548 INFO:tasks.workunit.client.1.smithi149.stdout:7/938: fdatasync d1/d1c/f98 0 2024-03-22T07:08:59.550 INFO:tasks.workunit.client.1.smithi149.stdout:1/911: truncate d4/d20/d2d/d64/d9f/dbe/fd5 589980 0 2024-03-22T07:08:59.551 INFO:tasks.workunit.client.1.smithi149.stdout:0/791: dwrite d1/d31/d52/dd4/f100 [0,4194304] 0 2024-03-22T07:08:59.551 INFO:tasks.workunit.client.1.smithi149.stdout:2/965: dwrite d1/f160 [0,4194304] 0 2024-03-22T07:08:59.551 INFO:tasks.workunit.client.1.smithi149.stdout:2/966: dread - d1/d8/fe9 zero size 2024-03-22T07:08:59.551 INFO:tasks.workunit.client.1.smithi149.stdout:2/967: dread - d1/d72/f121 zero size 2024-03-22T07:08:59.552 INFO:tasks.workunit.client.1.smithi149.stdout:7/939: creat d1/d1c/d12c/f136 x:0 0 0 2024-03-22T07:08:59.553 INFO:tasks.workunit.client.1.smithi149.stdout:8/911: truncate d1/d2/d18/f3c 344776 0 2024-03-22T07:08:59.554 INFO:tasks.workunit.client.1.smithi149.stdout:1/912: unlink d4/d7/d5a/dca/ccc 0 2024-03-22T07:08:59.562 INFO:tasks.workunit.client.1.smithi149.stdout:0/792: link d1/d20/d9f/dc6/dd9/f105 d1/d9/d14/d74/d77/f11a 0 2024-03-22T07:08:59.565 INFO:tasks.workunit.client.1.smithi149.stdout:0/793: dread d1/d1a/f2d [0,4194304] 0 2024-03-22T07:08:59.565 INFO:tasks.workunit.client.1.smithi149.stdout:0/794: chown d1/d31/l3b 1038 1 2024-03-22T07:08:59.566 INFO:tasks.workunit.client.1.smithi149.stdout:7/940: mknod d1/d1c/d49/d8e/d100/d1b/d1d/d72/c137 0 2024-03-22T07:08:59.566 INFO:tasks.workunit.client.1.smithi149.stdout:7/941: fdatasync d1/d1c/d49/d8e/d100/d1b/d51/d5a/d6a/fd6 0 2024-03-22T07:08:59.567 INFO:tasks.workunit.client.1.smithi149.stdout:2/968: dwrite d1/d3/d5/d1f/f39 [0,4194304] 0 2024-03-22T07:08:59.567 INFO:tasks.workunit.client.1.smithi149.stdout:2/969: truncate d1/d72/f124 476299 0 2024-03-22T07:08:59.567 INFO:tasks.workunit.client.1.smithi149.stdout:8/912: rmdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d82 39 2024-03-22T07:08:59.568 INFO:tasks.workunit.client.1.smithi149.stdout:1/913: mkdir d4/d7/dd3/d129/d93/d130 0 2024-03-22T07:08:59.568 INFO:tasks.workunit.client.1.smithi149.stdout:1/914: truncate d4/d7/dd3/d129/d93/f9c 5061330 0 2024-03-22T07:08:59.570 INFO:tasks.workunit.client.1.smithi149.stdout:0/795: creat d1/d27/d66/f11b x:0 0 0 2024-03-22T07:08:59.572 INFO:tasks.workunit.client.1.smithi149.stdout:7/942: creat d1/d1c/d49/d8e/d100/d1b/d11a/f138 x:0 0 0 2024-03-22T07:08:59.573 INFO:tasks.workunit.client.1.smithi149.stdout:8/913: creat d1/d2/d9/df/dd5/def/d128/f12a x:0 0 0 2024-03-22T07:08:59.576 INFO:tasks.workunit.client.1.smithi149.stdout:1/915: rename d4/d20/d2d/d64/fb3 to d4/d7/d5d/f131 0 2024-03-22T07:08:59.578 INFO:tasks.workunit.client.1.smithi149.stdout:2/970: read d1/d3/d9a/fd5 [3726204,13852] 0 2024-03-22T07:08:59.581 INFO:tasks.workunit.client.1.smithi149.stdout:8/914: creat d1/d2/d9/d11/d4d/dd6/f12b x:0 0 0 2024-03-22T07:08:59.581 INFO:tasks.workunit.client.1.smithi149.stdout:1/916: mkdir d4/d10/dd8/d3c/d132 0 2024-03-22T07:08:59.583 INFO:tasks.workunit.client.1.smithi149.stdout:7/943: write d1/d1c/d49/d8e/d100/f8c [3696691,18217] 0 2024-03-22T07:08:59.583 INFO:tasks.workunit.client.1.smithi149.stdout:7/944: chown d1/fdb 26960860 1 2024-03-22T07:08:59.587 INFO:tasks.workunit.client.1.smithi149.stdout:1/917: link d4/d10/d7c/d6a/f113 d4/d20/d2d/d64/d9f/dbe/f133 0 2024-03-22T07:08:59.587 INFO:tasks.workunit.client.1.smithi149.stdout:1/918: write d4/d7/f57 [1942895,2804] 0 2024-03-22T07:08:59.587 INFO:tasks.workunit.client.1.smithi149.stdout:8/915: mkdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d125/d12c 0 2024-03-22T07:08:59.587 INFO:tasks.workunit.client.1.smithi149.stdout:7/945: rename d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/db6/cb9 to d1/d1c/d49/d8e/d9c/df7/c139 0 2024-03-22T07:08:59.589 INFO:tasks.workunit.client.1.smithi149.stdout:0/796: dwrite d1/d20/d9f/dc6/ddc/fdd [0,4194304] 0 2024-03-22T07:08:59.590 INFO:tasks.workunit.client.1.smithi149.stdout:8/916: rmdir d1/d2/d9/d11/d8b/d8d/d105 0 2024-03-22T07:08:59.591 INFO:tasks.workunit.client.1.smithi149.stdout:0/797: write d1/d1a/f2d [9784,91029] 0 2024-03-22T07:08:59.592 INFO:tasks.workunit.client.1.smithi149.stdout:0/798: truncate d1/d9/d59/dc0/dc9/dcc/fe6 1159999 0 2024-03-22T07:08:59.592 INFO:tasks.workunit.client.1.smithi149.stdout:0/799: write d1/d9/d14/d1f/f24 [2766574,5805] 0 2024-03-22T07:08:59.594 INFO:tasks.workunit.client.1.smithi149.stdout:7/946: unlink d1/d1c/d49/d8e/d100/d1b/d51/d5a/d6a/da6/d93/fde 0 2024-03-22T07:08:59.595 INFO:tasks.workunit.client.1.smithi149.stdout:7/947: dread - d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/dee/f12a zero size 2024-03-22T07:08:59.595 INFO:tasks.workunit.client.1.smithi149.stdout:8/917: creat d1/d2/d9/d11/d8b/dee/f12d x:0 0 0 2024-03-22T07:08:59.595 INFO:tasks.workunit.client.1.smithi149.stdout:0/800: rename d1/d9/d14/d1f/l7b to d1/d20/d9f/dc6/dd9/l11c 0 2024-03-22T07:08:59.597 INFO:tasks.workunit.client.1.smithi149.stdout:0/801: mkdir d1/d31/d11d 0 2024-03-22T07:08:59.597 INFO:tasks.workunit.client.1.smithi149.stdout:8/918: dread d1/d2/d9/d11/d8b/fb1 [0,4194304] 0 2024-03-22T07:08:59.600 INFO:tasks.workunit.client.1.smithi149.stdout:8/919: dread d1/d2/d9/d11/d4d/dd6/f66 [4194304,4194304] 0 2024-03-22T07:08:59.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:08:59 smithi149 ceph-mon[36068]: pgmap v12: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 163 MiB/s rd, 210 MiB/s wr, 885 op/s 2024-03-22T07:08:59.609 INFO:tasks.workunit.client.1.smithi149.stdout:2/971: dwrite d1/d72/f124 [0,4194304] 0 2024-03-22T07:08:59.612 INFO:tasks.workunit.client.1.smithi149.stdout:1/919: dwrite d4/d20/da0/dc0/ff7 [0,4194304] 0 2024-03-22T07:08:59.614 INFO:tasks.workunit.client.1.smithi149.stdout:2/972: mknod d1/d3/d5/d41/d9f/da7/d157/d129/d74/c161 0 2024-03-22T07:08:59.630 INFO:tasks.workunit.client.1.smithi149.stdout:0/802: dwrite d1/d31/d52/da9/fce [0,4194304] 0 2024-03-22T07:08:59.631 INFO:tasks.workunit.client.1.smithi149.stdout:0/803: dread d1/d20/da3/fab [0,4194304] 0 2024-03-22T07:08:59.633 INFO:tasks.workunit.client.1.smithi149.stdout:0/804: creat d1/d9/d14/d1f/f11e x:0 0 0 2024-03-22T07:08:59.638 INFO:tasks.workunit.client.1.smithi149.stdout:1/920: dwrite d4/d10/d7c/d6a/ff9 [0,4194304] 0 2024-03-22T07:08:59.638 INFO:tasks.workunit.client.1.smithi149.stdout:8/920: dwrite d1/d2/f8f [0,4194304] 0 2024-03-22T07:08:59.639 INFO:tasks.workunit.client.1.smithi149.stdout:7/948: dwrite d1/d1c/d49/d8e/d100/d1b/d51/d5a/de7/f135 [0,4194304] 0 2024-03-22T07:08:59.650 INFO:tasks.workunit.client.1.smithi149.stdout:1/921: dread d4/d20/f52 [4194304,4194304] 0 2024-03-22T07:08:59.650 INFO:tasks.workunit.client.1.smithi149.stdout:1/922: fsync d4/d10/dd8/d3c/d8e/db5/db2/ffd 0 2024-03-22T07:08:59.650 INFO:tasks.workunit.client.1.smithi149.stdout:0/805: dwrite d1/d9/d14/d74/d77/f11a [0,4194304] 0 2024-03-22T07:08:59.655 INFO:tasks.workunit.client.1.smithi149.stdout:0/806: mknod d1/c11f 0 2024-03-22T07:08:59.656 INFO:tasks.workunit.client.1.smithi149.stdout:1/923: symlink d4/d7/d5a/l134 0 2024-03-22T07:08:59.657 INFO:tasks.workunit.client.1.smithi149.stdout:1/924: link d4/d7/d5d/c8b d4/d20/da0/d114/c135 0 2024-03-22T07:08:59.659 INFO:tasks.workunit.client.1.smithi149.stdout:1/925: rename d4/d7/dd3/d129/l115 to d4/d7/dd3/d129/d93/l136 0 2024-03-22T07:08:59.659 INFO:tasks.workunit.client.1.smithi149.stdout:1/926: dread - d4/d20/da0/fb0 zero size 2024-03-22T07:08:59.660 INFO:tasks.workunit.client.1.smithi149.stdout:1/927: creat d4/d7/d5d/d12f/f137 x:0 0 0 2024-03-22T07:08:59.663 INFO:tasks.workunit.client.1.smithi149.stdout:1/928: link d4/d7/d5d/d12f/f137 d4/d7/dd3/d129/d8c/daa/f138 0 2024-03-22T07:08:59.663 INFO:tasks.workunit.client.1.smithi149.stdout:1/929: fdatasync d4/d7/f50 0 2024-03-22T07:08:59.663 INFO:tasks.workunit.client.1.smithi149.stdout:1/930: chown d4/d10/d7c/la2 958743 1 2024-03-22T07:08:59.664 INFO:tasks.workunit.client.1.smithi149.stdout:1/931: mkdir d4/d10/dd8/d59/d139 0 2024-03-22T07:08:59.667 INFO:tasks.workunit.client.1.smithi149.stdout:1/932: rename d4/d20/ff4 to d4/d10/dd8/d3c/d8e/db5/db2/dbf/f13a 0 2024-03-22T07:08:59.667 INFO:tasks.workunit.client.1.smithi149.stdout:1/933: stat d4/d10/c4a 0 2024-03-22T07:08:59.667 INFO:tasks.workunit.client.1.smithi149.stdout:1/934: dread - d4/d10/dd8/d3c/d8e/db5/ddb/f12c zero size 2024-03-22T07:08:59.668 INFO:tasks.workunit.client.1.smithi149.stdout:1/935: write d4/d7/d5a/f87 [823973,56364] 0 2024-03-22T07:08:59.669 INFO:tasks.workunit.client.1.smithi149.stdout:1/936: creat d4/d7/d94/dd4/f13b x:0 0 0 2024-03-22T07:08:59.756 INFO:tasks.workunit.client.1.smithi149.stdout:0/807: dwrite d1/d20/d9f/dc6/dd9/fda [0,4194304] 0 2024-03-22T07:08:59.756 INFO:tasks.workunit.client.1.smithi149.stdout:0/808: dread - d1/d31/d51/d64/d8d/d9c/d9e/ff3 zero size 2024-03-22T07:08:59.756 INFO:tasks.workunit.client.1.smithi149.stdout:1/937: dwrite d4/d10/dd8/d3c/d85/d89/fe2 [0,4194304] 0 2024-03-22T07:08:59.756 INFO:tasks.workunit.client.1.smithi149.stdout:0/809: write d1/fd7 [403188,113082] 0 2024-03-22T07:08:59.760 INFO:tasks.workunit.client.1.smithi149.stdout:1/938: symlink d4/d7/d5a/dca/l13c 0 2024-03-22T07:08:59.760 INFO:tasks.workunit.client.1.smithi149.stdout:1/939: write d4/d7/dd3/d129/d8c/daa/f138 [573909,34580] 0 2024-03-22T07:08:59.761 INFO:tasks.workunit.client.1.smithi149.stdout:1/940: truncate d4/d7/d5d/d82/f102 2879042 0 2024-03-22T07:08:59.763 INFO:tasks.workunit.client.1.smithi149.stdout:1/941: link d4/d10/dd8/d3c/d8e/db5/ccb d4/d10/dd8/d3c/d8e/c13d 0 2024-03-22T07:08:59.773 INFO:tasks.workunit.client.1.smithi149.stdout:1/942: dread d4/d10/f4d [0,4194304] 0 2024-03-22T07:08:59.775 INFO:tasks.workunit.client.1.smithi149.stdout:0/810: dwrite d1/d9/d14/fa1 [0,4194304] 0 2024-03-22T07:08:59.784 INFO:tasks.workunit.client.1.smithi149.stdout:0/811: read d1/d9/d14/f18 [1013179,50023] 0 2024-03-22T07:08:59.795 INFO:tasks.workunit.client.1.smithi149.stdout:1/943: rename d4/d20/d2d/cd9 to d4/d7/d5d/c13e 0 2024-03-22T07:08:59.798 INFO:tasks.workunit.client.1.smithi149.stdout:0/812: rename d1/d9/d14/d1f/lbe to d1/d31/d51/l120 0 2024-03-22T07:08:59.799 INFO:tasks.workunit.client.1.smithi149.stdout:0/813: creat d1/d31/d51/d64/d8d/d9c/d9e/d10c/d108/f121 x:0 0 0 2024-03-22T07:08:59.801 INFO:tasks.workunit.client.1.smithi149.stdout:1/944: getdents d4/d7/d5a 0 2024-03-22T07:08:59.802 INFO:tasks.workunit.client.1.smithi149.stdout:0/814: creat d1/d31/d51/d64/d8d/d9c/d9e/d10c/f122 x:0 0 0 2024-03-22T07:08:59.804 INFO:tasks.workunit.client.1.smithi149.stdout:1/945: truncate d4/d10/dd8/d3c/d8e/db5/f34 582521 0 2024-03-22T07:08:59.805 INFO:tasks.workunit.client.1.smithi149.stdout:0/815: symlink d1/d31/d10f/l123 0 2024-03-22T07:08:59.805 INFO:tasks.workunit.client.1.smithi149.stdout:0/816: chown d1/d20/da3/f115 870063250 1 2024-03-22T07:08:59.805 INFO:tasks.workunit.client.1.smithi149.stdout:1/946: truncate d4/d7/f79 335025 0 2024-03-22T07:08:59.807 INFO:tasks.workunit.client.1.smithi149.stdout:0/817: symlink d1/d6b/l124 0 2024-03-22T07:08:59.808 INFO:tasks.workunit.client.1.smithi149.stdout:0/818: symlink d1/d31/d51/d64/d7e/l125 0 2024-03-22T07:08:59.813 INFO:tasks.workunit.client.1.smithi149.stdout:1/947: rename d4/d10/d58 to d4/d10/dd8/d3c/d109/d13f 0 2024-03-22T07:08:59.815 INFO:tasks.workunit.client.1.smithi149.stdout:1/948: link d4/f1b d4/d7/d61/f140 0 2024-03-22T07:08:59.815 INFO:tasks.workunit.client.1.smithi149.stdout:1/949: chown d4 6891 1 2024-03-22T07:08:59.815 INFO:tasks.workunit.client.1.smithi149.stdout:1/950: dread - d4/d7/dd3/d129/d93/fc9 zero size 2024-03-22T07:08:59.816 INFO:tasks.workunit.client.1.smithi149.stdout:1/951: symlink d4/d10/dd8/d3c/d8e/l141 0 2024-03-22T07:08:59.816 INFO:tasks.workunit.client.1.smithi149.stdout:1/952: readlink d4/d7/d61/l110 0 2024-03-22T07:08:59.816 INFO:tasks.workunit.client.1.smithi149.stdout:1/953: truncate d4/d7/d5d/d82/f102 3868444 0 2024-03-22T07:08:59.816 INFO:tasks.workunit.client.1.smithi149.stdout:0/819: dread d1/d9/d14/f18 [0,4194304] 0 2024-03-22T07:08:59.816 INFO:tasks.workunit.client.1.smithi149.stdout:0/820: mknod d1/d20/d9f/dc6/c126 0 2024-03-22T07:08:59.817 INFO:tasks.workunit.client.1.smithi149.stdout:1/954: getdents d4/d7/d5d/d82 0 2024-03-22T07:08:59.818 INFO:tasks.workunit.client.1.smithi149.stdout:0/821: symlink d1/d31/d51/l127 0 2024-03-22T07:08:59.818 INFO:tasks.workunit.client.1.smithi149.stdout:0/822: dread - d1/d31/d51/d64/d8d/d9c/d9e/ff3 zero size 2024-03-22T07:08:59.818 INFO:tasks.workunit.client.1.smithi149.stdout:1/955: symlink d4/d20/da0/d114/l142 0 2024-03-22T07:08:59.818 INFO:tasks.workunit.client.1.smithi149.stdout:0/823: stat d1/d31/d51/d64/d8d/d9c/d9e/cbf 0 2024-03-22T07:08:59.821 INFO:tasks.workunit.client.1.smithi149.stdout:6/975: dwrite d9/f3c [4194304,4194304] 0 2024-03-22T07:08:59.821 INFO:tasks.workunit.client.1.smithi149.stdout:6/976: write d9/df/d22/d6d/d7d/fb6 [1657220,40229] 0 2024-03-22T07:08:59.822 INFO:tasks.workunit.client.1.smithi149.stdout:0/824: dread d1/d9/d14/d1f/f99 [0,4194304] 0 2024-03-22T07:08:59.822 INFO:tasks.workunit.client.1.smithi149.stdout:0/825: creat d1/d31/d51/d64/f128 x:0 0 0 2024-03-22T07:08:59.825 INFO:tasks.workunit.client.1.smithi149.stdout:0/826: symlink d1/d9/d14/d74/l129 0 2024-03-22T07:08:59.825 INFO:tasks.workunit.client.1.smithi149.stdout:0/827: truncate d1/d9/d14/d74/d77/ff2 408646 0 2024-03-22T07:08:59.825 INFO:tasks.workunit.client.1.smithi149.stdout:0/828: truncate d1/d27/d66/f11b 994163 0 2024-03-22T07:08:59.825 INFO:tasks.workunit.client.1.smithi149.stdout:6/977: write d9/d47/f51 [8721702,94188] 0 2024-03-22T07:08:59.825 INFO:tasks.workunit.client.1.smithi149.stdout:6/978: chown d9/da2/cfd 7304552 1 2024-03-22T07:08:59.826 INFO:tasks.workunit.client.1.smithi149.stdout:6/979: stat d9/l3a 0 2024-03-22T07:08:59.826 INFO:tasks.workunit.client.1.smithi149.stdout:6/980: chown d9/df/d22/d6d/d7d/d96/d24/d68/d73/f13e 72332281 1 2024-03-22T07:08:59.828 INFO:tasks.workunit.client.1.smithi149.stdout:6/981: unlink d9/df/d22/d6d/deb/fcc 0 2024-03-22T07:08:59.849 INFO:tasks.workunit.client.1.smithi149.stdout:1/956: dwrite d4/d20/f52 [0,4194304] 0 2024-03-22T07:08:59.849 INFO:tasks.workunit.client.1.smithi149.stdout:1/957: chown d4/c4b 616697960 1 2024-03-22T07:08:59.850 INFO:tasks.workunit.client.1.smithi149.stdout:6/982: dwrite d9/da2/fff [0,4194304] 0 2024-03-22T07:08:59.851 INFO:tasks.workunit.client.1.smithi149.stdout:0/829: dwrite d1/d31/d52/f88 [4194304,4194304] 0 2024-03-22T07:08:59.852 INFO:tasks.workunit.client.1.smithi149.stdout:0/830: read d1/d20/d9f/dc6/fcf [215069,7156] 0 2024-03-22T07:08:59.853 INFO:tasks.workunit.client.1.smithi149.stdout:0/831: symlink d1/d20/da3/l12a 0 2024-03-22T07:08:59.854 INFO:tasks.workunit.client.1.smithi149.stdout:0/832: symlink d1/d9/d59/dc0/dc9/dcc/l12b 0 2024-03-22T07:08:59.855 INFO:tasks.workunit.client.1.smithi149.stdout:0/833: truncate d1/d9/d14/d74/d77/ff2 764501 0 2024-03-22T07:08:59.857 INFO:tasks.workunit.client.1.smithi149.stdout:0/834: symlink d1/d9/d14/d1f/l12c 0 2024-03-22T07:08:59.862 INFO:tasks.workunit.client.1.smithi149.stdout:1/958: dread d4/d10/dd8/d3c/f47 [0,4194304] 0 2024-03-22T07:08:59.862 INFO:tasks.workunit.client.1.smithi149.stdout:1/959: read d4/d10/dd8/d3c/d85/d89/f97 [3377049,54277] 0 2024-03-22T07:08:59.864 INFO:tasks.workunit.client.1.smithi149.stdout:1/960: mkdir d4/d10/dd8/d59/d143 0 2024-03-22T07:08:59.864 INFO:tasks.workunit.client.1.smithi149.stdout:1/961: stat d4/d7/dd3/d129/d8c 0 2024-03-22T07:08:59.865 INFO:tasks.workunit.client.1.smithi149.stdout:1/962: rmdir d4/d20/d2d/d64/d104 39 2024-03-22T07:08:59.865 INFO:tasks.workunit.client.1.smithi149.stdout:1/963: dread - d4/d10/dd8/d3c/d85/fd6 zero size 2024-03-22T07:08:59.867 INFO:tasks.workunit.client.1.smithi149.stdout:1/964: unlink d4/d7/f57 0 2024-03-22T07:08:59.867 INFO:tasks.workunit.client.1.smithi149.stdout:6/983: dwrite d9/df/d22/f67 [0,4194304] 0 2024-03-22T07:08:59.868 INFO:tasks.workunit.client.1.smithi149.stdout:1/965: symlink d4/d10/dd8/d3c/d8e/db5/ddb/l144 0 2024-03-22T07:08:59.869 INFO:tasks.workunit.client.1.smithi149.stdout:1/966: link d4/d10/dd8/d3c/d8e/db5/f65 d4/d20/da0/dc0/f145 0 2024-03-22T07:08:59.875 INFO:tasks.workunit.client.1.smithi149.stdout:0/835: dwrite d1/f39 [0,4194304] 0 2024-03-22T07:08:59.876 INFO:tasks.workunit.client.1.smithi149.stdout:6/984: mkdir d9/d47/d13f 0 2024-03-22T07:08:59.880 INFO:tasks.workunit.client.1.smithi149.stdout:6/985: rename d9/df/d22/d6d/d7d/d96/db1/d55/l105 to d9/df/d22/d6d/d7d/d96/d24/d34/d37/d134/l140 0 2024-03-22T07:08:59.880 INFO:tasks.workunit.client.1.smithi149.stdout:6/986: mknod d9/df/d22/d6d/deb/c141 0 2024-03-22T07:08:59.881 INFO:tasks.workunit.client.1.smithi149.stdout:6/987: mknod d9/d47/c142 0 2024-03-22T07:08:59.882 INFO:tasks.workunit.client.1.smithi149.stdout:6/988: symlink d9/d47/d13f/l143 0 2024-03-22T07:08:59.886 INFO:tasks.workunit.client.1.smithi149.stdout:1/967: dwrite d4/d10/dd8/f74 [0,4194304] 0 2024-03-22T07:08:59.887 INFO:tasks.workunit.client.1.smithi149.stdout:1/968: stat d4/d7/d5d/c13e 0 2024-03-22T07:08:59.889 INFO:tasks.workunit.client.1.smithi149.stdout:1/969: creat d4/f146 x:0 0 0 2024-03-22T07:08:59.889 INFO:tasks.workunit.client.1.smithi149.stdout:1/970: read d4/d10/d7c/f7d [9937,1555] 0 2024-03-22T07:08:59.889 INFO:tasks.workunit.client.1.smithi149.stdout:1/971: chown d4/d7/d61 1063673565 1 2024-03-22T07:08:59.893 INFO:tasks.workunit.client.1.smithi149.stdout:0/836: dwrite d1/d31/d51/d64/d87/f10a [0,4194304] 0 2024-03-22T07:08:59.894 INFO:tasks.workunit.client.1.smithi149.stdout:0/837: chown d1/d9/d59/cac 538 1 2024-03-22T07:08:59.913 INFO:tasks.workunit.client.1.smithi149.stdout:6/989: dwrite d9/df/d22/d6d/d7d/d96/d24/d42/fb7 [0,4194304] 0 2024-03-22T07:08:59.915 INFO:tasks.workunit.client.1.smithi149.stdout:0/838: dwrite d1/d1a/f2d [0,4194304] 0 2024-03-22T07:08:59.917 INFO:tasks.workunit.client.1.smithi149.stdout:0/839: creat d1/d31/d11d/f12d x:0 0 0 2024-03-22T07:08:59.917 INFO:tasks.workunit.client.1.smithi149.stdout:0/840: read d1/d9/f3a [370066,27396] 0 2024-03-22T07:08:59.917 INFO:tasks.workunit.client.1.smithi149.stdout:0/841: fsync d1/d31/d51/d64/d8d/d9c/fb5 0 2024-03-22T07:08:59.917 INFO:tasks.workunit.client.1.smithi149.stdout:0/842: read d1/fb1 [757098,82687] 0 2024-03-22T07:08:59.919 INFO:tasks.workunit.client.1.smithi149.stdout:0/843: rename d1/d9/d14/d1f/f99 to d1/d9/d14/d74/f12e 0 2024-03-22T07:08:59.920 INFO:tasks.workunit.client.1.smithi149.stdout:0/844: symlink d1/d9/d59/dc0/dc9/dcc/l12f 0 2024-03-22T07:08:59.920 INFO:tasks.workunit.client.1.smithi149.stdout:0/845: truncate d1/d9/d59/dc0/dc9/dcc/fe6 1438085 0 2024-03-22T07:08:59.920 INFO:tasks.workunit.client.1.smithi149.stdout:0/846: creat d1/d31/d11d/f130 x:0 0 0 2024-03-22T07:08:59.921 INFO:tasks.workunit.client.1.smithi149.stdout:6/990: dread d9/df/d2e/d63/f69 [0,4194304] 0 2024-03-22T07:08:59.922 INFO:tasks.workunit.client.1.smithi149.stdout:6/991: chown d9/df/d22/d6d/d7d/fb6 44390 1 2024-03-22T07:08:59.924 INFO:tasks.workunit.client.1.smithi149.stdout:6/992: rmdir d9/df 39 2024-03-22T07:08:59.925 INFO:tasks.workunit.client.1.smithi149.stdout:6/993: getdents d9/df/d22/d6d/d7d/d96/db1 0 2024-03-22T07:08:59.927 INFO:tasks.workunit.client.1.smithi149.stdout:6/994: link d9/df/d22/d6d/d7d/d96/d24/d34/d37/l122 d9/df/d22/d6d/d7d/d96/d24/d34/dac/l144 0 2024-03-22T07:08:59.927 INFO:tasks.workunit.client.1.smithi149.stdout:6/995: truncate d9/da2/fc2 4614075 0 2024-03-22T07:08:59.928 INFO:tasks.workunit.client.1.smithi149.stdout:6/996: creat d9/df/d22/dc0/dc8/de9/f145 x:0 0 0 2024-03-22T07:08:59.928 INFO:tasks.workunit.client.1.smithi149.stdout:6/997: fsync d9/d47/d52/f8a 0 2024-03-22T07:08:59.929 INFO:tasks.workunit.client.1.smithi149.stdout:6/998: mkdir d9/df/d22/d6d/d7d/d96/d24/d68/d73/d146 0 2024-03-22T07:08:59.930 INFO:tasks.workunit.client.1.smithi149.stdout:6/999: write d9/df/d22/d6d/d7d/d96/d24/d34/d37/d38/d98/f97 [762345,10610] 0 2024-03-22T07:08:59.940 INFO:tasks.workunit.client.1.smithi149.stdout:0/847: dwrite d1/d31/f33 [0,4194304] 0 2024-03-22T07:08:59.941 INFO:tasks.workunit.client.1.smithi149.stdout:0/848: mknod d1/d31/d51/d64/d8d/d9c/c131 0 2024-03-22T07:08:59.942 INFO:tasks.workunit.client.1.smithi149.stdout:0/849: unlink d1/fb1 0 2024-03-22T07:08:59.943 INFO:tasks.workunit.client.1.smithi149.stdout:0/850: rename d1/d9/d14/d74/l129 to d1/d20/l132 0 2024-03-22T07:08:59.944 INFO:tasks.workunit.client.1.smithi149.stdout:0/851: creat d1/d31/f133 x:0 0 0 2024-03-22T07:08:59.945 INFO:tasks.workunit.client.1.smithi149.stdout:0/852: truncate d1/d27/f62 2003520 0 2024-03-22T07:08:59.945 INFO:tasks.workunit.client.1.smithi149.stdout:0/853: write d1/d20/d9f/dc6/fcf [705028,20338] 0 2024-03-22T07:08:59.950 INFO:tasks.workunit.client.1.smithi149.stdout:0/854: link d1/d20/l132 d1/d9/d14/d74/l134 0 2024-03-22T07:08:59.951 INFO:tasks.workunit.client.1.smithi149.stdout:0/855: symlink d1/d31/d10f/l135 0 2024-03-22T07:08:59.952 INFO:tasks.workunit.client.1.smithi149.stdout:0/856: truncate d1/d31/d52/f88 5064208 0 2024-03-22T07:08:59.954 INFO:tasks.workunit.client.1.smithi149.stdout:0/857: rename d1/d31/d51/d64/d8d/d9c/d9e to d1/d136 0 2024-03-22T07:08:59.964 INFO:tasks.workunit.client.1.smithi149.stdout:0/858: dread d1/d31/f33 [0,4194304] 0 2024-03-22T07:08:59.964 INFO:tasks.workunit.client.1.smithi149.stdout:0/859: write d1/d9/d14/d1f/f4d [5148279,58002] 0 2024-03-22T07:08:59.979 INFO:tasks.workunit.client.1.smithi149.stdout:0/860: dwrite d1/d31/d51/d64/d8d/d9c/f101 [0,4194304] 0 2024-03-22T07:08:59.980 INFO:tasks.workunit.client.1.smithi149.stdout:0/861: mknod d1/d31/d52/c137 0 2024-03-22T07:08:59.981 INFO:tasks.workunit.client.1.smithi149.stdout:0/862: getdents d1/d31/d52/da9 0 2024-03-22T07:08:59.982 INFO:tasks.workunit.client.1.smithi149.stdout:0/863: chown d1/d31/d51/d64/d87/f10a 735 1 2024-03-22T07:08:59.999 INFO:tasks.workunit.client.1.smithi149.stdout:0/864: dwrite d1/d31/f8a [0,4194304] 0 2024-03-22T07:09:00.009 INFO:tasks.workunit.client.1.smithi149.stdout:0/865: write d1/d9/d59/f61 [2644559,92727] 0 2024-03-22T07:09:00.009 INFO:tasks.workunit.client.1.smithi149.stdout:0/866: readlink d1/d31/d10f/l106 0 2024-03-22T07:09:00.009 INFO:tasks.workunit.client.1.smithi149.stdout:0/867: fsync d1/d31/f33 0 2024-03-22T07:09:00.010 INFO:tasks.workunit.client.1.smithi149.stdout:0/868: write d1/f4a [608808,82308] 0 2024-03-22T07:09:00.011 INFO:tasks.workunit.client.1.smithi149.stdout:0/869: creat d1/d6b/f138 x:0 0 0 2024-03-22T07:09:00.019 INFO:tasks.workunit.client.1.smithi149.stdout:0/870: write d1/d31/d51/d64/d8d/d9c/fb5 [2184585,130570] 0 2024-03-22T07:09:00.020 INFO:tasks.workunit.client.1.smithi149.stdout:0/871: unlink d1/d9/d59/dc0/dc9/ff8 0 2024-03-22T07:09:00.020 INFO:tasks.workunit.client.1.smithi149.stdout:0/872: chown d1/d31/d11d/f130 48 1 2024-03-22T07:09:00.021 INFO:tasks.workunit.client.1.smithi149.stdout:0/873: mknod d1/d31/d51/d64/d8d/d9c/c139 0 2024-03-22T07:09:00.022 INFO:tasks.workunit.client.1.smithi149.stdout:0/874: mkdir d1/d9/d14/d74/d77/d13a 0 2024-03-22T07:09:00.022 INFO:tasks.workunit.client.1.smithi149.stdout:0/875: write d1/d27/d66/f11b [341913,58965] 0 2024-03-22T07:09:00.042 INFO:tasks.workunit.client.1.smithi149.stdout:0/876: dwrite d1/d9/d14/d1f/f11e [0,4194304] 0 2024-03-22T07:09:00.042 INFO:tasks.workunit.client.1.smithi149.stdout:0/877: stat d1/d31/d51/f5b 0 2024-03-22T07:09:00.043 INFO:tasks.workunit.client.1.smithi149.stdout:0/878: chown d1/d1a/f2d 303796740 1 2024-03-22T07:09:00.044 INFO:tasks.workunit.client.1.smithi149.stdout:0/879: mkdir d1/d20/d9f/dc6/dd9/d13b 0 2024-03-22T07:09:00.045 INFO:tasks.workunit.client.1.smithi149.stdout:0/880: rename d1/d9/d14/f85 to d1/d9/d14/f13c 0 2024-03-22T07:09:00.045 INFO:tasks.workunit.client.1.smithi149.stdout:0/881: dread - d1/d31/f133 zero size 2024-03-22T07:09:00.052 INFO:tasks.workunit.client.1.smithi149.stdout:0/882: dread d1/d31/d51/d64/d87/da6/fd6 [0,4194304] 0 2024-03-22T07:09:00.395 INFO:tasks.workunit.client.1.smithi149.stdout:5/900: sync 2024-03-22T07:09:00.401 INFO:tasks.workunit.client.1.smithi149.stdout:5/901: dread d7/da/d20/d30/d3a/d3d/d2c/f8b [0,4194304] 0 2024-03-22T07:09:00.404 INFO:tasks.workunit.client.1.smithi149.stdout:5/902: dread d7/da/d20/f5e [0,4194304] 0 2024-03-22T07:09:00.406 INFO:tasks.workunit.client.1.smithi149.stdout:5/903: dread d7/da/fd [0,4194304] 0 2024-03-22T07:09:00.406 INFO:tasks.workunit.client.1.smithi149.stdout:5/904: chown d7/da/d20/d30/d3a/d93/ce8 8096926 1 2024-03-22T07:09:00.407 INFO:tasks.workunit.client.1.smithi149.stdout:5/905: chown d7/d35/l39 7734650 1 2024-03-22T07:09:00.408 INFO:tasks.workunit.client.1.smithi149.stdout:5/906: rename d7/da/d20/d30/d3a/d3d/c75 to d7/d35/de0/dd7/dea/d102/c135 0 2024-03-22T07:09:00.408 INFO:tasks.workunit.client.1.smithi149.stdout:5/907: creat d7/da/d20/d30/d3a/f136 x:0 0 0 2024-03-22T07:09:00.410 INFO:tasks.workunit.client.1.smithi149.stdout:5/908: rename d7/d35/de0/f71 to d7/d35/de0/dd7/d7b/d2b/f137 0 2024-03-22T07:09:00.411 INFO:tasks.workunit.client.1.smithi149.stdout:5/909: mknod d7/da/d7d/d110/c138 0 2024-03-22T07:09:00.411 INFO:tasks.workunit.client.1.smithi149.stdout:5/910: write d7/da/d20/d30/d3a/f136 [811327,55906] 0 2024-03-22T07:09:00.412 INFO:tasks.workunit.client.1.smithi149.stdout:5/911: truncate d7/d35/f55 363770 0 2024-03-22T07:09:00.417 INFO:tasks.workunit.client.1.smithi149.stdout:5/912: unlink d7/d35/f55 0 2024-03-22T07:09:00.435 INFO:tasks.workunit.client.1.smithi149.stdout:5/913: dwrite d7/da/d20/d30/d3a/fc0 [0,4194304] 0 2024-03-22T07:09:00.437 INFO:tasks.workunit.client.1.smithi149.stdout:5/914: symlink d7/d35/d8e/l139 0 2024-03-22T07:09:00.438 INFO:tasks.workunit.client.1.smithi149.stdout:5/915: mkdir d7/da/d20/d30/d3a/d3d/d2c/db2/d13a 0 2024-03-22T07:09:00.452 INFO:tasks.workunit.client.1.smithi149.stdout:5/916: dwrite d7/da/d20/d30/d3a/d3d/d2c/fb1 [0,4194304] 0 2024-03-22T07:09:00.453 INFO:tasks.workunit.client.1.smithi149.stdout:5/917: dread d7/d35/d8e/fb5 [0,4194304] 0 2024-03-22T07:09:00.499 INFO:tasks.workunit.client.1.smithi149.stdout:4/923: sync 2024-03-22T07:09:00.499 INFO:tasks.workunit.client.1.smithi149.stdout:3/698: sync 2024-03-22T07:09:00.507 INFO:tasks.workunit.client.1.smithi149.stdout:4/924: dread d0/d8/d1d/d24/d82/fa1 [0,4194304] 0 2024-03-22T07:09:00.522 INFO:tasks.workunit.client.1.smithi149.stdout:4/925: dwrite d0/d8/f46 [0,4194304] 0 2024-03-22T07:09:00.522 INFO:tasks.workunit.client.1.smithi149.stdout:4/926: fsync d0/d8/fa2 0 2024-03-22T07:09:00.522 INFO:tasks.workunit.client.1.smithi149.stdout:4/927: readlink d0/d8/d4c/l109 0 2024-03-22T07:09:00.538 INFO:tasks.workunit.client.1.smithi149.stdout:4/928: dwrite d0/d8/d1d/d24/f54 [4194304,4194304] 0 2024-03-22T07:09:00.538 INFO:tasks.workunit.client.1.smithi149.stdout:4/929: fdatasync d0/d8/dc/d14/d3e/fd0 0 2024-03-22T07:09:00.539 INFO:tasks.workunit.client.1.smithi149.stdout:4/930: truncate d0/d8/dc/d4b/db3/ded/d121/f125 944491 0 2024-03-22T07:09:00.539 INFO:tasks.workunit.client.1.smithi149.stdout:4/931: fdatasync d0/d8/dc/d14/d3e/fb1 0 2024-03-22T07:09:00.540 INFO:tasks.workunit.client.1.smithi149.stdout:4/932: rename d0/d8/dc/d2e/f62 to d0/d8/dc/d4b/db3/f12d 0 2024-03-22T07:09:00.540 INFO:tasks.workunit.client.1.smithi149.stdout:4/933: chown d0/d29/cda 907914434 1 2024-03-22T07:09:00.541 INFO:tasks.workunit.client.1.smithi149.stdout:4/934: mkdir d0/d8/dc/d14/d10e/d12e 0 2024-03-22T07:09:00.545 INFO:tasks.workunit.client.1.smithi149.stdout:4/935: dread d0/d6b/db0/d110/f128 [0,4194304] 0 2024-03-22T07:09:00.547 INFO:tasks.workunit.client.1.smithi149.stdout:4/936: rename d0/d8/fa2 to d0/d8/dc/d74/d76/deb/f12f 0 2024-03-22T07:09:00.547 INFO:tasks.workunit.client.1.smithi149.stdout:4/937: fdatasync d0/d8/dc/d2e/f2f 0 2024-03-22T07:09:00.548 INFO:tasks.workunit.client.1.smithi149.stdout:4/938: mkdir d0/d8/dc/d4b/db3/dea/d130 0 2024-03-22T07:09:00.548 INFO:tasks.workunit.client.1.smithi149.stdout:4/939: read - d0/d8/d5c/dc1/ffa zero size 2024-03-22T07:09:00.548 INFO:tasks.workunit.client.1.smithi149.stdout:4/940: chown d0/d8/d4c/f111 644 1 2024-03-22T07:09:00.549 INFO:tasks.workunit.client.1.smithi149.stdout:4/941: rename d0/d8/f21 to d0/d6b/db0/d110/f131 0 2024-03-22T07:09:00.550 INFO:tasks.workunit.client.1.smithi149.stdout:4/942: mkdir d0/d132 0 2024-03-22T07:09:00.615 INFO:tasks.workunit.client.1.smithi149.stdout:2/973: sync 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:0/883: sync 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:9/906: sync 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:1/972: sync 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:7/949: sync 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/974: truncate d1/d72/f124 4482543 0 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:8/921: sync 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:9/907: read - db/f117 zero size 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:2/975: chown d1/d4d/d6a/de1/lc7 4154335 1 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:7/950: fdatasync d1/d1c/d49/d8e/d100/d1b/ff1 0 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:0/884: write d1/d27/d66/f11b [469668,100090] 0 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:0/885: chown d1/d27 10539 1 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:0/886: stat d1/d31/d52/da9/f113 0 2024-03-22T07:09:00.616 INFO:tasks.workunit.client.1.smithi149.stdout:0/887: fsync d1/d27/d66/df1/fb9 0 2024-03-22T07:09:00.618 INFO:tasks.workunit.client.1.smithi149.stdout:2/976: rename d1/d3/d5/d1f/d68/fb4 to d1/d3/d6/db2/d71/f162 0 2024-03-22T07:09:00.618 INFO:tasks.workunit.client.1.smithi149.stdout:7/951: truncate d1/d1c/d49/d8e/d100/d1b/d99/de5/db4/fd5 383863 0 2024-03-22T07:09:00.619 INFO:tasks.workunit.client.1.smithi149.stdout:0/888: symlink d1/d31/d52/da9/l13d 0 2024-03-22T07:09:00.619 INFO:tasks.workunit.client.1.smithi149.stdout:0/889: chown d1/d31/f53 2315 1 2024-03-22T07:09:00.619 INFO:tasks.workunit.client.1.smithi149.stdout:2/977: mkdir d1/d4d/d90/dbc/d163 0 2024-03-22T07:09:00.620 INFO:tasks.workunit.client.1.smithi149.stdout:2/978: write d1/d72/f12d [518046,85061] 0 2024-03-22T07:09:00.620 INFO:tasks.workunit.client.1.smithi149.stdout:2/979: stat d1/d3/d5/d41/d9f/da7/d157/d129/d74/c14a 0 2024-03-22T07:09:00.620 INFO:tasks.workunit.client.1.smithi149.stdout:2/980: write d1/d3/d6/d32/d6f/fcd [842444,47156] 0 2024-03-22T07:09:00.623 INFO:tasks.workunit.client.1.smithi149.stdout:7/952: creat d1/d1c/d49/d8e/d100/d1b/d51/d7a/f13a x:0 0 0 2024-03-22T07:09:00.624 INFO:tasks.workunit.client.1.smithi149.stdout:0/890: mknod d1/d20/d9f/c13e 0 2024-03-22T07:09:00.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/981: dread d1/d3/d6/f7 [0,4194304] 0 2024-03-22T07:09:00.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/982: chown d1/d3/d5/d1f/d5e/d120 257010 1 2024-03-22T07:09:00.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/983: write d1/d3/d5/d1f/d5e/f137 [1044726,25794] 0 2024-03-22T07:09:00.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/984: chown d1/d3/de0/f130 27956 1 2024-03-22T07:09:00.630 INFO:tasks.workunit.client.1.smithi149.stdout:2/985: fdatasync d1/d3/d5/d1f/d5e/f143 0 2024-03-22T07:09:00.634 INFO:tasks.workunit.client.1.smithi149.stdout:1/973: dread d4/d10/dd8/d3c/d8e/db5/f37 [0,4194304] 0 2024-03-22T07:09:00.640 INFO:tasks.workunit.client.1.smithi149.stdout:1/974: unlink d4/d7/d5d/d82/fe3 0 2024-03-22T07:09:00.640 INFO:tasks.workunit.client.1.smithi149.stdout:8/922: dread d1/d2/d3/f86 [0,4194304] 0 2024-03-22T07:09:00.640 INFO:tasks.workunit.client.1.smithi149.stdout:8/923: write d1/d2/d9/d11/d8b/dee/f12d [826128,40223] 0 2024-03-22T07:09:00.644 INFO:tasks.workunit.client.1.smithi149.stdout:0/891: write d1/d9/d14/d1f/f4d [842386,40817] 0 2024-03-22T07:09:00.647 INFO:tasks.workunit.client.1.smithi149.stdout:0/892: write d1/d27/f102 [722389,54292] 0 2024-03-22T07:09:00.647 INFO:tasks.workunit.client.1.smithi149.stdout:0/893: dread - d1/d6b/f138 zero size 2024-03-22T07:09:00.665 INFO:tasks.workunit.client.1.smithi149.stdout:8/924: dread d1/d2/d9/d11/fa3 [0,4194304] 0 2024-03-22T07:09:00.665 INFO:tasks.workunit.client.1.smithi149.stdout:8/925: write d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/fc9 [947760,41725] 0 2024-03-22T07:09:00.665 INFO:tasks.workunit.client.1.smithi149.stdout:8/926: dread - d1/d2/d9/d11/d4d/dd6/d35/d22/ff5 zero size 2024-03-22T07:09:00.665 INFO:tasks.workunit.client.1.smithi149.stdout:8/927: write d1/d2/d9/d11/d4d/dd6/d35/d22/d44/f52 [866625,85220] 0 2024-03-22T07:09:00.665 INFO:tasks.workunit.client.1.smithi149.stdout:8/928: write d1/d2/d9/d11/d8b/fdd [1067157,2895] 0 2024-03-22T07:09:00.667 INFO:tasks.workunit.client.1.smithi149.stdout:8/929: creat d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d82/f12e x:0 0 0 2024-03-22T07:09:00.668 INFO:tasks.workunit.client.1.smithi149.stdout:8/930: unlink d1/d2/d3/f96 0 2024-03-22T07:09:00.669 INFO:tasks.workunit.client.1.smithi149.stdout:8/931: mknod d1/d2/d9/c12f 0 2024-03-22T07:09:00.669 INFO:tasks.workunit.client.1.smithi149.stdout:8/932: fsync d1/d2/d9/d11/d4d/dd6/d51/f111 0 2024-03-22T07:09:00.672 INFO:tasks.workunit.client.1.smithi149.stdout:9/908: dwrite db/d19/d39/d68/d6f/fb7 [0,4194304] 0 2024-03-22T07:09:00.672 INFO:tasks.workunit.client.1.smithi149.stdout:2/986: dwrite d1/d4d/d6a/de1/fa8 [0,4194304] 0 2024-03-22T07:09:00.675 INFO:tasks.workunit.client.1.smithi149.stdout:9/909: mknod db/d21/d8c/c120 0 2024-03-22T07:09:00.675 INFO:tasks.workunit.client.1.smithi149.stdout:9/910: chown db/d19/d11d/c73 486854153 1 2024-03-22T07:09:00.675 INFO:tasks.workunit.client.1.smithi149.stdout:9/911: write db/d33/f92 [4229318,18275] 0 2024-03-22T07:09:00.677 INFO:tasks.workunit.client.1.smithi149.stdout:2/987: rename d1/d3/d9a/lc6 to d1/d4d/d6a/l164 0 2024-03-22T07:09:00.677 INFO:tasks.workunit.client.1.smithi149.stdout:2/988: chown d1/d8/d34/lf7 2 1 2024-03-22T07:09:00.677 INFO:tasks.workunit.client.1.smithi149.stdout:2/989: creat d1/d3/d5/d1f/d5e/d120/f165 x:0 0 0 2024-03-22T07:09:00.680 INFO:tasks.workunit.client.1.smithi149.stdout:0/894: dwrite d1/f39 [0,4194304] 0 2024-03-22T07:09:00.680 INFO:tasks.workunit.client.1.smithi149.stdout:1/975: dwrite d4/d7/dd3/d129/f62 [0,4194304] 0 2024-03-22T07:09:00.680 INFO:tasks.workunit.client.1.smithi149.stdout:1/976: dread - d4/d10/d7c/d6a/d6b/f123 zero size 2024-03-22T07:09:00.680 INFO:tasks.workunit.client.1.smithi149.stdout:1/977: write d4/d10/dd8/d3c/d85/d89/f97 [4641444,42400] 0 2024-03-22T07:09:00.683 INFO:tasks.workunit.client.1.smithi149.stdout:8/933: write d1/d2/d9/d11/d4d/dd6/d51/fe2 [2450839,59400] 0 2024-03-22T07:09:00.684 INFO:tasks.workunit.client.1.smithi149.stdout:0/895: symlink d1/l13f 0 2024-03-22T07:09:00.684 INFO:tasks.workunit.client.1.smithi149.stdout:0/896: write d1/d31/d51/d64/d87/f10d [243782,18184] 0 2024-03-22T07:09:00.690 INFO:tasks.workunit.client.1.smithi149.stdout:1/978: creat d4/d7/d5d/d82/f147 x:0 0 0 2024-03-22T07:09:00.693 INFO:tasks.workunit.client.1.smithi149.stdout:8/934: creat d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d84/f130 x:0 0 0 2024-03-22T07:09:00.693 INFO:tasks.workunit.client.1.smithi149.stdout:8/935: write d1/d2/d9/d11/d4d/dd6/f66 [4825447,643] 0 2024-03-22T07:09:00.693 INFO:tasks.workunit.client.1.smithi149.stdout:8/936: chown d1/d2/d9/d11/c2a 4 1 2024-03-22T07:09:00.694 INFO:tasks.workunit.client.1.smithi149.stdout:0/897: mkdir d1/d20/d9f/dc6/dd9/d140 0 2024-03-22T07:09:00.694 INFO:tasks.workunit.client.1.smithi149.stdout:0/898: write d1/d31/f8a [3570861,78877] 0 2024-03-22T07:09:00.695 INFO:tasks.workunit.client.1.smithi149.stdout:0/899: chown d1/d31/d52/da9/led 160 1 2024-03-22T07:09:00.700 INFO:tasks.workunit.client.1.smithi149.stdout:2/990: dwrite d1/d4d/d69/d75/d7e/fd6 [0,4194304] 0 2024-03-22T07:09:00.701 INFO:tasks.workunit.client.1.smithi149.stdout:1/979: read d4/d7/dd3/d129/f62 [3974654,73455] 0 2024-03-22T07:09:00.705 INFO:tasks.workunit.client.1.smithi149.stdout:2/991: mkdir d1/d3/d5/d41/d9f/da7/d157/d129/d74/d166 0 2024-03-22T07:09:00.707 INFO:tasks.workunit.client.1.smithi149.stdout:1/980: unlink d4/d10/d7c/f98 0 2024-03-22T07:09:00.707 INFO:tasks.workunit.client.1.smithi149.stdout:1/981: write d4/d20/d2d/fa1 [373904,75162] 0 2024-03-22T07:09:00.707 INFO:tasks.workunit.client.1.smithi149.stdout:3/699: sync 2024-03-22T07:09:00.708 INFO:tasks.workunit.client.1.smithi149.stdout:8/937: dread d1/d2/d9/d11/f55 [0,4194304] 0 2024-03-22T07:09:00.709 INFO:tasks.workunit.client.1.smithi149.stdout:8/938: fdatasync d1/d2/d9/d11/d4d/dd6/d35/d22/ff5 0 2024-03-22T07:09:00.709 INFO:tasks.workunit.client.1.smithi149.stdout:8/939: chown d1/d2/d9/d11/d4d/dd6/d51/c81 47 1 2024-03-22T07:09:00.709 INFO:tasks.workunit.client.1.smithi149.stdout:5/918: sync 2024-03-22T07:09:00.709 INFO:tasks.workunit.client.1.smithi149.stdout:4/943: sync 2024-03-22T07:09:00.709 INFO:tasks.workunit.client.1.smithi149.stdout:5/919: chown d7/da/d20/d30/d3a/d3d/d104/f126 0 1 2024-03-22T07:09:00.709 INFO:tasks.workunit.client.1.smithi149.stdout:5/920: write d7/da/d20/fdb [689235,114728] 0 2024-03-22T07:09:00.709 INFO:tasks.workunit.client.1.smithi149.stdout:1/982: rename d4/d10/d7c/l21 to d4/d7/d61/l148 0 2024-03-22T07:09:00.712 INFO:tasks.workunit.client.1.smithi149.stdout:8/940: mkdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d125/d12c/d131 0 2024-03-22T07:09:00.713 INFO:tasks.workunit.client.1.smithi149.stdout:4/944: unlink d0/d8/d1d/c52 0 2024-03-22T07:09:00.713 INFO:tasks.workunit.client.1.smithi149.stdout:5/921: stat d7/da/d20/c43 0 2024-03-22T07:09:00.714 INFO:tasks.workunit.client.1.smithi149.stdout:1/983: mknod d4/d7/d5a/c149 0 2024-03-22T07:09:00.715 INFO:tasks.workunit.client.1.smithi149.stdout:3/700: rmdir d0/d4/d6c 39 2024-03-22T07:09:00.716 INFO:tasks.workunit.client.1.smithi149.stdout:0/900: dwrite d1/d20/da3/fae [0,4194304] 0 2024-03-22T07:09:00.719 INFO:tasks.workunit.client.1.smithi149.stdout:0/901: dread d1/d31/d51/d64/d87/f10d [0,4194304] 0 2024-03-22T07:09:00.720 INFO:tasks.workunit.client.1.smithi149.stdout:5/922: truncate d7/da/d20/d36/fbd 469291 0 2024-03-22T07:09:00.720 INFO:tasks.workunit.client.1.smithi149.stdout:5/923: readlink d7/da/d20/d36/d44/l5a 0 2024-03-22T07:09:00.720 INFO:tasks.workunit.client.1.smithi149.stdout:5/924: creat d7/da/d20/d30/d3a/d93/f13b x:0 0 0 2024-03-22T07:09:00.720 INFO:tasks.workunit.client.1.smithi149.stdout:5/925: readlink d7/da/d20/d30/d3a/d3d/l120 0 2024-03-22T07:09:00.720 INFO:tasks.workunit.client.1.smithi149.stdout:5/926: readlink d7/d35/l39 0 2024-03-22T07:09:00.720 INFO:tasks.workunit.client.1.smithi149.stdout:3/701: dread d0/d2/d30/d5d/d84/d8f/feb [0,4194304] 0 2024-03-22T07:09:00.720 INFO:tasks.workunit.client.1.smithi149.stdout:3/702: readlink d0/lfd 0 2024-03-22T07:09:00.720 INFO:tasks.workunit.client.1.smithi149.stdout:1/984: getdents d4/d10/dd8/d3c/d8e 0 2024-03-22T07:09:00.721 INFO:tasks.workunit.client.1.smithi149.stdout:5/927: dread d7/da/d20/fdb [0,4194304] 0 2024-03-22T07:09:00.723 INFO:tasks.workunit.client.1.smithi149.stdout:4/945: rmdir d0/d8/dc/dbd 39 2024-03-22T07:09:00.723 INFO:tasks.workunit.client.1.smithi149.stdout:4/946: chown d0/d8/d1d/c22 165670 1 2024-03-22T07:09:00.724 INFO:tasks.workunit.client.1.smithi149.stdout:4/947: fdatasync d0/d8/f34 0 2024-03-22T07:09:00.724 INFO:tasks.workunit.client.1.smithi149.stdout:4/948: stat d0/la 0 2024-03-22T07:09:00.726 INFO:tasks.workunit.client.1.smithi149.stdout:0/902: link d1/d9/d59/l103 d1/d136/l141 0 2024-03-22T07:09:00.726 INFO:tasks.workunit.client.1.smithi149.stdout:0/903: fsync d1/d136/d10c/f122 0 2024-03-22T07:09:00.731 INFO:tasks.workunit.client.1.smithi149.stdout:3/703: mknod d0/d2/d5a/daf/c105 0 2024-03-22T07:09:00.732 INFO:tasks.workunit.client.1.smithi149.stdout:5/928: rmdir d7/da/d20/d45 39 2024-03-22T07:09:00.733 INFO:tasks.workunit.client.1.smithi149.stdout:1/985: dread d4/d7/d61/fd1 [0,4194304] 0 2024-03-22T07:09:00.734 INFO:tasks.workunit.client.1.smithi149.stdout:0/904: mknod d1/d27/d66/c142 0 2024-03-22T07:09:00.734 INFO:tasks.workunit.client.1.smithi149.stdout:0/905: fdatasync d1/d31/d51/d64/d87/da6/fd6 0 2024-03-22T07:09:00.734 INFO:tasks.workunit.client.1.smithi149.stdout:0/906: chown d1/d20/da3/lc2 0 1 2024-03-22T07:09:00.735 INFO:tasks.workunit.client.1.smithi149.stdout:3/704: creat d0/d4/d8/d24/db6/f106 x:0 0 0 2024-03-22T07:09:00.736 INFO:tasks.workunit.client.1.smithi149.stdout:1/986: rename d4/d10/dd8/d59/d143 to d4/d10/dd8/d3c/d85/d14a 0 2024-03-22T07:09:00.737 INFO:tasks.workunit.client.1.smithi149.stdout:0/907: mkdir d1/d20/d9f/dc6/dd9/d13b/d143 0 2024-03-22T07:09:00.738 INFO:tasks.workunit.client.1.smithi149.stdout:3/705: unlink d0/d4/d8/d24/d40/l88 0 2024-03-22T07:09:00.739 INFO:tasks.workunit.client.1.smithi149.stdout:1/987: mkdir d4/d7/d61/d14b 0 2024-03-22T07:09:00.739 INFO:tasks.workunit.client.1.smithi149.stdout:1/988: chown d4/d10/dd8/d3c/d8e/db5 31139587 1 2024-03-22T07:09:00.739 INFO:tasks.workunit.client.1.smithi149.stdout:0/908: mkdir d1/d9/d59/dc0/dc9/dcc/d144 0 2024-03-22T07:09:00.740 INFO:tasks.workunit.client.1.smithi149.stdout:3/706: creat d0/d2/d30/dd1/dfa/f107 x:0 0 0 2024-03-22T07:09:00.741 INFO:tasks.workunit.client.1.smithi149.stdout:1/989: mknod d4/d7/dd3/d129/d93/d130/c14c 0 2024-03-22T07:09:00.743 INFO:tasks.workunit.client.1.smithi149.stdout:0/909: rename d1/d31/l3b to d1/d20/d9f/dc6/l145 0 2024-03-22T07:09:00.743 INFO:tasks.workunit.client.1.smithi149.stdout:7/953: sync 2024-03-22T07:09:00.744 INFO:tasks.workunit.client.1.smithi149.stdout:5/929: dread d7/da/d20/d36/f91 [0,4194304] 0 2024-03-22T07:09:00.744 INFO:tasks.workunit.client.1.smithi149.stdout:0/910: creat d1/d136/f146 x:0 0 0 2024-03-22T07:09:00.745 INFO:tasks.workunit.client.1.smithi149.stdout:4/949: dwrite d0/d8/d1d/d24/d82/f88 [4194304,4194304] 0 2024-03-22T07:09:00.746 INFO:tasks.workunit.client.1.smithi149.stdout:7/954: rename d1/d1c/f9f to d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/d9e/f13b 0 2024-03-22T07:09:00.747 INFO:tasks.workunit.client.1.smithi149.stdout:0/911: link d1/d9/d14/d74/d77/f9a d1/d31/d11d/f147 0 2024-03-22T07:09:00.747 INFO:tasks.workunit.client.1.smithi149.stdout:0/912: chown d1/d9/d14/l17 13 1 2024-03-22T07:09:00.748 INFO:tasks.workunit.client.1.smithi149.stdout:4/950: truncate d0/d8/f10c 3865107 0 2024-03-22T07:09:00.750 INFO:tasks.workunit.client.1.smithi149.stdout:7/955: rename d1/d1c/dbc/cc3 to d1/d1c/dbc/c13c 0 2024-03-22T07:09:00.753 INFO:tasks.workunit.client.1.smithi149.stdout:0/913: rename d1/d31/d51/d64/d87/da6/cad to d1/d31/d52/dd4/c148 0 2024-03-22T07:09:00.756 INFO:tasks.workunit.client.1.smithi149.stdout:7/956: link d1/l12 d1/d1c/d49/dc0/dfc/l13d 0 2024-03-22T07:09:00.756 INFO:tasks.workunit.client.1.smithi149.stdout:4/951: getdents d0/d8/dc/d74/ddc 0 2024-03-22T07:09:00.757 INFO:tasks.workunit.client.1.smithi149.stdout:0/914: truncate d1/d9/f71 713006 0 2024-03-22T07:09:00.758 INFO:tasks.workunit.client.1.smithi149.stdout:4/952: truncate d0/d8/dc/d74/ddc/fe7 3650400 0 2024-03-22T07:09:00.760 INFO:tasks.workunit.client.1.smithi149.stdout:0/915: creat d1/d31/d51/d64/d8d/f149 x:0 0 0 2024-03-22T07:09:00.760 INFO:tasks.workunit.client.1.smithi149.stdout:0/916: fsync d1/d9/d14/d1f/f4d 0 2024-03-22T07:09:00.761 INFO:tasks.workunit.client.1.smithi149.stdout:0/917: chown d1/d31/d51/f45 64520301 1 2024-03-22T07:09:00.772 INFO:tasks.workunit.client.1.smithi149.stdout:1/990: dwrite d4/d20/d2d/fa1 [0,4194304] 0 2024-03-22T07:09:00.773 INFO:tasks.workunit.client.1.smithi149.stdout:1/991: write d4/d10/dd8/d3c/d109/d13f/d68/f128 [506402,87704] 0 2024-03-22T07:09:00.773 INFO:tasks.workunit.client.1.smithi149.stdout:1/992: creat d4/d10/dd8/d3c/d109/f14d x:0 0 0 2024-03-22T07:09:00.773 INFO:tasks.workunit.client.1.smithi149.stdout:1/993: stat d4/d10/d7c/d6a/d6b/c71 0 2024-03-22T07:09:00.774 INFO:tasks.workunit.client.1.smithi149.stdout:7/957: write d1/d1c/d49/d8e/d100/d1b/d51/d5a/fe9 [4171748,92610] 0 2024-03-22T07:09:00.774 INFO:tasks.workunit.client.1.smithi149.stdout:7/958: stat d1/d1c/d49/d8e/d100/de8 0 2024-03-22T07:09:00.774 INFO:tasks.workunit.client.1.smithi149.stdout:5/930: dwrite d7/da/d20/d36/fba [0,4194304] 0 2024-03-22T07:09:00.775 INFO:tasks.workunit.client.1.smithi149.stdout:7/959: mkdir d1/db8/d13e 0 2024-03-22T07:09:00.775 INFO:tasks.workunit.client.1.smithi149.stdout:7/960: fdatasync d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/d9e/dab/fce 0 2024-03-22T07:09:00.776 INFO:tasks.workunit.client.1.smithi149.stdout:3/707: dwrite d0/d4/d8/d24/d40/d54/d85/dbc/f80 [0,4194304] 0 2024-03-22T07:09:00.776 INFO:tasks.workunit.client.1.smithi149.stdout:3/708: fsync d0/d4/d8/fc8 0 2024-03-22T07:09:00.777 INFO:tasks.workunit.client.1.smithi149.stdout:0/918: dread d1/d9/d59/f6c [0,4194304] 0 2024-03-22T07:09:00.777 INFO:tasks.workunit.client.1.smithi149.stdout:5/931: truncate d7/da/d20/d30/d3a/f101 2183441 0 2024-03-22T07:09:00.778 INFO:tasks.workunit.client.1.smithi149.stdout:7/961: symlink d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/dfe/d120/l13f 0 2024-03-22T07:09:00.780 INFO:tasks.workunit.client.1.smithi149.stdout:0/919: creat d1/d20/d9f/dc6/dd9/d140/f14a x:0 0 0 2024-03-22T07:09:00.780 INFO:tasks.workunit.client.1.smithi149.stdout:0/920: stat d1/d27/c116 0 2024-03-22T07:09:00.780 INFO:tasks.workunit.client.1.smithi149.stdout:0/921: chown d1/d9/d14/d1f/l12c 36332 1 2024-03-22T07:09:00.782 INFO:tasks.workunit.client.1.smithi149.stdout:5/932: symlink d7/d35/de0/dd7/dea/l13c 0 2024-03-22T07:09:00.783 INFO:tasks.workunit.client.1.smithi149.stdout:5/933: fdatasync d7/da/d20/d30/f98 0 2024-03-22T07:09:00.783 INFO:tasks.workunit.client.1.smithi149.stdout:5/934: chown d7/da/d7d/f116 7315579 1 2024-03-22T07:09:00.787 INFO:tasks.workunit.client.1.smithi149.stdout:4/953: dwrite d0/d8/dc/f5b [0,4194304] 0 2024-03-22T07:09:00.787 INFO:tasks.workunit.client.1.smithi149.stdout:4/954: readlink d0/d8/d5c/l63 0 2024-03-22T07:09:00.788 INFO:tasks.workunit.client.1.smithi149.stdout:7/962: read d1/fa7 [7053062,128103] 0 2024-03-22T07:09:00.796 INFO:tasks.workunit.client.1.smithi149.stdout:0/922: dread d1/d9/d14/d1f/f89 [0,4194304] 0 2024-03-22T07:09:00.797 INFO:tasks.workunit.client.1.smithi149.stdout:3/709: dwrite d0/d2/f4d [0,4194304] 0 2024-03-22T07:09:00.800 INFO:tasks.workunit.client.1.smithi149.stdout:7/963: rename d1/d1c/d49/d8e/d100/d1b/d51 to d1/d1c/d49/d8e/d53/d140 0 2024-03-22T07:09:00.802 INFO:tasks.workunit.client.1.smithi149.stdout:0/923: symlink d1/d6b/l14b 0 2024-03-22T07:09:00.802 INFO:tasks.workunit.client.1.smithi149.stdout:0/924: fdatasync d1/d1a/f118 0 2024-03-22T07:09:00.806 INFO:tasks.workunit.client.1.smithi149.stdout:0/925: link d1/d31/d51/d64/d7e/ld5 d1/d136/l14c 0 2024-03-22T07:09:00.807 INFO:tasks.workunit.client.1.smithi149.stdout:3/710: getdents d0/d4/d8/d24/db6 0 2024-03-22T07:09:00.808 INFO:tasks.workunit.client.1.smithi149.stdout:0/926: write d1/d6b/f78 [219709,81272] 0 2024-03-22T07:09:00.809 INFO:tasks.workunit.client.1.smithi149.stdout:0/927: mknod d1/d31/d51/d64/c14d 0 2024-03-22T07:09:00.809 INFO:tasks.workunit.client.1.smithi149.stdout:0/928: stat d1/d9/d59/dc0/dc9/dcc/l12b 0 2024-03-22T07:09:00.817 INFO:tasks.workunit.client.1.smithi149.stdout:3/711: dread d0/d4/d8/d24/d40/d54/d85/fd5 [0,4194304] 0 2024-03-22T07:09:00.818 INFO:tasks.workunit.client.1.smithi149.stdout:3/712: mkdir d0/d4/d8/d24/d40/d62/d108 0 2024-03-22T07:09:00.819 INFO:tasks.workunit.client.1.smithi149.stdout:3/713: unlink d0/f1a 0 2024-03-22T07:09:00.820 INFO:tasks.workunit.client.1.smithi149.stdout:4/955: dwrite d0/d8/dc/fe [0,4194304] 0 2024-03-22T07:09:00.821 INFO:tasks.workunit.client.1.smithi149.stdout:4/956: fdatasync d0/d8/f1f 0 2024-03-22T07:09:00.821 INFO:tasks.workunit.client.1.smithi149.stdout:4/957: chown d0/d29/c123 6626454 1 2024-03-22T07:09:00.821 INFO:tasks.workunit.client.1.smithi149.stdout:7/964: dwrite d1/d1c/d49/d8e/d100/d1b/d1d/fd [4194304,4194304] 0 2024-03-22T07:09:00.821 INFO:tasks.workunit.client.1.smithi149.stdout:3/714: link d0/d2/l2c d0/d4/d8/d24/d40/d54/l109 0 2024-03-22T07:09:00.824 INFO:tasks.workunit.client.1.smithi149.stdout:0/929: dread d1/d1a/f90 [0,4194304] 0 2024-03-22T07:09:00.824 INFO:tasks.workunit.client.1.smithi149.stdout:0/930: chown d1/d9/lb8 68511346 1 2024-03-22T07:09:00.824 INFO:tasks.workunit.client.1.smithi149.stdout:3/715: dread d0/d4/d8/f99 [0,4194304] 0 2024-03-22T07:09:00.824 INFO:tasks.workunit.client.1.smithi149.stdout:3/716: write d0/d2/d42/d66/f7a [1415101,11430] 0 2024-03-22T07:09:00.824 INFO:tasks.workunit.client.1.smithi149.stdout:3/717: write d0/d2/d42/f7c [820666,75121] 0 2024-03-22T07:09:00.825 INFO:tasks.workunit.client.1.smithi149.stdout:4/958: symlink d0/d8/dc/d4b/db3/dea/d12c/l133 0 2024-03-22T07:09:00.825 INFO:tasks.workunit.client.1.smithi149.stdout:4/959: read - d0/d8/dc/d74/f95 zero size 2024-03-22T07:09:00.826 INFO:tasks.workunit.client.1.smithi149.stdout:4/960: mknod d0/d8/d5c/c134 0 2024-03-22T07:09:00.830 INFO:tasks.workunit.client.1.smithi149.stdout:0/931: write d1/d9/d59/f6c [2559500,108872] 0 2024-03-22T07:09:00.831 INFO:tasks.workunit.client.1.smithi149.stdout:4/961: mkdir d0/d8/dc/d14/d85/d97/d135 0 2024-03-22T07:09:00.831 INFO:tasks.workunit.client.1.smithi149.stdout:7/965: dread d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/fa9 [0,4194304] 0 2024-03-22T07:09:00.831 INFO:tasks.workunit.client.1.smithi149.stdout:4/962: chown d0/d8/d4c/f111 13 1 2024-03-22T07:09:00.834 INFO:tasks.workunit.client.1.smithi149.stdout:0/932: creat d1/d20/d9f/f14e x:0 0 0 2024-03-22T07:09:00.834 INFO:tasks.workunit.client.1.smithi149.stdout:3/718: read d0/d2/d30/d7e/fe8 [747077,101836] 0 2024-03-22T07:09:00.834 INFO:tasks.workunit.client.1.smithi149.stdout:7/966: mknod d1/d1c/d49/d8e/d9c/df7/c141 0 2024-03-22T07:09:00.834 INFO:tasks.workunit.client.1.smithi149.stdout:3/719: creat d0/d2/d30/d5d/d84/dc3/f10a x:0 0 0 2024-03-22T07:09:00.834 INFO:tasks.workunit.client.1.smithi149.stdout:3/720: fsync d0/d2/d30/d7e/f69 0 2024-03-22T07:09:00.834 INFO:tasks.workunit.client.1.smithi149.stdout:7/967: write d1/d1c/d49/d8e/d100/d1b/d1d/f40 [7436855,126448] 0 2024-03-22T07:09:00.834 INFO:tasks.workunit.client.1.smithi149.stdout:4/963: creat d0/d8/dc/d4b/dbc/f136 x:0 0 0 2024-03-22T07:09:00.835 INFO:tasks.workunit.client.1.smithi149.stdout:4/964: chown d0/d8/dc/d2e/d6f/def/c11b 0 1 2024-03-22T07:09:00.836 INFO:tasks.workunit.client.1.smithi149.stdout:7/968: dread d1/d1c/d49/d8e/d100/d1b/d1d/d72/fdc [0,4194304] 0 2024-03-22T07:09:00.841 INFO:tasks.workunit.client.1.smithi149.stdout:0/933: write d1/d31/d51/d64/d87/da6/fd6 [398052,93739] 0 2024-03-22T07:09:00.844 INFO:tasks.workunit.client.1.smithi149.stdout:0/934: mkdir d1/d31/d14f 0 2024-03-22T07:09:00.844 INFO:tasks.workunit.client.1.smithi149.stdout:9/912: sync 2024-03-22T07:09:00.849 INFO:tasks.workunit.client.1.smithi149.stdout:0/935: creat d1/d31/d14f/f150 x:0 0 0 2024-03-22T07:09:00.849 INFO:tasks.workunit.client.1.smithi149.stdout:9/913: creat db/d21/d8c/f121 x:0 0 0 2024-03-22T07:09:00.849 INFO:tasks.workunit.client.1.smithi149.stdout:9/914: chown db/d21/d8c 8013 1 2024-03-22T07:09:00.849 INFO:tasks.workunit.client.1.smithi149.stdout:7/969: dread d1/d1c/d49/d8e/d100/d1b/d1d/f16 [0,4194304] 0 2024-03-22T07:09:00.850 INFO:tasks.workunit.client.1.smithi149.stdout:7/970: symlink d1/d1c/d49/d8e/d53/d140/d7a/d7d/d5b/l142 0 2024-03-22T07:09:00.852 INFO:tasks.workunit.client.1.smithi149.stdout:0/936: link d1/d31/d51/d64/d8d/d9c/c139 d1/d20/d9f/dc6/dd9/d13b/c151 0 2024-03-22T07:09:00.854 INFO:tasks.workunit.client.1.smithi149.stdout:0/937: symlink d1/d20/da3/l152 0 2024-03-22T07:09:00.854 INFO:tasks.workunit.client.1.smithi149.stdout:0/938: chown d1/d9/d14/d74/c8f 178355 1 2024-03-22T07:09:00.855 INFO:tasks.workunit.client.1.smithi149.stdout:0/939: symlink d1/d9/d14/d74/d77/l153 0 2024-03-22T07:09:00.855 INFO:tasks.workunit.client.1.smithi149.stdout:0/940: write d1/d20/d9f/dc6/fcf [1368851,81003] 0 2024-03-22T07:09:00.857 INFO:tasks.workunit.client.1.smithi149.stdout:0/941: link d1/d31/d10f/l135 d1/d20/d9f/dc6/l154 0 2024-03-22T07:09:00.859 INFO:tasks.workunit.client.1.smithi149.stdout:4/965: dwrite d0/d6b/fc8 [0,4194304] 0 2024-03-22T07:09:00.859 INFO:tasks.workunit.client.1.smithi149.stdout:0/942: creat d1/d136/d10c/d108/f155 x:0 0 0 2024-03-22T07:09:00.859 INFO:tasks.workunit.client.1.smithi149.stdout:0/943: creat d1/d9/d14/d74/d77/f156 x:0 0 0 2024-03-22T07:09:00.859 INFO:tasks.workunit.client.1.smithi149.stdout:4/966: creat d0/f137 x:0 0 0 2024-03-22T07:09:00.859 INFO:tasks.workunit.client.1.smithi149.stdout:0/944: creat d1/d31/f157 x:0 0 0 2024-03-22T07:09:00.860 INFO:tasks.workunit.client.1.smithi149.stdout:9/915: write db/d33/f67 [876681,17739] 0 2024-03-22T07:09:00.861 INFO:tasks.workunit.client.1.smithi149.stdout:3/721: dwrite d0/d2/d30/d5d/d84/dc3/fe9 [0,4194304] 0 2024-03-22T07:09:00.871 INFO:tasks.workunit.client.1.smithi149.stdout:9/916: creat db/d42/d49/f122 x:0 0 0 2024-03-22T07:09:00.874 INFO:tasks.workunit.client.1.smithi149.stdout:9/917: rename db/d19/d1c/d85/fb6 to db/d21/d37/d3a/d104/d11b/f123 0 2024-03-22T07:09:00.876 INFO:tasks.workunit.client.1.smithi149.stdout:9/918: mknod db/d21/d9e/c124 0 2024-03-22T07:09:00.882 INFO:tasks.workunit.client.1.smithi149.stdout:9/919: dread db/d21/fd6 [0,4194304] 0 2024-03-22T07:09:00.884 INFO:tasks.workunit.client.1.smithi149.stdout:9/920: symlink db/d42/d49/d4b/l125 0 2024-03-22T07:09:00.884 INFO:tasks.workunit.client.1.smithi149.stdout:9/921: readlink db/d19/l47 0 2024-03-22T07:09:00.892 INFO:tasks.workunit.client.1.smithi149.stdout:3/722: dwrite d0/d2/d42/f7c [0,4194304] 0 2024-03-22T07:09:00.892 INFO:tasks.workunit.client.1.smithi149.stdout:3/723: chown d0/d4/d8/fc8 1835 1 2024-03-22T07:09:00.892 INFO:tasks.workunit.client.1.smithi149.stdout:0/945: dwrite d1/d31/d51/d64/d8d/d9c/f101 [0,4194304] 0 2024-03-22T07:09:00.892 INFO:tasks.workunit.client.1.smithi149.stdout:4/967: dwrite d0/d8/dc/d4b/db3/ded/d121/f125 [0,4194304] 0 2024-03-22T07:09:00.892 INFO:tasks.workunit.client.1.smithi149.stdout:4/968: truncate d0/d8/d5c/dc1/ffa 1021431 0 2024-03-22T07:09:00.894 INFO:tasks.workunit.client.1.smithi149.stdout:0/946: rmdir d1/d9/d59/dc0 39 2024-03-22T07:09:00.895 INFO:tasks.workunit.client.1.smithi149.stdout:0/947: write d1/d31/d51/d64/d87/da6/fd6 [177569,36108] 0 2024-03-22T07:09:00.900 INFO:tasks.workunit.client.1.smithi149.stdout:9/922: dread db/d21/d74/fdf [0,4194304] 0 2024-03-22T07:09:00.901 INFO:tasks.workunit.client.1.smithi149.stdout:3/724: dread d0/d2/d30/d5d/d84/dc3/fe9 [0,4194304] 0 2024-03-22T07:09:00.902 INFO:tasks.workunit.client.1.smithi149.stdout:9/923: mkdir db/d21/d2e/d126 0 2024-03-22T07:09:00.904 INFO:tasks.workunit.client.1.smithi149.stdout:3/725: getdents d0/d4/d8/d24/d40 0 2024-03-22T07:09:00.906 INFO:tasks.workunit.client.1.smithi149.stdout:4/969: dread d0/d29/f6e [0,4194304] 0 2024-03-22T07:09:00.906 INFO:tasks.workunit.client.1.smithi149.stdout:4/970: readlink d0/d8/l1b 0 2024-03-22T07:09:00.906 INFO:tasks.workunit.client.1.smithi149.stdout:4/971: chown d0/d29/l2a 0 1 2024-03-22T07:09:00.918 INFO:tasks.workunit.client.1.smithi149.stdout:9/924: write db/d19/d39/d68/d6f/fb7 [667163,82125] 0 2024-03-22T07:09:00.919 INFO:tasks.workunit.client.1.smithi149.stdout:0/948: dwrite d1/d9/f29 [0,4194304] 0 2024-03-22T07:09:00.921 INFO:tasks.workunit.client.1.smithi149.stdout:0/949: symlink d1/d31/d52/da9/l158 0 2024-03-22T07:09:00.949 INFO:tasks.workunit.client.1.smithi149.stdout:3/726: dwrite d0/d4/d8/d24/d40/d62/f64 [4194304,4194304] 0 2024-03-22T07:09:00.949 INFO:tasks.workunit.client.1.smithi149.stdout:4/972: dwrite d0/d8/d5c/dc1/dd2/f108 [0,4194304] 0 2024-03-22T07:09:00.949 INFO:tasks.workunit.client.1.smithi149.stdout:4/973: chown d0/c1 355005727 1 2024-03-22T07:09:00.949 INFO:tasks.workunit.client.1.smithi149.stdout:3/727: truncate d0/d4/d8/d24/d39/f71 632286 0 2024-03-22T07:09:00.949 INFO:tasks.workunit.client.1.smithi149.stdout:4/974: readlink d0/d8/dc/d74/l8a 0 2024-03-22T07:09:00.953 INFO:tasks.workunit.client.1.smithi149.stdout:9/925: dwrite db/d21/d37/d7c/dc7/f109 [0,4194304] 0 2024-03-22T07:09:00.953 INFO:tasks.workunit.client.1.smithi149.stdout:0/950: dwrite d1/d31/d11d/f130 [0,4194304] 0 2024-03-22T07:09:00.953 INFO:tasks.workunit.client.1.smithi149.stdout:0/951: dread d1/d31/d51/f5b [0,4194304] 0 2024-03-22T07:09:00.953 INFO:tasks.workunit.client.1.smithi149.stdout:0/952: fdatasync d1/d31/d51/d64/d7e/fca 0 2024-03-22T07:09:00.953 INFO:tasks.workunit.client.1.smithi149.stdout:4/975: truncate d0/d8/f1f 4309361 0 2024-03-22T07:09:00.954 INFO:tasks.workunit.client.1.smithi149.stdout:9/926: rename db/d42/d49/fdc to db/d21/f127 0 2024-03-22T07:09:00.954 INFO:tasks.workunit.client.1.smithi149.stdout:4/976: dread d0/d8/d1d/d24/d82/f91 [0,4194304] 0 2024-03-22T07:09:00.958 INFO:tasks.workunit.client.1.smithi149.stdout:9/927: rename db/lcc to db/d21/d9e/df4/l128 0 2024-03-22T07:09:00.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/953: getdents d1/d20/d9f/dc6/dd9 0 2024-03-22T07:09:00.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/954: dread - d1/d31/d52/da9/f113 zero size 2024-03-22T07:09:00.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/955: creat d1/d31/d51/d64/d8d/d9c/f159 x:0 0 0 2024-03-22T07:09:00.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/956: dread - d1/d31/f157 zero size 2024-03-22T07:09:00.960 INFO:tasks.workunit.client.1.smithi149.stdout:0/957: truncate d1/d31/d52/da9/f113 635625 0 2024-03-22T07:09:00.961 INFO:tasks.workunit.client.1.smithi149.stdout:9/928: stat db/d21/d37/d3a/c41 0 2024-03-22T07:09:00.967 INFO:tasks.workunit.client.1.smithi149.stdout:9/929: dread db/d19/f94 [0,4194304] 0 2024-03-22T07:09:00.967 INFO:tasks.workunit.client.1.smithi149.stdout:9/930: chown db/d19/d11d 49163 1 2024-03-22T07:09:00.968 INFO:tasks.workunit.client.1.smithi149.stdout:9/931: chown db/d21/d37/d3a/d104 14022 1 2024-03-22T07:09:00.969 INFO:tasks.workunit.client.1.smithi149.stdout:9/932: rmdir db/d19/da6 0 2024-03-22T07:09:00.970 INFO:tasks.workunit.client.1.smithi149.stdout:9/933: truncate db/d42/fad 2082252 0 2024-03-22T07:09:00.975 INFO:tasks.workunit.client.1.smithi149.stdout:8/941: sync 2024-03-22T07:09:00.975 INFO:tasks.workunit.client.1.smithi149.stdout:2/992: sync 2024-03-22T07:09:00.975 INFO:tasks.workunit.client.1.smithi149.stdout:8/942: stat d1/d2/d9/f4a 0 2024-03-22T07:09:00.975 INFO:tasks.workunit.client.1.smithi149.stdout:8/943: write d1/d2/d9/df/f15 [1637900,24237] 0 2024-03-22T07:09:00.976 INFO:tasks.workunit.client.1.smithi149.stdout:8/944: symlink d1/d2/d9/d11/d8b/d8d/d91/l132 0 2024-03-22T07:09:00.977 INFO:tasks.workunit.client.1.smithi149.stdout:8/945: stat d1/d2/d9/c9c 0 2024-03-22T07:09:00.978 INFO:tasks.workunit.client.1.smithi149.stdout:2/993: dread d1/d3/d5/d1f/d5e/f137 [0,4194304] 0 2024-03-22T07:09:00.978 INFO:tasks.workunit.client.1.smithi149.stdout:8/946: getdents d1/d2/d9/db2 0 2024-03-22T07:09:00.988 INFO:tasks.workunit.client.1.smithi149.stdout:3/728: dwrite d0/d4/d8/d24/f47 [4194304,4194304] 0 2024-03-22T07:09:00.989 INFO:tasks.workunit.client.1.smithi149.stdout:3/729: write d0/d4/d8/d24/d40/d54/d85/fb0 [1613715,104470] 0 2024-03-22T07:09:00.989 INFO:tasks.workunit.client.1.smithi149.stdout:4/977: dwrite d0/d8/d4c/db4/fb6 [0,4194304] 0 2024-03-22T07:09:00.989 INFO:tasks.workunit.client.1.smithi149.stdout:0/958: dwrite d1/d31/d51/d64/d7e/fca [0,4194304] 0 2024-03-22T07:09:00.993 INFO:tasks.workunit.client.1.smithi149.stdout:8/947: dread d1/d2/d9/d11/f17 [0,4194304] 0 2024-03-22T07:09:00.993 INFO:tasks.workunit.client.1.smithi149.stdout:2/994: dread d1/d8/f24 [0,4194304] 0 2024-03-22T07:09:00.994 INFO:tasks.workunit.client.1.smithi149.stdout:4/978: creat d0/d8/dc/d2e/f138 x:0 0 0 2024-03-22T07:09:00.995 INFO:tasks.workunit.client.1.smithi149.stdout:9/934: dwrite db/d19/d93/fe4 [0,4194304] 0 2024-03-22T07:09:00.997 INFO:tasks.workunit.client.1.smithi149.stdout:9/935: dread db/d21/d8c/f96 [0,4194304] 0 2024-03-22T07:09:00.998 INFO:tasks.workunit.client.1.smithi149.stdout:8/948: link d1/d2/d18/c5d d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/d10a/c133 0 2024-03-22T07:09:01.000 INFO:tasks.workunit.client.1.smithi149.stdout:2/995: link d1/d3/d5/d1f/d5e/f137 d1/d4d/d69/d75/df4/f167 0 2024-03-22T07:09:01.000 INFO:tasks.workunit.client.1.smithi149.stdout:2/996: truncate d1/d3/d5/d1f/d5e/f143 614918 0 2024-03-22T07:09:01.002 INFO:tasks.workunit.client.1.smithi149.stdout:4/979: symlink d0/d8/d1d/df1/l139 0 2024-03-22T07:09:01.002 INFO:tasks.workunit.client.1.smithi149.stdout:4/980: chown d0/d8/dc/d2e/fbf 1 1 2024-03-22T07:09:01.003 INFO:tasks.workunit.client.1.smithi149.stdout:0/959: getdents d1/d9/d14/d74/d77 0 2024-03-22T07:09:01.007 INFO:tasks.workunit.client.1.smithi149.stdout:9/936: creat db/d21/d9e/f129 x:0 0 0 2024-03-22T07:09:01.007 INFO:tasks.workunit.client.1.smithi149.stdout:9/937: readlink db/d21/d37/d3a/la9 0 2024-03-22T07:09:01.007 INFO:tasks.workunit.client.1.smithi149.stdout:9/938: fdatasync db/d42/d49/d4b/d4d/f55 0 2024-03-22T07:09:01.008 INFO:tasks.workunit.client.1.smithi149.stdout:3/730: truncate d0/d4/d8/d24/d40/d62/f64 5135454 0 2024-03-22T07:09:01.008 INFO:tasks.workunit.client.1.smithi149.stdout:8/949: dread d1/d2/d9/d11/f77 [0,4194304] 0 2024-03-22T07:09:01.011 INFO:tasks.workunit.client.1.smithi149.stdout:0/960: rename d1/d31/f8a to d1/d31/d10f/d109/f15a 0 2024-03-22T07:09:01.011 INFO:tasks.workunit.client.1.smithi149.stdout:9/939: creat db/d19/d39/f12a x:0 0 0 2024-03-22T07:09:01.012 INFO:tasks.workunit.client.1.smithi149.stdout:8/950: symlink d1/d2/d9/d11/d8b/dee/dff/l134 0 2024-03-22T07:09:01.013 INFO:tasks.workunit.client.1.smithi149.stdout:4/981: dread d0/d8/dc/dbd/fbe [4194304,4194304] 0 2024-03-22T07:09:01.016 INFO:tasks.workunit.client.1.smithi149.stdout:0/961: rmdir d1/d9/d59/dc0 39 2024-03-22T07:09:01.016 INFO:tasks.workunit.client.1.smithi149.stdout:0/962: creat d1/d31/d51/d64/d8d/f15b x:0 0 0 2024-03-22T07:09:01.017 INFO:tasks.workunit.client.1.smithi149.stdout:9/940: creat db/d19/d11d/ddb/dff/f12b x:0 0 0 2024-03-22T07:09:01.018 INFO:tasks.workunit.client.1.smithi149.stdout:8/951: symlink d1/d2/d9/d11/d4d/dd6/d51/l135 0 2024-03-22T07:09:01.021 INFO:tasks.workunit.client.1.smithi149.stdout:9/941: rename db/d12/fea to db/d19/d39/dd3/f12c 0 2024-03-22T07:09:01.025 INFO:tasks.workunit.client.1.smithi149.stdout:9/942: rename db/d42/d49/d4b/l7e to db/d21/d37/d7c/da3/l12d 0 2024-03-22T07:09:01.025 INFO:tasks.workunit.client.1.smithi149.stdout:9/943: chown db/d19/l34 27727374 1 2024-03-22T07:09:01.026 INFO:tasks.workunit.client.1.smithi149.stdout:8/952: getdents d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d84 0 2024-03-22T07:09:01.028 INFO:tasks.workunit.client.1.smithi149.stdout:2/997: dwrite d1/d3/d5/d1f/d5e/d120/f15d [0,4194304] 0 2024-03-22T07:09:01.028 INFO:tasks.workunit.client.1.smithi149.stdout:8/953: creat d1/d2/d9/d11/d8b/dee/df4/f136 x:0 0 0 2024-03-22T07:09:01.031 INFO:tasks.workunit.client.1.smithi149.stdout:4/982: write d0/d8/d1d/d24/f54 [4477847,110095] 0 2024-03-22T07:09:01.031 INFO:tasks.workunit.client.1.smithi149.stdout:4/983: chown d0/la 46 1 2024-03-22T07:09:01.034 INFO:tasks.workunit.client.1.smithi149.stdout:9/944: write db/d19/d1c/f10e [1265669,5521] 0 2024-03-22T07:09:01.036 INFO:tasks.workunit.client.1.smithi149.stdout:2/998: rename d1/d3/d5/d41/d9f/da7/d157/d129/f103 to d1/d3/de0/d147/f168 0 2024-03-22T07:09:01.036 INFO:tasks.workunit.client.1.smithi149.stdout:2/999: readlink d1/d3/d5/d1f/d5e/l31 0 2024-03-22T07:09:01.036 INFO:tasks.workunit.client.1.smithi149.stdout:4/984: mkdir d0/d8/dc/d4b/d119/d13a 0 2024-03-22T07:09:01.036 INFO:tasks.workunit.client.1.smithi149.stdout:4/985: write d0/d6b/fb9 [1112728,15485] 0 2024-03-22T07:09:01.038 INFO:tasks.workunit.client.1.smithi149.stdout:4/986: rename d0/d8/dc/d14/d10e to d0/d8/dc/d4b/d119/d13a/d13b 0 2024-03-22T07:09:01.040 INFO:tasks.workunit.client.1.smithi149.stdout:4/987: creat d0/d132/f13c x:0 0 0 2024-03-22T07:09:01.041 INFO:tasks.workunit.client.1.smithi149.stdout:4/988: symlink d0/d8/dc/d4b/d119/d13a/d13b/d12e/l13d 0 2024-03-22T07:09:01.043 INFO:tasks.workunit.client.1.smithi149.stdout:4/989: rename d0/d8/dc/d4b/db3/dea/fad to d0/d8/dc/d74/d76/f13e 0 2024-03-22T07:09:01.043 INFO:tasks.workunit.client.1.smithi149.stdout:4/990: stat d0/d8/d4c/fbb 0 2024-03-22T07:09:01.044 INFO:tasks.workunit.client.1.smithi149.stdout:0/963: dwrite d1/d31/f33 [0,4194304] 0 2024-03-22T07:09:01.044 INFO:tasks.workunit.client.1.smithi149.stdout:3/731: dwrite d0/d4/d8/d24/d39/f89 [0,4194304] 0 2024-03-22T07:09:01.044 INFO:tasks.workunit.client.1.smithi149.stdout:3/732: chown d0/d4/d8/d24/f6e 19730982 1 2024-03-22T07:09:01.044 INFO:tasks.workunit.client.1.smithi149.stdout:8/954: write d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f6e [1022092,114630] 0 2024-03-22T07:09:01.045 INFO:tasks.workunit.client.1.smithi149.stdout:4/991: symlink d0/d8/dc/d2e/d6f/def/l13f 0 2024-03-22T07:09:01.048 INFO:tasks.workunit.client.1.smithi149.stdout:3/733: dread d0/d4/d8/d24/d40/d54/d85/dbc/fc2 [0,4194304] 0 2024-03-22T07:09:01.048 INFO:tasks.workunit.client.1.smithi149.stdout:3/734: fsync d0/d4/d8/d24/d39/fb1 0 2024-03-22T07:09:01.048 INFO:tasks.workunit.client.1.smithi149.stdout:3/735: fdatasync d0/d2/d5a/daf/dce/fea 0 2024-03-22T07:09:01.048 INFO:tasks.workunit.client.1.smithi149.stdout:3/736: write d0/d2/f8c [910342,119610] 0 2024-03-22T07:09:01.055 INFO:tasks.workunit.client.1.smithi149.stdout:9/945: dwrite db/d19/f3c [0,4194304] 0 2024-03-22T07:09:01.078 INFO:tasks.workunit.client.1.smithi149.stdout:0/964: dwrite d1/d20/d9f/dc6/dd9/fda [4194304,4194304] 0 2024-03-22T07:09:01.078 INFO:tasks.workunit.client.1.smithi149.stdout:3/737: dwrite d0/d4/d8/d24/d40/d62/f64 [4194304,4194304] 0 2024-03-22T07:09:01.080 INFO:tasks.workunit.client.1.smithi149.stdout:0/965: mknod d1/d136/d10c/c15c 0 2024-03-22T07:09:01.086 INFO:tasks.workunit.client.1.smithi149.stdout:0/966: read d1/d9/fd2 [2874508,30468] 0 2024-03-22T07:09:01.086 INFO:tasks.workunit.client.1.smithi149.stdout:0/967: stat d1/d9/d14/d74/d77 0 2024-03-22T07:09:01.087 INFO:tasks.workunit.client.1.smithi149.stdout:0/968: chown d1/d9/d59/c7c 0 1 2024-03-22T07:09:01.087 INFO:tasks.workunit.client.1.smithi149.stdout:0/969: unlink d1/d20/d9f/dc6/ddc/l110 0 2024-03-22T07:09:01.142 INFO:tasks.workunit.client.1.smithi149.stdout:5/935: sync 2024-03-22T07:09:01.144 INFO:tasks.workunit.client.1.smithi149.stdout:5/936: creat d7/d35/de0/d92/df4/df5/d10b/d122/f13d x:0 0 0 2024-03-22T07:09:01.144 INFO:tasks.workunit.client.1.smithi149.stdout:1/994: sync 2024-03-22T07:09:01.144 INFO:tasks.workunit.client.1.smithi149.stdout:1/995: write d4/d20/d2d/d64/d9f/dbe/f133 [789167,99399] 0 2024-03-22T07:09:01.145 INFO:tasks.workunit.client.1.smithi149.stdout:5/937: creat d7/da/d20/d89/da2/f13e x:0 0 0 2024-03-22T07:09:01.147 INFO:tasks.workunit.client.1.smithi149.stdout:1/996: dread d4/d10/dd8/d3c/d8e/db5/db2/dbf/f13a [0,4194304] 0 2024-03-22T07:09:01.147 INFO:tasks.workunit.client.1.smithi149.stdout:1/997: read d4/d7/dd3/d129/f62 [2414073,20575] 0 2024-03-22T07:09:01.147 INFO:tasks.workunit.client.1.smithi149.stdout:1/998: chown d4/d7/d5a/dca/c112 1874797116 1 2024-03-22T07:09:01.148 INFO:tasks.workunit.client.1.smithi149.stdout:1/999: unlink d4/d10/dd8/d3c/d109/d13f/f5b 0 2024-03-22T07:09:01.156 INFO:tasks.workunit.client.1.smithi149.stdout:7/971: sync 2024-03-22T07:09:01.157 INFO:tasks.workunit.client.1.smithi149.stdout:5/938: read d7/d35/de0/dd7/d7b/f125 [3534383,108954] 0 2024-03-22T07:09:01.158 INFO:tasks.workunit.client.1.smithi149.stdout:5/939: dread d7/da/d20/d30/d3a/d3d/d2c/db2/d106/f109 [0,4194304] 0 2024-03-22T07:09:01.160 INFO:tasks.workunit.client.1.smithi149.stdout:9/946: dwrite db/d21/d8c/f121 [0,4194304] 0 2024-03-22T07:09:01.160 INFO:tasks.workunit.client.1.smithi149.stdout:0/970: dwrite d1/f6a [0,4194304] 0 2024-03-22T07:09:01.165 INFO:tasks.workunit.client.1.smithi149.stdout:7/972: dread d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/f6b [0,4194304] 0 2024-03-22T07:09:01.165 INFO:tasks.workunit.client.1.smithi149.stdout:8/955: dwrite d1/d2/d9/d11/d8b/dee/df4/f108 [0,4194304] 0 2024-03-22T07:09:01.166 INFO:tasks.workunit.client.1.smithi149.stdout:9/947: dread db/d19/d39/f5e [0,4194304] 0 2024-03-22T07:09:01.167 INFO:tasks.workunit.client.1.smithi149.stdout:7/973: symlink d1/d1c/d49/d8e/d100/d1b/d1d/d72/l143 0 2024-03-22T07:09:01.167 INFO:tasks.workunit.client.1.smithi149.stdout:7/974: readlink d1/d1c/d49/d8e/l65 0 2024-03-22T07:09:01.168 INFO:tasks.workunit.client.1.smithi149.stdout:9/948: mkdir db/d19/d12e 0 2024-03-22T07:09:01.169 INFO:tasks.workunit.client.1.smithi149.stdout:5/940: dread d7/da/d20/d36/f66 [0,4194304] 0 2024-03-22T07:09:01.169 INFO:tasks.workunit.client.1.smithi149.stdout:7/975: unlink d1/d1c/d49/d8e/d53/d140/d5a/d6a/fb0 0 2024-03-22T07:09:01.170 INFO:tasks.workunit.client.1.smithi149.stdout:9/949: truncate db/d21/d37/f5c 965636 0 2024-03-22T07:09:01.170 INFO:tasks.workunit.client.1.smithi149.stdout:0/971: dread d1/d9/d59/dc0/dc9/fcd [0,4194304] 0 2024-03-22T07:09:01.172 INFO:tasks.workunit.client.1.smithi149.stdout:5/941: rename d7/da/d20/d127 to d7/da/d20/d30/d3a/d3d/d2c/db2/d13f 0 2024-03-22T07:09:01.172 INFO:tasks.workunit.client.1.smithi149.stdout:5/942: creat d7/da/d20/d30/d3a/d3d/d104/f140 x:0 0 0 2024-03-22T07:09:01.173 INFO:tasks.workunit.client.1.smithi149.stdout:7/976: mknod d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/dfe/d120/c144 0 2024-03-22T07:09:01.173 INFO:tasks.workunit.client.1.smithi149.stdout:8/956: read d1/d2/ffe [222170,104631] 0 2024-03-22T07:09:01.174 INFO:tasks.workunit.client.1.smithi149.stdout:9/950: creat db/d19/d39/dd3/d11a/f12f x:0 0 0 2024-03-22T07:09:01.175 INFO:tasks.workunit.client.1.smithi149.stdout:9/951: chown db/f117 342967 1 2024-03-22T07:09:01.175 INFO:tasks.workunit.client.1.smithi149.stdout:9/952: fsync db/d21/d37/d7c/dc7/feb 0 2024-03-22T07:09:01.175 INFO:tasks.workunit.client.1.smithi149.stdout:5/943: mkdir d7/da/d20/d36/d141 0 2024-03-22T07:09:01.175 INFO:tasks.workunit.client.1.smithi149.stdout:5/944: write d7/da/d20/d30/d3a/d3d/d2c/db2/d106/f109 [683671,87640] 0 2024-03-22T07:09:01.178 INFO:tasks.workunit.client.1.smithi149.stdout:8/957: unlink d1/d2/l4e 0 2024-03-22T07:09:01.178 INFO:tasks.workunit.client.1.smithi149.stdout:8/958: fsync d1/d2/d3/fb0 0 2024-03-22T07:09:01.181 INFO:tasks.workunit.client.1.smithi149.stdout:9/953: rename db/d12/ffb to db/d21/d8c/f130 0 2024-03-22T07:09:01.181 INFO:tasks.workunit.client.1.smithi149.stdout:8/959: rmdir d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d84 39 2024-03-22T07:09:01.182 INFO:tasks.workunit.client.1.smithi149.stdout:9/954: mknod db/d21/d37/c131 0 2024-03-22T07:09:01.186 INFO:tasks.workunit.client.1.smithi149.stdout:9/955: mkdir db/d22/d11f/d132 0 2024-03-22T07:09:01.186 INFO:tasks.workunit.client.1.smithi149.stdout:9/956: chown db/d21/d2e 0 1 2024-03-22T07:09:01.186 INFO:tasks.workunit.client.1.smithi149.stdout:9/957: chown db/d42/d49/faa 14 1 2024-03-22T07:09:01.198 INFO:tasks.workunit.client.1.smithi149.stdout:9/958: dread db/d42/d49/f8e [0,4194304] 0 2024-03-22T07:09:01.200 INFO:tasks.workunit.client.1.smithi149.stdout:9/959: link db/d33/ld2 db/d21/d9e/df4/l133 0 2024-03-22T07:09:01.201 INFO:tasks.workunit.client.1.smithi149.stdout:9/960: mknod db/d42/d49/c134 0 2024-03-22T07:09:01.203 INFO:tasks.workunit.client.1.smithi149.stdout:9/961: symlink db/d42/d49/d4b/l135 0 2024-03-22T07:09:01.204 INFO:tasks.workunit.client.1.smithi149.stdout:9/962: link db/d22/lec db/d22/d11f/d132/l136 0 2024-03-22T07:09:01.204 INFO:tasks.workunit.client.1.smithi149.stdout:9/963: creat db/d19/d39/d3b/f137 x:0 0 0 2024-03-22T07:09:01.206 INFO:tasks.workunit.client.1.smithi149.stdout:9/964: creat db/d21/d2e/d126/f138 x:0 0 0 2024-03-22T07:09:01.208 INFO:tasks.workunit.client.1.smithi149.stdout:9/965: link db/d21/d37/d7c/dc7/f109 db/d21/d37/d7c/da3/f139 0 2024-03-22T07:09:01.209 INFO:tasks.workunit.client.1.smithi149.stdout:9/966: mkdir db/d19/d12e/d13a 0 2024-03-22T07:09:01.209 INFO:tasks.workunit.client.1.smithi149.stdout:9/967: truncate db/d21/d37/ff6 553951 0 2024-03-22T07:09:01.210 INFO:tasks.workunit.client.1.smithi149.stdout:9/968: creat db/d21/d37/d7c/dc7/f13b x:0 0 0 2024-03-22T07:09:01.212 INFO:tasks.workunit.client.1.smithi149.stdout:9/969: rename db/d19/d1c/cef to db/d21/d37/d3a/d104/c13c 0 2024-03-22T07:09:01.212 INFO:tasks.workunit.client.1.smithi149.stdout:7/977: dwrite d1/d1c/d49/d8e/d100/f1e [0,4194304] 0 2024-03-22T07:09:01.212 INFO:tasks.workunit.client.1.smithi149.stdout:5/945: dwrite d7/da/d20/d36/d44/d4a/f123 [0,4194304] 0 2024-03-22T07:09:01.213 INFO:tasks.workunit.client.1.smithi149.stdout:8/960: dwrite d1/d2/d9/df/f2d [0,4194304] 0 2024-03-22T07:09:01.217 INFO:tasks.workunit.client.1.smithi149.stdout:7/978: creat d1/d1c/d49/d8e/d53/d140/d5a/de7/d74/f145 x:0 0 0 2024-03-22T07:09:01.218 INFO:tasks.workunit.client.1.smithi149.stdout:5/946: creat d7/da/d20/d30/d3a/d3d/d2c/db2/d13a/f142 x:0 0 0 2024-03-22T07:09:01.220 INFO:tasks.workunit.client.1.smithi149.stdout:8/961: unlink d1/d2/d9/d11/fa3 0 2024-03-22T07:09:01.220 INFO:tasks.workunit.client.1.smithi149.stdout:8/962: truncate d1/d2/d9/d11/d4d/dd6/d51/f98 1765119 0 2024-03-22T07:09:01.235 INFO:tasks.workunit.client.1.smithi149.stdout:8/963: dwrite d1/d2/d9/d11/d8b/daf/f123 [0,4194304] 0 2024-03-22T07:09:01.237 INFO:tasks.workunit.client.1.smithi149.stdout:9/970: rmdir db 39 2024-03-22T07:09:01.238 INFO:tasks.workunit.client.1.smithi149.stdout:7/979: truncate d1/d1c/d49/d8e/d100/d1b/d1d/fe6 2640785 0 2024-03-22T07:09:01.252 INFO:tasks.workunit.client.1.smithi149.stdout:9/971: creat db/d21/d37/d3a/d104/f13d x:0 0 0 2024-03-22T07:09:01.252 INFO:tasks.workunit.client.1.smithi149.stdout:9/972: chown db/d21/d37/d7c/dc7/f109 1931 1 2024-03-22T07:09:01.252 INFO:tasks.workunit.client.1.smithi149.stdout:9/973: dread - db/d21/d9e/df4/f110 zero size 2024-03-22T07:09:01.253 INFO:tasks.workunit.client.1.smithi149.stdout:7/980: unlink d1/d1c/d49/d8e/d100/d1b/d99/de5/d92/d9e/f13b 0 2024-03-22T07:09:01.255 INFO:tasks.workunit.client.1.smithi149.stdout:5/947: rmdir d7/da/d20/d89/da2/dda 39 2024-03-22T07:09:01.256 INFO:tasks.workunit.client.1.smithi149.stdout:8/964: truncate d1/d2/d9/fea 198413 0 2024-03-22T07:09:01.257 INFO:tasks.workunit.client.1.smithi149.stdout:5/948: rename d7/da/d20/d36/d44/c82 to d7/d35/de0/d92/df4/df5/d10b/d122/c143 0 2024-03-22T07:09:01.259 INFO:tasks.workunit.client.1.smithi149.stdout:5/949: dread d7/da/d20/d30/d3a/d3d/d2c/db2/d106/f108 [0,4194304] 0 2024-03-22T07:09:01.260 INFO:tasks.workunit.client.1.smithi149.stdout:9/974: write db/d21/d37/d7c/da3/fc2 [1815315,6340] 0 2024-03-22T07:09:01.260 INFO:tasks.workunit.client.1.smithi149.stdout:9/975: read - db/d19/d28/fd5 zero size 2024-03-22T07:09:01.262 INFO:tasks.workunit.client.1.smithi149.stdout:9/976: mkdir db/d22/d11f/d132/d13e 0 2024-03-22T07:09:01.263 INFO:tasks.workunit.client.1.smithi149.stdout:9/977: creat db/d21/d9e/df4/f13f x:0 0 0 2024-03-22T07:09:01.263 INFO:tasks.workunit.client.1.smithi149.stdout:9/978: chown db/d21/d9e/l10d 47182692 1 2024-03-22T07:09:01.263 INFO:tasks.workunit.client.1.smithi149.stdout:9/979: stat db/d19/d28/fd5 0 2024-03-22T07:09:01.265 INFO:tasks.workunit.client.1.smithi149.stdout:9/980: creat db/d21/d8c/f140 x:0 0 0 2024-03-22T07:09:01.268 INFO:tasks.workunit.client.1.smithi149.stdout:8/965: write d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d84/f48 [299591,21922] 0 2024-03-22T07:09:01.269 INFO:tasks.workunit.client.1.smithi149.stdout:5/950: write d7/da/d20/d30/d3a/d3d/d2c/fb1 [1487648,5560] 0 2024-03-22T07:09:01.269 INFO:tasks.workunit.client.1.smithi149.stdout:5/951: creat d7/da/d7d/f144 x:0 0 0 2024-03-22T07:09:01.273 INFO:tasks.workunit.client.1.smithi149.stdout:7/981: dwrite d1/d1c/d49/d8e/d53/d140/d5a/fe9 [0,4194304] 0 2024-03-22T07:09:01.273 INFO:tasks.workunit.client.1.smithi149.stdout:7/982: write d1/d1c/d49/d8e/d53/d140/d5a/de7/d74/f145 [793760,23916] 0 2024-03-22T07:09:01.273 INFO:tasks.workunit.client.1.smithi149.stdout:7/983: readlink d1/lc 0 2024-03-22T07:09:01.273 INFO:tasks.workunit.client.1.smithi149.stdout:7/984: chown d1/d1c/d49/d8e/d53/d140/d5a/d6a/l117 10378789 1 2024-03-22T07:09:01.273 INFO:tasks.workunit.client.1.smithi149.stdout:7/985: fdatasync d1/d1c/d49/d8e/d100/d1b/d1d/f40 0 2024-03-22T07:09:01.274 INFO:tasks.workunit.client.1.smithi149.stdout:9/981: write db/d21/d37/d7c/da3/f139 [3413490,29078] 0 2024-03-22T07:09:01.275 INFO:tasks.workunit.client.1.smithi149.stdout:9/982: chown db/d21/d9e/lf9 17 1 2024-03-22T07:09:01.275 INFO:tasks.workunit.client.1.smithi149.stdout:5/952: dread d7/da/d20/d30/d3a/d93/fa0 [0,4194304] 0 2024-03-22T07:09:01.279 INFO:tasks.workunit.client.1.smithi149.stdout:9/983: link db/d19/d28/f84 db/d21/d37/d3a/f141 0 2024-03-22T07:09:01.281 INFO:tasks.workunit.client.1.smithi149.stdout:7/986: getdents d1/d1c/d49/d8e/d100/d1b/d11a 0 2024-03-22T07:09:01.281 INFO:tasks.workunit.client.1.smithi149.stdout:7/987: write d1/d1c/d49/d8e/d100/d1b/d99/de5/da3/dd7/d10f/de3/fc5 [4883314,104246] 0 2024-03-22T07:09:01.281 INFO:tasks.workunit.client.1.smithi149.stdout:7/988: chown d1/d1c/d49/d8e/d53/d140/d7a/d7d/d5b/lc2 1528647 1 2024-03-22T07:09:01.283 INFO:tasks.workunit.client.1.smithi149.stdout:9/984: creat db/d42/d49/d4b/f142 x:0 0 0 2024-03-22T07:09:01.283 INFO:tasks.workunit.client.1.smithi149.stdout:9/985: write db/d19/d39/f12a [677363,5096] 0 2024-03-22T07:09:01.284 INFO:tasks.workunit.client.1.smithi149.stdout:7/989: truncate d1/d1c/fbb 135556 0 2024-03-22T07:09:01.284 INFO:tasks.workunit.client.1.smithi149.stdout:7/990: fdatasync d1/d1c/d49/d8e/d100/d1b/f42 0 2024-03-22T07:09:01.284 INFO:tasks.workunit.client.1.smithi149.stdout:7/991: stat d1/d1c/d49/d8e/d53/d140 0 2024-03-22T07:09:01.284 INFO:tasks.workunit.client.1.smithi149.stdout:9/986: creat db/d22/f143 x:0 0 0 2024-03-22T07:09:01.285 INFO:tasks.workunit.client.1.smithi149.stdout:9/987: symlink db/d19/d12e/d13a/l144 0 2024-03-22T07:09:01.286 INFO:tasks.workunit.client.1.smithi149.stdout:9/988: symlink db/d19/d39/d68/l145 0 2024-03-22T07:09:01.288 INFO:tasks.workunit.client.1.smithi149.stdout:7/992: rename d1/d1c/d49/d8e/d9c/df7 to d1/d1c/d49/d8e/d100/d1b/d99/de5/db4/d146 0 2024-03-22T07:09:01.289 INFO:tasks.workunit.client.1.smithi149.stdout:7/993: symlink d1/d1c/d49/d8e/d53/l147 0 2024-03-22T07:09:01.289 INFO:tasks.workunit.client.1.smithi149.stdout:8/966: dwrite d1/d2/d9/d11/d4d/dd6/d51/fe2 [0,4194304] 0 2024-03-22T07:09:01.289 INFO:tasks.workunit.client.1.smithi149.stdout:8/967: chown d1/d2/d9/d11/d4d/ca8 0 1 2024-03-22T07:09:01.292 INFO:tasks.workunit.client.1.smithi149.stdout:9/989: dread db/d19/d39/dd3/f12c [4194304,4194304] 0 2024-03-22T07:09:01.315 INFO:tasks.workunit.client.1.smithi149.stdout:8/968: dwrite d1/d2/d9/d11/f17 [0,4194304] 0 2024-03-22T07:09:01.315 INFO:tasks.workunit.client.1.smithi149.stdout:9/990: dwrite db/d22/f69 [4194304,4194304] 0 2024-03-22T07:09:01.315 INFO:tasks.workunit.client.1.smithi149.stdout:9/991: write db/d21/d37/d3a/f6c [4949636,2450] 0 2024-03-22T07:09:01.324 INFO:tasks.workunit.client.1.smithi149.stdout:8/969: dread d1/d2/d9/d11/d4d/dd6/f72 [0,4194304] 0 2024-03-22T07:09:01.325 INFO:tasks.workunit.client.1.smithi149.stdout:8/970: stat d1/d2/d9/d11/d4d/dd6/d35/ccf 0 2024-03-22T07:09:01.325 INFO:tasks.workunit.client.1.smithi149.stdout:8/971: write d1/d2/d3/fb0 [1415939,38516] 0 2024-03-22T07:09:01.325 INFO:tasks.workunit.client.1.smithi149.stdout:8/972: readlink d1/d2/d18/ld1 0 2024-03-22T07:09:01.325 INFO:tasks.workunit.client.1.smithi149.stdout:8/973: creat d1/d2/d9/d11/d4d/dd6/d51/f137 x:0 0 0 2024-03-22T07:09:01.334 INFO:tasks.workunit.client.1.smithi149.stdout:9/992: dwrite db/d19/d39/d3b/dd8/f11e [0,4194304] 0 2024-03-22T07:09:01.334 INFO:tasks.workunit.client.1.smithi149.stdout:9/993: creat db/d21/d8c/f146 x:0 0 0 2024-03-22T07:09:01.338 INFO:tasks.workunit.client.1.smithi149.stdout:9/994: symlink db/d22/d11f/d132/l147 0 2024-03-22T07:09:01.339 INFO:tasks.workunit.client.1.smithi149.stdout:8/974: dread d1/d2/f8f [0,4194304] 0 2024-03-22T07:09:01.340 INFO:tasks.workunit.client.1.smithi149.stdout:9/995: mkdir db/d21/d37/d3a/d148 0 2024-03-22T07:09:01.340 INFO:tasks.workunit.client.1.smithi149.stdout:9/996: truncate db/d19/ffa 787947 0 2024-03-22T07:09:01.341 INFO:tasks.workunit.client.1.smithi149.stdout:8/975: mkdir d1/d2/d9/d11/d8b/daf/d138 0 2024-03-22T07:09:01.343 INFO:tasks.workunit.client.1.smithi149.stdout:9/997: dread db/d42/f5a [0,4194304] 0 2024-03-22T07:09:01.343 INFO:tasks.workunit.client.1.smithi149.stdout:8/976: unlink d1/d2/d9/df/dd5/def/d128/f12a 0 2024-03-22T07:09:01.344 INFO:tasks.workunit.client.1.smithi149.stdout:9/998: symlink db/d21/d2e/l149 0 2024-03-22T07:09:01.344 INFO:tasks.workunit.client.1.smithi149.stdout:9/999: readlink db/d21/d8c/l103 0 2024-03-22T07:09:01.357 INFO:tasks.workunit.client.1.smithi149.stdout:8/977: dwrite d1/d2/d9/d11/d4d/dd6/d35/f6f [0,4194304] 0 2024-03-22T07:09:01.359 INFO:tasks.workunit.client.1.smithi149.stdout:8/978: mknod d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/d9e/c139 0 2024-03-22T07:09:01.359 INFO:tasks.workunit.client.1.smithi149.stdout:8/979: chown d1/d2/d9/d11/d4d/dd6/d51/fa6 27 1 2024-03-22T07:09:01.359 INFO:tasks.workunit.client.1.smithi149.stdout:8/980: fsync d1/d2/d9/df/f2d 0 2024-03-22T07:09:01.367 INFO:tasks.workunit.client.1.smithi149.stdout:8/981: dread d1/d2/d9/d11/d4d/dd6/d51/fa6 [0,4194304] 0 2024-03-22T07:09:01.367 INFO:tasks.workunit.client.1.smithi149.stdout:8/982: truncate d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f117 1212280 0 2024-03-22T07:09:01.370 INFO:tasks.workunit.client.1.smithi149.stdout:0/972: sync 2024-03-22T07:09:01.370 INFO:tasks.workunit.client.1.smithi149.stdout:3/738: sync 2024-03-22T07:09:01.370 INFO:tasks.workunit.client.1.smithi149.stdout:4/992: sync 2024-03-22T07:09:01.371 INFO:tasks.workunit.client.1.smithi149.stdout:3/739: truncate d0/d2/d30/d5d/f65 5126721 0 2024-03-22T07:09:01.372 INFO:tasks.workunit.client.1.smithi149.stdout:0/973: rename d1/d136/d10c/fd0 to d1/d6b/f15d 0 2024-03-22T07:09:01.372 INFO:tasks.workunit.client.1.smithi149.stdout:4/993: stat d0/d8/f10c 0 2024-03-22T07:09:01.372 INFO:tasks.workunit.client.1.smithi149.stdout:4/994: fdatasync d0/d29/f31 0 2024-03-22T07:09:01.374 INFO:tasks.workunit.client.1.smithi149.stdout:3/740: unlink d0/d2/d30/d5d/d84/d8f/l28 0 2024-03-22T07:09:01.376 INFO:tasks.workunit.client.1.smithi149.stdout:0/974: rename d1/d31/d14f/f150 to d1/d31/d52/da9/f15e 0 2024-03-22T07:09:01.376 INFO:tasks.workunit.client.1.smithi149.stdout:0/975: write d1/d9/d14/d74/d77/f156 [77437,70771] 0 2024-03-22T07:09:01.377 INFO:tasks.workunit.client.1.smithi149.stdout:4/995: symlink d0/d8/dc/d74/d76/deb/l140 0 2024-03-22T07:09:01.377 INFO:tasks.workunit.client.1.smithi149.stdout:4/996: stat d0/d8/dc/d2e/fb7 0 2024-03-22T07:09:01.377 INFO:tasks.workunit.client.1.smithi149.stdout:3/741: symlink d0/d2/d5a/d93/l10b 0 2024-03-22T07:09:01.377 INFO:tasks.workunit.client.1.smithi149.stdout:3/742: truncate d0/fbe 4341252 0 2024-03-22T07:09:01.378 INFO:tasks.workunit.client.1.smithi149.stdout:0/976: getdents d1/d31/d10f 0 2024-03-22T07:09:01.378 INFO:tasks.workunit.client.1.smithi149.stdout:0/977: write d1/d27/ff6 [763043,52582] 0 2024-03-22T07:09:01.378 INFO:tasks.workunit.client.1.smithi149.stdout:0/978: creat d1/d31/d51/d64/d7e/f15f x:0 0 0 2024-03-22T07:09:01.378 INFO:tasks.workunit.client.1.smithi149.stdout:0/979: read - d1/d31/d51/d64/f128 zero size 2024-03-22T07:09:01.379 INFO:tasks.workunit.client.1.smithi149.stdout:0/980: stat d1/d9/d14/d1f/cb3 0 2024-03-22T07:09:01.379 INFO:tasks.workunit.client.1.smithi149.stdout:3/743: mkdir d0/d2/dfe/d10c 0 2024-03-22T07:09:01.380 INFO:tasks.workunit.client.1.smithi149.stdout:0/981: creat d1/d31/d10f/f160 x:0 0 0 2024-03-22T07:09:01.380 INFO:tasks.workunit.client.1.smithi149.stdout:0/982: readlink d1/d31/d10f/l135 0 2024-03-22T07:09:01.381 INFO:tasks.workunit.client.1.smithi149.stdout:0/983: symlink d1/d27/d66/df1/l161 0 2024-03-22T07:09:01.381 INFO:tasks.workunit.client.1.smithi149.stdout:4/997: write d0/d8/dc/d74/d76/fb8 [965133,102049] 0 2024-03-22T07:09:01.382 INFO:tasks.workunit.client.1.smithi149.stdout:0/984: creat d1/d31/d10f/d109/f162 x:0 0 0 2024-03-22T07:09:01.382 INFO:tasks.workunit.client.1.smithi149.stdout:0/985: write d1/d9/d59/fba [865193,12709] 0 2024-03-22T07:09:01.382 INFO:tasks.workunit.client.1.smithi149.stdout:0/986: write d1/d31/d51/d64/d87/da6/fcb [992801,5442] 0 2024-03-22T07:09:01.382 INFO:tasks.workunit.client.1.smithi149.stdout:0/987: truncate d1/d31/d52/da9/f15e 233047 0 2024-03-22T07:09:01.382 INFO:tasks.workunit.client.1.smithi149.stdout:0/988: write d1/d9/d14/fc4 [1340060,121153] 0 2024-03-22T07:09:01.384 INFO:tasks.workunit.client.1.smithi149.stdout:0/989: symlink d1/d31/d51/d64/d87/da6/l163 0 2024-03-22T07:09:01.386 INFO:tasks.workunit.client.1.smithi149.stdout:8/983: dwrite d1/d2/d9/d11/d4d/dd6/db4/f107 [0,4194304] 0 2024-03-22T07:09:01.386 INFO:tasks.workunit.client.1.smithi149.stdout:8/984: fsync d1/d2/d9/d11/d4d/dd6/db4/dce/f114 0 2024-03-22T07:09:01.389 INFO:tasks.workunit.client.1.smithi149.stdout:8/985: chown d1/d2/d9/d11/d4d/dd6/d35/c10 3390 1 2024-03-22T07:09:01.393 INFO:tasks.workunit.client.1.smithi149.stdout:0/990: write d1/d9/f29 [192973,34078] 0 2024-03-22T07:09:01.393 INFO:tasks.workunit.client.1.smithi149.stdout:0/991: creat d1/d20/d9f/f164 x:0 0 0 2024-03-22T07:09:01.397 INFO:tasks.workunit.client.1.smithi149.stdout:0/992: creat d1/d9/d59/dc0/f165 x:0 0 0 2024-03-22T07:09:01.397 INFO:tasks.workunit.client.1.smithi149.stdout:3/744: dread d0/d2/d30/dee/ffb [0,4194304] 0 2024-03-22T07:09:01.399 INFO:tasks.workunit.client.1.smithi149.stdout:0/993: unlink d1/d31/d51/d64/d8d/f15b 0 2024-03-22T07:09:01.400 INFO:tasks.workunit.client.1.smithi149.stdout:0/994: stat d1/d1a/c54 0 2024-03-22T07:09:01.403 INFO:tasks.workunit.client.1.smithi149.stdout:0/995: creat d1/d27/d66/df1/f166 x:0 0 0 2024-03-22T07:09:01.416 INFO:tasks.workunit.client.1.smithi149.stdout:0/996: write d1/d1a/f2d [2516064,95036] 0 2024-03-22T07:09:01.417 INFO:tasks.workunit.client.1.smithi149.stdout:0/997: unlink d1/f6a 0 2024-03-22T07:09:01.417 INFO:tasks.workunit.client.1.smithi149.stdout:0/998: chown d1/d9/d14/d1f 1347 1 2024-03-22T07:09:01.417 INFO:tasks.workunit.client.1.smithi149.stdout:0/999: chown d1/d31/d51/d64/d87/da6/fd6 5 1 2024-03-22T07:09:01.421 INFO:tasks.workunit.client.1.smithi149.stdout:4/998: dwrite d0/d8/dc/d74/fe3 [0,4194304] 0 2024-03-22T07:09:01.423 INFO:tasks.workunit.client.1.smithi149.stdout:8/986: dwrite d1/d2/d9/d11/d4d/dd6/f72 [0,4194304] 0 2024-03-22T07:09:01.423 INFO:tasks.workunit.client.1.smithi149.stdout:8/987: chown d1/d2/d18/f3c 9 1 2024-03-22T07:09:01.423 INFO:tasks.workunit.client.1.smithi149.stdout:8/988: stat d1/d2/d9/d11/d8b/daf/cb3 0 2024-03-22T07:09:01.424 INFO:tasks.workunit.client.1.smithi149.stdout:3/745: dwrite d0/d2/d30/f73 [0,4194304] 0 2024-03-22T07:09:01.425 INFO:tasks.workunit.client.1.smithi149.stdout:8/989: mknod d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d82/c13a 0 2024-03-22T07:09:01.425 INFO:tasks.workunit.client.1.smithi149.stdout:8/990: dread - d1/d2/d9/df/dd5/f10e zero size 2024-03-22T07:09:01.425 INFO:tasks.workunit.client.1.smithi149.stdout:8/991: truncate d1/d2/d9/d11/d8b/daf/f106 474518 0 2024-03-22T07:09:01.437 INFO:tasks.workunit.client.1.smithi149.stdout:8/992: rmdir d1/d2/d9/d11/d8b/d8d/df2 39 2024-03-22T07:09:01.439 INFO:tasks.workunit.client.1.smithi149.stdout:8/993: write d1/d2/d9/f29 [2435995,128554] 0 2024-03-22T07:09:01.445 INFO:tasks.workunit.client.1.smithi149.stdout:3/746: dwrite d0/f72 [0,4194304] 0 2024-03-22T07:09:01.445 INFO:tasks.workunit.client.1.smithi149.stdout:4/999: dwrite d0/d8/dc/dbd/f127 [0,4194304] 0 2024-03-22T07:09:01.448 INFO:tasks.workunit.client.1.smithi149.stdout:3/747: creat d0/d2/d30/d5d/f10d x:0 0 0 2024-03-22T07:09:01.448 INFO:tasks.workunit.client.1.smithi149.stdout:8/994: dread d1/d2/d9/d11/d4d/dd6/fc7 [0,4194304] 0 2024-03-22T07:09:01.450 INFO:tasks.workunit.client.1.smithi149.stdout:3/748: symlink d0/d2/d30/d5d/d84/dc3/l10e 0 2024-03-22T07:09:01.451 INFO:tasks.workunit.client.1.smithi149.stdout:8/995: mkdir d1/d2/d9/d11/d8b/dee/d13b 0 2024-03-22T07:09:01.452 INFO:tasks.workunit.client.1.smithi149.stdout:8/996: symlink d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d82/l13c 0 2024-03-22T07:09:01.481 INFO:tasks.workunit.client.1.smithi149.stdout:3/749: dwrite d0/d2/d30/d5d/f10d [0,4194304] 0 2024-03-22T07:09:01.481 INFO:tasks.workunit.client.1.smithi149.stdout:3/750: fsync d0/d4/d8/d24/d39/f89 0 2024-03-22T07:09:01.483 INFO:tasks.workunit.client.1.smithi149.stdout:3/751: mknod d0/d2/d5a/daf/c10f 0 2024-03-22T07:09:01.483 INFO:tasks.workunit.client.1.smithi149.stdout:3/752: chown d0/d2/d5a/cd7 79377437 1 2024-03-22T07:09:01.485 INFO:tasks.workunit.client.1.smithi149.stdout:3/753: creat d0/f110 x:0 0 0 2024-03-22T07:09:01.491 INFO:tasks.workunit.client.1.smithi149.stdout:3/754: dread d0/d2/d5a/daf/fdd [0,4194304] 0 2024-03-22T07:09:01.492 INFO:tasks.workunit.client.1.smithi149.stdout:3/755: getdents d0/d2/d5a 0 2024-03-22T07:09:01.492 INFO:tasks.workunit.client.1.smithi149.stdout:3/756: chown d0/d4/c27 478871242 1 2024-03-22T07:09:01.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:01 smithi081 ceph-mon[29245]: pgmap v13: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 160 MiB/s rd, 204 MiB/s wr, 294 op/s 2024-03-22T07:09:01.572 INFO:tasks.workunit.client.1.smithi149.stdout:7/994: sync 2024-03-22T07:09:01.574 INFO:tasks.workunit.client.1.smithi149.stdout:5/953: sync 2024-03-22T07:09:01.576 INFO:tasks.workunit.client.1.smithi149.stdout:5/954: getdents d7/d35/de0/dd7/d7b 0 2024-03-22T07:09:01.578 INFO:tasks.workunit.client.1.smithi149.stdout:5/955: creat d7/d35/de0/d92/df4/df5/f145 x:0 0 0 2024-03-22T07:09:01.578 INFO:tasks.workunit.client.1.smithi149.stdout:5/956: truncate d7/da/d7d/f116 949078 0 2024-03-22T07:09:01.579 INFO:tasks.workunit.client.1.smithi149.stdout:5/957: creat d7/da/d20/d30/d3a/d3d/d2c/db2/d13f/f146 x:0 0 0 2024-03-22T07:09:01.582 INFO:tasks.workunit.client.1.smithi149.stdout:5/958: stat d7/d35/c8f 0 2024-03-22T07:09:01.583 INFO:tasks.workunit.client.1.smithi149.stdout:5/959: getdents d7/da/d20/d89/da2 0 2024-03-22T07:09:01.588 INFO:tasks.workunit.client.1.smithi149.stdout:7/995: dwrite d1/d1c/d49/d8e/d53/d140/d5a/de7/f127 [0,4194304] 0 2024-03-22T07:09:01.592 INFO:tasks.workunit.client.1.smithi149.stdout:7/996: unlink d1/d1c/d49/d8e/d100/d1b/ff1 0 2024-03-22T07:09:01.595 INFO:tasks.workunit.client.1.smithi149.stdout:7/997: rmdir d1/d1c/d49/d8e/d53 39 2024-03-22T07:09:01.598 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:01 smithi149 ceph-mon[36068]: pgmap v13: 65 pgs: 65 active+clean; 2.1 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 160 MiB/s rd, 204 MiB/s wr, 294 op/s 2024-03-22T07:09:01.602 INFO:tasks.workunit.client.1.smithi149.stdout:7/998: dread d1/d1c/d49/d8e/d100/d1b/d99/f75 [0,4194304] 0 2024-03-22T07:09:01.603 INFO:tasks.workunit.client.1.smithi149.stdout:7/999: rmdir d1/d1c/d49/d8e/d100/d1b/d99/de5/d92 39 2024-03-22T07:09:01.605 INFO:tasks.workunit.client.1.smithi149.stdout:5/960: dwrite d7/da/d20/d36/f66 [4194304,4194304] 0 2024-03-22T07:09:01.607 INFO:tasks.workunit.client.1.smithi149.stdout:5/961: rename d7/da/d20/d89/l107 to d7/da/d20/d30/d3a/d3d/d2c/db2/l147 0 2024-03-22T07:09:01.607 INFO:tasks.workunit.client.1.smithi149.stdout:5/962: stat d7/da/d20/d30/d3a/d3d 0 2024-03-22T07:09:01.607 INFO:tasks.workunit.client.1.smithi149.stdout:5/963: mknod d7/da/d20/d30/c148 0 2024-03-22T07:09:01.608 INFO:tasks.workunit.client.1.smithi149.stdout:5/964: mknod d7/da/d20/d36/c149 0 2024-03-22T07:09:01.609 INFO:tasks.workunit.client.1.smithi149.stdout:5/965: link d7/d35/de0/dd7/dea/lf2 d7/l14a 0 2024-03-22T07:09:01.610 INFO:tasks.workunit.client.1.smithi149.stdout:5/966: readlink d7/da/d20/d89/da2/dda/ldf 0 2024-03-22T07:09:01.610 INFO:tasks.workunit.client.1.smithi149.stdout:5/967: write d7/d35/de0/dd7/d7b/fca [4321575,29297] 0 2024-03-22T07:09:01.626 INFO:tasks.workunit.client.1.smithi149.stdout:5/968: dwrite d7/da/d20/d36/fad [0,4194304] 0 2024-03-22T07:09:01.626 INFO:tasks.workunit.client.1.smithi149.stdout:5/969: fdatasync d7/da/d7d/f132 0 2024-03-22T07:09:01.634 INFO:tasks.workunit.client.1.smithi149.stdout:5/970: dread d7/da/d20/d36/f91 [0,4194304] 0 2024-03-22T07:09:01.634 INFO:tasks.workunit.client.1.smithi149.stdout:5/971: chown d7/da/l64 208316 1 2024-03-22T07:09:01.634 INFO:tasks.workunit.client.1.smithi149.stdout:5/972: fdatasync d7/da/d20/d30/d3a/f11b 0 2024-03-22T07:09:01.635 INFO:tasks.workunit.client.1.smithi149.stdout:5/973: truncate d7/da/d20/d36/fba 2695152 0 2024-03-22T07:09:01.636 INFO:tasks.workunit.client.1.smithi149.stdout:5/974: symlink d7/da/d20/d30/d3a/d93/l14b 0 2024-03-22T07:09:01.636 INFO:tasks.workunit.client.1.smithi149.stdout:5/975: stat d7/d35/c3b 0 2024-03-22T07:09:01.637 INFO:tasks.workunit.client.1.smithi149.stdout:5/976: symlink d7/d35/de0/d92/l14c 0 2024-03-22T07:09:01.638 INFO:tasks.workunit.client.1.smithi149.stdout:5/977: getdents d7/d35/de0/dd7/d7b/d81 0 2024-03-22T07:09:01.638 INFO:tasks.workunit.client.1.smithi149.stdout:5/978: chown d7/da/d20/d30/d3a/l130 504379 1 2024-03-22T07:09:01.639 INFO:tasks.workunit.client.1.smithi149.stdout:5/979: symlink d7/da/d20/d30/d3a/d3d/d2c/l14d 0 2024-03-22T07:09:01.642 INFO:tasks.workunit.client.1.smithi149.stdout:5/980: rmdir d7/d35/de0/d92/df4/df5/d10b 39 2024-03-22T07:09:01.656 INFO:tasks.workunit.client.1.smithi149.stdout:5/981: dwrite d7/da/d20/d89/da2/f13e [0,4194304] 0 2024-03-22T07:09:01.663 INFO:tasks.workunit.client.1.smithi149.stdout:5/982: rename d7/da/d20/d36/c149 to d7/da/d20/d30/c14e 0 2024-03-22T07:09:01.672 INFO:tasks.workunit.client.1.smithi149.stdout:5/983: dread d7/da/d20/d36/fad [0,4194304] 0 2024-03-22T07:09:01.673 INFO:tasks.workunit.client.1.smithi149.stdout:3/757: sync 2024-03-22T07:09:01.673 INFO:tasks.workunit.client.1.smithi149.stdout:8/997: sync 2024-03-22T07:09:01.673 INFO:tasks.workunit.client.1.smithi149.stdout:5/984: creat d7/da/d20/d30/d3a/d3d/d2c/d12b/f14f x:0 0 0 2024-03-22T07:09:01.674 INFO:tasks.workunit.client.1.smithi149.stdout:3/758: creat d0/d2/d30/f111 x:0 0 0 2024-03-22T07:09:01.674 INFO:tasks.workunit.client.1.smithi149.stdout:3/759: write d0/d2/d30/d5d/d84/d8e/d9d/fb9 [2631481,23723] 0 2024-03-22T07:09:01.675 INFO:tasks.workunit.client.1.smithi149.stdout:8/998: creat d1/d2/d9/d11/d4d/dd6/d35/d22/d44/d4f/f13d x:0 0 0 2024-03-22T07:09:01.676 INFO:tasks.workunit.client.1.smithi149.stdout:5/985: creat d7/da/d20/d36/f150 x:0 0 0 2024-03-22T07:09:01.676 INFO:tasks.workunit.client.1.smithi149.stdout:3/760: creat d0/d2/dfe/f112 x:0 0 0 2024-03-22T07:09:01.677 INFO:tasks.workunit.client.1.smithi149.stdout:8/999: truncate d1/d2/d9/db2/fc1 654971 0 2024-03-22T07:09:01.680 INFO:tasks.workunit.client.1.smithi149.stdout:3/761: rmdir d0/d2/d5a/daf/dce 39 2024-03-22T07:09:01.681 INFO:tasks.workunit.client.1.smithi149.stdout:3/762: mkdir d0/d2/d30/d5d/d84/d113 0 2024-03-22T07:09:01.681 INFO:tasks.workunit.client.1.smithi149.stdout:3/763: creat d0/d2/d30/d5d/f114 x:0 0 0 2024-03-22T07:09:01.681 INFO:tasks.workunit.client.1.smithi149.stdout:3/764: readlink d0/d4/d8/d24/d40/ld6 0 2024-03-22T07:09:01.700 INFO:tasks.workunit.client.1.smithi149.stdout:5/986: dwrite d7/d35/de0/dd7/f52 [0,4194304] 0 2024-03-22T07:09:01.700 INFO:tasks.workunit.client.1.smithi149.stdout:5/987: chown d7/da/d20/d30 32742272 1 2024-03-22T07:09:01.702 INFO:tasks.workunit.client.1.smithi149.stdout:5/988: rename d7/da/d20/d36/d44/l114 to d7/da/d20/d30/d3a/d3d/d2c/d12e/l151 0 2024-03-22T07:09:01.702 INFO:tasks.workunit.client.1.smithi149.stdout:5/989: creat d7/da/d20/d30/d3a/d3d/d2c/db2/d13a/f152 x:0 0 0 2024-03-22T07:09:01.702 INFO:tasks.workunit.client.1.smithi149.stdout:3/765: dwrite d0/d2/d30/d5d/f65 [0,4194304] 0 2024-03-22T07:09:01.702 INFO:tasks.workunit.client.1.smithi149.stdout:3/766: creat d0/d2/d30/d7e/f115 x:0 0 0 2024-03-22T07:09:01.710 INFO:tasks.workunit.client.1.smithi149.stdout:5/990: rmdir d7/da/d20/d30/d3a/d3d/d2c 39 2024-03-22T07:09:01.711 INFO:tasks.workunit.client.1.smithi149.stdout:5/991: creat d7/da/d20/d89/da2/dda/f153 x:0 0 0 2024-03-22T07:09:01.712 INFO:tasks.workunit.client.1.smithi149.stdout:5/992: unlink d7/ld6 0 2024-03-22T07:09:01.714 INFO:tasks.workunit.client.1.smithi149.stdout:5/993: link d7/da/d20/d30/d3a/d3d/d104/f126 d7/da/d20/d30/d3a/f154 0 2024-03-22T07:09:01.714 INFO:tasks.workunit.client.1.smithi149.stdout:5/994: creat d7/da/d20/d30/d3a/d3d/d104/f155 x:0 0 0 2024-03-22T07:09:01.714 INFO:tasks.workunit.client.1.smithi149.stdout:5/995: dread - d7/da/d20/d30/d3a/d3d/d104/f140 zero size 2024-03-22T07:09:01.714 INFO:tasks.workunit.client.1.smithi149.stdout:5/996: chown d7/da/d20/d30/d3a/c80 1171001 1 2024-03-22T07:09:01.715 INFO:tasks.workunit.client.1.smithi149.stdout:5/997: unlink d7/da/d20/d30/d3a/l130 0 2024-03-22T07:09:01.715 INFO:tasks.workunit.client.1.smithi149.stdout:5/998: fsync d7/da/d20/d30/d3a/f101 0 2024-03-22T07:09:01.716 INFO:tasks.workunit.client.1.smithi149.stdout:5/999: symlink d7/da/d20/d30/d3a/d3d/db0/l156 0 2024-03-22T07:09:01.725 INFO:tasks.workunit.client.1.smithi149.stdout:3/767: dwrite d0/d4/d8/d24/d39/f89 [0,4194304] 0 2024-03-22T07:09:01.725 INFO:tasks.workunit.client.1.smithi149.stdout:3/768: write d0/d4/d8/d24/db6/dcd/fe2 [5404639,127140] 0 2024-03-22T07:09:01.726 INFO:tasks.workunit.client.1.smithi149.stdout:3/769: symlink d0/d2/d30/d5d/d84/d8f/l116 0 2024-03-22T07:09:01.729 INFO:tasks.workunit.client.1.smithi149.stdout:3/770: rmdir d0/d2/d42/d66/d82 0 2024-03-22T07:09:01.729 INFO:tasks.workunit.client.1.smithi149.stdout:3/771: creat d0/d2/d30/d5d/f117 x:0 0 0 2024-03-22T07:09:01.729 INFO:tasks.workunit.client.1.smithi149.stdout:3/772: chown d0/d2/d5a/l97 95461 1 2024-03-22T07:09:01.729 INFO:tasks.workunit.client.1.smithi149.stdout:3/773: write d0/d4/d8/d24/db6/f106 [63327,7952] 0 2024-03-22T07:09:01.729 INFO:tasks.workunit.client.1.smithi149.stdout:3/774: truncate d0/d2/d42/f7c 2758658 0 2024-03-22T07:09:01.730 INFO:tasks.workunit.client.1.smithi149.stdout:3/775: mkdir d0/d2/d30/dd1/d118 0 2024-03-22T07:09:01.731 INFO:tasks.workunit.client.1.smithi149.stdout:3/776: mkdir d0/d2/dfe/d119 0 2024-03-22T07:09:01.732 INFO:tasks.workunit.client.1.smithi149.stdout:3/777: dread d0/d2/d5a/daf/ff4 [4194304,4194304] 0 2024-03-22T07:09:01.733 INFO:tasks.workunit.client.1.smithi149.stdout:3/778: unlink d0/d2/d5a/fbf 0 2024-03-22T07:09:01.735 INFO:tasks.workunit.client.1.smithi149.stdout:3/779: mkdir d0/d2/d30/dd1/d118/d11a 0 2024-03-22T07:09:01.737 INFO:tasks.workunit.client.1.smithi149.stdout:3/780: creat d0/d4/d8/d24/d40/d54/da0/f11b x:0 0 0 2024-03-22T07:09:01.737 INFO:tasks.workunit.client.1.smithi149.stdout:3/781: creat d0/d2/d30/d5d/d84/f11c x:0 0 0 2024-03-22T07:09:01.737 INFO:tasks.workunit.client.1.smithi149.stdout:3/782: chown d0/d2/d30/d7e/ce7 2 1 2024-03-22T07:09:01.919 INFO:tasks.workunit.client.1.smithi149.stdout:3/783: sync 2024-03-22T07:09:01.919 INFO:tasks.workunit.client.1.smithi149.stdout:3/784: chown d0/d2/d42/d66/ca4 130162797 1 2024-03-22T07:09:01.938 INFO:tasks.workunit.client.1.smithi149.stdout:3/785: dwrite d0/f110 [0,4194304] 0 2024-03-22T07:09:01.938 INFO:tasks.workunit.client.1.smithi149.stdout:3/786: dread - d0/d4/d8/d24/d40/fa5 zero size 2024-03-22T07:09:01.944 INFO:tasks.workunit.client.1.smithi149.stdout:3/787: dread d0/d2/f4d [0,4194304] 0 2024-03-22T07:09:01.945 INFO:tasks.workunit.client.1.smithi149.stdout:3/788: write d0/d4/d6c/ff5 [811918,87667] 0 2024-03-22T07:09:01.963 INFO:tasks.workunit.client.1.smithi149.stdout:3/789: dwrite d0/d4/d8/d24/d40/d54/f6f [0,4194304] 0 2024-03-22T07:09:01.964 INFO:tasks.workunit.client.1.smithi149.stdout:3/790: mknod d0/d4/d8/d24/db6/dcd/d103/c11d 0 2024-03-22T07:09:01.964 INFO:tasks.workunit.client.1.smithi149.stdout:3/791: dread - d0/d2/d5a/daf/dce/fea zero size 2024-03-22T07:09:01.978 INFO:tasks.workunit.client.1.smithi149.stdout:3/792: dwrite d0/d2/d30/dd1/fbb [0,4194304] 0 2024-03-22T07:09:01.980 INFO:tasks.workunit.client.1.smithi149.stdout:3/793: truncate d0/d2/d30/d5d/d84/dc3/f3d 3214485 0 2024-03-22T07:09:01.982 INFO:tasks.workunit.client.1.smithi149.stdout:3/794: write d0/d2/d30/d5d/d84/d8e/d9d/fb9 [340112,7022] 0 2024-03-22T07:09:01.982 INFO:tasks.workunit.client.1.smithi149.stdout:3/795: chown d0/d4/d6c/cb4 2002 1 2024-03-22T07:09:01.982 INFO:tasks.workunit.client.1.smithi149.stdout:3/796: write d0/d4/d8/d24/d40/d54/d85/dbc/fcf [94700,73659] 0 2024-03-22T07:09:01.991 INFO:tasks.workunit.client.1.smithi149.stdout:3/797: dread d0/d2/d30/d5d/d84/d8f/fc5 [0,4194304] 0 2024-03-22T07:09:01.993 INFO:tasks.workunit.client.1.smithi149.stdout:3/798: rename d0/d4/d8/d24/d40/d54/d85/f91 to d0/d2/d30/d5d/d84/d113/f11e 0 2024-03-22T07:09:01.993 INFO:tasks.workunit.client.1.smithi149.stdout:3/799: chown d0/d4/d8/d24/d40/d54/d85/dbc/f75 114225 1 2024-03-22T07:09:01.993 INFO:tasks.workunit.client.1.smithi149.stdout:3/800: chown d0/d2/d30/d7e/ce7 4452 1 2024-03-22T07:09:01.994 INFO:tasks.workunit.client.1.smithi149.stdout:3/801: mknod d0/d4/d8/d24/d40/d62/d108/c11f 0 2024-03-22T07:09:01.995 INFO:tasks.workunit.client.1.smithi149.stdout:3/802: mkdir d0/d2/d30/dee/d120 0 2024-03-22T07:09:01.995 INFO:tasks.workunit.client.1.smithi149.stdout:3/803: mkdir d0/d2/d30/dd1/dc9/d121 0 2024-03-22T07:09:01.997 INFO:tasks.workunit.client.1.smithi149.stdout:3/804: rename d0/d2/d30/d5d/d84/d8f/ld8 to d0/d2/d5a/l122 0 2024-03-22T07:09:02.014 INFO:tasks.workunit.client.1.smithi149.stdout:3/805: dwrite d0/d2/d30/f73 [0,4194304] 0 2024-03-22T07:09:02.015 INFO:tasks.workunit.client.1.smithi149.stdout:3/806: getdents d0/d4 0 2024-03-22T07:09:02.016 INFO:tasks.workunit.client.1.smithi149.stdout:3/807: mknod d0/d2/d5a/daf/dce/c123 0 2024-03-22T07:09:02.016 INFO:tasks.workunit.client.1.smithi149.stdout:3/808: fdatasync d0/f72 0 2024-03-22T07:09:02.017 INFO:tasks.workunit.client.1.smithi149.stdout:3/809: creat d0/d2/d30/d7e/db8/f124 x:0 0 0 2024-03-22T07:09:02.018 INFO:tasks.workunit.client.1.smithi149.stdout:3/810: rename d0/d2/f4d to d0/d4/d6c/f125 0 2024-03-22T07:09:02.019 INFO:tasks.workunit.client.1.smithi149.stdout:3/811: symlink d0/d2/d30/dd1/d118/l126 0 2024-03-22T07:09:02.020 INFO:tasks.workunit.client.1.smithi149.stdout:3/812: getdents d0/d4/d8/d24/d40/d62 0 2024-03-22T07:09:02.020 INFO:tasks.workunit.client.1.smithi149.stdout:3/813: dread - d0/d2/d30/d5d/d84/f11c zero size 2024-03-22T07:09:02.020 INFO:tasks.workunit.client.1.smithi149.stdout:3/814: write d0/d4/d8/d24/f4f [5217317,99980] 0 2024-03-22T07:09:02.022 INFO:tasks.workunit.client.1.smithi149.stdout:3/815: symlink d0/d2/d5a/daf/dce/l127 0 2024-03-22T07:09:02.025 INFO:tasks.workunit.client.1.smithi149.stdout:3/816: truncate d0/d2/d5a/fde 2803142 0 2024-03-22T07:09:02.025 INFO:tasks.workunit.client.1.smithi149.stdout:3/817: read - d0/d2/d5a/daf/fe6 zero size 2024-03-22T07:09:02.034 INFO:tasks.workunit.client.1.smithi149.stdout:3/818: write d0/d2/d30/d5d/d84/d8f/fc5 [3703093,78120] 0 2024-03-22T07:09:02.035 INFO:tasks.workunit.client.1.smithi149.stdout:3/819: mknod d0/d2/d42/c128 0 2024-03-22T07:09:02.035 INFO:tasks.workunit.client.1.smithi149.stdout:3/820: truncate d0/d2/d30/d5d/d84/d8f/feb 74647 0 2024-03-22T07:09:02.047 INFO:tasks.workunit.client.1.smithi149.stdout:3/821: dread d0/d2/d30/d5d/d84/d8f/fc5 [0,4194304] 0 2024-03-22T07:09:02.048 INFO:tasks.workunit.client.1.smithi149.stdout:3/822: mkdir d0/d2/d30/d5d/d84/dc3/d129 0 2024-03-22T07:09:02.049 INFO:tasks.workunit.client.1.smithi149.stdout:3/823: truncate d0/d4/d8/d24/f6e 1671914 0 2024-03-22T07:09:02.049 INFO:tasks.workunit.client.1.smithi149.stdout:3/824: read d0/d2/d30/d5d/d84/d8f/feb [14397,121407] 0 2024-03-22T07:09:02.050 INFO:tasks.workunit.client.1.smithi149.stdout:3/825: mkdir d0/d2/d30/d12a 0 2024-03-22T07:09:02.058 INFO:tasks.workunit.client.1.smithi149.stdout:3/826: dread d0/f72 [0,4194304] 0 2024-03-22T07:09:02.059 INFO:tasks.workunit.client.1.smithi149.stdout:3/827: rmdir d0/d2/d30/dd1/dc9 39 2024-03-22T07:09:02.059 INFO:tasks.workunit.client.1.smithi149.stdout:3/828: rmdir d0/d2/d5a/d93 39 2024-03-22T07:09:02.060 INFO:tasks.workunit.client.1.smithi149.stdout:3/829: creat d0/d4/d8/d24/db6/f12b x:0 0 0 2024-03-22T07:09:02.060 INFO:tasks.workunit.client.1.smithi149.stdout:3/830: stat d0/d4/d8/d24/d40/d54/da0 0 2024-03-22T07:09:02.061 INFO:tasks.workunit.client.1.smithi149.stdout:3/831: creat d0/d2/d30/dd1/f12c x:0 0 0 2024-03-22T07:09:02.070 INFO:tasks.workunit.client.1.smithi149.stdout:3/832: dread d0/d4/f2f [0,4194304] 0 2024-03-22T07:09:02.071 INFO:tasks.workunit.client.1.smithi149.stdout:3/833: creat d0/d2/d30/dd1/d118/d11a/f12d x:0 0 0 2024-03-22T07:09:02.089 INFO:tasks.workunit.client.1.smithi149.stdout:3/834: dwrite d0/d2/d30/f70 [0,4194304] 0 2024-03-22T07:09:02.172 INFO:tasks.workunit.client.1.smithi149.stdout:3/835: sync 2024-03-22T07:09:02.173 INFO:tasks.workunit.client.1.smithi149.stdout:3/836: creat d0/d2/d30/d5d/d84/dc3/d129/f12e x:0 0 0 2024-03-22T07:09:02.173 INFO:tasks.workunit.client.1.smithi149.stdout:3/837: dread - d0/d2/d30/d5d/d84/f11c zero size 2024-03-22T07:09:02.188 INFO:tasks.workunit.client.1.smithi149.stdout:3/838: dwrite d0/d2/d30/dd1/f12c [0,4194304] 0 2024-03-22T07:09:02.189 INFO:tasks.workunit.client.1.smithi149.stdout:3/839: creat d0/d2/d30/d5d/d84/d113/f12f x:0 0 0 2024-03-22T07:09:02.190 INFO:tasks.workunit.client.1.smithi149.stdout:3/840: getdents d0/d4/d8/d24/d40/d54/d85/dbc 0 2024-03-22T07:09:02.191 INFO:tasks.workunit.client.1.smithi149.stdout:3/841: mkdir d0/d2/d30/dd1/d130 0 2024-03-22T07:09:02.192 INFO:tasks.workunit.client.1.smithi149.stdout:3/842: write d0/d4/d8/d24/d40/fa5 [908754,8028] 0 2024-03-22T07:09:02.205 INFO:tasks.workunit.client.1.smithi149.stdout:3/843: dwrite d0/d2/d5a/fde [0,4194304] 0 2024-03-22T07:09:02.206 INFO:tasks.workunit.client.1.smithi149.stdout:3/844: creat d0/d2/d5a/d93/f131 x:0 0 0 2024-03-22T07:09:02.320 INFO:tasks.workunit.client.1.smithi149.stdout:3/845: sync 2024-03-22T07:09:02.321 INFO:tasks.workunit.client.1.smithi149.stdout:3/846: unlink d0/d2/d30/d7e/lb2 0 2024-03-22T07:09:02.323 INFO:tasks.workunit.client.1.smithi149.stdout:3/847: rename d0/d2/d30/d5d/d84/d113/f12f to d0/d2/d30/dd1/d130/f132 0 2024-03-22T07:09:02.323 INFO:tasks.workunit.client.1.smithi149.stdout:3/848: dread - d0/d2/d30/f9f zero size 2024-03-22T07:09:02.324 INFO:tasks.workunit.client.1.smithi149.stdout:3/849: creat d0/d2/d30/dee/d120/f133 x:0 0 0 2024-03-22T07:09:02.338 INFO:tasks.workunit.client.1.smithi149.stdout:3/850: dwrite d0/fbe [0,4194304] 0 2024-03-22T07:09:02.339 INFO:tasks.workunit.client.1.smithi149.stdout:3/851: mkdir d0/d4/d134 0 2024-03-22T07:09:02.444 INFO:tasks.workunit.client.1.smithi149.stdout:3/852: sync 2024-03-22T07:09:02.445 INFO:tasks.workunit.client.1.smithi149.stdout:3/853: creat d0/d2/dfe/d10c/f135 x:0 0 0 2024-03-22T07:09:02.445 INFO:tasks.workunit.client.1.smithi149.stdout:3/854: fsync d0/d2/d30/d5d/d84/dc3/d129/f12e 0 2024-03-22T07:09:02.544 INFO:tasks.workunit.client.1.smithi149.stdout:3/855: sync 2024-03-22T07:09:02.544 INFO:tasks.workunit.client.1.smithi149.stdout:3/856: dread - d0/d2/d30/d5d/d84/d8f/ff3 zero size 2024-03-22T07:09:02.544 INFO:tasks.workunit.client.1.smithi149.stdout:3/857: write d0/d4/d8/fd2 [2533332,43242] 0 2024-03-22T07:09:02.544 INFO:tasks.workunit.client.1.smithi149.stdout:3/858: creat d0/d2/d30/d5d/d84/d8e/d9d/f136 x:0 0 0 2024-03-22T07:09:02.562 INFO:tasks.workunit.client.1.smithi149.stdout:3/859: dwrite d0/d2/f76 [4194304,4194304] 0 2024-03-22T07:09:02.562 INFO:tasks.workunit.client.1.smithi149.stdout:3/860: write d0/d2/d30/dd1/dfa/f107 [295710,22761] 0 2024-03-22T07:09:02.578 INFO:tasks.workunit.client.1.smithi149.stdout:3/861: dwrite d0/d2/d30/dee/ffb [4194304,4194304] 0 2024-03-22T07:09:02.647 INFO:tasks.workunit.client.1.smithi149.stdout:3/862: symlink d0/d2/d30/d5d/d84/l137 0 2024-03-22T07:09:02.648 INFO:tasks.workunit.client.1.smithi149.stdout:3/863: creat d0/d4/d8/d24/db6/f138 x:0 0 0 2024-03-22T07:09:02.651 INFO:tasks.workunit.client.1.smithi149.stdout:3/864: link d0/d4/d8/d24/d40/d54/c6d d0/d4/d8/d24/d40/d54/d9e/c139 0 2024-03-22T07:09:02.651 INFO:tasks.workunit.client.1.smithi149.stdout:3/865: fdatasync d0/d4/d8/d24/db6/dcd/fe2 0 2024-03-22T07:09:02.651 INFO:tasks.workunit.client.1.smithi149.stdout:3/866: stat d0/d2/dfe/d10c 0 2024-03-22T07:09:02.651 INFO:tasks.workunit.client.1.smithi149.stdout:3/867: readlink d0/d2/lf0 0 2024-03-22T07:09:02.651 INFO:tasks.workunit.client.1.smithi149.stdout:3/868: mknod d0/d4/d8/d24/d40/d54/d85/dbc/df2/c13a 0 2024-03-22T07:09:02.677 INFO:tasks.workunit.client.1.smithi149.stdout:3/869: dwrite d0/d2/d30/d5d/d84/dc3/f3d [0,4194304] 0 2024-03-22T07:09:02.678 INFO:tasks.workunit.client.1.smithi149.stdout:3/870: creat d0/d4/d8/d24/f13b x:0 0 0 2024-03-22T07:09:02.679 INFO:tasks.workunit.client.1.smithi149.stdout:3/871: rmdir d0/d2/d30/dee 39 2024-03-22T07:09:02.683 INFO:tasks.workunit.client.1.smithi149.stdout:3/872: truncate d0/d4/d8/d24/d39/f89 648542 0 2024-03-22T07:09:02.683 INFO:tasks.workunit.client.1.smithi149.stdout:3/873: creat d0/d2/d30/dd1/d118/d11a/f13c x:0 0 0 2024-03-22T07:09:02.683 INFO:tasks.workunit.client.1.smithi149.stdout:3/874: dread - d0/d2/d30/d5d/f117 zero size 2024-03-22T07:09:02.684 INFO:tasks.workunit.client.1.smithi149.stdout:3/875: creat d0/d2/f13d x:0 0 0 2024-03-22T07:09:02.685 INFO:tasks.workunit.client.1.smithi149.stdout:3/876: creat d0/f13e x:0 0 0 2024-03-22T07:09:02.702 INFO:tasks.workunit.client.1.smithi149.stdout:3/877: dwrite d0/d4/d8/d24/d40/d54/d85/fd5 [0,4194304] 0 2024-03-22T07:09:02.703 INFO:tasks.workunit.client.1.smithi149.stdout:3/878: creat d0/d2/d30/dd1/f13f x:0 0 0 2024-03-22T07:09:02.712 INFO:tasks.workunit.client.1.smithi149.stdout:3/879: dread d0/d4/d8/d24/d40/d54/f6f [0,4194304] 0 2024-03-22T07:09:02.712 INFO:tasks.workunit.client.1.smithi149.stdout:3/880: fdatasync d0/d2/d30/d5d/f114 0 2024-03-22T07:09:02.713 INFO:tasks.workunit.client.1.smithi149.stdout:3/881: symlink d0/d4/d8/d24/d40/l140 0 2024-03-22T07:09:02.715 INFO:tasks.workunit.client.1.smithi149.stdout:3/882: link d0/d2/d30/dd1/dc9/f101 d0/d2/d30/dee/d120/f141 0 2024-03-22T07:09:02.715 INFO:tasks.workunit.client.1.smithi149.stdout:3/883: write d0/d2/d30/d5d/d84/d8f/fc5 [325218,59449] 0 2024-03-22T07:09:02.722 INFO:tasks.workunit.client.1.smithi149.stdout:3/884: dread d0/d2/d5a/daf/ff4 [0,4194304] 0 2024-03-22T07:09:02.722 INFO:tasks.workunit.client.1.smithi149.stdout:3/885: dread - d0/f13e zero size 2024-03-22T07:09:02.722 INFO:tasks.workunit.client.1.smithi149.stdout:3/886: write d0/d2/f13d [1017893,55618] 0 2024-03-22T07:09:02.722 INFO:tasks.workunit.client.1.smithi149.stdout:3/887: write d0/d4/d8/f8d [2466526,84454] 0 2024-03-22T07:09:02.723 INFO:tasks.workunit.client.1.smithi149.stdout:3/888: symlink d0/d2/d30/d5d/d84/d8f/l142 0 2024-03-22T07:09:02.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/889: mkdir d0/d2/d30/d143 0 2024-03-22T07:09:02.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/890: readlink d0/d4/d8/d24/d40/l78 0 2024-03-22T07:09:02.724 INFO:tasks.workunit.client.1.smithi149.stdout:3/891: fsync d0/d4/d8/d24/d40/fa5 0 2024-03-22T07:09:02.733 INFO:tasks.workunit.client.1.smithi149.stdout:3/892: read d0/d4/d8/d24/d40/d54/d9e/fab [2550883,22920] 0 2024-03-22T07:09:02.734 INFO:tasks.workunit.client.1.smithi149.stdout:3/893: link d0/d4/d6c/lc6 d0/d2/d5a/d93/l144 0 2024-03-22T07:09:02.734 INFO:tasks.workunit.client.1.smithi149.stdout:3/894: mkdir d0/d4/d8/d24/d145 0 2024-03-22T07:09:02.827 INFO:tasks.workunit.client.1.smithi149.stdout:3/895: sync 2024-03-22T07:09:02.828 INFO:tasks.workunit.client.1.smithi149.stdout:3/896: symlink d0/d2/d30/dd1/d118/d11a/l146 0 2024-03-22T07:09:02.829 INFO:tasks.workunit.client.1.smithi149.stdout:3/897: mknod d0/d2/d30/dee/c147 0 2024-03-22T07:09:02.830 INFO:tasks.workunit.client.1.smithi149.stdout:3/898: getdents d0 0 2024-03-22T07:09:02.840 INFO:tasks.workunit.client.1.smithi149.stdout:3/899: dread d0/d2/d30/d5d/d84/d8f/f2e [0,4194304] 0 2024-03-22T07:09:02.841 INFO:tasks.workunit.client.1.smithi149.stdout:3/900: getdents d0/d4/d8/d24/db6 0 2024-03-22T07:09:02.841 INFO:tasks.workunit.client.1.smithi149.stdout:3/901: creat d0/d2/dfe/f148 x:0 0 0 2024-03-22T07:09:02.843 INFO:tasks.workunit.client.1.smithi149.stdout:3/902: rename d0/d2/d30/dd1/d118/l126 to d0/d2/d5a/d93/l149 0 2024-03-22T07:09:02.845 INFO:tasks.workunit.client.1.smithi149.stdout:3/903: write d0/d4/d8/d24/d40/d54/d85/dbc/f87 [261556,55454] 0 2024-03-22T07:09:02.845 INFO:tasks.workunit.client.1.smithi149.stdout:3/904: fdatasync d0/d4/d8/d24/d40/d54/d85/dbc/f75 0 2024-03-22T07:09:02.845 INFO:tasks.workunit.client.1.smithi149.stdout:3/905: chown d0/d2/d30 207041 1 2024-03-22T07:09:02.937 INFO:tasks.workunit.client.1.smithi149.stdout:3/906: sync 2024-03-22T07:09:02.938 INFO:tasks.workunit.client.1.smithi149.stdout:3/907: symlink d0/d4/d8/d24/l14a 0 2024-03-22T07:09:02.938 INFO:tasks.workunit.client.1.smithi149.stdout:3/908: symlink d0/d2/d30/d5d/d84/dc3/l14b 0 2024-03-22T07:09:02.939 INFO:tasks.workunit.client.1.smithi149.stdout:3/909: chown d0/d2/f76 13896 1 2024-03-22T07:09:02.940 INFO:tasks.workunit.client.1.smithi149.stdout:3/910: dread d0/d4/d8/d24/db6/f106 [0,4194304] 0 2024-03-22T07:09:02.940 INFO:tasks.workunit.client.1.smithi149.stdout:3/911: fdatasync d0/d2/d5a/fde 0 2024-03-22T07:09:02.958 INFO:tasks.workunit.client.1.smithi149.stdout:3/912: dwrite d0/d4/d8/d24/d40/d54/d85/dbc/f87 [0,4194304] 0 2024-03-22T07:09:02.958 INFO:tasks.workunit.client.1.smithi149.stdout:3/913: write d0/d4/d8/d24/db6/dcd/fe0 [1122746,80812] 0 2024-03-22T07:09:02.972 INFO:tasks.workunit.client.1.smithi149.stdout:3/914: dwrite d0/d2/d30/d5d/d84/dc3/d129/f12e [0,4194304] 0 2024-03-22T07:09:02.974 INFO:tasks.workunit.client.1.smithi149.stdout:3/915: getdents d0/d2/d5a/d93 0 2024-03-22T07:09:02.975 INFO:tasks.workunit.client.1.smithi149.stdout:3/916: mkdir d0/d2/d30/dd1/d118/d11a/d14c 0 2024-03-22T07:09:03.002 INFO:tasks.workunit.client.1.smithi149.stdout:3/917: dread d0/d4/f58 [0,4194304] 0 2024-03-22T07:09:03.103 INFO:tasks.workunit.client.1.smithi149.stdout:3/918: sync 2024-03-22T07:09:03.103 INFO:tasks.workunit.client.1.smithi149.stdout:3/919: stat d0/d2/d5a/l97 0 2024-03-22T07:09:03.104 INFO:tasks.workunit.client.1.smithi149.stdout:3/920: rename d0/d2/d30/dee/c100 to d0/d4/d8/d24/d40/d54/d85/dbc/df2/c14d 0 2024-03-22T07:09:03.104 INFO:tasks.workunit.client.1.smithi149.stdout:3/921: stat d0/d2/d5a/d93/l10b 0 2024-03-22T07:09:03.105 INFO:tasks.workunit.client.1.smithi149.stdout:3/922: mknod d0/d2/d30/dd1/dc9/d121/c14e 0 2024-03-22T07:09:03.109 INFO:tasks.workunit.client.1.smithi149.stdout:3/923: read d0/fbe [1159487,77215] 0 2024-03-22T07:09:03.109 INFO:tasks.workunit.client.1.smithi149.stdout:3/924: chown d0/d2/d30/d5d/d84/d8f 137526 1 2024-03-22T07:09:03.110 INFO:tasks.workunit.client.1.smithi149.stdout:3/925: mkdir d0/d4/d8/d24/d40/d54/d85/dbc/d14f 0 2024-03-22T07:09:03.127 INFO:tasks.workunit.client.1.smithi149.stdout:3/926: dwrite d0/d4/d6c/f125 [0,4194304] 0 2024-03-22T07:09:03.145 INFO:tasks.workunit.client.1.smithi149.stdout:3/927: dwrite d0/d4/d8/d24/d40/d54/d85/dbc/df2/ff9 [0,4194304] 0 2024-03-22T07:09:03.152 INFO:tasks.workunit.client.1.smithi149.stdout:3/928: dread d0/d2/d30/d5d/d84/dc3/d129/f12e [0,4194304] 0 2024-03-22T07:09:03.153 INFO:tasks.workunit.client.1.smithi149.stdout:3/929: symlink d0/d4/d8/d24/db6/l150 0 2024-03-22T07:09:03.153 INFO:tasks.workunit.client.1.smithi149.stdout:3/930: chown d0/d2/d30/d5d/d84/d8e/d9d/fb9 502 1 2024-03-22T07:09:03.153 INFO:tasks.workunit.client.1.smithi149.stdout:3/931: stat d0/d4/d8/f8d 0 2024-03-22T07:09:03.166 INFO:tasks.workunit.client.1.smithi149.stdout:3/932: dwrite d0/d2/d30/d5d/d84/d8f/feb [0,4194304] 0 2024-03-22T07:09:03.173 INFO:tasks.workunit.client.1.smithi149.stdout:3/933: write d0/d2/d30/dee/ffb [7858400,110680] 0 2024-03-22T07:09:03.174 INFO:tasks.workunit.client.1.smithi149.stdout:3/934: mkdir d0/d4/d8/d24/d40/d54/d85/d151 0 2024-03-22T07:09:03.181 INFO:tasks.workunit.client.1.smithi149.stdout:3/935: dread d0/d2/d30/d7e/f69 [0,4194304] 0 2024-03-22T07:09:03.182 INFO:tasks.workunit.client.1.smithi149.stdout:3/936: mkdir d0/d2/d30/dee/d152 0 2024-03-22T07:09:03.198 INFO:tasks.workunit.client.1.smithi149.stdout:3/937: dwrite d0/d4/d8/d24/d40/d54/d85/fd5 [0,4194304] 0 2024-03-22T07:09:03.199 INFO:tasks.workunit.client.1.smithi149.stdout:3/938: symlink d0/d4/d8/d24/l153 0 2024-03-22T07:09:03.200 INFO:tasks.workunit.client.1.smithi149.stdout:3/939: mknod d0/d4/d8/d24/d40/d54/d85/dbc/c154 0 2024-03-22T07:09:03.206 INFO:tasks.workunit.client.1.smithi149.stdout:3/940: read d0/d4/d6c/f125 [4138108,98887] 0 2024-03-22T07:09:03.208 INFO:tasks.workunit.client.1.smithi149.stdout:3/941: mkdir d0/d2/dfe/d155 0 2024-03-22T07:09:03.208 INFO:tasks.workunit.client.1.smithi149.stdout:3/942: chown d0/d4/d8/d24/d40/d54/d9e/fab 1637318 1 2024-03-22T07:09:03.208 INFO:tasks.workunit.client.1.smithi149.stdout:3/943: chown d0/d2/d30/fcb 10 1 2024-03-22T07:09:03.208 INFO:tasks.workunit.client.1.smithi149.stdout:3/944: readlink d0/d4/d8/d24/la8 0 2024-03-22T07:09:03.208 INFO:tasks.workunit.client.1.smithi149.stdout:3/945: creat d0/d2/d42/f156 x:0 0 0 2024-03-22T07:09:03.209 INFO:tasks.workunit.client.1.smithi149.stdout:3/946: unlink d0/d4/d8/l44 0 2024-03-22T07:09:03.214 INFO:tasks.workunit.client.1.smithi149.stdout:3/947: dread d0/d2/d30/d5d/d84/dc3/d129/f12e [0,4194304] 0 2024-03-22T07:09:03.215 INFO:tasks.workunit.client.1.smithi149.stdout:3/948: mkdir d0/d4/d6c/d157 0 2024-03-22T07:09:03.215 INFO:tasks.workunit.client.1.smithi149.stdout:3/949: readlink d0/d2/d30/d5d/d84/d8e/lbd 0 2024-03-22T07:09:03.216 INFO:tasks.workunit.client.1.smithi149.stdout:3/950: creat d0/d4/d8/d24/d40/d62/f158 x:0 0 0 2024-03-22T07:09:03.217 INFO:tasks.workunit.client.1.smithi149.stdout:3/951: symlink d0/d2/d30/dee/l159 0 2024-03-22T07:09:03.217 INFO:tasks.workunit.client.1.smithi149.stdout:3/952: creat d0/d4/d8/d24/d39/f15a x:0 0 0 2024-03-22T07:09:03.218 INFO:tasks.workunit.client.1.smithi149.stdout:3/953: chown d0/d2/d30/d7e 5101 1 2024-03-22T07:09:03.218 INFO:tasks.workunit.client.1.smithi149.stdout:3/954: mkdir d0/d2/dfe/d10c/d15b 0 2024-03-22T07:09:03.218 INFO:tasks.workunit.client.1.smithi149.stdout:3/955: stat d0/d4/d8/d24/d40/d54/l109 0 2024-03-22T07:09:03.219 INFO:tasks.workunit.client.1.smithi149.stdout:3/956: symlink d0/d4/d8/d24/db6/dcd/l15c 0 2024-03-22T07:09:03.219 INFO:tasks.workunit.client.1.smithi149.stdout:3/957: write d0/d4/d6c/f9b [109183,94212] 0 2024-03-22T07:09:03.219 INFO:tasks.workunit.client.1.smithi149.stdout:3/958: chown d0/d4/d8/d24/d40/d62/f64 437 1 2024-03-22T07:09:03.221 INFO:tasks.workunit.client.1.smithi149.stdout:3/959: rename d0/d2/d30/c60 to d0/d4/d8/d24/d40/d54/da0/c15d 0 2024-03-22T07:09:03.224 INFO:tasks.workunit.client.1.smithi149.stdout:3/960: getdents d0/d4/d8/d24/d40/d54/d85 0 2024-03-22T07:09:03.322 INFO:tasks.workunit.client.1.smithi149.stdout:3/961: sync 2024-03-22T07:09:03.323 INFO:tasks.workunit.client.1.smithi149.stdout:3/962: creat d0/d2/dfe/d119/f15e x:0 0 0 2024-03-22T07:09:03.324 INFO:tasks.workunit.client.1.smithi149.stdout:3/963: creat d0/d4/d8/f15f x:0 0 0 2024-03-22T07:09:03.325 INFO:tasks.workunit.client.1.smithi149.stdout:3/964: mkdir d0/d4/d8/d24/d40/d62/d160 0 2024-03-22T07:09:03.327 INFO:tasks.workunit.client.1.smithi149.stdout:3/965: rename d0/d2/d30/dd1/d118/d11a/f12d to d0/d4/d8/f161 0 2024-03-22T07:09:03.328 INFO:tasks.workunit.client.1.smithi149.stdout:3/966: mkdir d0/d2/d30/dee/d152/d162 0 2024-03-22T07:09:03.350 INFO:tasks.workunit.client.1.smithi149.stdout:3/967: dwrite d0/d2/f13d [0,4194304] 0 2024-03-22T07:09:03.352 INFO:tasks.workunit.client.1.smithi149.stdout:3/968: rename d0/d2/d5a/daf/c105 to d0/d2/d30/d5d/c163 0 2024-03-22T07:09:03.354 INFO:tasks.workunit.client.1.smithi149.stdout:3/969: rename d0/d2/d30/dee/d120/f133 to d0/d4/d8/d24/d40/d54/da0/f164 0 2024-03-22T07:09:03.360 INFO:tasks.workunit.client.1.smithi149.stdout:3/970: read d0/d2/d30/d5d/d84/d8f/feb [776255,110204] 0 2024-03-22T07:09:03.362 INFO:tasks.workunit.client.1.smithi149.stdout:3/971: rename d0/d2/d30/dd1/ce4 to d0/d2/d30/d7e/db8/c165 0 2024-03-22T07:09:03.362 INFO:tasks.workunit.client.1.smithi149.stdout:3/972: truncate d0/d2/d5a/daf/dce/fda 5423 0 2024-03-22T07:09:03.362 INFO:tasks.workunit.client.1.smithi149.stdout:3/973: chown d0/d2/d30/dd1/dc9 3141337 1 2024-03-22T07:09:03.363 INFO:tasks.workunit.client.1.smithi149.stdout:3/974: link d0/d2/dfe/d10c/f135 d0/d2/d30/d5d/d84/d8f/f166 0 2024-03-22T07:09:03.364 INFO:tasks.workunit.client.1.smithi149.stdout:3/975: symlink d0/d2/d30/d143/l167 0 2024-03-22T07:09:03.364 INFO:tasks.workunit.client.1.smithi149.stdout:3/976: fsync d0/d2/d30/dd1/f12c 0 2024-03-22T07:09:03.364 INFO:tasks.workunit.client.1.smithi149.stdout:3/977: dread - d0/d2/d5a/d93/f131 zero size 2024-03-22T07:09:03.377 INFO:tasks.workunit.client.1.smithi149.stdout:3/978: dwrite d0/d2/d30/dd1/dfa/f107 [0,4194304] 0 2024-03-22T07:09:03.471 INFO:tasks.workunit.client.1.smithi149.stdout:3/979: sync 2024-03-22T07:09:03.471 INFO:tasks.workunit.client.1.smithi149.stdout:3/980: readlink d0/d4/d8/d24/d40/d54/l109 0 2024-03-22T07:09:03.472 INFO:tasks.workunit.client.1.smithi149.stdout:3/981: symlink d0/d4/d8/d24/db6/dcd/d103/l168 0 2024-03-22T07:09:03.486 INFO:tasks.workunit.client.1.smithi149.stdout:3/982: dwrite d0/d4/d8/d24/d40/d54/d85/dbc/fcf [0,4194304] 0 2024-03-22T07:09:03.494 INFO:tasks.workunit.client.1.smithi149.stdout:3/983: rmdir d0/d2 39 2024-03-22T07:09:03.494 INFO:tasks.workunit.client.1.smithi149.stdout:3/984: dread - d0/d2/d30/dd1/d130/f132 zero size 2024-03-22T07:09:03.507 INFO:tasks.workunit.client.1.smithi149.stdout:3/985: dwrite d0/d2/d30/d5d/d84/dc3/fba [0,4194304] 0 2024-03-22T07:09:03.509 INFO:tasks.workunit.client.1.smithi149.stdout:3/986: creat d0/d4/d8/d24/d40/d62/d108/f169 x:0 0 0 2024-03-22T07:09:03.516 INFO:tasks.workunit.client.1.smithi149.stdout:3/987: dread d0/f110 [0,4194304] 0 2024-03-22T07:09:03.525 INFO:tasks.workunit.client.1.smithi149.stdout:3/988: dread d0/d2/d30/d5d/d84/dc3/fe9 [0,4194304] 0 2024-03-22T07:09:03.525 INFO:tasks.workunit.client.1.smithi149.stdout:3/989: chown d0/d4/d8/d24/ldc 13 1 2024-03-22T07:09:03.525 INFO:tasks.workunit.client.1.smithi149.stdout:3/990: write d0/d2/d30/d5d/d84/dc3/fe9 [5083682,48888] 0 2024-03-22T07:09:03.539 INFO:tasks.workunit.client.1.smithi149.stdout:3/991: dwrite d0/d2/d30/d7e/db8/f124 [0,4194304] 0 2024-03-22T07:09:03.557 INFO:tasks.workunit.client.1.smithi149.stdout:3/992: dwrite d0/d4/d6c/f9b [0,4194304] 0 2024-03-22T07:09:03.559 INFO:tasks.workunit.client.1.smithi149.stdout:3/993: symlink d0/d4/d8/d24/d40/d62/d108/l16a 0 2024-03-22T07:09:03.559 INFO:tasks.workunit.client.1.smithi149.stdout:3/994: stat d0/d4/d8/d24/d40/d62/f64 0 2024-03-22T07:09:03.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:03 smithi081 ceph-mon[29245]: pgmap v14: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 235 MiB/s rd, 298 MiB/s wr, 434 op/s 2024-03-22T07:09:03.572 INFO:tasks.workunit.client.1.smithi149.stdout:3/995: dwrite d0/d2/d42/d66/f7a [4194304,4194304] 0 2024-03-22T07:09:03.574 INFO:tasks.workunit.client.1.smithi149.stdout:3/996: link d0/d2/d5a/daf/fdd d0/d2/d5a/daf/dce/f16b 0 2024-03-22T07:09:03.575 INFO:tasks.workunit.client.1.smithi149.stdout:3/997: symlink d0/d2/d30/dd1/d118/d11a/l16c 0 2024-03-22T07:09:03.575 INFO:tasks.workunit.client.1.smithi149.stdout:3/998: rmdir d0/d2/d30/d5d/d84/dc3/d129 39 2024-03-22T07:09:03.576 INFO:tasks.workunit.client.1.smithi149.stdout:3/999: rename d0/d4/d8/d24/d40/d54/d9e to d0/d2/d30/d5d/d84/d113/d16d 0 2024-03-22T07:09:03.581 INFO:tasks.workunit.client.1.smithi149.stderr:+ rm -rf -- ./tmp.n6qV8VL6Wb 2024-03-22T07:09:03.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:03 smithi149 ceph-mon[36068]: pgmap v14: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 235 MiB/s rd, 298 MiB/s wr, 434 op/s 2024-03-22T07:09:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: pgmap v15: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 184 MiB/s rd, 233 MiB/s wr, 322 op/s 2024-03-22T07:09:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:09:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:09:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-22T07:09:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:09:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: Updating smithi081:/etc/ceph/ceph.conf 2024-03-22T07:09:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:05 smithi149 ceph-mon[36068]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-22T07:09:05.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: pgmap v15: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 184 MiB/s rd, 233 MiB/s wr, 322 op/s 2024-03-22T07:09:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:09:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:09:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-22T07:09:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:09:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: Updating smithi081:/etc/ceph/ceph.conf 2024-03-22T07:09:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:05 smithi081 ceph-mon[29245]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-22T07:09:06.405 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:06.405 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:06.405 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:09:06.405 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:09:06.405 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:09:06.405 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:09:06.405 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.405 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.406 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.406 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.406 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.406 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.406 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.406 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.406 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:06 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:06.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:06 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:07.365 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:07 smithi081 ceph-mon[29245]: pgmap v16: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 184 MiB/s rd, 233 MiB/s wr, 322 op/s 2024-03-22T07:09:07.366 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:07 smithi081 ceph-mon[29245]: Reconfiguring node-exporter.smithi081 (dependencies changed)... 2024-03-22T07:09:07.366 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:07 smithi081 ceph-mon[29245]: Reconfiguring daemon node-exporter.smithi081 on smithi081 2024-03-22T07:09:07.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:07 smithi149 ceph-mon[36068]: pgmap v16: 65 pgs: 65 active+clean; 2.6 GiB data, 9.2 GiB used, 527 GiB / 536 GiB avail; 184 MiB/s rd, 233 MiB/s wr, 322 op/s 2024-03-22T07:09:07.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:07 smithi149 ceph-mon[36068]: Reconfiguring node-exporter.smithi081 (dependencies changed)... 2024-03-22T07:09:07.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:07 smithi149 ceph-mon[36068]: Reconfiguring daemon node-exporter.smithi081 on smithi081 2024-03-22T07:09:09.008 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:08 smithi081 ceph-mon[29245]: pgmap v17: 65 pgs: 65 active+clean; 2.6 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 211 MiB/s rd, 263 MiB/s wr, 392 op/s 2024-03-22T07:09:09.008 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:08 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:09.008 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:08 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:09.008 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:08 smithi081 ceph-mon[29245]: Reconfiguring alertmanager.smithi081 (dependencies changed)... 2024-03-22T07:09:09.008 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:08 smithi081 ceph-mon[29245]: Reconfiguring daemon alertmanager.smithi081 on smithi081 2024-03-22T07:09:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:08 smithi149 ceph-mon[36068]: pgmap v17: 65 pgs: 65 active+clean; 2.6 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 211 MiB/s rd, 263 MiB/s wr, 392 op/s 2024-03-22T07:09:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:08 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:08 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:08 smithi149 ceph-mon[36068]: Reconfiguring alertmanager.smithi081 (dependencies changed)... 2024-03-22T07:09:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:08 smithi149 ceph-mon[36068]: Reconfiguring daemon alertmanager.smithi081 on smithi081 2024-03-22T07:09:10.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:09 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:09:10.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:09 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:10.101 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:09 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:09:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:09 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:11.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:10 smithi081 ceph-mon[29245]: pgmap v18: 65 pgs: 65 active+clean; 2.6 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 103 MiB/s rd, 125 MiB/s wr, 209 op/s 2024-03-22T07:09:11.101 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:10 smithi149 ceph-mon[36068]: pgmap v18: 65 pgs: 65 active+clean; 2.6 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 103 MiB/s rd, 125 MiB/s wr, 209 op/s 2024-03-22T07:09:12.891 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-22T07:09:12.891 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-22T07:09:12.956 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:12 smithi149 ceph-mon[36068]: pgmap v19: 65 pgs: 65 active+clean; 1.5 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 103 MiB/s rd, 126 MiB/s wr, 288 op/s 2024-03-22T07:09:13.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:12 smithi081 ceph-mon[29245]: pgmap v19: 65 pgs: 65 active+clean; 1.5 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 103 MiB/s rd, 126 MiB/s wr, 288 op/s 2024-03-22T07:09:15.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:14 smithi081 ceph-mon[29245]: pgmap v20: 65 pgs: 65 active+clean; 1.5 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 31 MiB/s wr, 148 op/s 2024-03-22T07:09:15.101 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:14 smithi149 ceph-mon[36068]: pgmap v20: 65 pgs: 65 active+clean; 1.5 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 31 MiB/s wr, 148 op/s 2024-03-22T07:09:16.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.655+0000 7f8e53797700 1 -- 172.21.15.81:0/4071424212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 msgr2=0x7f8e4c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.655+0000 7f8e53797700 1 --2- 172.21.15.81:0/4071424212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 0x7f8e4c0fd3f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c004940 tx=0x7f8e3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.655+0000 7f8e53797700 1 -- 172.21.15.81:0/4071424212 shutdown_connections 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.655+0000 7f8e53797700 1 --2- 172.21.15.81:0/4071424212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 0x7f8e4c0fd3f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.655+0000 7f8e53797700 1 --2- 172.21.15.81:0/4071424212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e4c0fbcd0 0x7f8e4c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.655+0000 7f8e53797700 1 -- 172.21.15.81:0/4071424212 >> 172.21.15.81:0/4071424212 conn(0x7f8e4c0f7430 msgr2=0x7f8e4c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.655+0000 7f8e53797700 1 -- 172.21.15.81:0/4071424212 shutdown_connections 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e53797700 1 -- 172.21.15.81:0/4071424212 wait complete. 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e53797700 1 Processor -- start 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e53797700 1 -- start start 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e53797700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e4c0fbcd0 0x7f8e4c10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e53797700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 0x7f8e4c10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:16.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e53797700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c10e520 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e53797700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c10ed20 con 0x7f8e4c0fbcd0 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e50d32700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 0x7f8e4c10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e50d32700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 0x7f8e4c10df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50406/0 (socket says 172.21.15.81:50406) 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e50d32700 1 -- 172.21.15.81:0/2650216672 learned_addr learned my addr 172.21.15.81:0/2650216672 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.656+0000 7f8e51533700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e4c0fbcd0 0x7f8e4c10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.657+0000 7f8e50d32700 1 -- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e4c0fbcd0 msgr2=0x7f8e4c10d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.657+0000 7f8e50d32700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e4c0fbcd0 0x7f8e4c10d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.657+0000 7f8e50d32700 1 -- 172.21.15.81:0/2650216672 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c005040 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.657+0000 7f8e50d32700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 0x7f8e4c10df20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00b3b0 tx=0x7f8e3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.657+0000 7f8e41ffb700 1 -- 172.21.15.81:0/2650216672 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3c00e040 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.657+0000 7f8e41ffb700 1 -- 172.21.15.81:0/2650216672 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e3c00bbf0 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.657+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c10efa0 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.659 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.657+0000 7f8e41ffb700 1 -- 172.21.15.81:0/2650216672 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3c01c630 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.657+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c10f4e0 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.658+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c101b10 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.659+0000 7f8e41ffb700 1 -- 172.21.15.81:0/2650216672 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7f8e3c00bd60 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.659+0000 7f8e41ffb700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8e3806b0b0 0x7f8e3806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:16.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.659+0000 7f8e41ffb700 1 -- 172.21.15.81:0/2650216672 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8e3c08a5f0 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.659+0000 7f8e51533700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8e3806b0b0 0x7f8e3806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:16.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.660+0000 7f8e51533700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8e3806b0b0 0x7f8e3806d560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009fd0 tx=0x7f8e4800b040 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:16.664 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.664+0000 7f8e41ffb700 1 -- 172.21.15.81:0/2650216672 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f8e3c058550 con 0x7f8e4c0fcf80 2024-03-22T07:09:16.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.888+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e4c1048d0 con 0x7f8e3806b0b0 2024-03-22T07:09:16.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.889+0000 7f8e41ffb700 1 -- 172.21.15.81:0/2650216672 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f8e4c1048d0 con 0x7f8e3806b0b0 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8e3806b0b0 msgr2=0x7f8e3806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8e3806b0b0 0x7f8e3806d560 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009fd0 tx=0x7f8e4800b040 comp rx=0 tx=0).stop 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 msgr2=0x7f8e4c10df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 0x7f8e4c10df20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00b3b0 tx=0x7f8e3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 shutdown_connections 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f8e3806b0b0 0x7f8e3806d560 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e4c0fbcd0 0x7f8e4c10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 --2- 172.21.15.81:0/2650216672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e4c0fcf80 0x7f8e4c10df20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 >> 172.21.15.81:0/2650216672 conn(0x7f8e4c0f7430 msgr2=0x7f8e4c065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.891+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 shutdown_connections 2024-03-22T07:09:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.892+0000 7f8e53797700 1 -- 172.21.15.81:0/2650216672 wait complete. 2024-03-22T07:09:16.901 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:09:16.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.995+0000 7fabf6f81700 1 -- 172.21.15.81:0/4168240872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf0066f70 msgr2=0x7fabf00ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:16.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.995+0000 7fabf6f81700 1 --2- 172.21.15.81:0/4168240872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf0066f70 0x7fabf00ff9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fabe4004940 tx=0x7fabe4009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:16.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.995+0000 7fabf6f81700 1 -- 172.21.15.81:0/4168240872 shutdown_connections 2024-03-22T07:09:16.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.995+0000 7fabf6f81700 1 --2- 172.21.15.81:0/4168240872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf00fff00 0x7fabf0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:16.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.995+0000 7fabf6f81700 1 --2- 172.21.15.81:0/4168240872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf0066f70 0x7fabf00ff9c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:16.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.995+0000 7fabf6f81700 1 -- 172.21.15.81:0/4168240872 >> 172.21.15.81:0/4168240872 conn(0x7fabf00f7430 msgr2=0x7fabf00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:16.996 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:16 smithi081 ceph-mon[29245]: pgmap v21: 65 pgs: 65 active+clean; 1.5 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 31 MiB/s wr, 148 op/s 2024-03-22T07:09:16.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.996+0000 7fabf6f81700 1 -- 172.21.15.81:0/4168240872 shutdown_connections 2024-03-22T07:09:16.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.996+0000 7fabf6f81700 1 -- 172.21.15.81:0/4168240872 wait complete. 2024-03-22T07:09:16.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.996+0000 7fabf6f81700 1 Processor -- start 2024-03-22T07:09:16.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.996+0000 7fabf6f81700 1 -- start start 2024-03-22T07:09:16.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.996+0000 7fabf6f81700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf0066f70 0x7fabf010d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:16.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.996+0000 7fabf6f81700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf00fff00 0x7fabf010dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:16.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.996+0000 7fabf6f81700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf010e4c0 con 0x7fabf00fff00 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.996+0000 7fabf6f81700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf010e630 con 0x7fabf0066f70 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabeffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf00fff00 0x7fabf010dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabeffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf00fff00 0x7fabf010dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50418/0 (socket says 172.21.15.81:50418) 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabeffff700 1 -- 172.21.15.81:0/2162091668 learned_addr learned my addr 172.21.15.81:0/2162091668 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabf4d1d700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf0066f70 0x7fabf010d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabeffff700 1 -- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf0066f70 msgr2=0x7fabf010d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabeffff700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf0066f70 0x7fabf010d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabeffff700 1 -- 172.21.15.81:0/2162091668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe4005040 con 0x7fabf00fff00 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabeffff700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf00fff00 0x7fabf010dec0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fabdc002a80 tx=0x7fabdc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabed7fa700 1 -- 172.21.15.81:0/2162091668 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabdc007cc0 con 0x7fabf00fff00 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabed7fa700 1 -- 172.21.15.81:0/2162091668 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabdc00f040 con 0x7fabf00fff00 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf0112ff0 con 0x7fabf00fff00 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabed7fa700 1 -- 172.21.15.81:0/2162091668 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabdc00dac0 con 0x7fabf00fff00 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.997+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf0113580 con 0x7fabf00fff00 2024-03-22T07:09:16.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.998+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf004c710 con 0x7fabf00fff00 2024-03-22T07:09:17.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.999+0000 7fabed7fa700 1 -- 172.21.15.81:0/2162091668 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7fabdc00dc20 con 0x7fabf00fff00 2024-03-22T07:09:17.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.999+0000 7fabed7fa700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fabe006b0c0 0x7fabe006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:17.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:16.999+0000 7fabed7fa700 1 -- 172.21.15.81:0/2162091668 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fabdc0898b0 con 0x7fabf00fff00 2024-03-22T07:09:17.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.000+0000 7fabf4d1d700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fabe006b0c0 0x7fabe006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:17.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.000+0000 7fabf4d1d700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fabe006b0c0 0x7fabe006d570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b970 tx=0x7fabe4009e30 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:17.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.004+0000 7fabed7fa700 1 -- 172.21.15.81:0/2162091668 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fabdc0575b0 con 0x7fabf00fff00 2024-03-22T07:09:17.101 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:16 smithi149 ceph-mon[36068]: pgmap v21: 65 pgs: 65 active+clean; 1.5 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 31 MiB/s wr, 148 op/s 2024-03-22T07:09:17.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.222+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabf00f8f70 con 0x7fabe006b0c0 2024-03-22T07:09:17.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.223+0000 7fabed7fa700 1 -- 172.21.15.81:0/2162091668 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fabf00f8f70 con 0x7fabe006b0c0 2024-03-22T07:09:17.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fabe006b0c0 msgr2=0x7fabe006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:17.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fabe006b0c0 0x7fabe006d570 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b970 tx=0x7fabe4009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:17.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf00fff00 msgr2=0x7fabf010dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:17.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf00fff00 0x7fabf010dec0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fabdc002a80 tx=0x7fabdc0046b0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 shutdown_connections 2024-03-22T07:09:17.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fabe006b0c0 0x7fabe006d570 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabf0066f70 0x7fabf010d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 --2- 172.21.15.81:0/2162091668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabf00fff00 0x7fabf010dec0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 >> 172.21.15.81:0/2162091668 conn(0x7fabf00f7430 msgr2=0x7fabf00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:17.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 shutdown_connections 2024-03-22T07:09:17.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.225+0000 7fabf6f81700 1 -- 172.21.15.81:0/2162091668 wait complete. 2024-03-22T07:09:17.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.327+0000 7f9a93c66700 1 -- 172.21.15.81:0/3580406891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0662e0 msgr2=0x7f9a8c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:17.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.327+0000 7f9a93c66700 1 --2- 172.21.15.81:0/3580406891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0662e0 0x7f9a8c0ff9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9a88004940 tx=0x7f9a88009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:17.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.327+0000 7f9a93c66700 1 -- 172.21.15.81:0/3580406891 shutdown_connections 2024-03-22T07:09:17.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.327+0000 7f9a93c66700 1 --2- 172.21.15.81:0/3580406891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c0fff00 0x7f9a8c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.327+0000 7f9a93c66700 1 --2- 172.21.15.81:0/3580406891 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0662e0 0x7f9a8c0ff9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.327+0000 7f9a93c66700 1 -- 172.21.15.81:0/3580406891 >> 172.21.15.81:0/3580406891 conn(0x7f9a8c0f7430 msgr2=0x7f9a8c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:17.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.327+0000 7f9a93c66700 1 -- 172.21.15.81:0/3580406891 shutdown_connections 2024-03-22T07:09:17.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.327+0000 7f9a93c66700 1 -- 172.21.15.81:0/3580406891 wait complete. 2024-03-22T07:09:17.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a93c66700 1 Processor -- start 2024-03-22T07:09:17.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a93c66700 1 -- start start 2024-03-22T07:09:17.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a93c66700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c0662e0 0x7f9a8c10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:17.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a93c66700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0fff00 0x7f9a8c10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:17.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a93c66700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c10e560 con 0x7f9a8c0fff00 2024-03-22T07:09:17.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a93c66700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c10ed60 con 0x7f9a8c0662e0 2024-03-22T07:09:17.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a91201700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0fff00 0x7f9a8c10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:17.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a91201700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0fff00 0x7f9a8c10df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50426/0 (socket says 172.21.15.81:50426) 2024-03-22T07:09:17.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a91201700 1 -- 172.21.15.81:0/1140876408 learned_addr learned my addr 172.21.15.81:0/1140876408 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:17.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a91a02700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c0662e0 0x7f9a8c10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a91201700 1 -- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c0662e0 msgr2=0x7f9a8c10da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a91201700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c0662e0 0x7f9a8c10da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a91201700 1 -- 172.21.15.81:0/1140876408 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a88005040 con 0x7f9a8c0fff00 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a91201700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0fff00 0x7f9a8c10df60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00eb70 tx=0x7f9a7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a827fc700 1 -- 172.21.15.81:0/1140876408 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a7c007cc0 con 0x7f9a8c0fff00 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a827fc700 1 -- 172.21.15.81:0/1140876408 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a7c00f040 con 0x7f9a8c0fff00 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c10efe0 con 0x7f9a8c0fff00 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.328+0000 7f9a827fc700 1 -- 172.21.15.81:0/1140876408 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a7c00ca90 con 0x7f9a8c0fff00 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.329+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c10f520 con 0x7f9a8c0fff00 2024-03-22T07:09:17.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.329+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c05f5f0 con 0x7f9a8c0fff00 2024-03-22T07:09:17.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.330+0000 7f9a827fc700 1 -- 172.21.15.81:0/1140876408 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7f9a7c00cbf0 con 0x7f9a8c0fff00 2024-03-22T07:09:17.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.330+0000 7f9a827fc700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f9a7806b0b0 0x7f9a7806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:17.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.331+0000 7f9a827fc700 1 -- 172.21.15.81:0/1140876408 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9a7c0896c0 con 0x7f9a8c0fff00 2024-03-22T07:09:17.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.331+0000 7f9a91a02700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f9a7806b0b0 0x7f9a7806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:17.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.331+0000 7f9a91a02700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f9a7806b0b0 0x7f9a7806d560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a88013870 tx=0x7f9a88009e30 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:17.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.335+0000 7f9a827fc700 1 -- 172.21.15.81:0/1140876408 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f9a7c0573c0 con 0x7f9a8c0fff00 2024-03-22T07:09:17.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.580+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a8c10f850 con 0x7f9a7806b0b0 2024-03-22T07:09:17.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.598+0000 7f9a827fc700 1 -- 172.21.15.81:0/1140876408 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f9a8c10f850 con 0x7f9a7806b0b0 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (6m) 26s ago 8m 25.8M - 0.25.0 c8568f914cd2 f83c4e446eae 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (8m) 26s ago 8m 6660k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (7m) 23s ago 7m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (6m) 26s ago 7m 82.0M - 9.4.7 954c08fa6188 5b1c9b6c3c44 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (2m) 26s ago 2m 11.3M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (2m) 26s ago 2m 138M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (2m) 23s ago 2m 14.3M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (2m) 23s ago 2m 108M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:9283 running (9m) 26s ago 9m 387M - 17.2.7-726-g9c4e0c34 837124da804f 3ef26f13517a 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283 running (48s) 23s ago 7m 489M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ab0200c8a736 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (9m) 26s ago 9m 50.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:09:17.599 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (7m) 23s ago 7m 41.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:09:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 starting - - - - 2024-03-22T07:09:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (7m) 23s ago 7m 15.5M - 1.5.0 0da6a335fe13 fbdea05540dc 2024-03-22T07:09:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (6m) 26s ago 6m 173M 1557M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:09:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (6m) 26s ago 6m 178M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:09:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (5m) 26s ago 5m 155M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:09:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (5m) 23s ago 5m 170M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:09:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (4m) 23s ago 4m 148M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:09:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (4m) 23s ago 4m 155M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:09:17.600 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (6m) 26s ago 7m 58.2M - 2.43.0 a07b618ecd1d 92a93d9b8284 2024-03-22T07:09:17.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f9a7806b0b0 msgr2=0x7f9a7806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:17.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f9a7806b0b0 0x7f9a7806d560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9a88013870 tx=0x7f9a88009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:17.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0fff00 msgr2=0x7f9a8c10df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:17.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0fff00 0x7f9a8c10df60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00eb70 tx=0x7f9a7c00b040 comp rx=0 tx=0).stop 2024-03-22T07:09:17.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 shutdown_connections 2024-03-22T07:09:17.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f9a7806b0b0 0x7f9a7806d560 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c0662e0 0x7f9a8c10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 --2- 172.21.15.81:0/1140876408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a8c0fff00 0x7f9a8c10df60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 >> 172.21.15.81:0/1140876408 conn(0x7f9a8c0f7430 msgr2=0x7f9a8c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:17.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 shutdown_connections 2024-03-22T07:09:17.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.602+0000 7f9a93c66700 1 -- 172.21.15.81:0/1140876408 wait complete. 2024-03-22T07:09:17.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 -- 172.21.15.81:0/4143029813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fce40 msgr2=0x7fdb180fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:17.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 --2- 172.21.15.81:0/4143029813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fce40 0x7fdb180fd250 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdb08004940 tx=0x7fdb08009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:17.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 -- 172.21.15.81:0/4143029813 shutdown_connections 2024-03-22T07:09:17.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 --2- 172.21.15.81:0/4143029813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb180fe040 0x7fdb180fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 --2- 172.21.15.81:0/4143029813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fce40 0x7fdb180fd250 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 -- 172.21.15.81:0/4143029813 >> 172.21.15.81:0/4143029813 conn(0x7fdb180f8510 msgr2=0x7fdb180fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:17.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 -- 172.21.15.81:0/4143029813 shutdown_connections 2024-03-22T07:09:17.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 -- 172.21.15.81:0/4143029813 wait complete. 2024-03-22T07:09:17.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 Processor -- start 2024-03-22T07:09:17.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 -- start start 2024-03-22T07:09:17.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb180fce40 0x7fdb1810d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:17.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fe040 0x7fdb1810dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:17.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1810e4e0 con 0x7fdb180fe040 2024-03-22T07:09:17.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.722+0000 7fdb1f85e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1810ed30 con 0x7fdb180fce40 2024-03-22T07:09:17.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1cdf9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fe040 0x7fdb1810dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1cdf9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fe040 0x7fdb1810dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50440/0 (socket says 172.21.15.81:50440) 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1cdf9700 1 -- 172.21.15.81:0/1488360728 learned_addr learned my addr 172.21.15.81:0/1488360728 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1d5fa700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb180fce40 0x7fdb1810d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1cdf9700 1 -- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb180fce40 msgr2=0x7fdb1810d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1cdf9700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb180fce40 0x7fdb1810d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1cdf9700 1 -- 172.21.15.81:0/1488360728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb08005040 con 0x7fdb180fe040 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1cdf9700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fe040 0x7fdb1810dee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdb14002a80 tx=0x7fdb140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb0dffb700 1 -- 172.21.15.81:0/1488360728 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb14007cc0 con 0x7fdb180fe040 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb0dffb700 1 -- 172.21.15.81:0/1488360728 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb1400f040 con 0x7fdb180fe040 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1810f010 con 0x7fdb180fe040 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb0dffb700 1 -- 172.21.15.81:0/1488360728 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb1400dac0 con 0x7fdb180fe040 2024-03-22T07:09:17.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.723+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1810f550 con 0x7fdb180fe040 2024-03-22T07:09:17.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.724+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1805f5f0 con 0x7fdb180fe040 2024-03-22T07:09:17.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.725+0000 7fdb0dffb700 1 -- 172.21.15.81:0/1488360728 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7fdb1400dc20 con 0x7fdb180fe040 2024-03-22T07:09:17.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.725+0000 7fdb0dffb700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fdb0406b0b0 0x7fdb0406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:17.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.725+0000 7fdb0dffb700 1 -- 172.21.15.81:0/1488360728 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fdb140896b0 con 0x7fdb180fe040 2024-03-22T07:09:17.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.725+0000 7fdb1d5fa700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fdb0406b0b0 0x7fdb0406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:17.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.726+0000 7fdb1d5fa700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fdb0406b0b0 0x7fdb0406d560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800b970 tx=0x7fdb08009e30 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:17.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:17.729+0000 7fdb0dffb700 1 -- 172.21.15.81:0/1488360728 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fdb14057430 con 0x7fdb180fe040 2024-03-22T07:09:18.002 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:17 smithi149 ceph-mon[36068]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:18.021 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.021+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdb1810f7c0 con 0x7fdb180fe040 2024-03-22T07:09:18.022 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:17 smithi081 ceph-mon[29245]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:18.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.021+0000 7fdb0dffb700 1 -- 172.21.15.81:0/1488360728 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fdb14056fc0 con 0x7fdb180fe040 2024-03-22T07:09:18.022 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:09:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:09:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-22T07:09:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:09:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:09:18.024 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fdb0406b0b0 msgr2=0x7fdb0406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fdb0406b0b0 0x7fdb0406d560 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800b970 tx=0x7fdb08009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fe040 msgr2=0x7fdb1810dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fe040 0x7fdb1810dee0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdb14002a80 tx=0x7fdb140046b0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 shutdown_connections 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fdb0406b0b0 0x7fdb0406d560 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb180fce40 0x7fdb1810d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 --2- 172.21.15.81:0/1488360728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb180fe040 0x7fdb1810dee0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 >> 172.21.15.81:0/1488360728 conn(0x7fdb180f8510 msgr2=0x7fdb180f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 shutdown_connections 2024-03-22T07:09:18.025 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.024+0000 7fdb1f85e700 1 -- 172.21.15.81:0/1488360728 wait complete. 2024-03-22T07:09:18.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.127+0000 7fe14333f700 1 -- 172.21.15.81:0/1422260566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 msgr2=0x7fe13c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.127+0000 7fe14333f700 1 --2- 172.21.15.81:0/1422260566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 0x7fe13c0fd3f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe12c004940 tx=0x7fe12c009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:18.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.127+0000 7fe14333f700 1 -- 172.21.15.81:0/1422260566 shutdown_connections 2024-03-22T07:09:18.127 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.127+0000 7fe14333f700 1 --2- 172.21.15.81:0/1422260566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 0x7fe13c0fd3f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.127+0000 7fe14333f700 1 --2- 172.21.15.81:0/1422260566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe13c0fbcd0 0x7fe13c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.127+0000 7fe14333f700 1 -- 172.21.15.81:0/1422260566 >> 172.21.15.81:0/1422260566 conn(0x7fe13c0f7430 msgr2=0x7fe13c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:18.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.127+0000 7fe14333f700 1 -- 172.21.15.81:0/1422260566 shutdown_connections 2024-03-22T07:09:18.128 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.127+0000 7fe14333f700 1 -- 172.21.15.81:0/1422260566 wait complete. 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe14333f700 1 Processor -- start 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe14333f700 1 -- start start 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe14333f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe13c0fbcd0 0x7fe13c10c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe14333f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 0x7fe13c10ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe14333f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c10d440 con 0x7fe13c0fcf80 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe14333f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c10dc90 con 0x7fe13c0fbcd0 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe1408da700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 0x7fe13c10ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe1408da700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 0x7fe13c10ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50456/0 (socket says 172.21.15.81:50456) 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe1408da700 1 -- 172.21.15.81:0/363669595 learned_addr learned my addr 172.21.15.81:0/363669595 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe1410db700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe13c0fbcd0 0x7fe13c10c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:18.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe1408da700 1 -- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe13c0fbcd0 msgr2=0x7fe13c10c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe1408da700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe13c0fbcd0 0x7fe13c10c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe1408da700 1 -- 172.21.15.81:0/363669595 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c005040 con 0x7fe13c0fcf80 2024-03-22T07:09:18.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe1408da700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 0x7fe13c10ce40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00b3b0 tx=0x7fe12c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:18.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe131ffb700 1 -- 172.21.15.81:0/363669595 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12c00e040 con 0x7fe13c0fcf80 2024-03-22T07:09:18.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c10df10 con 0x7fe13c0fcf80 2024-03-22T07:09:18.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe131ffb700 1 -- 172.21.15.81:0/363669595 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe12c004290 con 0x7fe13c0fcf80 2024-03-22T07:09:18.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.128+0000 7fe131ffb700 1 -- 172.21.15.81:0/363669595 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12c01c3f0 con 0x7fe13c0fcf80 2024-03-22T07:09:18.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.129+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c10e450 con 0x7fe13c0fcf80 2024-03-22T07:09:18.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.130+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c05f5f0 con 0x7fe13c0fcf80 2024-03-22T07:09:18.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.130+0000 7fe131ffb700 1 -- 172.21.15.81:0/363669595 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7fe12c00bbf0 con 0x7fe13c0fcf80 2024-03-22T07:09:18.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.130+0000 7fe131ffb700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fe12806d2c0 0x7fe12806f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:18.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.131+0000 7fe131ffb700 1 -- 172.21.15.81:0/363669595 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe12c08ae30 con 0x7fe13c0fcf80 2024-03-22T07:09:18.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.131+0000 7fe1410db700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fe12806d2c0 0x7fe12806f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:18.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.131+0000 7fe1410db700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fe12806d2c0 0x7fe12806f770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe138004f40 tx=0x7fe13800b040 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:18.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.135+0000 7fe131ffb700 1 -- 172.21.15.81:0/363669595 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe12c058bb0 con 0x7fe13c0fcf80 2024-03-22T07:09:18.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.355+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe13c10e7f0 con 0x7fe12806d2c0 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.357+0000 7fe131ffb700 1 -- 172.21.15.81:0/363669595 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fe13c10e7f0 con 0x7fe12806d2c0 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [], 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "1/2 daemons upgraded", 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "", 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:09:18.357 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.358+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fe12806d2c0 msgr2=0x7fe12806f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.358+0000 7fe14333f700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fe12806d2c0 0x7fe12806f770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe138004f40 tx=0x7fe13800b040 comp rx=0 tx=0).stop 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.359+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 msgr2=0x7fe13c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.359+0000 7fe14333f700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 0x7fe13c10ce40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00b3b0 tx=0x7fe12c009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.359+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 shutdown_connections 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.359+0000 7fe14333f700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fe12806d2c0 0x7fe12806f770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.359+0000 7fe14333f700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe13c0fbcd0 0x7fe13c10c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.359+0000 7fe14333f700 1 --2- 172.21.15.81:0/363669595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe13c0fcf80 0x7fe13c10ce40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.359+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 >> 172.21.15.81:0/363669595 conn(0x7fe13c0f7430 msgr2=0x7fe13c0f8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:18.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.359+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 shutdown_connections 2024-03-22T07:09:18.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:18.359+0000 7fe14333f700 1 -- 172.21.15.81:0/363669595 wait complete. 2024-03-22T07:09:19.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:18 smithi081 ceph-mon[29245]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:19.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:18 smithi081 ceph-mon[29245]: pgmap v22: 65 pgs: 65 active+clean; 1012 MiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 27 MiB/s rd, 32 MiB/s wr, 214 op/s 2024-03-22T07:09:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:18 smithi081 ceph-mon[29245]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:18 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1488360728' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:09:19.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:18 smithi149 ceph-mon[36068]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:19.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:18 smithi149 ceph-mon[36068]: pgmap v22: 65 pgs: 65 active+clean; 1012 MiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 27 MiB/s rd, 32 MiB/s wr, 214 op/s 2024-03-22T07:09:19.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:18 smithi149 ceph-mon[36068]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:19.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:18 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1488360728' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:09:19.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:19 smithi149 ceph-mon[36068]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:20.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:19 smithi081 ceph-mon[29245]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:21.059 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:20 smithi081 ceph-mon[29245]: pgmap v23: 65 pgs: 65 active+clean; 1012 MiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-03-22T07:09:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:20 smithi149 ceph-mon[36068]: pgmap v23: 65 pgs: 65 active+clean; 1012 MiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-03-22T07:09:23.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:22 smithi081 ceph-mon[29245]: pgmap v24: 65 pgs: 65 active+clean; 895 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-03-22T07:09:23.101 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:22 smithi149 ceph-mon[36068]: pgmap v24: 65 pgs: 65 active+clean; 895 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 38 KiB/s rd, 1.8 MiB/s wr, 290 op/s 2024-03-22T07:09:25.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:25 smithi081 ceph-mon[29245]: pgmap v25: 65 pgs: 65 active+clean; 895 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-22T07:09:25.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:25 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:25.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:25 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:25.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:25 smithi081 ceph-mon[29245]: Reconfiguring grafana.smithi081 (dependencies changed)... 2024-03-22T07:09:25.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:25 smithi081 ceph-mon[29245]: Reconfiguring daemon grafana.smithi081 on smithi081 2024-03-22T07:09:25.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:25 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:25.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:25 smithi149 ceph-mon[36068]: pgmap v25: 65 pgs: 65 active+clean; 895 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-22T07:09:25.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:25 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:25.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:25 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:25.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:25 smithi149 ceph-mon[36068]: Reconfiguring grafana.smithi081 (dependencies changed)... 2024-03-22T07:09:25.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:25 smithi149 ceph-mon[36068]: Reconfiguring daemon grafana.smithi081 on smithi081 2024-03-22T07:09:25.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:25 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:27.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:27 smithi081 ceph-mon[29245]: pgmap v26: 65 pgs: 65 active+clean; 895 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-22T07:09:27.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:27 smithi149 ceph-mon[36068]: pgmap v26: 65 pgs: 65 active+clean; 895 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-03-22T07:09:29.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:29 smithi081 ceph-mon[29245]: pgmap v27: 65 pgs: 65 active+clean; 901 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 41 KiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-03-22T07:09:29.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:29 smithi149 ceph-mon[36068]: pgmap v27: 65 pgs: 65 active+clean; 901 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 41 KiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-03-22T07:09:31.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:31 smithi081 ceph-mon[29245]: pgmap v28: 65 pgs: 65 active+clean; 901 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 31 KiB/s rd, 1.4 MiB/s wr, 296 op/s 2024-03-22T07:09:31.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:31 smithi149 ceph-mon[36068]: pgmap v28: 65 pgs: 65 active+clean; 901 MiB data, 4.4 GiB used, 532 GiB / 536 GiB avail; 31 KiB/s rd, 1.4 MiB/s wr, 296 op/s 2024-03-22T07:09:33.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:33 smithi081 ceph-mon[29245]: pgmap v29: 65 pgs: 65 active+clean; 888 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 47 KiB/s rd, 2.0 MiB/s wr, 436 op/s 2024-03-22T07:09:33.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:33 smithi149 ceph-mon[36068]: pgmap v29: 65 pgs: 65 active+clean; 888 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 47 KiB/s rd, 2.0 MiB/s wr, 436 op/s 2024-03-22T07:09:35.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:35 smithi081 ceph-mon[29245]: pgmap v30: 65 pgs: 65 active+clean; 888 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 31 KiB/s rd, 1.4 MiB/s wr, 291 op/s 2024-03-22T07:09:35.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:35 smithi149 ceph-mon[36068]: pgmap v30: 65 pgs: 65 active+clean; 888 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 31 KiB/s rd, 1.4 MiB/s wr, 291 op/s 2024-03-22T07:09:37.424 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:37 smithi081 ceph-mon[29245]: pgmap v31: 65 pgs: 65 active+clean; 888 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 31 KiB/s rd, 1.4 MiB/s wr, 291 op/s 2024-03-22T07:09:37.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:37 smithi149 ceph-mon[36068]: pgmap v31: 65 pgs: 65 active+clean; 888 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 31 KiB/s rd, 1.4 MiB/s wr, 291 op/s 2024-03-22T07:09:39.254 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:39 smithi081 ceph-mon[29245]: pgmap v32: 65 pgs: 65 active+clean; 886 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 48 KiB/s rd, 1.9 MiB/s wr, 462 op/s 2024-03-22T07:09:39.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:39 smithi149 ceph-mon[36068]: pgmap v32: 65 pgs: 65 active+clean; 886 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 48 KiB/s rd, 1.9 MiB/s wr, 462 op/s 2024-03-22T07:09:40.259 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:40 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:40.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:40 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:41.421 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:41 smithi081 ceph-mon[29245]: pgmap v33: 65 pgs: 65 active+clean; 886 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 32 KiB/s rd, 1.2 MiB/s wr, 311 op/s 2024-03-22T07:09:41.421 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:41 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:41.421 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:41 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:41.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:41 smithi149 ceph-mon[36068]: pgmap v33: 65 pgs: 65 active+clean; 886 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 32 KiB/s rd, 1.2 MiB/s wr, 311 op/s 2024-03-22T07:09:41.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:41 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:41.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:41 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:42.474 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:42 smithi081 ceph-mon[29245]: Reconfiguring prometheus.smithi081 (dependencies changed)... 2024-03-22T07:09:42.474 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:42 smithi081 ceph-mon[29245]: Reconfiguring daemon prometheus.smithi081 on smithi081 2024-03-22T07:09:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:42 smithi149 ceph-mon[36068]: Reconfiguring prometheus.smithi081 (dependencies changed)... 2024-03-22T07:09:42.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:42 smithi149 ceph-mon[36068]: Reconfiguring daemon prometheus.smithi081 on smithi081 2024-03-22T07:09:43.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:43 smithi081 ceph-mon[29245]: pgmap v34: 65 pgs: 65 active+clean; 889 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 45 KiB/s rd, 1.8 MiB/s wr, 437 op/s 2024-03-22T07:09:43.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:43 smithi149 ceph-mon[36068]: pgmap v34: 65 pgs: 65 active+clean; 889 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 45 KiB/s rd, 1.8 MiB/s wr, 437 op/s 2024-03-22T07:09:45.405 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:45 smithi081 ceph-mon[29245]: pgmap v35: 65 pgs: 65 active+clean; 889 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-03-22T07:09:45.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:45 smithi149 ceph-mon[36068]: pgmap v35: 65 pgs: 65 active+clean; 889 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-03-22T07:09:47.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:47 smithi081 ceph-mon[29245]: pgmap v36: 65 pgs: 65 active+clean; 889 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-03-22T07:09:47.601 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:47 smithi149 ceph-mon[36068]: pgmap v36: 65 pgs: 65 active+clean; 889 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 297 op/s 2024-03-22T07:09:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.465+0000 7fa9201ec700 1 -- 172.21.15.81:0/3176461461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 msgr2=0x7fa9180fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.465+0000 7fa9201ec700 1 --2- 172.21.15.81:0/3176461461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 0x7fa9180fe450 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa908004940 tx=0x7fa908009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.465+0000 7fa9201ec700 1 -- 172.21.15.81:0/3176461461 shutdown_connections 2024-03-22T07:09:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.465+0000 7fa9201ec700 1 --2- 172.21.15.81:0/3176461461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 0x7fa9180fe450 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.465+0000 7fa9201ec700 1 --2- 172.21.15.81:0/3176461461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9180fce70 0x7fa9180fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.465+0000 7fa9201ec700 1 -- 172.21.15.81:0/3176461461 >> 172.21.15.81:0/3176461461 conn(0x7fa9180f8550 msgr2=0x7fa9180fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.465+0000 7fa9201ec700 1 -- 172.21.15.81:0/3176461461 shutdown_connections 2024-03-22T07:09:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.465+0000 7fa9201ec700 1 -- 172.21.15.81:0/3176461461 wait complete. 2024-03-22T07:09:48.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa9201ec700 1 Processor -- start 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa9201ec700 1 -- start start 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa9201ec700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9180fce70 0x7fa91810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa9201ec700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 0x7fa91810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa9201ec700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91810e460 con 0x7fa9180fdfe0 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa9201ec700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91810e5d0 con 0x7fa9180fce70 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa91d787700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 0x7fa91810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa91d787700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 0x7fa91810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60342/0 (socket says 172.21.15.81:60342) 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa91d787700 1 -- 172.21.15.81:0/837032688 learned_addr learned my addr 172.21.15.81:0/837032688 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa91df88700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9180fce70 0x7fa91810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa91d787700 1 -- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9180fce70 msgr2=0x7fa91810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.466+0000 7fa91d787700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9180fce70 0x7fa91810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.467+0000 7fa91d787700 1 -- 172.21.15.81:0/837032688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908005040 con 0x7fa9180fdfe0 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.467+0000 7fa91d787700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 0x7fa91810de60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa908009150 tx=0x7fa908009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.467+0000 7fa90e7fc700 1 -- 172.21.15.81:0/837032688 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90800e040 con 0x7fa9180fdfe0 2024-03-22T07:09:48.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.467+0000 7fa90e7fc700 1 -- 172.21.15.81:0/837032688 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa908004290 con 0x7fa9180fdfe0 2024-03-22T07:09:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.467+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa918112fd0 con 0x7fa9180fdfe0 2024-03-22T07:09:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.467+0000 7fa90e7fc700 1 -- 172.21.15.81:0/837032688 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90801c630 con 0x7fa9180fdfe0 2024-03-22T07:09:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.467+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa918113560 con 0x7fa9180fdfe0 2024-03-22T07:09:48.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.468+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91804c710 con 0x7fa9180fdfe0 2024-03-22T07:09:48.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.468+0000 7fa90e7fc700 1 -- 172.21.15.81:0/837032688 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7fa90800bbf0 con 0x7fa9180fdfe0 2024-03-22T07:09:48.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.469+0000 7fa90e7fc700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fa90406b0c0 0x7fa90406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:48.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.469+0000 7fa90e7fc700 1 -- 172.21.15.81:0/837032688 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa90808a6b0 con 0x7fa9180fdfe0 2024-03-22T07:09:48.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.469+0000 7fa91df88700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fa90406b0c0 0x7fa90406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:48.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.469+0000 7fa91df88700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fa90406b0c0 0x7fa90406d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa914004f40 tx=0x7fa91400b040 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:48.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.473+0000 7fa90e7fc700 1 -- 172.21.15.81:0/837032688 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa908058550 con 0x7fa9180fdfe0 2024-03-22T07:09:48.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.689+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9181023a0 con 0x7fa90406b0c0 2024-03-22T07:09:48.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.690+0000 7fa90e7fc700 1 -- 172.21.15.81:0/837032688 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fa9181023a0 con 0x7fa90406b0c0 2024-03-22T07:09:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.692+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fa90406b0c0 msgr2=0x7fa90406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.692+0000 7fa9201ec700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fa90406b0c0 0x7fa90406d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa914004f40 tx=0x7fa91400b040 comp rx=0 tx=0).stop 2024-03-22T07:09:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.692+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 msgr2=0x7fa91810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.692+0000 7fa9201ec700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 0x7fa91810de60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa908009150 tx=0x7fa908009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.692+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 shutdown_connections 2024-03-22T07:09:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.692+0000 7fa9201ec700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fa90406b0c0 0x7fa90406d570 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.692+0000 7fa9201ec700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9180fce70 0x7fa91810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.692+0000 7fa9201ec700 1 --2- 172.21.15.81:0/837032688 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9180fdfe0 0x7fa91810de60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.692+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 >> 172.21.15.81:0/837032688 conn(0x7fa9180f8550 msgr2=0x7fa9180f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:48.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.693+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 shutdown_connections 2024-03-22T07:09:48.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.693+0000 7fa9201ec700 1 -- 172.21.15.81:0/837032688 wait complete. 2024-03-22T07:09:48.702 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:09:48.797 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.796+0000 7f2f20d11700 1 -- 172.21.15.81:0/1144168166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fe440 msgr2=0x7f2f1c0fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.796+0000 7f2f20d11700 1 --2- 172.21.15.81:0/1144168166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fe440 0x7f2f1c0fe850 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c004940 tx=0x7f2f0c009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.796+0000 7f2f20d11700 1 -- 172.21.15.81:0/1144168166 shutdown_connections 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.796+0000 7f2f20d11700 1 --2- 172.21.15.81:0/1144168166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f1c0fed90 0x7f2f1c0faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.796+0000 7f2f20d11700 1 --2- 172.21.15.81:0/1144168166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fe440 0x7f2f1c0fe850 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.796+0000 7f2f20d11700 1 -- 172.21.15.81:0/1144168166 >> 172.21.15.81:0/1144168166 conn(0x7f2f1c0f6be0 msgr2=0x7f2f1c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.796+0000 7f2f20d11700 1 -- 172.21.15.81:0/1144168166 shutdown_connections 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.796+0000 7f2f20d11700 1 -- 172.21.15.81:0/1144168166 wait complete. 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f20d11700 1 Processor -- start 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f20d11700 1 -- start start 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f20d11700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f1c0fe440 0x7f2f1c10d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:48.798 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f20d11700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fed90 0x7f2f1c10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f20d11700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c10e3f0 con 0x7f2f1c0fed90 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f20d11700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c10e560 con 0x7f2f1c0fe440 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f19d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fed90 0x7f2f1c10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f19d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fed90 0x7f2f1c10ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60346/0 (socket says 172.21.15.81:60346) 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f19d9b700 1 -- 172.21.15.81:0/441087961 learned_addr learned my addr 172.21.15.81:0/441087961 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f1a59c700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f1c0fe440 0x7f2f1c10d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f19d9b700 1 -- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f1c0fe440 msgr2=0x7f2f1c10d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f19d9b700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f1c0fe440 0x7f2f1c10d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f19d9b700 1 -- 172.21.15.81:0/441087961 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c005040 con 0x7f2f1c0fed90 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f19d9b700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fed90 0x7f2f1c10ddf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f10002a80 tx=0x7f2f100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f0affd700 1 -- 172.21.15.81:0/441087961 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f10007cc0 con 0x7f2f1c0fed90 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f0affd700 1 -- 172.21.15.81:0/441087961 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f1000f040 con 0x7f2f1c0fed90 2024-03-22T07:09:48.799 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.797+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c112fc0 con 0x7f2f1c0fed90 2024-03-22T07:09:48.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.798+0000 7f2f0affd700 1 -- 172.21.15.81:0/441087961 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f1000dac0 con 0x7f2f1c0fed90 2024-03-22T07:09:48.800 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.798+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c113550 con 0x7f2f1c0fed90 2024-03-22T07:09:48.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.798+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c04c710 con 0x7f2f1c0fed90 2024-03-22T07:09:48.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.799+0000 7f2f0affd700 1 -- 172.21.15.81:0/441087961 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7f2f1000dc20 con 0x7f2f1c0fed90 2024-03-22T07:09:48.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.799+0000 7f2f0affd700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f2f0406b0c0 0x7f2f0406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:48.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.799+0000 7f2f0affd700 1 -- 172.21.15.81:0/441087961 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f2f100896c0 con 0x7f2f1c0fed90 2024-03-22T07:09:48.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.800+0000 7f2f1a59c700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f2f0406b0c0 0x7f2f0406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:48.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.800+0000 7f2f1a59c700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f2f0406b0c0 0x7f2f0406d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00b3b0 tx=0x7f2f0c009e30 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:48.805 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:48.805+0000 7f2f0affd700 1 -- 172.21.15.81:0/441087961 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2f10057440 con 0x7f2f1c0fed90 2024-03-22T07:09:49.094 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.094+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f1c102490 con 0x7f2f0406b0c0 2024-03-22T07:09:49.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.095+0000 7f2f0affd700 1 -- 172.21.15.81:0/441087961 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f2f1c102490 con 0x7f2f0406b0c0 2024-03-22T07:09:49.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f2f0406b0c0 msgr2=0x7f2f0406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f2f0406b0c0 0x7f2f0406d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00b3b0 tx=0x7f2f0c009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:49.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fed90 msgr2=0x7f2f1c10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fed90 0x7f2f1c10ddf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f10002a80 tx=0x7f2f100046b0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 shutdown_connections 2024-03-22T07:09:49.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f2f0406b0c0 0x7f2f0406d570 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f1c0fe440 0x7f2f1c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 --2- 172.21.15.81:0/441087961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1c0fed90 0x7f2f1c10ddf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 >> 172.21.15.81:0/441087961 conn(0x7f2f1c0f6be0 msgr2=0x7f2f1c0f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:49.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 shutdown_connections 2024-03-22T07:09:49.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.098+0000 7f2f20d11700 1 -- 172.21.15.81:0/441087961 wait complete. 2024-03-22T07:09:49.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.199+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/232589219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a4066260 msgr2=0x7fb2a40ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.199+0000 7fb2ac2ce700 1 --2- 172.21.15.81:0/232589219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a4066260 0x7fb2a40ffa40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0004940 tx=0x7fb2a0009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:49.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/232589219 shutdown_connections 2024-03-22T07:09:49.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 --2- 172.21.15.81:0/232589219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2a40fff80 0x7fb2a4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 --2- 172.21.15.81:0/232589219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a4066260 0x7fb2a40ffa40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/232589219 >> 172.21.15.81:0/232589219 conn(0x7fb2a40f7430 msgr2=0x7fb2a40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/232589219 shutdown_connections 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/232589219 wait complete. 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 Processor -- start 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 -- start start 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2a4066260 0x7fb2a410da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a40fff80 0x7fb2a410dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a410e5b0 con 0x7fb2a40fff80 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2ac2ce700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a410ee00 con 0x7fb2a4066260 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2a9869700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a40fff80 0x7fb2a410dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2a9869700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a40fff80 0x7fb2a410dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60360/0 (socket says 172.21.15.81:60360) 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2a9869700 1 -- 172.21.15.81:0/3720536766 learned_addr learned my addr 172.21.15.81:0/3720536766 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.200+0000 7fb2aa06a700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2a4066260 0x7fb2a410da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.201+0000 7fb2a9869700 1 -- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2a4066260 msgr2=0x7fb2a410da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.201+0000 7fb2a9869700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2a4066260 0x7fb2a410da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.201+0000 7fb2a9869700 1 -- 172.21.15.81:0/3720536766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a0005040 con 0x7fb2a40fff80 2024-03-22T07:09:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.201+0000 7fb2a9869700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a40fff80 0x7fb2a410dfb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb294002a50 tx=0x7fb2940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.201+0000 7fb29affd700 1 -- 172.21.15.81:0/3720536766 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb294007cc0 con 0x7fb2a40fff80 2024-03-22T07:09:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.201+0000 7fb29affd700 1 -- 172.21.15.81:0/3720536766 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb29400f040 con 0x7fb2a40fff80 2024-03-22T07:09:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.201+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a410f0e0 con 0x7fb2a40fff80 2024-03-22T07:09:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.201+0000 7fb29affd700 1 -- 172.21.15.81:0/3720536766 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29400da30 con 0x7fb2a40fff80 2024-03-22T07:09:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.201+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a410f620 con 0x7fb2a40fff80 2024-03-22T07:09:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.202+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a405f5f0 con 0x7fb2a40fff80 2024-03-22T07:09:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.203+0000 7fb29affd700 1 -- 172.21.15.81:0/3720536766 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7fb29400db90 con 0x7fb2a40fff80 2024-03-22T07:09:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.203+0000 7fb29affd700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fb29006b0b0 0x7fb29006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.203+0000 7fb29affd700 1 -- 172.21.15.81:0/3720536766 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fb294089840 con 0x7fb2a40fff80 2024-03-22T07:09:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.203+0000 7fb2aa06a700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fb29006b0b0 0x7fb29006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.204+0000 7fb2aa06a700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fb29006b0b0 0x7fb29006d560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b970 tx=0x7fb2a0009e30 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:49.208 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.208+0000 7fb29affd700 1 -- 172.21.15.81:0/3720536766 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb294057580 con 0x7fb2a40fff80 2024-03-22T07:09:49.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.417+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2a410fa10 con 0x7fb29006b0b0 2024-03-22T07:09:49.418 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:49 smithi081 ceph-mon[29245]: pgmap v37: 65 pgs: 65 active+clean; 887 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 44 KiB/s rd, 1.8 MiB/s wr, 432 op/s 2024-03-22T07:09:49.418 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:49 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:49.418 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:49 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:49.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.434+0000 7fb29affd700 1 -- 172.21.15.81:0/3720536766 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fb2a410fa10 con 0x7fb29006b0b0 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 starting - - - - 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (8m) 58s ago 8m 6660k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (8m) 55s ago 8m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 starting - - - - 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (3m) 58s ago 3m 11.3M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (3m) 58s ago 3m 138M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (3m) 55s ago 3m 14.3M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (3m) 55s ago 3m 108M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:9283 running (10m) 58s ago 10m 387M - 17.2.7-726-g9c4e0c34 837124da804f 3ef26f13517a 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283 running (79s) 55s ago 7m 489M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ab0200c8a736 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (10m) 58s ago 10m 50.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (7m) 55s ago 7m 41.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:09:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 starting - - - - 2024-03-22T07:09:49.436 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (7m) 55s ago 7m 15.5M - 1.5.0 0da6a335fe13 fbdea05540dc 2024-03-22T07:09:49.436 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (7m) 58s ago 7m 173M 1557M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:09:49.436 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (6m) 58s ago 6m 178M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:09:49.436 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (6m) 58s ago 6m 155M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:09:49.436 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (5m) 55s ago 5m 170M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:09:49.436 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (5m) 55s ago 5m 148M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:09:49.436 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (4m) 55s ago 4m 155M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:09:49.436 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 starting - - - - 2024-03-22T07:09:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.436+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fb29006b0b0 msgr2=0x7fb29006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.436+0000 7fb2ac2ce700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fb29006b0b0 0x7fb29006d560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b970 tx=0x7fb2a0009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.436+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a40fff80 msgr2=0x7fb2a410dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.436+0000 7fb2ac2ce700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a40fff80 0x7fb2a410dfb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb294002a50 tx=0x7fb2940046b0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.437+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 shutdown_connections 2024-03-22T07:09:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.437+0000 7fb2ac2ce700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7fb29006b0b0 0x7fb29006d560 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.437+0000 7fb2ac2ce700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2a4066260 0x7fb2a410da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.437+0000 7fb2ac2ce700 1 --2- 172.21.15.81:0/3720536766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a40fff80 0x7fb2a410dfb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.437+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 >> 172.21.15.81:0/3720536766 conn(0x7fb2a40f7430 msgr2=0x7fb2a40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.437+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 shutdown_connections 2024-03-22T07:09:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.437+0000 7fb2ac2ce700 1 -- 172.21.15.81:0/3720536766 wait complete. 2024-03-22T07:09:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 -- 172.21.15.81:0/648748217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 msgr2=0x7f04740fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 --2- 172.21.15.81:0/648748217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 0x7f04740fd3f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0464004940 tx=0x7f0464009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 -- 172.21.15.81:0/648748217 shutdown_connections 2024-03-22T07:09:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 --2- 172.21.15.81:0/648748217 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 0x7f04740fd3f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 --2- 172.21.15.81:0/648748217 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04740fbcd0 0x7f04740fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 -- 172.21.15.81:0/648748217 >> 172.21.15.81:0/648748217 conn(0x7f04740f7430 msgr2=0x7f04740f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:49.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 -- 172.21.15.81:0/648748217 shutdown_connections 2024-03-22T07:09:49.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 -- 172.21.15.81:0/648748217 wait complete. 2024-03-22T07:09:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 Processor -- start 2024-03-22T07:09:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.540+0000 7f047bfa5700 1 -- start start 2024-03-22T07:09:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f047bfa5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04740fbcd0 0x7f04741106a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f047bfa5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 0x7f0474112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f047bfa5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0474066350 con 0x7f04740fcf80 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f047bfa5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0474066490 con 0x7f04740fbcd0 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f0479540700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 0x7f0474112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f0479540700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 0x7f0474112bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60382/0 (socket says 172.21.15.81:60382) 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f0479540700 1 -- 172.21.15.81:0/468322785 learned_addr learned my addr 172.21.15.81:0/468322785 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f0479540700 1 -- 172.21.15.81:0/468322785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04740fbcd0 msgr2=0x7f04741106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f0479540700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04740fbcd0 0x7f04741106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f0479540700 1 -- 172.21.15.81:0/468322785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0464005040 con 0x7f04740fcf80 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f0479540700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 0x7f0474112bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0464013870 tx=0x7f0464009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f046a7fc700 1 -- 172.21.15.81:0/468322785 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0464021070 con 0x7f04740fcf80 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0474113130 con 0x7f04740fcf80 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0474113670 con 0x7f04740fcf80 2024-03-22T07:09:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f046a7fc700 1 -- 172.21.15.81:0/468322785 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f046401c400 con 0x7f04740fcf80 2024-03-22T07:09:49.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.541+0000 7f046a7fc700 1 -- 172.21.15.81:0/468322785 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f046400e070 con 0x7f04740fcf80 2024-03-22T07:09:49.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.542+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04580052e0 con 0x7f04740fcf80 2024-03-22T07:09:49.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.543+0000 7f046a7fc700 1 -- 172.21.15.81:0/468322785 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7f046401ca20 con 0x7f04740fcf80 2024-03-22T07:09:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.543+0000 7f046a7fc700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f046006b0b0 0x7f046006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.543+0000 7f046a7fc700 1 -- 172.21.15.81:0/468322785 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f046408ac20 con 0x7f04740fcf80 2024-03-22T07:09:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.543+0000 7f0479d41700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f046006b0b0 0x7f046006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.544+0000 7f0479d41700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f046006b0b0 0x7f046006d560 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0474065fd0 tx=0x7f047000b040 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:49.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.548+0000 7f046a7fc700 1 -- 172.21.15.81:0/468322785 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f0464055050 con 0x7f04740fcf80 2024-03-22T07:09:49.588 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:49 smithi149 ceph-mon[36068]: pgmap v37: 65 pgs: 65 active+clean; 887 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 44 KiB/s rd, 1.8 MiB/s wr, 432 op/s 2024-03-22T07:09:49.588 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:49 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:49.588 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:49 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:49.734 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-22T07:09:49.734 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-22T07:09:49.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.855+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0458005860 con 0x7f04740fcf80 2024-03-22T07:09:49.856 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.856+0000 7f046a7fc700 1 -- 172.21.15.81:0/468322785 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f0464004490 con 0x7f04740fcf80 2024-03-22T07:09:49.856 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:09:49.856 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:09:49.856 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:09:49.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:09:49.857 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:09:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.859+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f046006b0b0 msgr2=0x7f046006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.859+0000 7f047bfa5700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f046006b0b0 0x7f046006d560 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0474065fd0 tx=0x7f047000b040 comp rx=0 tx=0).stop 2024-03-22T07:09:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.859+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 msgr2=0x7f0474112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.859+0000 7f047bfa5700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 0x7f0474112bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0464013870 tx=0x7f0464009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.860+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 shutdown_connections 2024-03-22T07:09:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.860+0000 7f047bfa5700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f046006b0b0 0x7f046006d560 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.860+0000 7f047bfa5700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04740fbcd0 0x7f04741106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.860+0000 7f047bfa5700 1 --2- 172.21.15.81:0/468322785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04740fcf80 0x7f0474112bf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.860+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 >> 172.21.15.81:0/468322785 conn(0x7f04740f7430 msgr2=0x7f04740f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:49.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.860+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 shutdown_connections 2024-03-22T07:09:49.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.861+0000 7f047bfa5700 1 -- 172.21.15.81:0/468322785 wait complete. 2024-03-22T07:09:49.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 -- 172.21.15.81:0/1007700718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9208066240 msgr2=0x7f92080ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:49.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 --2- 172.21.15.81:0/1007700718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9208066240 0x7f92080ffa60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f91fc004940 tx=0x7f91fc009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:49.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 -- 172.21.15.81:0/1007700718 shutdown_connections 2024-03-22T07:09:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 --2- 172.21.15.81:0/1007700718 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92080fffa0 0x7f9208102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 --2- 172.21.15.81:0/1007700718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9208066240 0x7f92080ffa60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 -- 172.21.15.81:0/1007700718 >> 172.21.15.81:0/1007700718 conn(0x7f92080f7430 msgr2=0x7f92080f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 -- 172.21.15.81:0/1007700718 shutdown_connections 2024-03-22T07:09:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 -- 172.21.15.81:0/1007700718 wait complete. 2024-03-22T07:09:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 Processor -- start 2024-03-22T07:09:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:49.999+0000 7f920ef6d700 1 -- start start 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f920ef6d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9208066240 0x7f92080fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f920ef6d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92080fffa0 0x7f92080fb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f920ef6d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080fbf10 con 0x7f92080fffa0 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f920ef6d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080fc080 con 0x7f9208066240 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f9207fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92080fffa0 0x7f92080fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f9207fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92080fffa0 0x7f92080fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60398/0 (socket says 172.21.15.81:60398) 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f9207fff700 1 -- 172.21.15.81:0/262168700 learned_addr learned my addr 172.21.15.81:0/262168700 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f920cd09700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9208066240 0x7f92080fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f9207fff700 1 -- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9208066240 msgr2=0x7f92080fd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f9207fff700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9208066240 0x7f92080fd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f9207fff700 1 -- 172.21.15.81:0/262168700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc005040 con 0x7f92080fffa0 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f9207fff700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92080fffa0 0x7f92080fb960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f91f4002a50 tx=0x7f91f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f92057fa700 1 -- 172.21.15.81:0/262168700 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91f4007cc0 con 0x7f92080fffa0 2024-03-22T07:09:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.000+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92080fc360 con 0x7f92080fffa0 2024-03-22T07:09:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.001+0000 7f92057fa700 1 -- 172.21.15.81:0/262168700 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91f400f040 con 0x7f92080fffa0 2024-03-22T07:09:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.001+0000 7f92057fa700 1 -- 172.21.15.81:0/262168700 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91f400da30 con 0x7f92080fffa0 2024-03-22T07:09:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.001+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9208115ab0 con 0x7f92080fffa0 2024-03-22T07:09:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.001+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920805f5f0 con 0x7f92080fffa0 2024-03-22T07:09:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.002+0000 7f92057fa700 1 -- 172.21.15.81:0/262168700 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 25) v1 ==== 88855+0+0 (secure 0 0 0) 0x7f91f400db90 con 0x7f92080fffa0 2024-03-22T07:09:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.002+0000 7f92057fa700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f91f806b0b0 0x7f91f806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.002+0000 7f92057fa700 1 -- 172.21.15.81:0/262168700 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f91f40898e0 con 0x7f92080fffa0 2024-03-22T07:09:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.003+0000 7f920cd09700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f91f806b0b0 0x7f91f806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.003+0000 7f920cd09700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f91f806b0b0 0x7f91f806d560 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91fc004890 tx=0x7f91fc009e30 comp rx=0 tx=0).ready entity=mgr.24337 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:50.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.007+0000 7f92057fa700 1 -- 172.21.15.81:0/262168700 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f91f4053ab0 con 0x7f92080fffa0 2024-03-22T07:09:50.139 DEBUG:teuthology.parallel:result is None 2024-03-22T07:09:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.221+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 --> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92080f8f70 con 0x7f91f806b0b0 2024-03-22T07:09:50.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.223+0000 7f92057fa700 1 -- 172.21.15.81:0/262168700 <== mgr.24337 v2:172.21.15.149:6828/18108693 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f92080f8f70 con 0x7f91f806b0b0 2024-03-22T07:09:50.223 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:09:50.223 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:09:50.223 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:09:50.223 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:09:50.223 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [], 2024-03-22T07:09:50.223 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "1/2 daemons upgraded", 2024-03-22T07:09:50.224 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "", 2024-03-22T07:09:50.224 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:09:50.224 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:09:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f91f806b0b0 msgr2=0x7f91f806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f91f806b0b0 0x7f91f806d560 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91fc004890 tx=0x7f91fc009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92080fffa0 msgr2=0x7f92080fb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92080fffa0 0x7f92080fb960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f91f4002a50 tx=0x7f91f40046b0 comp rx=0 tx=0).stop 2024-03-22T07:09:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 shutdown_connections 2024-03-22T07:09:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:6828/18108693,v1:172.21.15.149:6829/18108693] conn(0x7f91f806b0b0 0x7f91f806d560 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9208066240 0x7f92080fd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 --2- 172.21.15.81:0/262168700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92080fffa0 0x7f92080fb960 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 >> 172.21.15.81:0/262168700 conn(0x7f92080f7430 msgr2=0x7f92080f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 shutdown_connections 2024-03-22T07:09:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:09:50.225+0000 7f920ef6d700 1 -- 172.21.15.81:0/262168700 wait complete. 2024-03-22T07:09:50.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:50 smithi081 ceph-mon[29245]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:50.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:50 smithi081 ceph-mon[29245]: Reconfiguring node-exporter.smithi149 (dependencies changed)... 2024-03-22T07:09:50.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:50 smithi081 ceph-mon[29245]: Reconfiguring daemon node-exporter.smithi149 on smithi149 2024-03-22T07:09:50.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:50 smithi081 ceph-mon[29245]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:50.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:50 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/468322785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:09:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:50 smithi149 ceph-mon[36068]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:50 smithi149 ceph-mon[36068]: Reconfiguring node-exporter.smithi149 (dependencies changed)... 2024-03-22T07:09:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:50 smithi149 ceph-mon[36068]: Reconfiguring daemon node-exporter.smithi149 on smithi149 2024-03-22T07:09:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:50 smithi149 ceph-mon[36068]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:50 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/468322785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:09:51.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:51 smithi081 ceph-mon[29245]: pgmap v38: 65 pgs: 65 active+clean; 887 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 261 op/s 2024-03-22T07:09:51.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:51 smithi081 ceph-mon[29245]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:51 smithi149 ceph-mon[36068]: pgmap v38: 65 pgs: 65 active+clean; 887 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 261 op/s 2024-03-22T07:09:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:51 smithi149 ceph-mon[36068]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:52.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:52 smithi081 ceph-mon[29245]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:52.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:52 smithi081 ceph-mon[29245]: pgmap v39: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 399 op/s 2024-03-22T07:09:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:52 smithi149 ceph-mon[36068]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:52 smithi149 ceph-mon[36068]: pgmap v39: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 41 KiB/s rd, 1.7 MiB/s wr, 399 op/s 2024-03-22T07:09:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:54 smithi081 ceph-mon[29245]: pgmap v40: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 273 op/s 2024-03-22T07:09:54.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:54 smithi149 ceph-mon[36068]: pgmap v40: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 273 op/s 2024-03-22T07:09:55.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:55 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:55.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:55 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:56.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:56 smithi081 ceph-mon[29245]: pgmap v41: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 273 op/s 2024-03-22T07:09:56.577 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:56 smithi149 ceph-mon[36068]: pgmap v41: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 273 op/s 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:09:57.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: Upgrade: Need to upgrade myself (mgr.smithi149.sgmtoa) 2024-03-22T07:09:57.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:57 smithi081 ceph-mon[29245]: Upgrade: Need to upgrade myself (mgr.smithi149.sgmtoa) 2024-03-22T07:09:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:09:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:09:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:09:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:09:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:09:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:09:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:09:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: Upgrade: Need to upgrade myself (mgr.smithi149.sgmtoa) 2024-03-22T07:09:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:57 smithi149 ceph-mon[36068]: Upgrade: Need to upgrade myself (mgr.smithi149.sgmtoa) 2024-03-22T07:09:58.884 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:58 smithi081 ceph-mon[29245]: pgmap v42: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 275 op/s 2024-03-22T07:09:58.884 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:58 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:58.884 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:58 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi081.bcxarw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:58.884 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:58 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi081.bcxarw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:58.885 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:58 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:09:58.885 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:58 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:59.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:58 smithi149 ceph-mon[36068]: pgmap v42: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 275 op/s 2024-03-22T07:09:59.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:58 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:09:59.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:58 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi081.bcxarw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:59.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:58 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi081.bcxarw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:59.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:58 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:09:59.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:58 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:59.865 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:59 smithi081 ceph-mon[29245]: Upgrade: Updating mgr.smithi081.bcxarw 2024-03-22T07:09:59.865 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:09:59 smithi081 ceph-mon[29245]: Deploying daemon mgr.smithi081.bcxarw on smithi081 2024-03-22T07:10:00.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:59 smithi149 ceph-mon[36068]: Upgrade: Updating mgr.smithi081.bcxarw 2024-03-22T07:10:00.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:09:59 smithi149 ceph-mon[36068]: Deploying daemon mgr.smithi081.bcxarw on smithi081 2024-03-22T07:10:00.712 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:00 smithi081 ceph-mon[29245]: pgmap v43: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 15 KiB/s rd, 509 KiB/s wr, 139 op/s 2024-03-22T07:10:00.713 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:00 smithi081 ceph-mon[29245]: overall HEALTH_OK 2024-03-22T07:10:01.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:00 smithi149 ceph-mon[36068]: pgmap v43: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 15 KiB/s rd, 509 KiB/s wr, 139 op/s 2024-03-22T07:10:01.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:00 smithi149 ceph-mon[36068]: overall HEALTH_OK 2024-03-22T07:10:03.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:03 smithi081 ceph-mon[29245]: pgmap v44: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 15 KiB/s rd, 509 KiB/s wr, 140 op/s 2024-03-22T07:10:03.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:03 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:03.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:03 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:03.155 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:03 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:03 smithi149 ceph-mon[36068]: pgmap v44: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 15 KiB/s rd, 509 KiB/s wr, 140 op/s 2024-03-22T07:10:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:03 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:03 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:03 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:05.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:05 smithi081 ceph-mon[29245]: pgmap v45: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 255 B/s rd, 8.2 KiB/s wr, 2 op/s 2024-03-22T07:10:05.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:05 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:05.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:05 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:05 smithi149 ceph-mon[36068]: pgmap v45: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 255 B/s rd, 8.2 KiB/s wr, 2 op/s 2024-03-22T07:10:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:05 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:05 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:07.262 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:07 smithi081 ceph-mon[29245]: pgmap v46: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 255 B/s rd, 8.2 KiB/s wr, 2 op/s 2024-03-22T07:10:07.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:07 smithi149 ceph-mon[36068]: pgmap v46: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 255 B/s rd, 8.2 KiB/s wr, 2 op/s 2024-03-22T07:10:08.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:08 smithi081 ceph-mon[29245]: pgmap v47: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 255 B/s rd, 8.2 KiB/s wr, 2 op/s 2024-03-22T07:10:08.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:08 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:08.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:08 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:08.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:08 smithi149 ceph-mon[36068]: pgmap v47: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 255 B/s rd, 8.2 KiB/s wr, 2 op/s 2024-03-22T07:10:08.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:08 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:08.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:08 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:09 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:09 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:09 smithi081 ceph-mon[29245]: Standby manager daemon smithi081.bcxarw restarted 2024-03-22T07:10:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:09 smithi081 ceph-mon[29245]: Standby manager daemon smithi081.bcxarw started 2024-03-22T07:10:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:09 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi081.bcxarw/crt"}]: dispatch 2024-03-22T07:10:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:09 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:10:09.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:09 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi081.bcxarw/key"}]: dispatch 2024-03-22T07:10:09.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:09 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:10:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:09 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:09 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:09 smithi149 ceph-mon[36068]: Standby manager daemon smithi081.bcxarw restarted 2024-03-22T07:10:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:09 smithi149 ceph-mon[36068]: Standby manager daemon smithi081.bcxarw started 2024-03-22T07:10:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:09 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi081.bcxarw/crt"}]: dispatch 2024-03-22T07:10:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:09 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:10:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:09 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi081.bcxarw/key"}]: dispatch 2024-03-22T07:10:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:09 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:10:10.611 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:10 smithi081 ceph-mon[29245]: pgmap v48: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-22T07:10:10.611 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:10 smithi081 ceph-mon[29245]: mgrmap e26: smithi149.sgmtoa(active, since 90s), standbys: smithi081.bcxarw 2024-03-22T07:10:10.611 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:10 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:10.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:10 smithi149 ceph-mon[36068]: pgmap v48: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-03-22T07:10:10.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:10 smithi149 ceph-mon[36068]: mgrmap e26: smithi149.sgmtoa(active, since 90s), standbys: smithi081.bcxarw 2024-03-22T07:10:10.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:10 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:11.647 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:11 smithi081 ceph-mon[29245]: mgrmap e27: smithi149.sgmtoa(active, since 91s), standbys: smithi081.bcxarw 2024-03-22T07:10:11.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:11 smithi149 ceph-mon[36068]: mgrmap e27: smithi149.sgmtoa(active, since 91s), standbys: smithi081.bcxarw 2024-03-22T07:10:13.112 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: pgmap v49: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.1 KiB/s wr, 1 op/s 2024-03-22T07:10:13.112 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr fail", "who": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr fail", "who": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd='[{"prefix": "mgr fail", "who": "smithi149.sgmtoa"}]': finished 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: mgrmap e28: smithi081.bcxarw(active, starting, since 0.0103268s) 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:10:13.113 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: Manager daemon smithi081.bcxarw is now available 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:13.114 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:13.115 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:10:13.115 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/trash_purge_schedule"}]: dispatch 2024-03-22T07:10:13.115 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: [22/Mar/2024:07:10:12] ENGINE Bus STARTING 2024-03-22T07:10:13.115 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: [22/Mar/2024:07:10:12] ENGINE Serving on https://172.21.15.81:7150 2024-03-22T07:10:13.115 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: [22/Mar/2024:07:10:12] ENGINE Serving on http://172.21.15.81:8765 2024-03-22T07:10:13.115 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:13 smithi149 ceph-mon[36068]: [22/Mar/2024:07:10:12] ENGINE Bus STARTED 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: pgmap v49: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.1 KiB/s wr, 1 op/s 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.24337 172.21.15.149:0/3391238910' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr fail", "who": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "mgr fail", "who": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.24337 ' entity='mgr.smithi149.sgmtoa' cmd='[{"prefix": "mgr fail", "who": "smithi149.sgmtoa"}]': finished 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: mgrmap e28: smithi081.bcxarw(active, starting, since 0.0103268s) 2024-03-22T07:10:13.414 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:10:13.415 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: Manager daemon smithi081.bcxarw is now available 2024-03-22T07:10:13.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:13.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:13.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:13.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:10:13.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/trash_purge_schedule"}]: dispatch 2024-03-22T07:10:13.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: [22/Mar/2024:07:10:12] ENGINE Bus STARTING 2024-03-22T07:10:13.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: [22/Mar/2024:07:10:12] ENGINE Serving on https://172.21.15.81:7150 2024-03-22T07:10:13.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: [22/Mar/2024:07:10:12] ENGINE Serving on http://172.21.15.81:8765 2024-03-22T07:10:13.416 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:13 smithi081 ceph-mon[29245]: [22/Mar/2024:07:10:12] ENGINE Bus STARTED 2024-03-22T07:10:14.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:14 smithi081 ceph-mon[29245]: mgrmap e29: smithi081.bcxarw(active, since 1.01144s) 2024-03-22T07:10:14.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:14 smithi081 ceph-mon[29245]: pgmap v3: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:14.577 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:14 smithi149 ceph-mon[36068]: mgrmap e29: smithi081.bcxarw(active, since 1.01144s) 2024-03-22T07:10:14.577 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:14 smithi149 ceph-mon[36068]: pgmap v3: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:15.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:15 smithi081 ceph-mon[29245]: pgmap v4: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:15.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:15 smithi149 ceph-mon[36068]: pgmap v4: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:16.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:16 smithi081 ceph-mon[29245]: mgrmap e30: smithi081.bcxarw(active, since 3s) 2024-03-22T07:10:16.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:16 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:16.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:16 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:16.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:16 smithi149 ceph-mon[36068]: mgrmap e30: smithi081.bcxarw(active, since 3s) 2024-03-22T07:10:16.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:16 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:16.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:16 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:18.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:17 smithi081 ceph-mon[29245]: pgmap v5: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:17 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:17 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:17 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:17 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:17 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:18.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:17 smithi149 ceph-mon[36068]: pgmap v5: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:18.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:17 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:18.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:17 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:18.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:17 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:18.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:17 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:18.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:17 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:19 smithi081 ceph-mon[29245]: pgmap v6: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:19 smithi081 ceph-mon[29245]: Standby manager daemon smithi149.sgmtoa started 2024-03-22T07:10:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:19 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/1194347184' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/crt"}]: dispatch 2024-03-22T07:10:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:19 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/1194347184' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:10:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:19 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/1194347184' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/key"}]: dispatch 2024-03-22T07:10:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:19 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/1194347184' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:10:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:19 smithi149 ceph-mon[36068]: pgmap v6: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:19 smithi149 ceph-mon[36068]: Standby manager daemon smithi149.sgmtoa started 2024-03-22T07:10:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:19 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/1194347184' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/crt"}]: dispatch 2024-03-22T07:10:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:19 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/1194347184' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:10:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:19 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/1194347184' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/key"}]: dispatch 2024-03-22T07:10:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:19 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/1194347184' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:10:20.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 -- 172.21.15.81:0/3064102647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 msgr2=0x7fc11c0fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 --2- 172.21.15.81:0/3064102647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 0x7fc11c0fe860 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc118004940 tx=0x7fc118009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 -- 172.21.15.81:0/3064102647 shutdown_connections 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 --2- 172.21.15.81:0/3064102647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc11c0feda0 0x7fc11c0faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 --2- 172.21.15.81:0/3064102647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 0x7fc11c0fe860 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 -- 172.21.15.81:0/3064102647 >> 172.21.15.81:0/3064102647 conn(0x7fc11c0f6c10 msgr2=0x7fc11c0f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 -- 172.21.15.81:0/3064102647 shutdown_connections 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 -- 172.21.15.81:0/3064102647 wait complete. 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 Processor -- start 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 -- start start 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 0x7fc11c10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc11c0feda0 0x7fc11c10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c10e470 con 0x7fc11c0fe450 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.329+0000 7fc1235f6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c10e5e0 con 0x7fc11c0feda0 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc121392700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 0x7fc11c10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc121392700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 0x7fc11c10d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59648/0 (socket says 172.21.15.81:59648) 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc121392700 1 -- 172.21.15.81:0/3298216865 learned_addr learned my addr 172.21.15.81:0/3298216865 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc120b91700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc11c0feda0 0x7fc11c10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc121392700 1 -- 172.21.15.81:0/3298216865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc11c0feda0 msgr2=0x7fc11c10de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc121392700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc11c0feda0 0x7fc11c10de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc121392700 1 -- 172.21.15.81:0/3298216865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc118005040 con 0x7fc11c0fe450 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc121392700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 0x7fc11c10d930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc118009150 tx=0x7fc118009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc111ffb700 1 -- 172.21.15.81:0/3298216865 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11800e040 con 0x7fc11c0fe450 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c112fe0 con 0x7fc11c0fe450 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc111ffb700 1 -- 172.21.15.81:0/3298216865 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc118004290 con 0x7fc11c0fe450 2024-03-22T07:10:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc111ffb700 1 -- 172.21.15.81:0/3298216865 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc11801c630 con 0x7fc11c0fe450 2024-03-22T07:10:20.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.330+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c113510 con 0x7fc11c0fe450 2024-03-22T07:10:20.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.331+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c04c710 con 0x7fc11c0fe450 2024-03-22T07:10:20.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.332+0000 7fc111ffb700 1 -- 172.21.15.81:0/3298216865 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fc11800bbf0 con 0x7fc11c0fe450 2024-03-22T07:10:20.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.332+0000 7fc111ffb700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc10806cdb0 0x7fc10806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:20.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.332+0000 7fc111ffb700 1 -- 172.21.15.81:0/3298216865 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc11808cdb0 con 0x7fc11c0fe450 2024-03-22T07:10:20.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.332+0000 7fc120b91700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc10806cdb0 0x7fc10806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:20.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.333+0000 7fc120b91700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc10806cdb0 0x7fc10806f260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc10c004f40 tx=0x7fc10c00b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:20.337 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.337+0000 7fc111ffb700 1 -- 172.21.15.81:0/3298216865 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fc11805aa20 con 0x7fc11c0fe450 2024-03-22T07:10:20.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.556+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc11c113730 con 0x7fc10806cdb0 2024-03-22T07:10:20.558 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.558+0000 7fc111ffb700 1 -- 172.21.15.81:0/3298216865 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fc11c113730 con 0x7fc10806cdb0 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc10806cdb0 msgr2=0x7fc10806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc10806cdb0 0x7fc10806f260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc10c004f40 tx=0x7fc10c00b040 comp rx=0 tx=0).stop 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 msgr2=0x7fc11c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 0x7fc11c10d930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc118009150 tx=0x7fc118009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 shutdown_connections 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc10806cdb0 0x7fc10806f260 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc11c0fe450 0x7fc11c10d930 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 --2- 172.21.15.81:0/3298216865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc11c0feda0 0x7fc11c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 >> 172.21.15.81:0/3298216865 conn(0x7fc11c0f6c10 msgr2=0x7fc11c0f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:20.562 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.561+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 shutdown_connections 2024-03-22T07:10:20.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.562+0000 7fc1235f6700 1 -- 172.21.15.81:0/3298216865 wait complete. 2024-03-22T07:10:20.576 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:10:20.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.670+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/307951902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 msgr2=0x7f5c080fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:20.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.670+0000 7f5c0fd0a700 1 --2- 172.21.15.81:0/307951902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 0x7f5c080fe450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8004940 tx=0x7f5bf8009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:20.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.670+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/307951902 shutdown_connections 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.670+0000 7f5c0fd0a700 1 --2- 172.21.15.81:0/307951902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 0x7f5c080fe450 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.670+0000 7f5c0fd0a700 1 --2- 172.21.15.81:0/307951902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c080fce70 0x7f5c080fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.670+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/307951902 >> 172.21.15.81:0/307951902 conn(0x7f5c080f8550 msgr2=0x7f5c080fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.670+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/307951902 shutdown_connections 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.670+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/307951902 wait complete. 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0fd0a700 1 Processor -- start 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0fd0a700 1 -- start start 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0fd0a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c080fce70 0x7f5c0810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0fd0a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 0x7f5c0810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0fd0a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0810e460 con 0x7f5c080fdfe0 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0fd0a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0810e5d0 con 0x7f5c080fce70 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0d2a5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 0x7f5c0810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0d2a5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 0x7f5c0810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59664/0 (socket says 172.21.15.81:59664) 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0d2a5700 1 -- 172.21.15.81:0/1235297191 learned_addr learned my addr 172.21.15.81:0/1235297191 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:20.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.671+0000 7f5c0daa6700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c080fce70 0x7f5c0810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:20.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.672+0000 7f5c0d2a5700 1 -- 172.21.15.81:0/1235297191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c080fce70 msgr2=0x7f5c0810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:20.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.672+0000 7f5c0d2a5700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c080fce70 0x7f5c0810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.672+0000 7f5c0d2a5700 1 -- 172.21.15.81:0/1235297191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf8005040 con 0x7f5c080fdfe0 2024-03-22T07:10:20.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.672+0000 7f5c0daa6700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c080fce70 0x7f5c0810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:10:20.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.672+0000 7f5c0d2a5700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 0x7f5c0810de60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009150 tx=0x7f5bf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:20.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.672+0000 7f5bfe7fc700 1 -- 172.21.15.81:0/1235297191 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bf800e040 con 0x7f5c080fdfe0 2024-03-22T07:10:20.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.672+0000 7f5bfe7fc700 1 -- 172.21.15.81:0/1235297191 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bf800bcc0 con 0x7f5c080fdfe0 2024-03-22T07:10:20.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.672+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c08112fd0 con 0x7f5c080fdfe0 2024-03-22T07:10:20.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.672+0000 7f5bfe7fc700 1 -- 172.21.15.81:0/1235297191 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bf801c630 con 0x7f5c080fdfe0 2024-03-22T07:10:20.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.673+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c08113560 con 0x7f5c080fdfe0 2024-03-22T07:10:20.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.673+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0804c710 con 0x7f5c080fdfe0 2024-03-22T07:10:20.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.674+0000 7f5bfe7fc700 1 -- 172.21.15.81:0/1235297191 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f5bf8013420 con 0x7f5c080fdfe0 2024-03-22T07:10:20.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.674+0000 7f5bfe7fc700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5bf406cdb0 0x7f5bf406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:20.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.674+0000 7f5bfe7fc700 1 -- 172.21.15.81:0/1235297191 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5bf808cee0 con 0x7f5c080fdfe0 2024-03-22T07:10:20.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.674+0000 7f5c0daa6700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5bf406cdb0 0x7f5bf406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:20.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.675+0000 7f5c0daa6700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5bf406cdb0 0x7f5bf406f260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5c04007fd0 tx=0x7f5c04009040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:20.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.679+0000 7f5bfe7fc700 1 -- 172.21.15.81:0/1235297191 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f5bf805ab50 con 0x7f5c080fdfe0 2024-03-22T07:10:20.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.899+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c081023f0 con 0x7f5bf406cdb0 2024-03-22T07:10:20.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.900+0000 7f5bfe7fc700 1 -- 172.21.15.81:0/1235297191 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f5c081023f0 con 0x7f5bf406cdb0 2024-03-22T07:10:20.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5bf406cdb0 msgr2=0x7f5bf406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:20.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5bf406cdb0 0x7f5bf406f260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5c04007fd0 tx=0x7f5c04009040 comp rx=0 tx=0).stop 2024-03-22T07:10:20.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 msgr2=0x7f5c0810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:20.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 0x7f5c0810de60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009150 tx=0x7f5bf8009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:20.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 shutdown_connections 2024-03-22T07:10:20.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5bf406cdb0 0x7f5bf406f260 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c080fce70 0x7f5c0810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 --2- 172.21.15.81:0/1235297191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c080fdfe0 0x7f5c0810de60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:20.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 >> 172.21.15.81:0/1235297191 conn(0x7f5c080f8550 msgr2=0x7f5c080f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:20.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 shutdown_connections 2024-03-22T07:10:20.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:20.903+0000 7f5c0fd0a700 1 -- 172.21.15.81:0/1235297191 wait complete. 2024-03-22T07:10:20.941 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:20 smithi081 ceph-mon[29245]: mgrmap e31: smithi081.bcxarw(active, since 7s), standbys: smithi149.sgmtoa 2024-03-22T07:10:20.941 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:20 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi149.sgmtoa", "id": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/1729157219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 msgr2=0x7f4c98102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 --2- 172.21.15.81:0/1729157219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 0x7f4c98102320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4c84004940 tx=0x7f4c84009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/1729157219 shutdown_connections 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 --2- 172.21.15.81:0/1729157219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 0x7f4c98102320 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 --2- 172.21.15.81:0/1729157219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c980662e0 0x7f4c980ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/1729157219 >> 172.21.15.81:0/1729157219 conn(0x7f4c980f7430 msgr2=0x7f4c980f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/1729157219 shutdown_connections 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/1729157219 wait complete. 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 Processor -- start 2024-03-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.008+0000 7f4c9ee7d700 1 -- start start 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c9ee7d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c980662e0 0x7f4c9810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c9ee7d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 0x7f4c9810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c9ee7d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9810e5a0 con 0x7f4c980fff00 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c9ee7d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9810edf0 con 0x7f4c980662e0 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c97fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 0x7f4c9810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c97fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 0x7f4c9810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59696/0 (socket says 172.21.15.81:59696) 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c97fff700 1 -- 172.21.15.81:0/2741701159 learned_addr learned my addr 172.21.15.81:0/2741701159 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c9cc19700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c980662e0 0x7f4c9810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c97fff700 1 -- 172.21.15.81:0/2741701159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c980662e0 msgr2=0x7f4c9810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c97fff700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c980662e0 0x7f4c9810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c97fff700 1 -- 172.21.15.81:0/2741701159 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c84005040 con 0x7f4c980fff00 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c97fff700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 0x7f4c9810dfa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b3b0 tx=0x7f4c84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.009+0000 7f4c957fa700 1 -- 172.21.15.81:0/2741701159 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8400e040 con 0x7f4c980fff00 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.010+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9810f070 con 0x7f4c980fff00 2024-03-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.010+0000 7f4c957fa700 1 -- 172.21.15.81:0/2741701159 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c8400bcc0 con 0x7f4c980fff00 2024-03-22T07:10:21.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.010+0000 7f4c957fa700 1 -- 172.21.15.81:0/2741701159 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8401c630 con 0x7f4c980fff00 2024-03-22T07:10:21.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.010+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9810f5b0 con 0x7f4c980fff00 2024-03-22T07:10:21.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.011+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9805f5f0 con 0x7f4c980fff00 2024-03-22T07:10:21.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.011+0000 7f4c957fa700 1 -- 172.21.15.81:0/2741701159 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f4c84026440 con 0x7f4c980fff00 2024-03-22T07:10:21.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.012+0000 7f4c957fa700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4c8806cda0 0x7f4c8806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:21.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.012+0000 7f4c957fa700 1 -- 172.21.15.81:0/2741701159 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4c8408ce50 con 0x7f4c980fff00 2024-03-22T07:10:21.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.012+0000 7f4c9cc19700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4c8806cda0 0x7f4c8806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:21.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.012+0000 7f4c9cc19700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4c8806cda0 0x7f4c8806f250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c98066020 tx=0x7f4c8c009ee0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:21.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.016+0000 7f4c957fa700 1 -- 172.21.15.81:0/2741701159 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f4c8405aac0 con 0x7f4c980fff00 2024-03-22T07:10:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:20 smithi149 ceph-mon[36068]: mgrmap e31: smithi081.bcxarw(active, since 7s), standbys: smithi149.sgmtoa 2024-03-22T07:10:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:20 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi149.sgmtoa", "id": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:10:21.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.226+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c9810f980 con 0x7f4c8806cda0 2024-03-22T07:10:21.238 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.238+0000 7f4c957fa700 1 -- 172.21.15.81:0/2741701159 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4c9810f980 con 0x7f4c8806cda0 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (57s) 3s ago 9m 23.5M - 0.25.0 c8568f914cd2 ffe7f846ca44 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (9m) 3s ago 9m 6660k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (8m) 5s ago 8m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (41s) 3s ago 8m 80.0M - 9.4.7 954c08fa6188 c959485196e4 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (3m) 3s ago 3m 12.1M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (3m) 3s ago 3m 132M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (3m) 5s ago 3m 15.2M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (3m) 5s ago 3m 148M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (19s) 3s ago 10m 474M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283 running (111s) 5s ago 8m 71.8M - 19.0.0-1667-gdb0330b1 4f325f8440e9 ab0200c8a736 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (10m) 3s ago 10m 50.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (8m) 5s ago 8m 45.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:10:21.239 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (73s) 3s ago 9m 17.4M - 1.5.0 0da6a335fe13 85d3835de2c8 2024-03-22T07:10:21.240 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (25s) 5s ago 8m 8476k - 1.5.0 0da6a335fe13 0c642c5716fb 2024-03-22T07:10:21.240 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (7m) 3s ago 7m 182M 1557M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:10:21.240 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (7m) 3s ago 7m 197M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:10:21.240 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (6m) 3s ago 6m 160M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:10:21.240 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (6m) 5s ago 6m 214M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:10:21.240 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (5m) 5s ago 5m 192M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:10:21.240 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (5m) 5s ago 5m 180M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:10:21.240 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (35s) 3s ago 8m 43.3M - 2.43.0 a07b618ecd1d e254716ab266 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.240+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4c8806cda0 msgr2=0x7f4c8806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.240+0000 7f4c9ee7d700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4c8806cda0 0x7f4c8806f250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4c98066020 tx=0x7f4c8c009ee0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.240+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 msgr2=0x7f4c9810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.240+0000 7f4c9ee7d700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 0x7f4c9810dfa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b3b0 tx=0x7f4c84009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.241+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 shutdown_connections 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.241+0000 7f4c9ee7d700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4c8806cda0 0x7f4c8806f250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.241+0000 7f4c9ee7d700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4c980662e0 0x7f4c9810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.241+0000 7f4c9ee7d700 1 --2- 172.21.15.81:0/2741701159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c980fff00 0x7f4c9810dfa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.241+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 >> 172.21.15.81:0/2741701159 conn(0x7f4c980f7430 msgr2=0x7f4c980f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.241+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 shutdown_connections 2024-03-22T07:10:21.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.241+0000 7f4c9ee7d700 1 -- 172.21.15.81:0/2741701159 wait complete. 2024-03-22T07:10:21.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.343+0000 7f7504cb5700 1 -- 172.21.15.81:0/3290880619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fcf00 msgr2=0x7f75000fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.343+0000 7f7504cb5700 1 --2- 172.21.15.81:0/3290880619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fcf00 0x7f75000fd310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f74f0004940 tx=0x7f74f0009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.343+0000 7f7504cb5700 1 -- 172.21.15.81:0/3290880619 shutdown_connections 2024-03-22T07:10:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.343+0000 7f7504cb5700 1 --2- 172.21.15.81:0/3290880619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75000fe070 0x7f75000fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.343+0000 7f7504cb5700 1 --2- 172.21.15.81:0/3290880619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fcf00 0x7f75000fd310 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.343+0000 7f7504cb5700 1 -- 172.21.15.81:0/3290880619 >> 172.21.15.81:0/3290880619 conn(0x7f75000f85a0 msgr2=0x7f75000fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.343+0000 7f7504cb5700 1 -- 172.21.15.81:0/3290880619 shutdown_connections 2024-03-22T07:10:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.343+0000 7f7504cb5700 1 -- 172.21.15.81:0/3290880619 wait complete. 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f7504cb5700 1 Processor -- start 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f7504cb5700 1 -- start start 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f7504cb5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75000fcf00 0x7f750010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f7504cb5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fe070 0x7f750010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f7504cb5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750010e520 con 0x7f75000fe070 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f7504cb5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750010ed70 con 0x7f75000fcf00 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f74fdd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fe070 0x7f750010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f74fdd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fe070 0x7f750010df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59720/0 (socket says 172.21.15.81:59720) 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f74fdd9b700 1 -- 172.21.15.81:0/243152799 learned_addr learned my addr 172.21.15.81:0/243152799 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.344+0000 7f74fe59c700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75000fcf00 0x7f750010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.345+0000 7f74fdd9b700 1 -- 172.21.15.81:0/243152799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75000fcf00 msgr2=0x7f750010d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.345+0000 7f74fdd9b700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75000fcf00 0x7f750010d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.345+0000 7f74fdd9b700 1 -- 172.21.15.81:0/243152799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0005040 con 0x7f75000fe070 2024-03-22T07:10:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.345+0000 7f74fdd9b700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fe070 0x7f750010df20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f74f400eb70 tx=0x7f74f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.345+0000 7f74eeffd700 1 -- 172.21.15.81:0/243152799 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74f4007cc0 con 0x7f75000fe070 2024-03-22T07:10:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.345+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f750010eff0 con 0x7f75000fe070 2024-03-22T07:10:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.345+0000 7f74eeffd700 1 -- 172.21.15.81:0/243152799 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74f400f040 con 0x7f75000fe070 2024-03-22T07:10:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.345+0000 7f74eeffd700 1 -- 172.21.15.81:0/243152799 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74f400ca90 con 0x7f75000fe070 2024-03-22T07:10:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.345+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750010f530 con 0x7f75000fe070 2024-03-22T07:10:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.346+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f750005f5f0 con 0x7f75000fe070 2024-03-22T07:10:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.346+0000 7f74eeffd700 1 -- 172.21.15.81:0/243152799 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f74f400cbf0 con 0x7f75000fe070 2024-03-22T07:10:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.347+0000 7f74eeffd700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74e806cda0 0x7f74e806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.347+0000 7f74eeffd700 1 -- 172.21.15.81:0/243152799 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f74f408bc90 con 0x7f75000fe070 2024-03-22T07:10:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.347+0000 7f74fe59c700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74e806cda0 0x7f74e806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.347+0000 7f74fe59c700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74e806cda0 0x7f74e806f250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009b30 tx=0x7f74f0009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:21.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.352+0000 7f74eeffd700 1 -- 172.21.15.81:0/243152799 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f74f4059930 con 0x7f75000fe070 2024-03-22T07:10:21.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.674+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7500105910 con 0x7f75000fe070 2024-03-22T07:10:21.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.675+0000 7f74eeffd700 1 -- 172.21.15.81:0/243152799 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f74f40594c0 con 0x7f75000fe070 2024-03-22T07:10:21.675 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:10:21.675 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:10:21.675 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:10:21.676 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74e806cda0 msgr2=0x7f74e806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74e806cda0 0x7f74e806f250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009b30 tx=0x7f74f0009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fe070 msgr2=0x7f750010df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fe070 0x7f750010df20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f74f400eb70 tx=0x7f74f400b040 comp rx=0 tx=0).stop 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 shutdown_connections 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74e806cda0 0x7f74e806f250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75000fcf00 0x7f750010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 --2- 172.21.15.81:0/243152799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75000fe070 0x7f750010df20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 >> 172.21.15.81:0/243152799 conn(0x7f75000f85a0 msgr2=0x7f75000f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:21.678 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 shutdown_connections 2024-03-22T07:10:21.679 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.677+0000 7f7504cb5700 1 -- 172.21.15.81:0/243152799 wait complete. 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.787+0000 7f9b84db5700 1 -- 172.21.15.81:0/604841578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800662e0 msgr2=0x7f9b800fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.787+0000 7f9b84db5700 1 --2- 172.21.15.81:0/604841578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800662e0 0x7f9b800fd7b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9b70004940 tx=0x7f9b70009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.787+0000 7f9b84db5700 1 -- 172.21.15.81:0/604841578 shutdown_connections 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.787+0000 7f9b84db5700 1 --2- 172.21.15.81:0/604841578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b800fdcf0 0x7f9b80100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.787+0000 7f9b84db5700 1 --2- 172.21.15.81:0/604841578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800662e0 0x7f9b800fd7b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.787+0000 7f9b84db5700 1 -- 172.21.15.81:0/604841578 >> 172.21.15.81:0/604841578 conn(0x7f9b800f7430 msgr2=0x7f9b800f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.787+0000 7f9b84db5700 1 -- 172.21.15.81:0/604841578 shutdown_connections 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.787+0000 7f9b84db5700 1 -- 172.21.15.81:0/604841578 wait complete. 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b84db5700 1 Processor -- start 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b84db5700 1 -- start start 2024-03-22T07:10:21.789 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b84db5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b800662e0 0x7f9b80114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b84db5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800fdcf0 0x7f9b801145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b84db5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80114c30 con 0x7f9b800fdcf0 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b84db5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b801153d0 con 0x7f9b800662e0 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b7dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800fdcf0 0x7f9b801145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b7dd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800fdcf0 0x7f9b801145a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59748/0 (socket says 172.21.15.81:59748) 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b7dd9b700 1 -- 172.21.15.81:0/2414492627 learned_addr learned my addr 172.21.15.81:0/2414492627 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b7dd9b700 1 -- 172.21.15.81:0/2414492627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b800662e0 msgr2=0x7f9b80114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b7dd9b700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b800662e0 0x7f9b80114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b7dd9b700 1 -- 172.21.15.81:0/2414492627 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b70005040 con 0x7f9b800fdcf0 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.788+0000 7f9b7dd9b700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800fdcf0 0x7f9b801145a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b790 tx=0x7f9b74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.789+0000 7f9b6effd700 1 -- 172.21.15.81:0/2414492627 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7400f770 con 0x7f9b800fdcf0 2024-03-22T07:10:21.790 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.789+0000 7f9b6effd700 1 -- 172.21.15.81:0/2414492627 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b7400fda0 con 0x7f9b800fdcf0 2024-03-22T07:10:21.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.789+0000 7f9b6effd700 1 -- 172.21.15.81:0/2414492627 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7400d310 con 0x7f9b800fdcf0 2024-03-22T07:10:21.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.789+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b801156b0 con 0x7f9b800fdcf0 2024-03-22T07:10:21.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.789+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b80115c70 con 0x7f9b800fdcf0 2024-03-22T07:10:21.791 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.790+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8005f5f0 con 0x7f9b800fdcf0 2024-03-22T07:10:21.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.790+0000 7f9b6effd700 1 -- 172.21.15.81:0/2414492627 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f9b74010420 con 0x7f9b800fdcf0 2024-03-22T07:10:21.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.790+0000 7f9b6effd700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f9b6806cda0 0x7f9b6806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:21.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.791+0000 7f9b6effd700 1 -- 172.21.15.81:0/2414492627 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f9b7408c080 con 0x7f9b800fdcf0 2024-03-22T07:10:21.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.791+0000 7f9b7e59c700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f9b6806cda0 0x7f9b6806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:21.792 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.791+0000 7f9b7e59c700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f9b6806cda0 0x7f9b6806f250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b70004f90 tx=0x7f9b70009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:21.795 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:21.795+0000 7f9b6effd700 1 -- 172.21.15.81:0/2414492627 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f9b74059ea0 con 0x7f9b800fdcf0 2024-03-22T07:10:22.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.010+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b80115fe0 con 0x7f9b6806cda0 2024-03-22T07:10:22.011 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:21 smithi081 ceph-mon[29245]: pgmap v7: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-22T07:10:22.011 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:21 smithi081 ceph-mon[29245]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:22.011 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:21 smithi081 ceph-mon[29245]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:22.011 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:21 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/243152799' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:22.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.011+0000 7f9b6effd700 1 -- 172.21.15.81:0/2414492627 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f9b80115fe0 con 0x7f9b6806cda0 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr" 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "2/2 daemons upgraded", 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "", 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:10:22.012 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:10:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.013+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f9b6806cda0 msgr2=0x7f9b6806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.013+0000 7f9b84db5700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f9b6806cda0 0x7f9b6806f250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9b70004f90 tx=0x7f9b70009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.013+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800fdcf0 msgr2=0x7f9b801145a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.013+0000 7f9b84db5700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800fdcf0 0x7f9b801145a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b790 tx=0x7f9b74009250 comp rx=0 tx=0).stop 2024-03-22T07:10:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.013+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 shutdown_connections 2024-03-22T07:10:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.013+0000 7f9b84db5700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f9b6806cda0 0x7f9b6806f250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.013+0000 7f9b84db5700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b800662e0 0x7f9b80114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.013+0000 7f9b84db5700 1 --2- 172.21.15.81:0/2414492627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b800fdcf0 0x7f9b801145a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.013+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 >> 172.21.15.81:0/2414492627 conn(0x7f9b800f7430 msgr2=0x7f9b800f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:22.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.014+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 shutdown_connections 2024-03-22T07:10:22.015 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:22.014+0000 7f9b84db5700 1 -- 172.21.15.81:0/2414492627 wait complete. 2024-03-22T07:10:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:21 smithi149 ceph-mon[36068]: pgmap v7: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-22T07:10:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:21 smithi149 ceph-mon[36068]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:21 smithi149 ceph-mon[36068]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:21 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/243152799' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:23.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:22 smithi081 ceph-mon[29245]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:23.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:22 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:23.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:22 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:23.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:22 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:23.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:22 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:22 smithi149 ceph-mon[36068]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:22 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:22 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:22 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:22 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:24.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:23 smithi081 ceph-mon[29245]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:23 smithi081 ceph-mon[29245]: pgmap v8: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T07:10:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:23 smithi081 ceph-mon[29245]: Updating smithi081:/etc/ceph/ceph.conf 2024-03-22T07:10:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:23 smithi081 ceph-mon[29245]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-22T07:10:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:23 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:23 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:23 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:23 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:23 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:23 smithi149 ceph-mon[36068]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:23 smithi149 ceph-mon[36068]: pgmap v8: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T07:10:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:23 smithi149 ceph-mon[36068]: Updating smithi081:/etc/ceph/ceph.conf 2024-03-22T07:10:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:23 smithi149 ceph-mon[36068]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-22T07:10:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:23 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:23 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:23 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:23 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:23 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:25.138 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:24 smithi081 ceph-mon[29245]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:10:25.138 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:24 smithi081 ceph-mon[29245]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:10:25.138 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:24 smithi081 ceph-mon[29245]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:10:25.138 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:24 smithi081 ceph-mon[29245]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:10:25.138 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:24 smithi081 ceph-mon[29245]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:10:25.138 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:24 smithi081 ceph-mon[29245]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:10:25.138 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:24 smithi081 ceph-mon[29245]: Reconfiguring prometheus.smithi081 (dependencies changed)... 2024-03-22T07:10:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:24 smithi149 ceph-mon[36068]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:10:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:24 smithi149 ceph-mon[36068]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:10:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:24 smithi149 ceph-mon[36068]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:10:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:24 smithi149 ceph-mon[36068]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:10:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:24 smithi149 ceph-mon[36068]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:10:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:24 smithi149 ceph-mon[36068]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:10:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:24 smithi149 ceph-mon[36068]: Reconfiguring prometheus.smithi081 (dependencies changed)... 2024-03-22T07:10:25.968 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:25 smithi081 ceph-mon[29245]: Reconfiguring daemon prometheus.smithi081 on smithi081 2024-03-22T07:10:25.968 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:25 smithi081 ceph-mon[29245]: pgmap v9: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-22T07:10:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:25 smithi149 ceph-mon[36068]: Reconfiguring daemon prometheus.smithi081 on smithi081 2024-03-22T07:10:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:25 smithi149 ceph-mon[36068]: pgmap v9: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-22T07:10:27.990 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:27 smithi081 ceph-mon[29245]: pgmap v10: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:10:27.990 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:27 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:27 smithi149 ceph-mon[36068]: pgmap v10: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:10:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:27 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:30.020 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:29 smithi149 ceph-mon[36068]: pgmap v11: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:10:30.021 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:29 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:30.021 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:29 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:30.021 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:29 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:10:30.021 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:29 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:30.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:29 smithi081 ceph-mon[29245]: pgmap v11: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:10:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:29 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:29 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:29 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:10:30.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:29 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:31.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:30 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:10:31.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:30 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:10:31.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:30 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:10:31.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:30 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:31.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:30 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:10:31.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:30 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:10:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:30 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:10:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:30 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:32.189 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:31 smithi149 ceph-mon[36068]: pgmap v12: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:10:32.189 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:31 smithi149 ceph-mon[36068]: Upgrade: Updating mgr.smithi149.sgmtoa 2024-03-22T07:10:32.189 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:31 smithi149 ceph-mon[36068]: Deploying daemon mgr.smithi149.sgmtoa on smithi149 2024-03-22T07:10:32.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:31 smithi081 ceph-mon[29245]: pgmap v12: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:10:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:31 smithi081 ceph-mon[29245]: Upgrade: Updating mgr.smithi149.sgmtoa 2024-03-22T07:10:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:31 smithi081 ceph-mon[29245]: Deploying daemon mgr.smithi149.sgmtoa on smithi149 2024-03-22T07:10:34.027 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:33 smithi149 ceph-mon[36068]: pgmap v13: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:34.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:33 smithi081 ceph-mon[29245]: pgmap v13: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:35.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:35 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:35.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:35 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:35.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:35 smithi149 ceph-mon[36068]: pgmap v14: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:35.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:35 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:35.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:35 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:35.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:35 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:35 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:35 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:35 smithi081 ceph-mon[29245]: pgmap v14: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:35 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:35 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:35.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:35 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:37.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:37 smithi081 ceph-mon[29245]: pgmap v15: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:37.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:37 smithi149 ceph-mon[36068]: pgmap v15: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:38 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:38 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:38.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:38 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:38.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:38 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:39.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:39 smithi081 ceph-mon[29245]: pgmap v16: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:39.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:39 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:39.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:39 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:39.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:39 smithi149 ceph-mon[36068]: pgmap v16: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:39.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:39 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:39.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:39 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:41.202 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:40 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:41.202 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:40 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:41.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:40 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:41.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:40 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:42.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:41 smithi081 ceph-mon[29245]: pgmap v17: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:41 smithi081 ceph-mon[29245]: Standby manager daemon smithi149.sgmtoa restarted 2024-03-22T07:10:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:41 smithi081 ceph-mon[29245]: Standby manager daemon smithi149.sgmtoa started 2024-03-22T07:10:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:41 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/3498838428' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/crt"}]: dispatch 2024-03-22T07:10:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:41 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/3498838428' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:10:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:41 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/3498838428' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/key"}]: dispatch 2024-03-22T07:10:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:41 smithi081 ceph-mon[29245]: from='mgr.? 172.21.15.149:0/3498838428' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:10:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:41 smithi149 ceph-mon[36068]: pgmap v17: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:41 smithi149 ceph-mon[36068]: Standby manager daemon smithi149.sgmtoa restarted 2024-03-22T07:10:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:41 smithi149 ceph-mon[36068]: Standby manager daemon smithi149.sgmtoa started 2024-03-22T07:10:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:41 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/3498838428' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/crt"}]: dispatch 2024-03-22T07:10:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:41 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/3498838428' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:10:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:41 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/3498838428' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/key"}]: dispatch 2024-03-22T07:10:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:41 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/3498838428' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:10:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: mgrmap e32: smithi081.bcxarw(active, since 29s), standbys: smithi149.sgmtoa 2024-03-22T07:10:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: Upgrade: Setting container_image for all mgr 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi081.bcxarw"}]: dispatch 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi081.bcxarw"}]': finished 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi149.sgmtoa"}]: dispatch 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi149.sgmtoa"}]': finished 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: Upgrade: Setting container_image for all rgw 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: Upgrade: Setting container_image for all iscsi 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: Upgrade: Setting container_image for all nfs 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: Upgrade: Setting container_image for all nvmeof 2024-03-22T07:10:43.072 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.072 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: pgmap v18: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:43.072 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:42 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:43.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: mgrmap e32: smithi081.bcxarw(active, since 29s), standbys: smithi149.sgmtoa 2024-03-22T07:10:43.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: Upgrade: Setting container_image for all mgr 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi081.bcxarw"}]: dispatch 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi081.bcxarw"}]': finished 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi149.sgmtoa"}]: dispatch 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi149.sgmtoa"}]': finished 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: Upgrade: Setting container_image for all rgw 2024-03-22T07:10:43.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: Upgrade: Setting container_image for all iscsi 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: Upgrade: Setting container_image for all nfs 2024-03-22T07:10:43.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:43.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: Upgrade: Setting container_image for all nvmeof 2024-03-22T07:10:43.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:43.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: pgmap v18: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:43.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:42 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:45.437 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:45 smithi081 ceph-mon[29245]: Upgrade: Updating node-exporter.smithi081 (1/2) 2024-03-22T07:10:45.438 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:45 smithi081 ceph-mon[29245]: Deploying daemon node-exporter.smithi081 on smithi081 2024-03-22T07:10:45.438 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:45 smithi081 ceph-mon[29245]: pgmap v19: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:45.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:45 smithi149 ceph-mon[36068]: Upgrade: Updating node-exporter.smithi081 (1/2) 2024-03-22T07:10:45.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:45 smithi149 ceph-mon[36068]: Deploying daemon node-exporter.smithi081 on smithi081 2024-03-22T07:10:45.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:45 smithi149 ceph-mon[36068]: pgmap v19: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:48.000 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:47 smithi149 ceph-mon[36068]: pgmap v20: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:48.000 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:47 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:48.001 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:47 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:48.086 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:47 smithi081 ceph-mon[29245]: pgmap v20: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:48.086 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:47 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:48.086 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:47 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:50.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:49 smithi149 ceph-mon[36068]: pgmap v21: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:50.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:49 smithi149 ceph-mon[36068]: Upgrade: Updating node-exporter.smithi149 (2/2) 2024-03-22T07:10:50.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:49 smithi149 ceph-mon[36068]: Deploying daemon node-exporter.smithi149 on smithi149 2024-03-22T07:10:50.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:49 smithi081 ceph-mon[29245]: pgmap v21: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:50.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:49 smithi081 ceph-mon[29245]: Upgrade: Updating node-exporter.smithi149 (2/2) 2024-03-22T07:10:50.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:49 smithi081 ceph-mon[29245]: Deploying daemon node-exporter.smithi149 on smithi149 2024-03-22T07:10:52.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:51 smithi081 ceph-mon[29245]: pgmap v22: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:52.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:51 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:52.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:51 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:52.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:51 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:52.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:51 smithi149 ceph-mon[36068]: pgmap v22: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:52.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:51 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:52.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:51 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:52.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:51 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 -- 172.21.15.81:0/3831036244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 msgr2=0x7f4ab4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 --2- 172.21.15.81:0/3831036244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 0x7f4ab4102320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8004940 tx=0x7f4aa8009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 -- 172.21.15.81:0/3831036244 shutdown_connections 2024-03-22T07:10:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 --2- 172.21.15.81:0/3831036244 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 0x7f4ab4102320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 --2- 172.21.15.81:0/3831036244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ab40662e0 0x7f4ab40ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 -- 172.21.15.81:0/3831036244 >> 172.21.15.81:0/3831036244 conn(0x7f4ab40f7430 msgr2=0x7f4ab40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:52.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 -- 172.21.15.81:0/3831036244 shutdown_connections 2024-03-22T07:10:52.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 -- 172.21.15.81:0/3831036244 wait complete. 2024-03-22T07:10:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 Processor -- start 2024-03-22T07:10:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.130+0000 7f4ab9506700 1 -- start start 2024-03-22T07:10:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab9506700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ab40662e0 0x7f4ab40fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab9506700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 0x7f4ab40fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab9506700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab41116f0 con 0x7f4ab40fff00 2024-03-22T07:10:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab9506700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab4111860 con 0x7f4ab40662e0 2024-03-22T07:10:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ab40662e0 0x7f4ab40fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 0x7f4ab40fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ab40662e0 0x7f4ab40fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.81:58754/0 (socket says 172.21.15.81:58754) 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab27fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 0x7f4ab40fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39480/0 (socket says 172.21.15.81:39480) 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab2ffd700 1 -- 172.21.15.81:0/475998264 learned_addr learned my addr 172.21.15.81:0/475998264 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab27fc700 1 -- 172.21.15.81:0/475998264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ab40662e0 msgr2=0x7f4ab40fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab27fc700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ab40662e0 0x7f4ab40fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab27fc700 1 -- 172.21.15.81:0/475998264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa8005040 con 0x7f4ab40fff00 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.131+0000 7f4ab27fc700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 0x7f4ab40fb8c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8000c00 tx=0x7f4aa8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.132+0000 7f4a9b7fe700 1 -- 172.21.15.81:0/475998264 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4aa800e040 con 0x7f4ab40fff00 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.132+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab40fbec0 con 0x7f4ab40fff00 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.132+0000 7f4a9b7fe700 1 -- 172.21.15.81:0/475998264 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4aa800bbf0 con 0x7f4ab40fff00 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.132+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab40fc430 con 0x7f4ab40fff00 2024-03-22T07:10:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.132+0000 7f4a9b7fe700 1 -- 172.21.15.81:0/475998264 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4aa801c650 con 0x7f4ab40fff00 2024-03-22T07:10:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.133+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab405f5f0 con 0x7f4ab40fff00 2024-03-22T07:10:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.133+0000 7f4a9b7fe700 1 -- 172.21.15.81:0/475998264 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f4aa800bd60 con 0x7f4ab40fff00 2024-03-22T07:10:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.134+0000 7f4a9b7fe700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4a9c06cda0 0x7f4a9c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.134+0000 7f4a9b7fe700 1 -- 172.21.15.81:0/475998264 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4aa808df20 con 0x7f4ab40fff00 2024-03-22T07:10:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.134+0000 7f4ab2ffd700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4a9c06cda0 0x7f4a9c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:52.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.134+0000 7f4ab2ffd700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4a9c06cda0 0x7f4a9c06f250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40fd180 tx=0x7f4aa400b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:52.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.138+0000 7f4a9b7fe700 1 -- 172.21.15.81:0/475998264 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f4aa8058100 con 0x7f4ab40fff00 2024-03-22T07:10:52.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.355+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ab4105a80 con 0x7f4a9c06cda0 2024-03-22T07:10:52.357 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.357+0000 7f4a9b7fe700 1 -- 172.21.15.81:0/475998264 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f4aa400f040 con 0x7f4a9c06cda0 2024-03-22T07:10:52.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4a9c06cda0 msgr2=0x7f4a9c06f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4a9c06cda0 0x7f4a9c06f250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40fd180 tx=0x7f4aa400b040 comp rx=0 tx=0).stop 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 msgr2=0x7f4ab40fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 0x7f4ab40fb8c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8000c00 tx=0x7f4aa8009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 shutdown_connections 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4a9c06cda0 0x7f4a9c06f250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ab40662e0 0x7f4ab40fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 --2- 172.21.15.81:0/475998264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ab40fff00 0x7f4ab40fb8c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 >> 172.21.15.81:0/475998264 conn(0x7f4ab40f7430 msgr2=0x7f4ab40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 shutdown_connections 2024-03-22T07:10:52.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.360+0000 7f4ab9506700 1 -- 172.21.15.81:0/475998264 wait complete. 2024-03-22T07:10:52.375 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:10:52.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.470+0000 7f2d86038700 1 -- 172.21.15.81:0/2124839508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fe430 msgr2=0x7f2d800fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.470+0000 7f2d86038700 1 --2- 172.21.15.81:0/2124839508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fe430 0x7f2d800fe840 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2d70004940 tx=0x7f2d70009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:52.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.470+0000 7f2d86038700 1 -- 172.21.15.81:0/2124839508 shutdown_connections 2024-03-22T07:10:52.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.470+0000 7f2d86038700 1 --2- 172.21.15.81:0/2124839508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d800fed80 0x7f2d800faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.470+0000 7f2d86038700 1 --2- 172.21.15.81:0/2124839508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fe430 0x7f2d800fe840 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.470+0000 7f2d86038700 1 -- 172.21.15.81:0/2124839508 >> 172.21.15.81:0/2124839508 conn(0x7f2d800f6bd0 msgr2=0x7f2d800f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:52.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.470+0000 7f2d86038700 1 -- 172.21.15.81:0/2124839508 shutdown_connections 2024-03-22T07:10:52.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.470+0000 7f2d86038700 1 -- 172.21.15.81:0/2124839508 wait complete. 2024-03-22T07:10:52.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.471+0000 7f2d86038700 1 Processor -- start 2024-03-22T07:10:52.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.471+0000 7f2d86038700 1 -- start start 2024-03-22T07:10:52.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.471+0000 7f2d86038700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d800fe430 0x7f2d800fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.471+0000 7f2d86038700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fed80 0x7f2d800fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.471+0000 7f2d86038700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800fc630 con 0x7f2d800fed80 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.471+0000 7f2d86038700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800fc7a0 con 0x7f2d800fe430 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.471+0000 7f2d7effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fed80 0x7f2d800fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d7effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fed80 0x7f2d800fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39498/0 (socket says 172.21.15.81:39498) 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d7effd700 1 -- 172.21.15.81:0/1167941710 learned_addr learned my addr 172.21.15.81:0/1167941710 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d7f7fe700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d800fe430 0x7f2d800fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d7effd700 1 -- 172.21.15.81:0/1167941710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d800fe430 msgr2=0x7f2d800fda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d7effd700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d800fe430 0x7f2d800fda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d7effd700 1 -- 172.21.15.81:0/1167941710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70005040 con 0x7f2d800fed80 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d7effd700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fed80 0x7f2d800fc0f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400b790 tx=0x7f2d74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d67fff700 1 -- 172.21.15.81:0/1167941710 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d7400f6f0 con 0x7f2d800fed80 2024-03-22T07:10:52.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d67fff700 1 -- 172.21.15.81:0/1167941710 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d7400fd20 con 0x7f2d800fed80 2024-03-22T07:10:52.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d67fff700 1 -- 172.21.15.81:0/1167941710 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d740103a0 con 0x7f2d800fed80 2024-03-22T07:10:52.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d800fca80 con 0x7f2d800fed80 2024-03-22T07:10:52.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.472+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d80115a30 con 0x7f2d800fed80 2024-03-22T07:10:52.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.474+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8004c710 con 0x7f2d800fed80 2024-03-22T07:10:52.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.474+0000 7f2d67fff700 1 -- 172.21.15.81:0/1167941710 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f2d74011400 con 0x7f2d800fed80 2024-03-22T07:10:52.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.474+0000 7f2d67fff700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2d6806cdb0 0x7f2d6806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:52.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.474+0000 7f2d67fff700 1 -- 172.21.15.81:0/1167941710 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2d7408bf00 con 0x7f2d800fed80 2024-03-22T07:10:52.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.475+0000 7f2d7f7fe700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2d6806cdb0 0x7f2d6806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:52.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.475+0000 7f2d7f7fe700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2d6806cdb0 0x7f2d6806f260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b3b0 tx=0x7f2d70009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:52.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.479+0000 7f2d67fff700 1 -- 172.21.15.81:0/1167941710 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2d74059c80 con 0x7f2d800fed80 2024-03-22T07:10:52.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.694+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d80102480 con 0x7f2d6806cdb0 2024-03-22T07:10:52.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.696+0000 7f2d67fff700 1 -- 172.21.15.81:0/1167941710 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f2d80102480 con 0x7f2d6806cdb0 2024-03-22T07:10:52.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.697+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2d6806cdb0 msgr2=0x7f2d6806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.697+0000 7f2d86038700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2d6806cdb0 0x7f2d6806f260 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000b3b0 tx=0x7f2d70009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:52.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.697+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fed80 msgr2=0x7f2d800fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.697+0000 7f2d86038700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fed80 0x7f2d800fc0f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400b790 tx=0x7f2d74009250 comp rx=0 tx=0).stop 2024-03-22T07:10:52.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.698+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 shutdown_connections 2024-03-22T07:10:52.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.698+0000 7f2d86038700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2d6806cdb0 0x7f2d6806f260 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.698+0000 7f2d86038700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d800fe430 0x7f2d800fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.698+0000 7f2d86038700 1 --2- 172.21.15.81:0/1167941710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d800fed80 0x7f2d800fc0f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.698+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 >> 172.21.15.81:0/1167941710 conn(0x7f2d800f6bd0 msgr2=0x7f2d800f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:52.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.698+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 shutdown_connections 2024-03-22T07:10:52.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.698+0000 7f2d86038700 1 -- 172.21.15.81:0/1167941710 wait complete. 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.799+0000 7f026329c700 1 -- 172.21.15.81:0/2252184062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fcf80 msgr2=0x7f025c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.799+0000 7f026329c700 1 --2- 172.21.15.81:0/2252184062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fcf80 0x7f025c0fd3f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f024c004940 tx=0x7f024c009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.799+0000 7f026329c700 1 -- 172.21.15.81:0/2252184062 shutdown_connections 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.799+0000 7f026329c700 1 --2- 172.21.15.81:0/2252184062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fcf80 0x7f025c0fd3f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.799+0000 7f026329c700 1 --2- 172.21.15.81:0/2252184062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f025c0fbcd0 0x7f025c0fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.799+0000 7f026329c700 1 -- 172.21.15.81:0/2252184062 >> 172.21.15.81:0/2252184062 conn(0x7f025c0f7430 msgr2=0x7f025c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.799+0000 7f026329c700 1 -- 172.21.15.81:0/2252184062 shutdown_connections 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.799+0000 7f026329c700 1 -- 172.21.15.81:0/2252184062 wait complete. 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.799+0000 7f026329c700 1 Processor -- start 2024-03-22T07:10:52.801 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f026329c700 1 -- start start 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f026329c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fbcd0 0x7f025c0ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f026329c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f025c0fcf80 0x7f025c0fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f026329c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0fe870 con 0x7f025c0fbcd0 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f026329c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0fe9e0 con 0x7f025c0fcf80 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0261038700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fbcd0 0x7f025c0ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0261038700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fbcd0 0x7f025c0ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39516/0 (socket says 172.21.15.81:39516) 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0261038700 1 -- 172.21.15.81:0/3265874751 learned_addr learned my addr 172.21.15.81:0/3265874751 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0260837700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f025c0fcf80 0x7f025c0fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0261038700 1 -- 172.21.15.81:0/3265874751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f025c0fcf80 msgr2=0x7f025c0fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0261038700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f025c0fcf80 0x7f025c0fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.802 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0261038700 1 -- 172.21.15.81:0/3265874751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c005040 con 0x7f025c0fbcd0 2024-03-22T07:10:52.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0261038700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fbcd0 0x7f025c0ffdf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0258002a50 tx=0x7f02580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:52.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0251ffb700 1 -- 172.21.15.81:0/3265874751 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0258007cc0 con 0x7f025c0fbcd0 2024-03-22T07:10:52.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.800+0000 7f0251ffb700 1 -- 172.21.15.81:0/3265874751 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f025800f040 con 0x7f025c0fbcd0 2024-03-22T07:10:52.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.801+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c0fecc0 con 0x7f025c0fbcd0 2024-03-22T07:10:52.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.801+0000 7f0251ffb700 1 -- 172.21.15.81:0/3265874751 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025800da30 con 0x7f025c0fbcd0 2024-03-22T07:10:52.803 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.801+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c115a40 con 0x7f025c0fbcd0 2024-03-22T07:10:52.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.802+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c05f5f0 con 0x7f025c0fbcd0 2024-03-22T07:10:52.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.802+0000 7f0251ffb700 1 -- 172.21.15.81:0/3265874751 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f025800db90 con 0x7f025c0fbcd0 2024-03-22T07:10:52.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.802+0000 7f0251ffb700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f024806cda0 0x7f024806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:52.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.802+0000 7f0251ffb700 1 -- 172.21.15.81:0/3265874751 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f025808bfc0 con 0x7f025c0fbcd0 2024-03-22T07:10:52.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.803+0000 7f0260837700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f024806cda0 0x7f024806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:52.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.803+0000 7f0260837700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f024806cda0 0x7f024806f250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f024c00b970 tx=0x7f024c009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:52.807 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:52.807+0000 7f0251ffb700 1 -- 172.21.15.81:0/3265874751 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f0258056080 con 0x7f025c0fbcd0 2024-03-22T07:10:53.029 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.029+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f025c05e400 con 0x7f024806cda0 2024-03-22T07:10:53.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.039+0000 7f0251ffb700 1 -- 172.21.15.81:0/3265874751 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f025c05e400 con 0x7f024806cda0 2024-03-22T07:10:53.039 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:10:53.039 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (89s) 13s ago 9m 25.6M - 0.25.0 c8568f914cd2 ffe7f846ca44 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (10m) 13s ago 9m 6660k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (9m) 15s ago 9m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (73s) 13s ago 9m 82.3M - 9.4.7 954c08fa6188 c959485196e4 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (4m) 13s ago 4m 12.3M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (4m) 13s ago 4m 132M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (4m) 15s ago 4m 15.4M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (4m) 15s ago 4m 148M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (51s) 13s ago 11m 491M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (19s) 15s ago 8m 65.2M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (11m) 13s ago 11m 50.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (8m) 15s ago 8m 44.5M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 starting - - - - 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 starting - - - - 2024-03-22T07:10:53.040 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (8m) 13s ago 8m 183M 1557M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:10:53.041 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (7m) 13s ago 7m 197M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:10:53.041 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (7m) 13s ago 7m 160M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:10:53.041 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (6m) 15s ago 6m 214M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:10:53.041 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (6m) 15s ago 6m 192M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:10:53.041 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (5m) 15s ago 5m 181M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:10:53.041 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (24s) 13s ago 9m 38.7M - 2.43.0 a07b618ecd1d c4c318a3f03b 2024-03-22T07:10:53.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f024806cda0 msgr2=0x7f024806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f024806cda0 0x7f024806f250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f024c00b970 tx=0x7f024c009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:53.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fbcd0 msgr2=0x7f025c0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fbcd0 0x7f025c0ffdf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0258002a50 tx=0x7f02580046b0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 shutdown_connections 2024-03-22T07:10:53.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f024806cda0 0x7f024806f250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f025c0fbcd0 0x7f025c0ffdf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 --2- 172.21.15.81:0/3265874751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f025c0fcf80 0x7f025c0fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 >> 172.21.15.81:0/3265874751 conn(0x7f025c0f7430 msgr2=0x7f025c065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:53.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 shutdown_connections 2024-03-22T07:10:53.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.041+0000 7f026329c700 1 -- 172.21.15.81:0/3265874751 wait complete. 2024-03-22T07:10:53.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.142+0000 7f40d2477700 1 -- 172.21.15.81:0/1108680378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 msgr2=0x7f40cc0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.142+0000 7f40d2477700 1 --2- 172.21.15.81:0/1108680378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 0x7f40cc0fd700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f40bc004940 tx=0x7f40bc009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:53.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.142+0000 7f40d2477700 1 -- 172.21.15.81:0/1108680378 shutdown_connections 2024-03-22T07:10:53.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.142+0000 7f40d2477700 1 --2- 172.21.15.81:0/1108680378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40cc0fdc40 0x7f40cc0fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.142+0000 7f40d2477700 1 --2- 172.21.15.81:0/1108680378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 0x7f40cc0fd700 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.142+0000 7f40d2477700 1 -- 172.21.15.81:0/1108680378 >> 172.21.15.81:0/1108680378 conn(0x7f40cc0f6bd0 msgr2=0x7f40cc0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.142+0000 7f40d2477700 1 -- 172.21.15.81:0/1108680378 shutdown_connections 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.142+0000 7f40d2477700 1 -- 172.21.15.81:0/1108680378 wait complete. 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40d2477700 1 Processor -- start 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40d2477700 1 -- start start 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40d2477700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 0x7f40cc103d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40d2477700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40cc0fdc40 0x7f40cc1023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40d2477700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc104310 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40d2477700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc102900 con 0x7f40cc0fdc40 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40cbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 0x7f40cc103d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40cbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 0x7f40cc103d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39544/0 (socket says 172.21.15.81:39544) 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40cbfff700 1 -- 172.21.15.81:0/1447292332 learned_addr learned my addr 172.21.15.81:0/1447292332 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40cb7fe700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40cc0fdc40 0x7f40cc1023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:53.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40cbfff700 1 -- 172.21.15.81:0/1447292332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40cc0fdc40 msgr2=0x7f40cc1023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40cbfff700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40cc0fdc40 0x7f40cc1023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.143+0000 7f40cbfff700 1 -- 172.21.15.81:0/1447292332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc005040 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.144+0000 7f40cbfff700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 0x7f40cc103d10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00b3b0 tx=0x7f40bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:53.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.144+0000 7f40c8ff9700 1 -- 172.21.15.81:0/1447292332 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40bc00e040 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.144+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc102b50 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.144+0000 7f40c8ff9700 1 -- 172.21.15.81:0/1447292332 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40bc00bbf0 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.144+0000 7f40c8ff9700 1 -- 172.21.15.81:0/1447292332 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40bc01c630 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.144+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc103110 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.145+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc04c710 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.145+0000 7f40c8ff9700 1 -- 172.21.15.81:0/1447292332 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f40bc00bd60 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.146+0000 7f40c8ff9700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f40b406cdb0 0x7f40b406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:53.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.146+0000 7f40c8ff9700 1 -- 172.21.15.81:0/1447292332 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f40bc08ce10 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.146+0000 7f40cb7fe700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f40b406cdb0 0x7f40b406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:53.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.146+0000 7f40cb7fe700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f40b406cdb0 0x7f40b406f260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0662a0 tx=0x7f40c000b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:53.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.150+0000 7f40c8ff9700 1 -- 172.21.15.81:0/1447292332 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f40bc05aa80 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.432+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f40cc115be0 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.432+0000 7f40c8ff9700 1 -- 172.21.15.81:0/1447292332 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f40bc05a610 con 0x7f40cc0fd2f0 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:10:53.433 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:10:53.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:10:53.434 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:10:53.434 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:10:53.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:10:53.434 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:10:53.434 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T07:10:53.434 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:10:53.434 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:10:53.434 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:10:53.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f40b406cdb0 msgr2=0x7f40b406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f40b406cdb0 0x7f40b406f260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0662a0 tx=0x7f40c000b040 comp rx=0 tx=0).stop 2024-03-22T07:10:53.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 msgr2=0x7f40cc103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 0x7f40cc103d10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00b3b0 tx=0x7f40bc009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:53.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 shutdown_connections 2024-03-22T07:10:53.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f40b406cdb0 0x7f40b406f260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc0fd2f0 0x7f40cc103d10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 --2- 172.21.15.81:0/1447292332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40cc0fdc40 0x7f40cc1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 >> 172.21.15.81:0/1447292332 conn(0x7f40cc0f6bd0 msgr2=0x7f40cc0f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:53.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 shutdown_connections 2024-03-22T07:10:53.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.435+0000 7f40d2477700 1 -- 172.21.15.81:0/1447292332 wait complete. 2024-03-22T07:10:53.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 -- 172.21.15.81:0/1311189212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 msgr2=0x7fabc00ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 --2- 172.21.15.81:0/1311189212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 0x7fabc00ffa20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fabb4004940 tx=0x7fabb4009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 -- 172.21.15.81:0/1311189212 shutdown_connections 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 --2- 172.21.15.81:0/1311189212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc00fff60 0x7fabc0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 --2- 172.21.15.81:0/1311189212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 0x7fabc00ffa20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 -- 172.21.15.81:0/1311189212 >> 172.21.15.81:0/1311189212 conn(0x7fabc00f7450 msgr2=0x7fabc00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 -- 172.21.15.81:0/1311189212 shutdown_connections 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 -- 172.21.15.81:0/1311189212 wait complete. 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 Processor -- start 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.540+0000 7fabc6e54700 1 -- start start 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc6e54700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 0x7fabc010fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc6e54700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc00fff60 0x7fabc0110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:53.542 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc6e54700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc0110620 con 0x7fabc0066f50 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc6e54700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc0110790 con 0x7fabc00fff60 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc4bf0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 0x7fabc010fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc4bf0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 0x7fabc010fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39566/0 (socket says 172.21.15.81:39566) 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc4bf0700 1 -- 172.21.15.81:0/362978984 learned_addr learned my addr 172.21.15.81:0/362978984 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabbffff700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc00fff60 0x7fabc0110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc4bf0700 1 -- 172.21.15.81:0/362978984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc00fff60 msgr2=0x7fabc0110020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc4bf0700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc00fff60 0x7fabc0110020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc4bf0700 1 -- 172.21.15.81:0/362978984 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb4005040 con 0x7fabc0066f50 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc4bf0700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 0x7fabc010fae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fabb4000c00 tx=0x7fabb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabbd7fa700 1 -- 172.21.15.81:0/362978984 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb400e030 con 0x7fabc0066f50 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabbd7fa700 1 -- 172.21.15.81:0/362978984 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabb4013de0 con 0x7fabc0066f50 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc0115190 con 0x7fabc0066f50 2024-03-22T07:10:53.543 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabbd7fa700 1 -- 172.21.15.81:0/362978984 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb401ce50 con 0x7fabc0066f50 2024-03-22T07:10:53.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.541+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc01156d0 con 0x7fabc0066f50 2024-03-22T07:10:53.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.542+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc004c710 con 0x7fabc0066f50 2024-03-22T07:10:53.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.543+0000 7fabbd7fa700 1 -- 172.21.15.81:0/362978984 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fabb401c400 con 0x7fabc0066f50 2024-03-22T07:10:53.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.543+0000 7fabbd7fa700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fabb006cd40 0x7fabb006f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:53.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.543+0000 7fabbd7fa700 1 -- 172.21.15.81:0/362978984 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fabb408dbe0 con 0x7fabc0066f50 2024-03-22T07:10:53.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.543+0000 7fabbffff700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fabb006cd40 0x7fabb006f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:53.545 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.544+0000 7fabbffff700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fabb006cd40 0x7fabb006f1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fabac009b80 tx=0x7fabac00b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:53.548 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.548+0000 7fabbd7fa700 1 -- 172.21.15.81:0/362978984 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fabb405b7d0 con 0x7fabc0066f50 2024-03-22T07:10:53.781 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.781+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabc00fe9d0 con 0x7fabb006cd40 2024-03-22T07:10:53.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.782+0000 7fabbd7fa700 1 -- 172.21.15.81:0/362978984 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fabc00fe9d0 con 0x7fabb006cd40 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr" 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "2/2 daemons upgraded", 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:10:53.783 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:10:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.785+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fabb006cd40 msgr2=0x7fabb006f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.785+0000 7fabc6e54700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fabb006cd40 0x7fabb006f1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fabac009b80 tx=0x7fabac00b040 comp rx=0 tx=0).stop 2024-03-22T07:10:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.785+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 msgr2=0x7fabc010fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.785+0000 7fabc6e54700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 0x7fabc010fae0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fabb4000c00 tx=0x7fabb4009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.786+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 shutdown_connections 2024-03-22T07:10:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.786+0000 7fabc6e54700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fabb006cd40 0x7fabb006f1f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.786+0000 7fabc6e54700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc0066f50 0x7fabc010fae0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.786+0000 7fabc6e54700 1 --2- 172.21.15.81:0/362978984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabc00fff60 0x7fabc0110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:53.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.786+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 >> 172.21.15.81:0/362978984 conn(0x7fabc00f7450 msgr2=0x7fabc00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:53.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.786+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 shutdown_connections 2024-03-22T07:10:53.787 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:10:53.786+0000 7fabc6e54700 1 -- 172.21.15.81:0/362978984 wait complete. 2024-03-22T07:10:54.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:53 smithi081 ceph-mon[29245]: pgmap v23: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:54.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:53 smithi081 ceph-mon[29245]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:54.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:53 smithi081 ceph-mon[29245]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:54.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:53 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1447292332' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:54.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:53 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:54.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:53 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:53 smithi149 ceph-mon[36068]: pgmap v23: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:53 smithi149 ceph-mon[36068]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:53 smithi149 ceph-mon[36068]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:53 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1447292332' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:10:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:53 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:54.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:53 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:55.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:54 smithi081 ceph-mon[29245]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:55.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:54 smithi081 ceph-mon[29245]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:55.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:54 smithi149 ceph-mon[36068]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:55.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:54 smithi149 ceph-mon[36068]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:56.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:55 smithi081 ceph-mon[29245]: pgmap v24: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:56.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:55 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:56.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:55 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:56.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:55 smithi149 ceph-mon[36068]: pgmap v24: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:56.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:55 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:56.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:55 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:57.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:57 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:57.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:57 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:57.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:57 smithi081 ceph-mon[29245]: pgmap v25: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:57.582 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:57 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:57.582 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:57 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:57.582 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:57 smithi149 ceph-mon[36068]: pgmap v25: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:58.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:58 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:58.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:58 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:58.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:58 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:58.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:58 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:58.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:58 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:58.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:58 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:59.562 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:59 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:59.563 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:59 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:59.563 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:10:59 smithi081 ceph-mon[29245]: pgmap v26: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:10:59.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:59 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:59.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:59 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:10:59.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:10:59 smithi149 ceph-mon[36068]: pgmap v26: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:01.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: pgmap v27: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:01.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:01.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:01.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:01.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:01 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: pgmap v27: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.669 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.670 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.670 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:01.670 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:01 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:03.635 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:03 smithi081 ceph-mon[29245]: pgmap v28: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:03.635 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:03 smithi081 ceph-mon[29245]: Upgrade: Updating prometheus.smithi081 2024-03-22T07:11:03.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:03 smithi149 ceph-mon[36068]: pgmap v28: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:03.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:03 smithi149 ceph-mon[36068]: Upgrade: Updating prometheus.smithi081 2024-03-22T07:11:04.769 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:04 smithi081 ceph-mon[29245]: Deploying daemon prometheus.smithi081 on smithi081 2024-03-22T07:11:04.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:04 smithi149 ceph-mon[36068]: Deploying daemon prometheus.smithi081 on smithi081 2024-03-22T07:11:05.447 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:05 smithi081 ceph-mon[29245]: pgmap v29: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:05.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:05 smithi149 ceph-mon[36068]: pgmap v29: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:07.467 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:07 smithi081 ceph-mon[29245]: pgmap v30: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:07.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:07 smithi149 ceph-mon[36068]: pgmap v30: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:09 smithi081 ceph-mon[29245]: pgmap v31: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:09 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:09 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:09.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:09 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:09 smithi149 ceph-mon[36068]: pgmap v31: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:09 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:09 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:09 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:12.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:11 smithi081 ceph-mon[29245]: pgmap v32: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:12.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:11 smithi149 ceph-mon[36068]: pgmap v32: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:13.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:12 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:13.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:12 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:14.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:13 smithi081 ceph-mon[29245]: pgmap v33: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:14.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:13 smithi149 ceph-mon[36068]: pgmap v33: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:15.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:14 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:15.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:14 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:14 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:14 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:16.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:15 smithi081 ceph-mon[29245]: pgmap v34: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:15 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:15 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:16.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:15 smithi149 ceph-mon[36068]: pgmap v34: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:16.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:15 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:16.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:15 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:18.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:17 smithi081 ceph-mon[29245]: pgmap v35: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:17 smithi149 ceph-mon[36068]: pgmap v35: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:19.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: pgmap v36: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:19.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:19.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:19.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:19.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:19.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:19 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: pgmap v36: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:19.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:19.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:19.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:19 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:21.584 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:21 smithi081 ceph-mon[29245]: pgmap v37: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:21.585 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:21 smithi081 ceph-mon[29245]: Upgrade: Updating alertmanager.smithi081 2024-03-22T07:11:21.585 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:21 smithi081 ceph-mon[29245]: Deploying daemon alertmanager.smithi081 on smithi081 2024-03-22T07:11:21.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:21 smithi149 ceph-mon[36068]: pgmap v37: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:21.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:21 smithi149 ceph-mon[36068]: Upgrade: Updating alertmanager.smithi081 2024-03-22T07:11:21.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:21 smithi149 ceph-mon[36068]: Deploying daemon alertmanager.smithi081 on smithi081 2024-03-22T07:11:23.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:23 smithi149 ceph-mon[36068]: pgmap v38: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:23.876 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:23 smithi081 ceph-mon[29245]: pgmap v38: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:23.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.901+0000 7fe38f66f700 1 -- 172.21.15.81:0/1747275498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fcf80 msgr2=0x7fe3880fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.901+0000 7fe38f66f700 1 --2- 172.21.15.81:0/1747275498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fcf80 0x7fe3880fd3f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe378004940 tx=0x7fe378009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.901+0000 7fe38f66f700 1 -- 172.21.15.81:0/1747275498 shutdown_connections 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.901+0000 7fe38f66f700 1 --2- 172.21.15.81:0/1747275498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fcf80 0x7fe3880fd3f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.901+0000 7fe38f66f700 1 --2- 172.21.15.81:0/1747275498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3880fbcd0 0x7fe3880fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.901+0000 7fe38f66f700 1 -- 172.21.15.81:0/1747275498 >> 172.21.15.81:0/1747275498 conn(0x7fe3880f7430 msgr2=0x7fe3880f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.901+0000 7fe38f66f700 1 -- 172.21.15.81:0/1747275498 shutdown_connections 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.901+0000 7fe38f66f700 1 -- 172.21.15.81:0/1747275498 wait complete. 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38f66f700 1 Processor -- start 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38f66f700 1 -- start start 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38f66f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fbcd0 0x7fe388066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38f66f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3880fcf80 0x7fe388112760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38f66f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe388112ca0 con 0x7fe3880fbcd0 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38f66f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe388112de0 con 0x7fe3880fcf80 2024-03-22T07:11:23.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38d40b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fbcd0 0x7fe388066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38d40b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fbcd0 0x7fe388066cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35300/0 (socket says 172.21.15.81:35300) 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38d40b700 1 -- 172.21.15.81:0/2257655205 learned_addr learned my addr 172.21.15.81:0/2257655205 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38cc0a700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3880fcf80 0x7fe388112760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38d40b700 1 -- 172.21.15.81:0/2257655205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3880fcf80 msgr2=0x7fe388112760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38d40b700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3880fcf80 0x7fe388112760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.902+0000 7fe38d40b700 1 -- 172.21.15.81:0/2257655205 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe378005040 con 0x7fe3880fbcd0 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.903+0000 7fe38d40b700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fbcd0 0x7fe388066cf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe38400cb10 tx=0x7fe384007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.903+0000 7fe37dffb700 1 -- 172.21.15.81:0/2257655205 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe384004200 con 0x7fe3880fbcd0 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.903+0000 7fe37dffb700 1 -- 172.21.15.81:0/2257655205 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe38400acc0 con 0x7fe3880fbcd0 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.903+0000 7fe37dffb700 1 -- 172.21.15.81:0/2257655205 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe384003d90 con 0x7fe3880fbcd0 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.903+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe388112fe0 con 0x7fe3880fbcd0 2024-03-22T07:11:23.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.903+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe388113570 con 0x7fe3880fbcd0 2024-03-22T07:11:23.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.904+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38804c710 con 0x7fe3880fbcd0 2024-03-22T07:11:23.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.904+0000 7fe37dffb700 1 -- 172.21.15.81:0/2257655205 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fe384003430 con 0x7fe3880fbcd0 2024-03-22T07:11:23.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.905+0000 7fe37dffb700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe37406cdb0 0x7fe37406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:23.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.905+0000 7fe38cc0a700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe37406cdb0 0x7fe37406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:23.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.905+0000 7fe37dffb700 1 -- 172.21.15.81:0/2257655205 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe38408c6d0 con 0x7fe3880fbcd0 2024-03-22T07:11:23.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.905+0000 7fe38cc0a700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe37406cdb0 0x7fe37406f260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b3b0 tx=0x7fe378009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:23.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:23.909+0000 7fe37dffb700 1 -- 172.21.15.81:0/2257655205 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe38405a3d0 con 0x7fe3880fbcd0 2024-03-22T07:11:24.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.132+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3881011c0 con 0x7fe37406cdb0 2024-03-22T07:11:24.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.133+0000 7fe37dffb700 1 -- 172.21.15.81:0/2257655205 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fe3881011c0 con 0x7fe37406cdb0 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.135+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe37406cdb0 msgr2=0x7fe37406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.135+0000 7fe38f66f700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe37406cdb0 0x7fe37406f260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b3b0 tx=0x7fe378009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.135+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fbcd0 msgr2=0x7fe388066cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.135+0000 7fe38f66f700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fbcd0 0x7fe388066cf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe38400cb10 tx=0x7fe384007f90 comp rx=0 tx=0).stop 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.135+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 shutdown_connections 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.135+0000 7fe38f66f700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe37406cdb0 0x7fe37406f260 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.135+0000 7fe38f66f700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe3880fbcd0 0x7fe388066cf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.135+0000 7fe38f66f700 1 --2- 172.21.15.81:0/2257655205 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe3880fcf80 0x7fe388112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.135+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 >> 172.21.15.81:0/2257655205 conn(0x7fe3880f7430 msgr2=0x7fe3880f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.136+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 shutdown_connections 2024-03-22T07:11:24.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.136+0000 7fe38f66f700 1 -- 172.21.15.81:0/2257655205 wait complete. 2024-03-22T07:11:24.146 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.250+0000 7fb6dbdec700 1 -- 172.21.15.81:0/3710716663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 msgr2=0x7fb6d4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.250+0000 7fb6dbdec700 1 --2- 172.21.15.81:0/3710716663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 0x7fb6d4070330 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4004940 tx=0x7fb6c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.250+0000 7fb6dbdec700 1 -- 172.21.15.81:0/3710716663 shutdown_connections 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.250+0000 7fb6dbdec700 1 --2- 172.21.15.81:0/3710716663 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d4070870 0x7fb6d4070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.250+0000 7fb6dbdec700 1 --2- 172.21.15.81:0/3710716663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 0x7fb6d4070330 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.250+0000 7fb6dbdec700 1 -- 172.21.15.81:0/3710716663 >> 172.21.15.81:0/3710716663 conn(0x7fb6d40f8550 msgr2=0x7fb6d40fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.251+0000 7fb6dbdec700 1 -- 172.21.15.81:0/3710716663 shutdown_connections 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.251+0000 7fb6dbdec700 1 -- 172.21.15.81:0/3710716663 wait complete. 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.251+0000 7fb6dbdec700 1 Processor -- start 2024-03-22T07:11:24.254 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.251+0000 7fb6dbdec700 1 -- start start 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6dbdec700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d4070870 0x7fb6d410fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6dbdec700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 0x7fb6d41100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6dbdec700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4110750 con 0x7fb6d4071e20 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6dbdec700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d41150f0 con 0x7fb6d4070870 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6d9387700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 0x7fb6d41100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6d9387700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 0x7fb6d41100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35310/0 (socket says 172.21.15.81:35310) 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6d9387700 1 -- 172.21.15.81:0/878181200 learned_addr learned my addr 172.21.15.81:0/878181200 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6d9b88700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d4070870 0x7fb6d410fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6d9387700 1 -- 172.21.15.81:0/878181200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d4070870 msgr2=0x7fb6d410fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6d9387700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d4070870 0x7fb6d410fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6d9387700 1 -- 172.21.15.81:0/878181200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c4005040 con 0x7fb6d4071e20 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6d9387700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 0x7fb6d41100c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000b790 tx=0x7fb6d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6ca7fc700 1 -- 172.21.15.81:0/878181200 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d000f6f0 con 0x7fb6d4071e20 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d4115350 con 0x7fb6d4071e20 2024-03-22T07:11:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6ca7fc700 1 -- 172.21.15.81:0/878181200 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6d000fd20 con 0x7fb6d4071e20 2024-03-22T07:11:24.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6ca7fc700 1 -- 172.21.15.81:0/878181200 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d00103a0 con 0x7fb6d4071e20 2024-03-22T07:11:24.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.252+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d41158c0 con 0x7fb6d4071e20 2024-03-22T07:11:24.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.254+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d404c710 con 0x7fb6d4071e20 2024-03-22T07:11:24.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.254+0000 7fb6ca7fc700 1 -- 172.21.15.81:0/878181200 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fb6d0011400 con 0x7fb6d4071e20 2024-03-22T07:11:24.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.254+0000 7fb6ca7fc700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fb6c006cdb0 0x7fb6c006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:24.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.254+0000 7fb6ca7fc700 1 -- 172.21.15.81:0/878181200 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb6d008bf00 con 0x7fb6d4071e20 2024-03-22T07:11:24.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.254+0000 7fb6d9b88700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fb6c006cdb0 0x7fb6c006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:24.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.255+0000 7fb6d9b88700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fb6c006cdb0 0x7fb6c006f260 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400b970 tx=0x7fb6c4009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:24.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.259+0000 7fb6ca7fc700 1 -- 172.21.15.81:0/878181200 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb6d0059c80 con 0x7fb6d4071e20 2024-03-22T07:11:24.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.479+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6d4103e40 con 0x7fb6c006cdb0 2024-03-22T07:11:24.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.481+0000 7fb6ca7fc700 1 -- 172.21.15.81:0/878181200 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fb6d4103e40 con 0x7fb6c006cdb0 2024-03-22T07:11:24.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fb6c006cdb0 msgr2=0x7fb6c006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fb6c006cdb0 0x7fb6c006f260 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400b970 tx=0x7fb6c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:24.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 msgr2=0x7fb6d41100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 0x7fb6d41100c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000b790 tx=0x7fb6d0009250 comp rx=0 tx=0).stop 2024-03-22T07:11:24.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 shutdown_connections 2024-03-22T07:11:24.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fb6c006cdb0 0x7fb6c006f260 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6d4070870 0x7fb6d410fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 --2- 172.21.15.81:0/878181200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6d4071e20 0x7fb6d41100c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 >> 172.21.15.81:0/878181200 conn(0x7fb6d40f8550 msgr2=0x7fb6d40f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:24.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 shutdown_connections 2024-03-22T07:11:24.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.483+0000 7fb6dbdec700 1 -- 172.21.15.81:0/878181200 wait complete. 2024-03-22T07:11:24.587 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.585+0000 7f3a08e79700 1 -- 172.21.15.81:0/814891232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fcf90 msgr2=0x7f3a040fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.585+0000 7f3a08e79700 1 --2- 172.21.15.81:0/814891232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fcf90 0x7f3a040fd400 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f39f8004940 tx=0x7f39f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.585+0000 7f3a08e79700 1 -- 172.21.15.81:0/814891232 shutdown_connections 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.585+0000 7f3a08e79700 1 --2- 172.21.15.81:0/814891232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fcf90 0x7f3a040fd400 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.585+0000 7f3a08e79700 1 --2- 172.21.15.81:0/814891232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a040fbce0 0x7f3a040fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.585+0000 7f3a08e79700 1 -- 172.21.15.81:0/814891232 >> 172.21.15.81:0/814891232 conn(0x7f3a040f7440 msgr2=0x7f3a040f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a08e79700 1 -- 172.21.15.81:0/814891232 shutdown_connections 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a08e79700 1 -- 172.21.15.81:0/814891232 wait complete. 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a08e79700 1 Processor -- start 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a08e79700 1 -- start start 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a08e79700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fbce0 0x7f3a0410c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a08e79700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a040fcf90 0x7f3a0410ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:24.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a08e79700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0410d470 con 0x7f3a040fbce0 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a08e79700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0410dcc0 con 0x7f3a040fcf90 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a0259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fbce0 0x7f3a0410c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a0259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fbce0 0x7f3a0410c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35316/0 (socket says 172.21.15.81:35316) 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.586+0000 7f3a0259c700 1 -- 172.21.15.81:0/2429321473 learned_addr learned my addr 172.21.15.81:0/2429321473 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.587+0000 7f3a0259c700 1 -- 172.21.15.81:0/2429321473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a040fcf90 msgr2=0x7f3a0410ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.587+0000 7f3a0259c700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a040fcf90 0x7f3a0410ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.587+0000 7f3a0259c700 1 -- 172.21.15.81:0/2429321473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f8005040 con 0x7f3a040fbce0 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.587+0000 7f3a0259c700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fbce0 0x7f3a0410c930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f39f400c8c0 tx=0x7f39f40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.587+0000 7f39f2ffd700 1 -- 172.21.15.81:0/2429321473 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39f400a980 con 0x7f3a040fbce0 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.587+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0410dfa0 con 0x7f3a040fbce0 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.587+0000 7f39f2ffd700 1 -- 172.21.15.81:0/2429321473 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39f400ce60 con 0x7f3a040fbce0 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.587+0000 7f39f2ffd700 1 -- 172.21.15.81:0/2429321473 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39f40115e0 con 0x7f3a040fbce0 2024-03-22T07:11:24.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.587+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0410e4e0 con 0x7f3a040fbce0 2024-03-22T07:11:24.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.588+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0405f5f0 con 0x7f3a040fbce0 2024-03-22T07:11:24.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.589+0000 7f39f2ffd700 1 -- 172.21.15.81:0/2429321473 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f39f4012400 con 0x7f3a040fbce0 2024-03-22T07:11:24.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.589+0000 7f39f2ffd700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f39ec06efb0 0x7f39ec071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:24.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.589+0000 7f39f2ffd700 1 -- 172.21.15.81:0/2429321473 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f39f408c1f0 con 0x7f3a040fbce0 2024-03-22T07:11:24.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.589+0000 7f3a01d9b700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f39ec06efb0 0x7f39ec071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:24.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.589+0000 7f3a01d9b700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f39ec06efb0 0x7f39ec071460 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f39f8004f90 tx=0x7f39f8009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:24.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.595+0000 7f39f2ffd700 1 -- 172.21.15.81:0/2429321473 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f39f4059f70 con 0x7f3a040fbce0 2024-03-22T07:11:24.804 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.803+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a0410e8d0 con 0x7f39ec06efb0 2024-03-22T07:11:24.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.813+0000 7f39f2ffd700 1 -- 172.21.15.81:0/2429321473 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f3a0410e8d0 con 0x7f39ec06efb0 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (2m) 11s ago 10m 25.6M - 0.25.0 c8568f914cd2 ffe7f846ca44 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (10m) 11s ago 10m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (9m) 31s ago 9m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (104s) 11s ago 10m 80.9M - 9.4.7 954c08fa6188 c959485196e4 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (4m) 11s ago 4m 12.5M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (4m) 11s ago 4m 132M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (4m) 31s ago 4m 15.5M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (4m) 31s ago 4m 148M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (83s) 11s ago 11m 496M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (51s) 31s ago 9m 447M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (11m) 11s ago 11m 52.4M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:11:24.814 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (9m) 31s ago 9m 44.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:11:24.815 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (38s) 11s ago 10m 8296k - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:11:24.815 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (34s) 31s ago 9m 3640k - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:11:24.815 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (8m) 11s ago 8m 183M 1557M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:11:24.815 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (8m) 11s ago 8m 198M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:11:24.815 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (7m) 11s ago 7m 160M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:11:24.815 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (7m) 31s ago 7m 214M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:11:24.815 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (6m) 31s ago 6m 191M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:11:24.815 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (6m) 31s ago 6m 181M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:11:24.815 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (16s) 11s ago 9m 40.1M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:11:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f39ec06efb0 msgr2=0x7f39ec071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f39ec06efb0 0x7f39ec071460 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f39f8004f90 tx=0x7f39f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fbce0 msgr2=0x7f3a0410c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fbce0 0x7f3a0410c930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f39f400c8c0 tx=0x7f39f40044c0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 shutdown_connections 2024-03-22T07:11:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f39ec06efb0 0x7f39ec071460 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a040fbce0 0x7f3a0410c930 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 --2- 172.21.15.81:0/2429321473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a040fcf90 0x7f3a0410ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 >> 172.21.15.81:0/2429321473 conn(0x7f3a040f7440 msgr2=0x7f3a040f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 shutdown_connections 2024-03-22T07:11:24.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.815+0000 7f3a08e79700 1 -- 172.21.15.81:0/2429321473 wait complete. 2024-03-22T07:11:24.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.915+0000 7fe565b6e700 1 -- 172.21.15.81:0/1718501989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 msgr2=0x7fe5600fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.915+0000 7fe565b6e700 1 --2- 172.21.15.81:0/1718501989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 0x7fe5600fb650 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe550004940 tx=0x7fe550009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:24.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.915+0000 7fe565b6e700 1 -- 172.21.15.81:0/1718501989 shutdown_connections 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.915+0000 7fe565b6e700 1 --2- 172.21.15.81:0/1718501989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5600fbb90 0x7fe5601023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.915+0000 7fe565b6e700 1 --2- 172.21.15.81:0/1718501989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 0x7fe5600fb650 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.915+0000 7fe565b6e700 1 -- 172.21.15.81:0/1718501989 >> 172.21.15.81:0/1718501989 conn(0x7fe5600f6bc0 msgr2=0x7fe5600f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.915+0000 7fe565b6e700 1 -- 172.21.15.81:0/1718501989 shutdown_connections 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.915+0000 7fe565b6e700 1 -- 172.21.15.81:0/1718501989 wait complete. 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe565b6e700 1 Processor -- start 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe565b6e700 1 -- start start 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe565b6e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 0x7fe56010d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe565b6e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5600fbb90 0x7fe56010ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe565b6e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56010e3d0 con 0x7fe5600fb240 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe565b6e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56010e540 con 0x7fe5600fbb90 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe55f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 0x7fe56010d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe55f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 0x7fe56010d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58536/0 (socket says 172.21.15.81:58536) 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe55f7fe700 1 -- 172.21.15.81:0/1722095471 learned_addr learned my addr 172.21.15.81:0/1722095471 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe557fff700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5600fbb90 0x7fe56010ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe55f7fe700 1 -- 172.21.15.81:0/1722095471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5600fbb90 msgr2=0x7fe56010ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe55f7fe700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5600fbb90 0x7fe56010ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe55f7fe700 1 -- 172.21.15.81:0/1722095471 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe550005040 con 0x7fe5600fb240 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.916+0000 7fe557fff700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5600fbb90 0x7fe56010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.917+0000 7fe55f7fe700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 0x7fe56010d890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe5500043d0 tx=0x7fe550009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.917+0000 7fe55cff9700 1 -- 172.21.15.81:0/1722095471 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe550020070 con 0x7fe5600fb240 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.917+0000 7fe55cff9700 1 -- 172.21.15.81:0/1722095471 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe55001c400 con 0x7fe5600fb240 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.917+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe560112f40 con 0x7fe5600fb240 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.917+0000 7fe55cff9700 1 -- 172.21.15.81:0/1722095471 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe55000e070 con 0x7fe5600fb240 2024-03-22T07:11:24.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.917+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5601133a0 con 0x7fe5600fb240 2024-03-22T07:11:24.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.918+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56004c710 con 0x7fe5600fb240 2024-03-22T07:11:24.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.918+0000 7fe55cff9700 1 -- 172.21.15.81:0/1722095471 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fe55001c9e0 con 0x7fe5600fb240 2024-03-22T07:11:24.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.919+0000 7fe55cff9700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe54c06cdb0 0x7fe54c06f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:24.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.919+0000 7fe55cff9700 1 -- 172.21.15.81:0/1722095471 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe55008d4d0 con 0x7fe5600fb240 2024-03-22T07:11:24.920 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.919+0000 7fe557fff700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe54c06cdb0 0x7fe54c06f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:24.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.919+0000 7fe557fff700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe54c06cdb0 0x7fe54c06f260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe548009fd0 tx=0x7fe54800b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:24.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:24.923+0000 7fe55cff9700 1 -- 172.21.15.81:0/1722095471 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe55005b140 con 0x7fe5600fb240 2024-03-22T07:11:25.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.201+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe5601138d0 con 0x7fe5600fb240 2024-03-22T07:11:25.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.202+0000 7fe55cff9700 1 -- 172.21.15.81:0/1722095471 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe550004490 con 0x7fe5600fb240 2024-03-22T07:11:25.202 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:11:25.202 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:11:25.202 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:11:25.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:11:25.202 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:11:25.202 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:11:25.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:11:25.202 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:11:25.203 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.203+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe54c06cdb0 msgr2=0x7fe54c06f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.203+0000 7fe565b6e700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe54c06cdb0 0x7fe54c06f260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe548009fd0 tx=0x7fe54800b040 comp rx=0 tx=0).stop 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.203+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 msgr2=0x7fe56010d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.203+0000 7fe565b6e700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 0x7fe56010d890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe5500043d0 tx=0x7fe550009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.203+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 shutdown_connections 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.204+0000 7fe565b6e700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe54c06cdb0 0x7fe54c06f260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.204+0000 7fe565b6e700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5600fb240 0x7fe56010d890 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.204+0000 7fe565b6e700 1 --2- 172.21.15.81:0/1722095471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5600fbb90 0x7fe56010ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.204+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 >> 172.21.15.81:0/1722095471 conn(0x7fe5600f6bc0 msgr2=0x7fe5601003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.204+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 shutdown_connections 2024-03-22T07:11:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.204+0000 7fe565b6e700 1 -- 172.21.15.81:0/1722095471 wait complete. 2024-03-22T07:11:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 -- 172.21.15.81:0/1115977673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 msgr2=0x7efe780fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 --2- 172.21.15.81:0/1115977673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 0x7efe780fe850 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efe6c004940 tx=0x7efe6c009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 -- 172.21.15.81:0/1115977673 shutdown_connections 2024-03-22T07:11:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 --2- 172.21.15.81:0/1115977673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe780fed90 0x7efe780faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 --2- 172.21.15.81:0/1115977673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 0x7efe780fe850 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 -- 172.21.15.81:0/1115977673 >> 172.21.15.81:0/1115977673 conn(0x7efe780f6be0 msgr2=0x7efe780f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 -- 172.21.15.81:0/1115977673 shutdown_connections 2024-03-22T07:11:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 -- 172.21.15.81:0/1115977673 wait complete. 2024-03-22T07:11:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 Processor -- start 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.305+0000 7efe7e373700 1 -- start start 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe7e373700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 0x7efe7810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe7e373700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe780fed90 0x7efe7810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe7e373700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7810e480 con 0x7efe780fe440 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe7e373700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7810e5f0 con 0x7efe780fed90 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe77fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 0x7efe7810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe77fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 0x7efe7810d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58550/0 (socket says 172.21.15.81:58550) 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe77fff700 1 -- 172.21.15.81:0/1842476316 learned_addr learned my addr 172.21.15.81:0/1842476316 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe777fe700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe780fed90 0x7efe7810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe77fff700 1 -- 172.21.15.81:0/1842476316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe780fed90 msgr2=0x7efe7810de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe77fff700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe780fed90 0x7efe7810de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe77fff700 1 -- 172.21.15.81:0/1842476316 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c005040 con 0x7efe780fe440 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe77fff700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 0x7efe7810d940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00b3b0 tx=0x7efe6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe74ff9700 1 -- 172.21.15.81:0/1842476316 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe6c00e040 con 0x7efe780fe440 2024-03-22T07:11:25.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe78112ff0 con 0x7efe780fe440 2024-03-22T07:11:25.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe74ff9700 1 -- 172.21.15.81:0/1842476316 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe6c00bcc0 con 0x7efe780fe440 2024-03-22T07:11:25.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe74ff9700 1 -- 172.21.15.81:0/1842476316 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe6c01c630 con 0x7efe780fe440 2024-03-22T07:11:25.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.306+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe78113520 con 0x7efe780fe440 2024-03-22T07:11:25.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.307+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7804c710 con 0x7efe780fe440 2024-03-22T07:11:25.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.308+0000 7efe74ff9700 1 -- 172.21.15.81:0/1842476316 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7efe6c013420 con 0x7efe780fe440 2024-03-22T07:11:25.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.308+0000 7efe74ff9700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7efe6006cdb0 0x7efe6006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:25.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.308+0000 7efe777fe700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7efe6006cdb0 0x7efe6006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:25.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.308+0000 7efe74ff9700 1 -- 172.21.15.81:0/1842476316 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7efe6c08cf10 con 0x7efe780fe440 2024-03-22T07:11:25.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.309+0000 7efe777fe700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7efe6006cdb0 0x7efe6006f260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe68007fd0 tx=0x7efe68009040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:25.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.313+0000 7efe74ff9700 1 -- 172.21.15.81:0/1842476316 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7efe6c05ab00 con 0x7efe780fe440 2024-03-22T07:11:25.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.524+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe78113740 con 0x7efe6006cdb0 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.525+0000 7efe74ff9700 1 -- 172.21.15.81:0/1842476316 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7efe78113740 con 0x7efe6006cdb0 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr" 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "2/2 daemons upgraded", 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading alertmanager daemons", 2024-03-22T07:11:25.526 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:11:25.527 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:11:25.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.527+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7efe6006cdb0 msgr2=0x7efe6006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:25.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.527+0000 7efe7e373700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7efe6006cdb0 0x7efe6006f260 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe68007fd0 tx=0x7efe68009040 comp rx=0 tx=0).stop 2024-03-22T07:11:25.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.527+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 msgr2=0x7efe7810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:25.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.527+0000 7efe7e373700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 0x7efe7810d940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00b3b0 tx=0x7efe6c009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:25.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.527+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 shutdown_connections 2024-03-22T07:11:25.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.527+0000 7efe7e373700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7efe6006cdb0 0x7efe6006f260 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:25.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.527+0000 7efe7e373700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe780fe440 0x7efe7810d940 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:25.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.527+0000 7efe7e373700 1 --2- 172.21.15.81:0/1842476316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe780fed90 0x7efe7810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:25.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.527+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 >> 172.21.15.81:0/1842476316 conn(0x7efe780f6be0 msgr2=0x7efe780f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:25.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.528+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 shutdown_connections 2024-03-22T07:11:25.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:25.528+0000 7efe7e373700 1 -- 172.21.15.81:0/1842476316 wait complete. 2024-03-22T07:11:25.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:25 smithi081 ceph-mon[29245]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:25.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:25 smithi081 ceph-mon[29245]: pgmap v39: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:25.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:25 smithi081 ceph-mon[29245]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:25.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:25 smithi081 ceph-mon[29245]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:25.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:25 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1722095471' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:25.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:25 smithi149 ceph-mon[36068]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:25.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:25 smithi149 ceph-mon[36068]: pgmap v39: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:25.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:25 smithi149 ceph-mon[36068]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:25.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:25 smithi149 ceph-mon[36068]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:25.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:25 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1722095471' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:26.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:26 smithi149 ceph-mon[36068]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:26.857 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:26 smithi081 ceph-mon[29245]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:27.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:27 smithi149 ceph-mon[36068]: pgmap v40: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:27.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:27 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:27.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:27 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:27.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:27 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:27.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:27 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:27.879 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:27 smithi081 ceph-mon[29245]: pgmap v40: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:27.880 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:27 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:27.880 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:27 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:27.880 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:27 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:27.880 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:27 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:29.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:29 smithi149 ceph-mon[36068]: pgmap v41: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:29.984 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:29 smithi081 ceph-mon[29245]: pgmap v41: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:31.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:31 smithi081 ceph-mon[29245]: pgmap v42: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:32.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:31 smithi149 ceph-mon[36068]: pgmap v42: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:33.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:33 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:33 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:33 smithi081 ceph-mon[29245]: pgmap v43: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:33.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:33 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:33.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:33 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:33.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:33 smithi149 ceph-mon[36068]: pgmap v43: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:34.537 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:34 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:34.537 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:34 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:34.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:34 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:34.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:34 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:35.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:35 smithi081 ceph-mon[29245]: pgmap v44: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:35.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:35 smithi149 ceph-mon[36068]: pgmap v44: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: pgmap v45: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:38.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:37 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: pgmap v45: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:11:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:38.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:37 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:39.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:38 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:11:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:38 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:11:40.045 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:39 smithi081 ceph-mon[29245]: pgmap v46: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:40.045 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:39 smithi081 ceph-mon[29245]: Upgrade: Updating grafana.smithi081 2024-03-22T07:11:40.045 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:39 smithi081 ceph-mon[29245]: Deploying daemon grafana.smithi081 on smithi081 2024-03-22T07:11:40.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:39 smithi149 ceph-mon[36068]: pgmap v46: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:40.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:39 smithi149 ceph-mon[36068]: Upgrade: Updating grafana.smithi081 2024-03-22T07:11:40.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:39 smithi149 ceph-mon[36068]: Deploying daemon grafana.smithi081 on smithi081 2024-03-22T07:11:41.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:41 smithi081 ceph-mon[29245]: pgmap v47: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:41.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:41 smithi149 ceph-mon[36068]: pgmap v47: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:43.721 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:43 smithi081 ceph-mon[29245]: pgmap v48: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:43.721 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:43 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:43.722 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:43 smithi081 ceph-mon[29245]: 1 slow requests, 1 included below; oldest blocked for > 182.624726 secs 2024-03-22T07:11:43.722 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:43 smithi081 ceph-mon[29245]: slow request 182.624726 seconds old, received at 2024-03-22T07:08:39.885715+0000: client_request(client.14534:13454 rename #0x100000034b7/d56 #0x100000033f3/d2e 2024-03-22T07:08:39.884274+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently acquired locks 2024-03-22T07:11:43.722 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:43 smithi081 ceph-mon[29245]: 1 slow requests, 1 included below; oldest blocked for > 182.964278 secs 2024-03-22T07:11:43.722 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:43 smithi081 ceph-mon[29245]: slow request 182.964277 seconds old, received at 2024-03-22T07:08:39.886131+0000: peer_request:client.14534:13454 currently acquired locks 2024-03-22T07:11:43.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:43 smithi149 ceph-mon[36068]: pgmap v48: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:43.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:43 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:43.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:43 smithi149 ceph-mon[36068]: 1 slow requests, 1 included below; oldest blocked for > 182.624726 secs 2024-03-22T07:11:43.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:43 smithi149 ceph-mon[36068]: slow request 182.624726 seconds old, received at 2024-03-22T07:08:39.885715+0000: client_request(client.14534:13454 rename #0x100000034b7/d56 #0x100000033f3/d2e 2024-03-22T07:08:39.884274+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently acquired locks 2024-03-22T07:11:43.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:43 smithi149 ceph-mon[36068]: 1 slow requests, 1 included below; oldest blocked for > 182.964278 secs 2024-03-22T07:11:43.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:43 smithi149 ceph-mon[36068]: slow request 182.964277 seconds old, received at 2024-03-22T07:08:39.886131+0000: peer_request:client.14534:13454 currently acquired locks 2024-03-22T07:11:45.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:44 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:45.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:44 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:45.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:44 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:45.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:44 smithi081 ceph-mon[29245]: pgmap v49: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:45.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:44 smithi081 ceph-mon[29245]: mgrmap e33: smithi081.bcxarw(active, since 92s), standbys: smithi149.sgmtoa 2024-03-22T07:11:45.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:44 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:45.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:44 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:44 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:44 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:44 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:45.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:44 smithi149 ceph-mon[36068]: pgmap v49: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:45.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:44 smithi149 ceph-mon[36068]: mgrmap e33: smithi081.bcxarw(active, since 92s), standbys: smithi149.sgmtoa 2024-03-22T07:11:45.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:44 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:45.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:44 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:46.231 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:45 smithi081 ceph-mon[29245]: Health check failed: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-03-22T07:11:46.231 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:45 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:active 2024-03-22T07:11:46.231 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:45 smithi081 ceph-mon[29245]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:active 2024-03-22T07:11:46.231 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:45 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:11:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:45 smithi149 ceph-mon[36068]: Health check failed: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-03-22T07:11:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:45 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:active 2024-03-22T07:11:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:45 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:active 2024-03-22T07:11:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:45 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:11:47.231 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:46 smithi081 ceph-mon[29245]: pgmap v50: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:47.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:46 smithi149 ceph-mon[36068]: pgmap v50: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:48.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:47 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 187.624791 secs 2024-03-22T07:11:48.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:47 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 187.964358 secs 2024-03-22T07:11:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:47 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 187.624791 secs 2024-03-22T07:11:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:47 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 187.964358 secs 2024-03-22T07:11:49.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:48 smithi149 ceph-mon[36068]: pgmap v51: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:49.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:48 smithi081 ceph-mon[29245]: pgmap v51: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:50.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:50 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:50.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:50 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:50.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:50 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:50.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:50 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:51.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:51 smithi081 ceph-mon[29245]: pgmap v52: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:51.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:51 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:51.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:51 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:51 smithi149 ceph-mon[36068]: pgmap v52: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:51 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:51.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:51 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:53.562 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:53 smithi081 ceph-mon[29245]: pgmap v53: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:53.562 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:53 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 192.624908 secs 2024-03-22T07:11:53.563 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:53 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 192.964513 secs 2024-03-22T07:11:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:53 smithi149 ceph-mon[36068]: pgmap v53: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:53 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 192.624908 secs 2024-03-22T07:11:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:53 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 192.964513 secs 2024-03-22T07:11:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: Upgrade: Finalizing container_image settings 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T07:11:55.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: Upgrade: Complete! 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:55.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.324 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:54 smithi081 ceph-mon[29245]: pgmap v54: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:55.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: Upgrade: Finalizing container_image settings 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T07:11:55.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T07:11:55.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T07:11:55.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T07:11:55.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T07:11:55.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T07:11:55.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T07:11:55.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: Upgrade: Complete! 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:55.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:55.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:55.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:54 smithi149 ceph-mon[36068]: pgmap v54: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:11:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.643+0000 7fe03459c700 1 -- 172.21.15.81:0/623661613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fbcd0 msgr2=0x7fe02c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.643+0000 7fe03459c700 1 --2- 172.21.15.81:0/623661613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fbcd0 0x7fe02c0fc0e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe024009ce0 tx=0x7fe024004d10 comp rx=0 tx=0).stop 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.643+0000 7fe03459c700 1 -- 172.21.15.81:0/623661613 shutdown_connections 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.643+0000 7fe03459c700 1 --2- 172.21.15.81:0/623661613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe02c0fcf80 0x7fe02c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.643+0000 7fe03459c700 1 --2- 172.21.15.81:0/623661613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fbcd0 0x7fe02c0fc0e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.643+0000 7fe03459c700 1 -- 172.21.15.81:0/623661613 >> 172.21.15.81:0/623661613 conn(0x7fe02c0f7430 msgr2=0x7fe02c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.643+0000 7fe03459c700 1 -- 172.21.15.81:0/623661613 shutdown_connections 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.643+0000 7fe03459c700 1 -- 172.21.15.81:0/623661613 wait complete. 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe03459c700 1 Processor -- start 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe03459c700 1 -- start start 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe03459c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe02c0fbcd0 0x7fe02c10dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe03459c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fcf80 0x7fe02c10e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe03459c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c10e740 con 0x7fe02c0fcf80 2024-03-22T07:11:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe03459c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c10ef40 con 0x7fe02c0fbcd0 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe031b37700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fcf80 0x7fe02c10e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe031b37700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fcf80 0x7fe02c10e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40578/0 (socket says 172.21.15.81:40578) 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe031b37700 1 -- 172.21.15.81:0/2643748285 learned_addr learned my addr 172.21.15.81:0/2643748285 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.644+0000 7fe032338700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe02c0fbcd0 0x7fe02c10dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.645+0000 7fe031b37700 1 -- 172.21.15.81:0/2643748285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe02c0fbcd0 msgr2=0x7fe02c10dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.645+0000 7fe031b37700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe02c0fbcd0 0x7fe02c10dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.645+0000 7fe031b37700 1 -- 172.21.15.81:0/2643748285 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe024009940 con 0x7fe02c0fcf80 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.645+0000 7fe031b37700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fcf80 0x7fe02c10e140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe020002a50 tx=0x7fe0200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.645+0000 7fe02affd700 1 -- 172.21.15.81:0/2643748285 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe020007cc0 con 0x7fe02c0fcf80 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.645+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c10f220 con 0x7fe02c0fcf80 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.645+0000 7fe02affd700 1 -- 172.21.15.81:0/2643748285 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe02000f040 con 0x7fe02c0fcf80 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.645+0000 7fe02affd700 1 -- 172.21.15.81:0/2643748285 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe02000da30 con 0x7fe02c0fcf80 2024-03-22T07:11:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.645+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c10f760 con 0x7fe02c0fcf80 2024-03-22T07:11:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.646+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02c101b10 con 0x7fe02c0fcf80 2024-03-22T07:11:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.646+0000 7fe02affd700 1 -- 172.21.15.81:0/2643748285 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fe02000db90 con 0x7fe02c0fcf80 2024-03-22T07:11:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.647+0000 7fe02affd700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe01406cc80 0x7fe01406f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.647+0000 7fe02affd700 1 -- 172.21.15.81:0/2643748285 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe02008bd40 con 0x7fe02c0fcf80 2024-03-22T07:11:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.647+0000 7fe032338700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe01406cc80 0x7fe01406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:55.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.647+0000 7fe032338700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe01406cc80 0x7fe01406f130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe024015040 tx=0x7fe024004d10 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:55.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.652+0000 7fe02affd700 1 -- 172.21.15.81:0/2643748285 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe020059b50 con 0x7fe02c0fcf80 2024-03-22T07:11:55.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.862+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe02c10f980 con 0x7fe01406cc80 2024-03-22T07:11:55.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.863+0000 7fe02affd700 1 -- 172.21.15.81:0/2643748285 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fe02c10f980 con 0x7fe01406cc80 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe01406cc80 msgr2=0x7fe01406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe01406cc80 0x7fe01406f130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe024015040 tx=0x7fe024004d10 comp rx=0 tx=0).stop 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fcf80 msgr2=0x7fe02c10e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fcf80 0x7fe02c10e140 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe020002a50 tx=0x7fe0200046b0 comp rx=0 tx=0).stop 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 shutdown_connections 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe01406cc80 0x7fe01406f130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe02c0fbcd0 0x7fe02c10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 --2- 172.21.15.81:0/2643748285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0fcf80 0x7fe02c10e140 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 >> 172.21.15.81:0/2643748285 conn(0x7fe02c0f7430 msgr2=0x7fe02c065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 shutdown_connections 2024-03-22T07:11:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:55.865+0000 7fe03459c700 1 -- 172.21.15.81:0/2643748285 wait complete. 2024-03-22T07:11:56.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:56 smithi081 ceph-mon[29245]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:56.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:56 smithi149 ceph-mon[36068]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:56.607 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-22T07:11:56.911 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:11:57.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:57 smithi081 ceph-mon[29245]: pgmap v55: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:57.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:57 smithi149 ceph-mon[36068]: pgmap v55: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:58.649 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:58 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:58.650 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:58 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:58.650 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:58 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:58.650 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:58 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 197.624978 secs 2024-03-22T07:11:58.650 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:58 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 197.964640 secs 2024-03-22T07:11:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:58 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:58 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:58 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:11:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:58 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 197.624978 secs 2024-03-22T07:11:58.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:58 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 197.964640 secs 2024-03-22T07:11:58.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.867+0000 7f751d1f4700 1 -- 172.21.15.81:0/1885985247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 msgr2=0x7f75181020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:58.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.867+0000 7f751d1f4700 1 --2- 172.21.15.81:0/1885985247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 0x7f75181020f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7508004940 tx=0x7f7508009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:58.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.867+0000 7f751d1f4700 1 -- 172.21.15.81:0/1885985247 shutdown_connections 2024-03-22T07:11:58.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.867+0000 7f751d1f4700 1 --2- 172.21.15.81:0/1885985247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75181026c0 0x7f75180fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:58.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.867+0000 7f751d1f4700 1 --2- 172.21.15.81:0/1885985247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 0x7f75181020f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:58.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.867+0000 7f751d1f4700 1 -- 172.21.15.81:0/1885985247 >> 172.21.15.81:0/1885985247 conn(0x7f75180f7870 msgr2=0x7f75180f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:58.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.867+0000 7f751d1f4700 1 -- 172.21.15.81:0/1885985247 shutdown_connections 2024-03-22T07:11:58.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.867+0000 7f751d1f4700 1 -- 172.21.15.81:0/1885985247 wait complete. 2024-03-22T07:11:58.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f751d1f4700 1 Processor -- start 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f751d1f4700 1 -- start start 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f751d1f4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 0x7f7518102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f751d1f4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75181026c0 0x7f75181030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f751d1f4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518104ad0 con 0x7f7518101d20 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f751d1f4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518103600 con 0x7f75181026c0 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f7516d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 0x7f7518102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f7516d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 0x7f7518102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40594/0 (socket says 172.21.15.81:40594) 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f7516d9d700 1 -- 172.21.15.81:0/1414034450 learned_addr learned my addr 172.21.15.81:0/1414034450 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.868+0000 7f751659c700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75181026c0 0x7f75181030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.869+0000 7f7516d9d700 1 -- 172.21.15.81:0/1414034450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75181026c0 msgr2=0x7f75181030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.869+0000 7f7516d9d700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75181026c0 0x7f75181030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.869+0000 7f7516d9d700 1 -- 172.21.15.81:0/1414034450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7508005040 con 0x7f7518101d20 2024-03-22T07:11:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.869+0000 7f7516d9d700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 0x7f7518102b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f750800b3b0 tx=0x7f7508009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:58.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.869+0000 7f74ff7fe700 1 -- 172.21.15.81:0/1414034450 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f750800e040 con 0x7f7518101d20 2024-03-22T07:11:58.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.869+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7518103880 con 0x7f7518101d20 2024-03-22T07:11:58.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.869+0000 7f74ff7fe700 1 -- 172.21.15.81:0/1414034450 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7508013420 con 0x7f7518101d20 2024-03-22T07:11:58.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.869+0000 7f74ff7fe700 1 -- 172.21.15.81:0/1414034450 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f750801c630 con 0x7f7518101d20 2024-03-22T07:11:58.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.869+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751811a130 con 0x7f7518101d20 2024-03-22T07:11:58.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.870+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75180fd470 con 0x7f7518101d20 2024-03-22T07:11:58.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.870+0000 7f74ff7fe700 1 -- 172.21.15.81:0/1414034450 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f7508013590 con 0x7f7518101d20 2024-03-22T07:11:58.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.871+0000 7f74ff7fe700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f750006cc80 0x7f750006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:58.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.871+0000 7f751659c700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f750006cc80 0x7f750006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:58.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.871+0000 7f74ff7fe700 1 -- 172.21.15.81:0/1414034450 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f750808d1b0 con 0x7f7518101d20 2024-03-22T07:11:58.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.871+0000 7f751659c700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f750006cc80 0x7f750006f130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75181042b0 tx=0x7f750c0043c0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:58.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:58.875+0000 7f74ff7fe700 1 -- 172.21.15.81:0/1414034450 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f7508057510 con 0x7f7518101d20 2024-03-22T07:11:59.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.149+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7518104090 con 0x7f7518101d20 2024-03-22T07:11:59.150 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.150+0000 7f74ff7fe700 1 -- 172.21.15.81:0/1414034450 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f750805abd0 con 0x7f7518101d20 2024-03-22T07:11:59.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f750006cc80 msgr2=0x7f750006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:59.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f750006cc80 0x7f750006f130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f75181042b0 tx=0x7f750c0043c0 comp rx=0 tx=0).stop 2024-03-22T07:11:59.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 msgr2=0x7f7518102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:59.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 0x7f7518102b80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f750800b3b0 tx=0x7f7508009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 shutdown_connections 2024-03-22T07:11:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f750006cc80 0x7f750006f130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7518101d20 0x7f7518102b80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 --2- 172.21.15.81:0/1414034450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75181026c0 0x7f75181030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 >> 172.21.15.81:0/1414034450 conn(0x7f75180f7870 msgr2=0x7f75180f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 shutdown_connections 2024-03-22T07:11:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:11:59.152+0000 7f751d1f4700 1 -- 172.21.15.81:0/1414034450 wait complete. 2024-03-22T07:11:59.162 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:11:59.537 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:59 smithi081 ceph-mon[29245]: pgmap v56: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:59.537 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:59 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:11:59.537 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:11:59 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1414034450' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:11:59.782 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-22T07:11:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:59 smithi149 ceph-mon[36068]: pgmap v56: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:11:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:59 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:11:59.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:11:59 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1414034450' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:00.042 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:01.566 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:01 smithi081 ceph-mon[29245]: pgmap v57: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:01.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.817+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1905246466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 msgr2=0x7f5a68102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:01.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.817+0000 7f5a6d61e700 1 --2- 172.21.15.81:0/1905246466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 0x7f5a68102140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5a58004940 tx=0x7f5a58009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:01.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.817+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1905246466 shutdown_connections 2024-03-22T07:12:01.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.817+0000 7f5a6d61e700 1 --2- 172.21.15.81:0/1905246466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a68102710 0x7f5a680fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:01.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.817+0000 7f5a6d61e700 1 --2- 172.21.15.81:0/1905246466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 0x7f5a68102140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:01.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.817+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1905246466 >> 172.21.15.81:0/1905246466 conn(0x7f5a68072bc0 msgr2=0x7f5a68072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:01.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.817+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1905246466 shutdown_connections 2024-03-22T07:12:01.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.817+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1905246466 wait complete. 2024-03-22T07:12:01.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a6d61e700 1 Processor -- start 2024-03-22T07:12:01.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a6d61e700 1 -- start start 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a6d61e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 0x7f5a6810ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a6d61e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a68102710 0x7f5a6810f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a6d61e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6810f6d0 con 0x7f5a68101d70 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a6d61e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6810f810 con 0x7f5a68102710 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a66ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 0x7f5a6810ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a66ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 0x7f5a6810ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40618/0 (socket says 172.21.15.81:40618) 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a66ffd700 1 -- 172.21.15.81:0/1006737308 learned_addr learned my addr 172.21.15.81:0/1006737308 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a667fc700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a68102710 0x7f5a6810f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a66ffd700 1 -- 172.21.15.81:0/1006737308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a68102710 msgr2=0x7f5a6810f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a66ffd700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a68102710 0x7f5a6810f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.818+0000 7f5a66ffd700 1 -- 172.21.15.81:0/1006737308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a58005040 con 0x7f5a68101d70 2024-03-22T07:12:01.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.819+0000 7f5a66ffd700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 0x7f5a6810ec50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800b3b0 tx=0x7f5a58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:01.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.819+0000 7f5a4f7fe700 1 -- 172.21.15.81:0/1006737308 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a5800e040 con 0x7f5a68101d70 2024-03-22T07:12:01.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.819+0000 7f5a4f7fe700 1 -- 172.21.15.81:0/1006737308 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a5800bbc0 con 0x7f5a68101d70 2024-03-22T07:12:01.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.819+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a68117b60 con 0x7f5a68101d70 2024-03-22T07:12:01.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.819+0000 7f5a4f7fe700 1 -- 172.21.15.81:0/1006737308 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a5801c3f0 con 0x7f5a68101d70 2024-03-22T07:12:01.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.819+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a68118120 con 0x7f5a68101d70 2024-03-22T07:12:01.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.820+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a680ff080 con 0x7f5a68101d70 2024-03-22T07:12:01.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.820+0000 7f5a4f7fe700 1 -- 172.21.15.81:0/1006737308 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f5a5800bd30 con 0x7f5a68101d70 2024-03-22T07:12:01.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.821+0000 7f5a4f7fe700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5a5006cc80 0x7f5a5006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:01.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.821+0000 7f5a4f7fe700 1 -- 172.21.15.81:0/1006737308 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5a5808d240 con 0x7f5a68101d70 2024-03-22T07:12:01.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.821+0000 7f5a667fc700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5a5006cc80 0x7f5a5006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:01.822 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.821+0000 7f5a667fc700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5a5006cc80 0x7f5a5006f130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c006fd0 tx=0x7f5a5c008040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:01.825 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:01.825+0000 7f5a4f7fe700 1 -- 172.21.15.81:0/1006737308 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f5a58057520 con 0x7f5a68101d70 2024-03-22T07:12:01.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:01 smithi149 ceph-mon[36068]: pgmap v57: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:02.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.102+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5a6804bed0 con 0x7f5a68101d70 2024-03-22T07:12:02.103 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.103+0000 7f5a4f7fe700 1 -- 172.21.15.81:0/1006737308 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f5a5805abe0 con 0x7f5a68101d70 2024-03-22T07:12:02.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5a5006cc80 msgr2=0x7f5a5006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:02.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5a5006cc80 0x7f5a5006f130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c006fd0 tx=0x7f5a5c008040 comp rx=0 tx=0).stop 2024-03-22T07:12:02.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 msgr2=0x7f5a6810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:02.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 0x7f5a6810ec50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800b3b0 tx=0x7f5a58009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:02.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 shutdown_connections 2024-03-22T07:12:02.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5a5006cc80 0x7f5a5006f130 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:02.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a68101d70 0x7f5a6810ec50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:02.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 --2- 172.21.15.81:0/1006737308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a68102710 0x7f5a6810f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:02.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 >> 172.21.15.81:0/1006737308 conn(0x7f5a68072bc0 msgr2=0x7f5a680fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:02.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 shutdown_connections 2024-03-22T07:12:02.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:02.105+0000 7f5a6d61e700 1 -- 172.21.15.81:0/1006737308 wait complete. 2024-03-22T07:12:02.115 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)" 2024-03-22T07:12:02.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:02 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1006737308' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:02.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:02 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1006737308' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:03.018 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-22T07:12:03.279 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:03.736 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:03 smithi081 ceph-mon[29245]: pgmap v58: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:03.736 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:03 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 202.625092 secs 2024-03-22T07:12:03.736 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:03 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 202.964756 secs 2024-03-22T07:12:03.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:03 smithi149 ceph-mon[36068]: pgmap v58: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:03.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:03 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 202.625092 secs 2024-03-22T07:12:03.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:03 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 202.964756 secs 2024-03-22T07:12:04.995 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.993+0000 7f4b909bf700 1 -- 172.21.15.81:0/221047687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 msgr2=0x7f4b8c1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.993+0000 7f4b909bf700 1 --2- 172.21.15.81:0/221047687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 0x7f4b8c1018f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009ce0 tx=0x7f4b84004d10 comp rx=0 tx=0).stop 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.993+0000 7f4b909bf700 1 -- 172.21.15.81:0/221047687 shutdown_connections 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.993+0000 7f4b909bf700 1 --2- 172.21.15.81:0/221047687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c0fb4c0 0x7f4b8c0fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.993+0000 7f4b909bf700 1 --2- 172.21.15.81:0/221047687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 0x7f4b8c1018f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.993+0000 7f4b909bf700 1 -- 172.21.15.81:0/221047687 >> 172.21.15.81:0/221047687 conn(0x7f4b8c072310 msgr2=0x7f4b8c072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b909bf700 1 -- 172.21.15.81:0/221047687 shutdown_connections 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b909bf700 1 -- 172.21.15.81:0/221047687 wait complete. 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b909bf700 1 Processor -- start 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b909bf700 1 -- start start 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b909bf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c0fb4c0 0x7f4b8c112990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b909bf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 0x7f4b8c112ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b909bf700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c113520 con 0x7f4b8c101520 2024-03-22T07:12:04.996 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b909bf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c10cb40 con 0x7f4b8c0fb4c0 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b89d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 0x7f4b8c112ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b89d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 0x7f4b8c112ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34098/0 (socket says 172.21.15.81:34098) 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.994+0000 7f4b89d9b700 1 -- 172.21.15.81:0/2004101911 learned_addr learned my addr 172.21.15.81:0/2004101911 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b8a59c700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c0fb4c0 0x7f4b8c112990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b89d9b700 1 -- 172.21.15.81:0/2004101911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c0fb4c0 msgr2=0x7f4b8c112990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b89d9b700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c0fb4c0 0x7f4b8c112990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b89d9b700 1 -- 172.21.15.81:0/2004101911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b84009940 con 0x7f4b8c101520 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b89d9b700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 0x7f4b8c112ed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400c830 tx=0x7f4b74004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b82ffd700 1 -- 172.21.15.81:0/2004101911 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b7400a760 con 0x7f4b8c101520 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b82ffd700 1 -- 172.21.15.81:0/2004101911 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b7400ad90 con 0x7f4b8c101520 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c10ce20 con 0x7f4b8c101520 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b82ffd700 1 -- 172.21.15.81:0/2004101911 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b740113a0 con 0x7f4b8c101520 2024-03-22T07:12:04.997 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.995+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c10d3e0 con 0x7f4b8c101520 2024-03-22T07:12:04.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.996+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c103900 con 0x7f4b8c101520 2024-03-22T07:12:04.998 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.996+0000 7f4b82ffd700 1 -- 172.21.15.81:0/2004101911 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f4b74012400 con 0x7f4b8c101520 2024-03-22T07:12:04.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.997+0000 7f4b82ffd700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4b7806cc80 0x7f4b7806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:04.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.997+0000 7f4b82ffd700 1 -- 172.21.15.81:0/2004101911 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4b7408bf20 con 0x7f4b8c101520 2024-03-22T07:12:04.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.997+0000 7f4b8a59c700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4b7806cc80 0x7f4b7806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:04.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:04.997+0000 7f4b8a59c700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4b7806cc80 0x7f4b7806f130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400b740 tx=0x7f4b84004d10 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:05.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.001+0000 7f4b82ffd700 1 -- 172.21.15.81:0/2004101911 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f4b740bf620 con 0x7f4b8c101520 2024-03-22T07:12:05.274 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.274+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b8c10dfe0 con 0x7f4b8c101520 2024-03-22T07:12:05.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.274+0000 7f4b82ffd700 1 -- 172.21.15.81:0/2004101911 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f4b74056320 con 0x7f4b8c101520 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.276+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4b7806cc80 msgr2=0x7f4b7806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.276+0000 7f4b909bf700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4b7806cc80 0x7f4b7806f130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4b8400b740 tx=0x7f4b84004d10 comp rx=0 tx=0).stop 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.276+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 msgr2=0x7f4b8c112ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.276+0000 7f4b909bf700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 0x7f4b8c112ed0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400c830 tx=0x7f4b74004470 comp rx=0 tx=0).stop 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.276+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 shutdown_connections 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.276+0000 7f4b909bf700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f4b7806cc80 0x7f4b7806f130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.276+0000 7f4b909bf700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b8c0fb4c0 0x7f4b8c112990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.276+0000 7f4b909bf700 1 --2- 172.21.15.81:0/2004101911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b8c101520 0x7f4b8c112ed0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.276+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 >> 172.21.15.81:0/2004101911 conn(0x7f4b8c072310 msgr2=0x7f4b8c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.277+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 shutdown_connections 2024-03-22T07:12:05.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:05.277+0000 7f4b909bf700 1 -- 172.21.15.81:0/2004101911 wait complete. 2024-03-22T07:12:05.287 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:12:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:05 smithi081 ceph-mon[29245]: pgmap v59: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:05.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:05 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2004101911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:05.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:05 smithi149 ceph-mon[36068]: pgmap v59: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:05.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:05 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2004101911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:06.091 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-22T07:12:06.351 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:07.815 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:07 smithi081 ceph-mon[29245]: pgmap v60: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:07.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:07 smithi149 ceph-mon[36068]: pgmap v60: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:08.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.312+0000 7f5ebe36f700 1 -- 172.21.15.81:0/1404442384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb81037d0 msgr2=0x7f5eb80fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:08.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.312+0000 7f5ebe36f700 1 --2- 172.21.15.81:0/1404442384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb81037d0 0x7f5eb80fce40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5eac004940 tx=0x7f5eac009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:08.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.312+0000 7f5ebe36f700 1 -- 172.21.15.81:0/1404442384 shutdown_connections 2024-03-22T07:12:08.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.312+0000 7f5ebe36f700 1 --2- 172.21.15.81:0/1404442384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb81037d0 0x7f5eb80fce40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:08.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.312+0000 7f5ebe36f700 1 --2- 172.21.15.81:0/1404442384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eb8102e30 0x7f5eb8103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:08.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.312+0000 7f5ebe36f700 1 -- 172.21.15.81:0/1404442384 >> 172.21.15.81:0/1404442384 conn(0x7f5eb80f8a20 msgr2=0x7f5eb80fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:08.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.312+0000 7f5ebe36f700 1 -- 172.21.15.81:0/1404442384 shutdown_connections 2024-03-22T07:12:08.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.312+0000 7f5ebe36f700 1 -- 172.21.15.81:0/1404442384 wait complete. 2024-03-22T07:12:08.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.313+0000 7f5ebe36f700 1 Processor -- start 2024-03-22T07:12:08.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.313+0000 7f5ebe36f700 1 -- start start 2024-03-22T07:12:08.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.313+0000 7f5ebe36f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e30 0x7f5eb8113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:08.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.313+0000 7f5ebe36f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eb81037d0 0x7f5eb8113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:08.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.313+0000 7f5ebe36f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8114520 con 0x7f5eb8102e30 2024-03-22T07:12:08.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.313+0000 7f5ebe36f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb810db20 con 0x7f5eb81037d0 2024-03-22T07:12:08.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.313+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e30 0x7f5eb8113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:08.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.313+0000 7f5eb77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eb81037d0 0x7f5eb8113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5eb7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e30 0x7f5eb8113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34116/0 (socket says 172.21.15.81:34116) 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5eb7fff700 1 -- 172.21.15.81:0/3178687665 learned_addr learned my addr 172.21.15.81:0/3178687665 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5eb7fff700 1 -- 172.21.15.81:0/3178687665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eb81037d0 msgr2=0x7f5eb8113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5eb7fff700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eb81037d0 0x7f5eb8113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5eb7fff700 1 -- 172.21.15.81:0/3178687665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac005040 con 0x7f5eb8102e30 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5eb7fff700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e30 0x7f5eb8113990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c830 tx=0x7f5ea8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5eb4ff9700 1 -- 172.21.15.81:0/3178687665 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea80106f0 con 0x7f5eb8102e30 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5eb4ff9700 1 -- 172.21.15.81:0/3178687665 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ea8010d20 con 0x7f5eb8102e30 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb810de00 con 0x7f5eb8102e30 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5eb4ff9700 1 -- 172.21.15.81:0/3178687665 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ea80113a0 con 0x7f5eb8102e30 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.314+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb810e390 con 0x7f5eb8102e30 2024-03-22T07:12:08.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.315+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb8105300 con 0x7f5eb8102e30 2024-03-22T07:12:08.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.316+0000 7f5eb4ff9700 1 -- 172.21.15.81:0/3178687665 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f5ea8012400 con 0x7f5eb8102e30 2024-03-22T07:12:08.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.316+0000 7f5eb4ff9700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5ea006cc80 0x7f5ea006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:08.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.316+0000 7f5eb4ff9700 1 -- 172.21.15.81:0/3178687665 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f5ea808bf00 con 0x7f5eb8102e30 2024-03-22T07:12:08.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.316+0000 7f5eb77fe700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5ea006cc80 0x7f5ea006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:08.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.316+0000 7f5eb77fe700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5ea006cc80 0x7f5ea006f130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5eac004890 tx=0x7f5eac009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:08.320 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.320+0000 7f5eb4ff9700 1 -- 172.21.15.81:0/3178687665 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f5ea8059d90 con 0x7f5eb8102e30 2024-03-22T07:12:08.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:08.534+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}) v1 -- 0x7f5eb810f050 con 0x7f5ea006cc80 2024-03-22T07:12:08.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:08 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 207.625228 secs 2024-03-22T07:12:08.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:08 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 207.964876 secs 2024-03-22T07:12:08.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:08 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 207.625228 secs 2024-03-22T07:12:08.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:08 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 207.964876 secs 2024-03-22T07:12:09.681 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:09 smithi081 ceph-mon[29245]: pgmap v61: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:09.681 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:09 smithi081 ceph-mon[29245]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:09 smithi149 ceph-mon[36068]: pgmap v61: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:09.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:09 smithi149 ceph-mon[36068]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:10.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.654+0000 7f5eb4ff9700 1 -- 172.21.15.81:0/3178687665 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7f5eb810f050 con 0x7f5ea006cc80 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.656+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5ea006cc80 msgr2=0x7f5ea006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.656+0000 7f5ebe36f700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5ea006cc80 0x7f5ea006f130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5eac004890 tx=0x7f5eac009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.656+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e30 msgr2=0x7f5eb8113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.656+0000 7f5ebe36f700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e30 0x7f5eb8113990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c830 tx=0x7f5ea8004470 comp rx=0 tx=0).stop 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.656+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 shutdown_connections 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.656+0000 7f5ebe36f700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f5ea006cc80 0x7f5ea006f130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.656+0000 7f5ebe36f700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5eb8102e30 0x7f5eb8113990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.656+0000 7f5ebe36f700 1 --2- 172.21.15.81:0/3178687665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eb81037d0 0x7f5eb8113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.656+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 >> 172.21.15.81:0/3178687665 conn(0x7f5eb80f8a20 msgr2=0x7f5eb81021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.657+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 shutdown_connections 2024-03-22T07:12:10.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:10.657+0000 7f5ebe36f700 1 -- 172.21.15.81:0/3178687665 wait complete. 2024-03-22T07:12:10.668 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:12:11.300 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch ps' 2024-03-22T07:12:11.560 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:11.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:11 smithi081 ceph-mon[29245]: pgmap v62: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:11.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:11 smithi149 ceph-mon[36068]: pgmap v62: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:12.631 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:12 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:12 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:13.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.144+0000 7f6605369700 1 -- 172.21.15.81:0/371812189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6600102fd0 msgr2=0x7f66001033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:13.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.144+0000 7f6605369700 1 --2- 172.21.15.81:0/371812189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6600102fd0 0x7f66001033a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f65f0004940 tx=0x7f65f0009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:13.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.144+0000 7f6605369700 1 -- 172.21.15.81:0/371812189 shutdown_connections 2024-03-22T07:12:13.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.144+0000 7f6605369700 1 --2- 172.21.15.81:0/371812189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66000fb540 0x7f66000fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:13.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.144+0000 7f6605369700 1 --2- 172.21.15.81:0/371812189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6600102fd0 0x7f66001033a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:13.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.144+0000 7f6605369700 1 -- 172.21.15.81:0/371812189 >> 172.21.15.81:0/371812189 conn(0x7f6600071530 msgr2=0x7f6600071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:13.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.144+0000 7f6605369700 1 -- 172.21.15.81:0/371812189 shutdown_connections 2024-03-22T07:12:13.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.144+0000 7f6605369700 1 -- 172.21.15.81:0/371812189 wait complete. 2024-03-22T07:12:13.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.144+0000 7f6605369700 1 Processor -- start 2024-03-22T07:12:13.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f6605369700 1 -- start start 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f6605369700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66000fb540 0x7f6600113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f6605369700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6600102fd0 0x7f6600113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f6605369700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66001144b0 con 0x7f66000fb540 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f6605369700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660010dab0 con 0x7f6600102fd0 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f65feffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66000fb540 0x7f6600113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f65feffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66000fb540 0x7f6600113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34130/0 (socket says 172.21.15.81:34130) 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f65feffd700 1 -- 172.21.15.81:0/2238786932 learned_addr learned my addr 172.21.15.81:0/2238786932 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f65f67fc700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6600102fd0 0x7f6600113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f65feffd700 1 -- 172.21.15.81:0/2238786932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6600102fd0 msgr2=0x7f6600113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f65feffd700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6600102fd0 0x7f6600113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f65feffd700 1 -- 172.21.15.81:0/2238786932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f0005040 con 0x7f66000fb540 2024-03-22T07:12:13.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f65feffd700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66000fb540 0x7f6600113920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65f000b8b0 tx=0x7f65f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:13.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.145+0000 7f65f7fff700 1 -- 172.21.15.81:0/2238786932 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65f000e040 con 0x7f66000fb540 2024-03-22T07:12:13.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.146+0000 7f65f7fff700 1 -- 172.21.15.81:0/2238786932 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65f0004290 con 0x7f66000fb540 2024-03-22T07:12:13.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.146+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660010dd30 con 0x7f66000fb540 2024-03-22T07:12:13.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.146+0000 7f65f7fff700 1 -- 172.21.15.81:0/2238786932 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65f001c3f0 con 0x7f66000fb540 2024-03-22T07:12:13.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.146+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660010e2f0 con 0x7f66000fb540 2024-03-22T07:12:13.148 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.146+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6600100270 con 0x7f66000fb540 2024-03-22T07:12:13.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.147+0000 7f65f7fff700 1 -- 172.21.15.81:0/2238786932 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f65f00137b0 con 0x7f66000fb540 2024-03-22T07:12:13.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.147+0000 7f65f7fff700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f65ec06cc80 0x7f65ec06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:13.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.148+0000 7f65f67fc700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f65ec06cc80 0x7f65ec06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:13.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.148+0000 7f65f7fff700 1 -- 172.21.15.81:0/2238786932 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f65f008d2e0 con 0x7f66000fb540 2024-03-22T07:12:13.149 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.148+0000 7f65f67fc700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f65ec06cc80 0x7f65ec06f130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65e8007fd0 tx=0x7f65e8009040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:13.152 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.152+0000 7f65f7fff700 1 -- 172.21.15.81:0/2238786932 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f65f005b170 con 0x7f66000fb540 2024-03-22T07:12:13.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.353+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f660010ef70 con 0x7f65ec06cc80 2024-03-22T07:12:13.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.363+0000 7f65f7fff700 1 -- 172.21.15.81:0/2238786932 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f660010ef70 con 0x7f65ec06cc80 2024-03-22T07:12:13.363 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (47s) 24s ago 11m 16.7M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (11m) 24s ago 11m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (10m) 79s ago 10m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (29s) 24s ago 10m 56.8M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (5m) 24s ago 5m 12.6M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (5m) 24s ago 5m 132M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (5m) 79s ago 5m 15.5M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (5m) 79s ago 5m 148M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (2m) 24s ago 12m 498M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (99s) 79s ago 10m 447M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (12m) 24s ago 12m 53.2M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (10m) 79s ago 10m 44.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (87s) 24s ago 11m 8254k - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (83s) 79s ago 10m 3640k - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:12:13.364 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (9m) 24s ago 9m 184M 1557M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:12:13.365 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (8m) 24s ago 8m 198M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:12:13.365 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (8m) 24s ago 8m 161M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:12:13.365 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (8m) 79s ago 8m 214M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:12:13.365 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (7m) 79s ago 7m 191M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:12:13.365 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (7m) 79s ago 7m 181M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:12:13.365 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (65s) 24s ago 10m 47.0M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:12:13.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f65ec06cc80 msgr2=0x7f65ec06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:13.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f65ec06cc80 0x7f65ec06f130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65e8007fd0 tx=0x7f65e8009040 comp rx=0 tx=0).stop 2024-03-22T07:12:13.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66000fb540 msgr2=0x7f6600113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:13.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66000fb540 0x7f6600113920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f65f000b8b0 tx=0x7f65f0009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:13.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 shutdown_connections 2024-03-22T07:12:13.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f65ec06cc80 0x7f65ec06f130 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:13.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66000fb540 0x7f6600113920 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:13.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 --2- 172.21.15.81:0/2238786932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6600102fd0 0x7f6600113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:13.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 >> 172.21.15.81:0/2238786932 conn(0x7f6600071530 msgr2=0x7f66000fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:13.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 shutdown_connections 2024-03-22T07:12:13.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:13.366+0000 7f6605369700 1 -- 172.21.15.81:0/2238786932 wait complete. 2024-03-22T07:12:13.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:13 smithi081 ceph-mon[29245]: pgmap v63: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:13.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:13 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 212.625335 secs 2024-03-22T07:12:13.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:13 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 212.964997 secs 2024-03-22T07:12:13.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:13 smithi149 ceph-mon[36068]: pgmap v63: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:13.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:13 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 212.625335 secs 2024-03-22T07:12:13.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:13 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 212.964997 secs 2024-03-22T07:12:14.302 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T07:12:14.303 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:12:14.303 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-22T07:12:14.568 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:14.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:14 smithi081 ceph-mon[29245]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:14.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:14 smithi149 ceph-mon[36068]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:15.705 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:15 smithi081 ceph-mon[29245]: pgmap v64: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:15.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:15 smithi149 ceph-mon[36068]: pgmap v64: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:16.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.260+0000 7fd185223700 1 -- 172.21.15.81:0/621164319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 msgr2=0x7fd1800fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:16.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.260+0000 7fd185223700 1 --2- 172.21.15.81:0/621164319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 0x7fd1800fe260 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd170009ce0 tx=0x7fd170004d10 comp rx=0 tx=0).stop 2024-03-22T07:12:16.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.260+0000 7fd185223700 1 -- 172.21.15.81:0/621164319 shutdown_connections 2024-03-22T07:12:16.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.260+0000 7fd185223700 1 --2- 172.21.15.81:0/621164319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1800fe830 0x7fd180106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.260+0000 7fd185223700 1 --2- 172.21.15.81:0/621164319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 0x7fd1800fe260 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.260+0000 7fd185223700 1 -- 172.21.15.81:0/621164319 >> 172.21.15.81:0/621164319 conn(0x7fd180072bc0 msgr2=0x7fd180072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.260+0000 7fd185223700 1 -- 172.21.15.81:0/621164319 shutdown_connections 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.260+0000 7fd185223700 1 -- 172.21.15.81:0/621164319 wait complete. 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.260+0000 7fd185223700 1 Processor -- start 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd185223700 1 -- start start 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd185223700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 0x7fd180113c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd185223700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1800fe830 0x7fd180114150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd185223700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180114830 con 0x7fd1800fde90 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd185223700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18010db90 con 0x7fd1800fe830 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd17ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 0x7fd180113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd17ed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 0x7fd180113c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60204/0 (socket says 172.21.15.81:60204) 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd17ed9d700 1 -- 172.21.15.81:0/1226608754 learned_addr learned my addr 172.21.15.81:0/1226608754 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd17659c700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1800fe830 0x7fd180114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd17ed9d700 1 -- 172.21.15.81:0/1226608754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1800fe830 msgr2=0x7fd180114150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:16.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd17ed9d700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1800fe830 0x7fd180114150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:16.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd17ed9d700 1 -- 172.21.15.81:0/1226608754 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170009940 con 0x7fd1800fde90 2024-03-22T07:12:16.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd17659c700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1800fe830 0x7fd180114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:12:16.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.261+0000 7fd17ed9d700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 0x7fd180113c10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd17000b740 tx=0x7fd170004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:16.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.262+0000 7fd177fff700 1 -- 172.21.15.81:0/1226608754 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd17000e040 con 0x7fd1800fde90 2024-03-22T07:12:16.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.262+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18010de10 con 0x7fd1800fde90 2024-03-22T07:12:16.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.262+0000 7fd177fff700 1 -- 172.21.15.81:0/1226608754 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1700128e0 con 0x7fd1800fde90 2024-03-22T07:12:16.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.262+0000 7fd177fff700 1 -- 172.21.15.81:0/1226608754 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd17001c7c0 con 0x7fd1800fde90 2024-03-22T07:12:16.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.262+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18010e350 con 0x7fd1800fde90 2024-03-22T07:12:16.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.263+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd180065e30 con 0x7fd1800fde90 2024-03-22T07:12:16.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.263+0000 7fd177fff700 1 -- 172.21.15.81:0/1226608754 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fd170012410 con 0x7fd1800fde90 2024-03-22T07:12:16.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.263+0000 7fd177fff700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fd16c06cc80 0x7fd16c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:16.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.264+0000 7fd177fff700 1 -- 172.21.15.81:0/1226608754 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd17008dd70 con 0x7fd1800fde90 2024-03-22T07:12:16.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.264+0000 7fd17659c700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fd16c06cc80 0x7fd16c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:16.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.264+0000 7fd17659c700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fd16c06cc80 0x7fd16c06f130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd168006fd0 tx=0x7fd168008040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:16.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.268+0000 7fd177fff700 1 -- 172.21.15.81:0/1226608754 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fd17005bb80 con 0x7fd1800fde90 2024-03-22T07:12:16.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.462+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fd18010dfa0 con 0x7fd1800fde90 2024-03-22T07:12:16.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.467+0000 7fd177fff700 1 -- 172.21.15.81:0/1226608754 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v46)=0 v46) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd18010dfa0 con 0x7fd1800fde90 2024-03-22T07:12:16.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.469+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fd16c06cc80 msgr2=0x7fd16c06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:16.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.469+0000 7fd185223700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fd16c06cc80 0x7fd16c06f130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd168006fd0 tx=0x7fd168008040 comp rx=0 tx=0).stop 2024-03-22T07:12:16.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.469+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 msgr2=0x7fd180113c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:16.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.469+0000 7fd185223700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 0x7fd180113c10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd17000b740 tx=0x7fd170004d10 comp rx=0 tx=0).stop 2024-03-22T07:12:16.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.470+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 shutdown_connections 2024-03-22T07:12:16.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.470+0000 7fd185223700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fd16c06cc80 0x7fd16c06f130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:16.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.470+0000 7fd185223700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1800fde90 0x7fd180113c10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:16.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.470+0000 7fd185223700 1 --2- 172.21.15.81:0/1226608754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1800fe830 0x7fd180114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:16.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.470+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 >> 172.21.15.81:0/1226608754 conn(0x7fd180072bc0 msgr2=0x7fd1801037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:16.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.470+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 shutdown_connections 2024-03-22T07:12:16.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:16.470+0000 7fd185223700 1 -- 172.21.15.81:0/1226608754 wait complete. 2024-03-22T07:12:17.140 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T07:12:17.140 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:12:17.141 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-22T07:12:17.401 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:17.783 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:17 smithi081 ceph-mon[29245]: pgmap v65: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:17.784 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:17 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1226608754' entity='client.admin' 2024-03-22T07:12:17.784 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:17 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:17.784 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:17 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:17.784 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:17 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:17.784 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:17 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:17.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:17 smithi149 ceph-mon[36068]: pgmap v65: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:17.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:17 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1226608754' entity='client.admin' 2024-03-22T07:12:17.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:17 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:17.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:17 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:17.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:17 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:17.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:17 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:18.728 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:18 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 217.625405 secs 2024-03-22T07:12:18.729 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:18 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 217.965095 secs 2024-03-22T07:12:18.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:18 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 217.625405 secs 2024-03-22T07:12:18.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:18 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 217.965095 secs 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.058+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/2840779414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 msgr2=0x7f2bb41020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.058+0000 7f2bbc5e6700 1 --2- 172.21.15.81:0/2840779414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 0x7f2bb41020e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0004940 tx=0x7f2bb0009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.058+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/2840779414 shutdown_connections 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.058+0000 7f2bbc5e6700 1 --2- 172.21.15.81:0/2840779414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb41026b0 0x7f2bb40fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.058+0000 7f2bbc5e6700 1 --2- 172.21.15.81:0/2840779414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 0x7f2bb41020e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.058+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/2840779414 >> 172.21.15.81:0/2840779414 conn(0x7f2bb40f7880 msgr2=0x7f2bb40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/2840779414 shutdown_connections 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/2840779414 wait complete. 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bbc5e6700 1 Processor -- start 2024-03-22T07:12:19.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bbc5e6700 1 -- start start 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bbc5e6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 0x7f2bb410dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bbc5e6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb41026b0 0x7f2bb410bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bbc5e6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb410c140 con 0x7f2bb4101d10 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bbc5e6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb410c2b0 con 0x7f2bb41026b0 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bba382700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 0x7f2bb410dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bba382700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 0x7f2bb410dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60222/0 (socket says 172.21.15.81:60222) 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bba382700 1 -- 172.21.15.81:0/1813409210 learned_addr learned my addr 172.21.15.81:0/1813409210 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.059+0000 7f2bb9b81700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb41026b0 0x7f2bb410bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.060+0000 7f2bba382700 1 -- 172.21.15.81:0/1813409210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb41026b0 msgr2=0x7f2bb410bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.060+0000 7f2bba382700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb41026b0 0x7f2bb410bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.060+0000 7f2bba382700 1 -- 172.21.15.81:0/1813409210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0005040 con 0x7f2bb4101d10 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.060+0000 7f2bba382700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 0x7f2bb410dc60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000b3b0 tx=0x7f2bb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:19.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.060+0000 7f2baaffd700 1 -- 172.21.15.81:0/1813409210 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb000e040 con 0x7f2bb4101d10 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.060+0000 7f2baaffd700 1 -- 172.21.15.81:0/1813409210 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bb000bbc0 con 0x7f2bb4101d10 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.060+0000 7f2baaffd700 1 -- 172.21.15.81:0/1813409210 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bb001c3f0 con 0x7f2bb4101d10 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.060+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb410c4e0 con 0x7f2bb4101d10 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.060+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb410ca20 con 0x7f2bb4101d10 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.061+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb40fd460 con 0x7f2bb4101d10 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.061+0000 7f2baaffd700 1 -- 172.21.15.81:0/1813409210 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f2bb000bd30 con 0x7f2bb4101d10 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.062+0000 7f2baaffd700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2ba006cc80 0x7f2ba006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.062+0000 7f2baaffd700 1 -- 172.21.15.81:0/1813409210 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2bb008d190 con 0x7f2bb4101d10 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.062+0000 7f2bb9b81700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2ba006cc80 0x7f2ba006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.062+0000 7f2bb9b81700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2ba006cc80 0x7f2ba006f130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4007fd0 tx=0x7f2ba4009040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:19.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.066+0000 7f2baaffd700 1 -- 172.21.15.81:0/1813409210 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2bb00574f0 con 0x7f2bb4101d10 2024-03-22T07:12:19.264 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.264+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f2bb405f5f0 con 0x7f2bb4101d10 2024-03-22T07:12:19.265 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.264+0000 7f2baaffd700 1 -- 172.21.15.81:0/1813409210 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v46)=0 v46) v1 ==== 155+0+0 (secure 0 0 0) 0x7f2bb005abb0 con 0x7f2bb4101d10 2024-03-22T07:12:19.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.266+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2ba006cc80 msgr2=0x7f2ba006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:19.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.266+0000 7f2bbc5e6700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2ba006cc80 0x7f2ba006f130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4007fd0 tx=0x7f2ba4009040 comp rx=0 tx=0).stop 2024-03-22T07:12:19.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.266+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 msgr2=0x7f2bb410dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:19.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.267+0000 7f2bbc5e6700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 0x7f2bb410dc60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000b3b0 tx=0x7f2bb0009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:19.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.267+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 shutdown_connections 2024-03-22T07:12:19.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.267+0000 7f2bbc5e6700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f2ba006cc80 0x7f2ba006f130 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:19.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.267+0000 7f2bbc5e6700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bb4101d10 0x7f2bb410dc60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:19.267 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.267+0000 7f2bbc5e6700 1 --2- 172.21.15.81:0/1813409210 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2bb41026b0 0x7f2bb410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:19.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.267+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 >> 172.21.15.81:0/1813409210 conn(0x7f2bb40f7880 msgr2=0x7f2bb40f88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:19.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.267+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 shutdown_connections 2024-03-22T07:12:19.268 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:19.267+0000 7f2bbc5e6700 1 -- 172.21.15.81:0/1813409210 wait complete. 2024-03-22T07:12:19.753 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:19 smithi149 ceph-mon[36068]: pgmap v66: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:19.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:19 smithi081 ceph-mon[29245]: pgmap v66: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:19.945 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-22T07:12:20.205 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:21.672 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:21 smithi081 ceph-mon[29245]: pgmap v67: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:21.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:21 smithi149 ceph-mon[36068]: pgmap v67: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:21.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.913+0000 7f78bdf2f700 1 -- 172.21.15.81:0/2292127156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 msgr2=0x7f78b8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:21.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.913+0000 7f78bdf2f700 1 --2- 172.21.15.81:0/2292127156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 0x7f78b8102150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f78a8004940 tx=0x7f78a8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:21.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.913+0000 7f78bdf2f700 1 -- 172.21.15.81:0/2292127156 shutdown_connections 2024-03-22T07:12:21.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.913+0000 7f78bdf2f700 1 --2- 172.21.15.81:0/2292127156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8102720 0x7f78b80fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:21.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.913+0000 7f78bdf2f700 1 --2- 172.21.15.81:0/2292127156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 0x7f78b8102150 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.913+0000 7f78bdf2f700 1 -- 172.21.15.81:0/2292127156 >> 172.21.15.81:0/2292127156 conn(0x7f78b80f78f0 msgr2=0x7f78b80f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.913+0000 7f78bdf2f700 1 -- 172.21.15.81:0/2292127156 shutdown_connections 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.913+0000 7f78bdf2f700 1 -- 172.21.15.81:0/2292127156 wait complete. 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78bdf2f700 1 Processor -- start 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78bdf2f700 1 -- start start 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78bdf2f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 0x7f78b8112820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78bdf2f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8102720 0x7f78b8112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78bdf2f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b81133b0 con 0x7f78b8101d80 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78bdf2f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b810c9b0 con 0x7f78b8102720 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78b77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 0x7f78b8112820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78b77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 0x7f78b8112820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60248/0 (socket says 172.21.15.81:60248) 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78b77fe700 1 -- 172.21.15.81:0/4197362498 learned_addr learned my addr 172.21.15.81:0/4197362498 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:21.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78b6ffd700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8102720 0x7f78b8112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78b77fe700 1 -- 172.21.15.81:0/4197362498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8102720 msgr2=0x7f78b8112d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78b77fe700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8102720 0x7f78b8112d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78b77fe700 1 -- 172.21.15.81:0/4197362498 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8005040 con 0x7f78b8101d80 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.914+0000 7f78b6ffd700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8102720 0x7f78b8112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.915+0000 7f78b77fe700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 0x7f78b8112820 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f78a800b3b0 tx=0x7f78a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.915+0000 7f789ffff700 1 -- 172.21.15.81:0/4197362498 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a800e040 con 0x7f78b8101d80 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.915+0000 7f789ffff700 1 -- 172.21.15.81:0/4197362498 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78a800bbc0 con 0x7f78b8101d80 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.915+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b810cc30 con 0x7f78b8101d80 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.915+0000 7f789ffff700 1 -- 172.21.15.81:0/4197362498 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a801c3f0 con 0x7f78b8101d80 2024-03-22T07:12:21.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.915+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b810d1f0 con 0x7f78b8101d80 2024-03-22T07:12:21.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.916+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b80ff090 con 0x7f78b8101d80 2024-03-22T07:12:21.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.916+0000 7f789ffff700 1 -- 172.21.15.81:0/4197362498 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f78a800bd30 con 0x7f78b8101d80 2024-03-22T07:12:21.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.917+0000 7f789ffff700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f78a006cc80 0x7f78a006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:21.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.917+0000 7f789ffff700 1 -- 172.21.15.81:0/4197362498 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f78a808d290 con 0x7f78b8101d80 2024-03-22T07:12:21.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.917+0000 7f78b6ffd700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f78a006cc80 0x7f78a006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:21.919 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.917+0000 7f78b6ffd700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f78a006cc80 0x7f78a006f130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009fd0 tx=0x7f78ac009ee0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:21.921 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:21.921+0000 7f789ffff700 1 -- 172.21.15.81:0/4197362498 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f78a805b0a0 con 0x7f78b8101d80 2024-03-22T07:12:22.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.113+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f78b810cdc0 con 0x7f78b8101d80 2024-03-22T07:12:22.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.114+0000 7f789ffff700 1 -- 172.21.15.81:0/4197362498 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v46)=0 v46) v1 ==== 163+0+0 (secure 0 0 0) 0x7f78b810cdc0 con 0x7f78b8101d80 2024-03-22T07:12:22.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f78a006cc80 msgr2=0x7f78a006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f78a006cc80 0x7f78a006f130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009fd0 tx=0x7f78ac009ee0 comp rx=0 tx=0).stop 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 msgr2=0x7f78b8112820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 0x7f78b8112820 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f78a800b3b0 tx=0x7f78a8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 shutdown_connections 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f78a006cc80 0x7f78a006f130 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78b8101d80 0x7f78b8112820 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 --2- 172.21.15.81:0/4197362498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8102720 0x7f78b8112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 >> 172.21.15.81:0/4197362498 conn(0x7f78b80f78f0 msgr2=0x7f78b8106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 shutdown_connections 2024-03-22T07:12:22.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:22.116+0000 7f78bdf2f700 1 -- 172.21.15.81:0/4197362498 wait complete. 2024-03-22T07:12:22.787 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-22T07:12:23.057 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:23.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:23 smithi081 ceph-mon[29245]: pgmap v68: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:23.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:23 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 222.625468 secs 2024-03-22T07:12:23.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:23 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 222.965172 secs 2024-03-22T07:12:23.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:23 smithi149 ceph-mon[36068]: pgmap v68: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:23.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:23 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 222.625468 secs 2024-03-22T07:12:23.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:23 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 222.965172 secs 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.716+0000 7f90da3c3700 1 -- 172.21.15.81:0/1113916988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 msgr2=0x7f90d40fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.716+0000 7f90da3c3700 1 --2- 172.21.15.81:0/1113916988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 0x7f90d40fce10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f90c8004940 tx=0x7f90c8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.716+0000 7f90da3c3700 1 -- 172.21.15.81:0/1113916988 shutdown_connections 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.716+0000 7f90da3c3700 1 --2- 172.21.15.81:0/1113916988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 0x7f90d40fce10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.716+0000 7f90da3c3700 1 --2- 172.21.15.81:0/1113916988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d4102e00 0x7f90d41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.716+0000 7f90da3c3700 1 -- 172.21.15.81:0/1113916988 >> 172.21.15.81:0/1113916988 conn(0x7f90d40f89b0 msgr2=0x7f90d40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.716+0000 7f90da3c3700 1 -- 172.21.15.81:0/1113916988 shutdown_connections 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.716+0000 7f90da3c3700 1 -- 172.21.15.81:0/1113916988 wait complete. 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.716+0000 7f90da3c3700 1 Processor -- start 2024-03-22T07:12:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90da3c3700 1 -- start start 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90da3c3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d4102e00 0x7f90d4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90da3c3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 0x7f90d4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90da3c3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d4114590 con 0x7f90d41037a0 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90da3c3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d410db00 con 0x7f90d4102e00 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90d37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 0x7f90d4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90d37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 0x7f90d4113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49646/0 (socket says 172.21.15.81:49646) 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90d37fe700 1 -- 172.21.15.81:0/2324928139 learned_addr learned my addr 172.21.15.81:0/2324928139 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90d3fff700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d4102e00 0x7f90d4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90d37fe700 1 -- 172.21.15.81:0/2324928139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d4102e00 msgr2=0x7f90d4113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90d37fe700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d4102e00 0x7f90d4113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90d37fe700 1 -- 172.21.15.81:0/2324928139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c8005040 con 0x7f90d41037a0 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90d37fe700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 0x7f90d4113eb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009100 tx=0x7f90c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.717+0000 7f90d0ff9700 1 -- 172.21.15.81:0/2324928139 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90c800e040 con 0x7f90d41037a0 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.718+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d410dd80 con 0x7f90d41037a0 2024-03-22T07:12:24.719 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.718+0000 7f90d0ff9700 1 -- 172.21.15.81:0/2324928139 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90c8013420 con 0x7f90d41037a0 2024-03-22T07:12:24.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.718+0000 7f90d0ff9700 1 -- 172.21.15.81:0/2324928139 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90c801c610 con 0x7f90d41037a0 2024-03-22T07:12:24.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.718+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d410e340 con 0x7f90d41037a0 2024-03-22T07:12:24.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.718+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d4100110 con 0x7f90d41037a0 2024-03-22T07:12:24.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.719+0000 7f90d0ff9700 1 -- 172.21.15.81:0/2324928139 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f90c8013590 con 0x7f90d41037a0 2024-03-22T07:12:24.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.719+0000 7f90d0ff9700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f90bc06cc80 0x7f90bc06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:24.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.720+0000 7f90d0ff9700 1 -- 172.21.15.81:0/2324928139 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f90c808d100 con 0x7f90d41037a0 2024-03-22T07:12:24.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.720+0000 7f90d3fff700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f90bc06cc80 0x7f90bc06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:24.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.720+0000 7f90d3fff700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f90bc06cc80 0x7f90bc06f130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f90c4006fd0 tx=0x7f90c4008040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:24.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.724+0000 7f90d0ff9700 1 -- 172.21.15.81:0/2324928139 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f90c805b030 con 0x7f90d41037a0 2024-03-22T07:12:24.724 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:24 smithi081 ceph-mon[29245]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:12:24.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:24 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:12:24.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.914+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f90d410efc0 con 0x7f90d41037a0 2024-03-22T07:12:24.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.915+0000 7f90d0ff9700 1 -- 172.21.15.81:0/2324928139 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v46)=0 v46) v1 ==== 135+0+0 (secure 0 0 0) 0x7f90c805abc0 con 0x7f90d41037a0 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.916+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f90bc06cc80 msgr2=0x7f90bc06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f90bc06cc80 0x7f90bc06f130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f90c4006fd0 tx=0x7f90c4008040 comp rx=0 tx=0).stop 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 msgr2=0x7f90d4113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 0x7f90d4113eb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009100 tx=0x7f90c8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 shutdown_connections 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f90bc06cc80 0x7f90bc06f130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90d4102e00 0x7f90d4113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 --2- 172.21.15.81:0/2324928139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90d41037a0 0x7f90d4113eb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 >> 172.21.15.81:0/2324928139 conn(0x7f90d40f89b0 msgr2=0x7f90d40fa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 shutdown_connections 2024-03-22T07:12:24.918 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:24.917+0000 7f90da3c3700 1 -- 172.21.15.81:0/2324928139 wait complete. 2024-03-22T07:12:25.527 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-22T07:12:25.786 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:25.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:25 smithi081 ceph-mon[29245]: pgmap v69: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:25.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:25 smithi149 ceph-mon[36068]: pgmap v69: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:27.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.362+0000 7f74efab7700 1 -- 172.21.15.81:0/3975269564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 msgr2=0x7f74e80fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:27.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.362+0000 7f74efab7700 1 --2- 172.21.15.81:0/3975269564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 0x7f74e80fb990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f74dc004940 tx=0x7f74dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:27.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.362+0000 7f74efab7700 1 -- 172.21.15.81:0/3975269564 shutdown_connections 2024-03-22T07:12:27.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.362+0000 7f74efab7700 1 --2- 172.21.15.81:0/3975269564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 0x7f74e80fb990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:27.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.362+0000 7f74efab7700 1 --2- 172.21.15.81:0/3975269564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8102fd0 0x7f74e81033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:27.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.362+0000 7f74efab7700 1 -- 172.21.15.81:0/3975269564 >> 172.21.15.81:0/3975269564 conn(0x7f74e8071530 msgr2=0x7f74e8071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:27.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.362+0000 7f74efab7700 1 -- 172.21.15.81:0/3975269564 shutdown_connections 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74efab7700 1 -- 172.21.15.81:0/3975269564 wait complete. 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74efab7700 1 Processor -- start 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74efab7700 1 -- start start 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74efab7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 0x7f74e80fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74efab7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8102fd0 0x7f74e80fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74efab7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80fe650 con 0x7f74e80fb540 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74efab7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e80fd1a0 con 0x7f74e8102fd0 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74ed853700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 0x7f74e80fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74ed853700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 0x7f74e80fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49670/0 (socket says 172.21.15.81:49670) 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74ed853700 1 -- 172.21.15.81:0/969617429 learned_addr learned my addr 172.21.15.81:0/969617429 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.363+0000 7f74ed052700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8102fd0 0x7f74e80fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.364+0000 7f74ed853700 1 -- 172.21.15.81:0/969617429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8102fd0 msgr2=0x7f74e80fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.364+0000 7f74ed853700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8102fd0 0x7f74e80fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.364+0000 7f74ed853700 1 -- 172.21.15.81:0/969617429 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc005040 con 0x7f74e80fb540 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.364+0000 7f74ed853700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 0x7f74e80fc720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f74e4002a50 tx=0x7f74e40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:27.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.364+0000 7f74da7fc700 1 -- 172.21.15.81:0/969617429 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74e4007cc0 con 0x7f74e80fb540 2024-03-22T07:12:27.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.364+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e80fd480 con 0x7f74e80fb540 2024-03-22T07:12:27.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.364+0000 7f74da7fc700 1 -- 172.21.15.81:0/969617429 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74e400f040 con 0x7f74e80fb540 2024-03-22T07:12:27.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.364+0000 7f74da7fc700 1 -- 172.21.15.81:0/969617429 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74e400da30 con 0x7f74e80fb540 2024-03-22T07:12:27.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.364+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e811a0d0 con 0x7f74e80fb540 2024-03-22T07:12:27.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.365+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e81053b0 con 0x7f74e80fb540 2024-03-22T07:12:27.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.365+0000 7f74da7fc700 1 -- 172.21.15.81:0/969617429 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f74e400db90 con 0x7f74e80fb540 2024-03-22T07:12:27.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.366+0000 7f74da7fc700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74d406cc80 0x7f74d406f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:27.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.366+0000 7f74da7fc700 1 -- 172.21.15.81:0/969617429 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f74e408bd10 con 0x7f74e80fb540 2024-03-22T07:12:27.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.366+0000 7f74ed052700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74d406cc80 0x7f74d406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:27.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.366+0000 7f74ed052700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74d406cc80 0x7f74d406f130 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f74e80f9f40 tx=0x7f74dc009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:27.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.370+0000 7f74da7fc700 1 -- 172.21.15.81:0/969617429 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f74e4055ff0 con 0x7f74e80fb540 2024-03-22T07:12:27.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.583+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}) v1 -- 0x7f74e80fe830 con 0x7f74d406cc80 2024-03-22T07:12:27.588 INFO:teuthology.orchestra.run.smithi081.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:12:27.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.588+0000 7f74da7fc700 1 -- 172.21.15.81:0/969617429 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f74e80fe830 con 0x7f74d406cc80 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.590+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74d406cc80 msgr2=0x7f74d406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.590+0000 7f74efab7700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74d406cc80 0x7f74d406f130 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f74e80f9f40 tx=0x7f74dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.590+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 msgr2=0x7f74e80fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.590+0000 7f74efab7700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 0x7f74e80fc720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f74e4002a50 tx=0x7f74e40046b0 comp rx=0 tx=0).stop 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.590+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 shutdown_connections 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.590+0000 7f74efab7700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f74d406cc80 0x7f74d406f130 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.590+0000 7f74efab7700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e80fb540 0x7f74e80fc720 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.590+0000 7f74efab7700 1 --2- 172.21.15.81:0/969617429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e8102fd0 0x7f74e80fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.590+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 >> 172.21.15.81:0/969617429 conn(0x7f74e8071530 msgr2=0x7f74e80fa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:27.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.591+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 shutdown_connections 2024-03-22T07:12:27.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:27.591+0000 7f74efab7700 1 -- 172.21.15.81:0/969617429 wait complete. 2024-03-22T07:12:27.601 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:27 smithi081 ceph-mon[29245]: pgmap v70: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:27.601 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:27 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:27.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:27 smithi149 ceph-mon[36068]: pgmap v70: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:27.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:27 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:28.324 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T07:12:28.325 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:12:28.325 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 227.625555 secs 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:12:28.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:28 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 227.965293 secs 2024-03-22T07:12:28.593 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:12:28.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 227.625555 secs 2024-03-22T07:12:28.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:28.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:12:28.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:28.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:28.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:28.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:28.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:28.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:12:28.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:28 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 227.965293 secs 2024-03-22T07:12:29.986 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:29 smithi081 ceph-mon[29245]: pgmap v71: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:30.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:29 smithi149 ceph-mon[36068]: pgmap v71: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:30.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.532+0000 7f195b154700 1 -- 172.21.15.81:0/3691360033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19541037a0 msgr2=0x7f19540fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:30.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.532+0000 7f195b154700 1 --2- 172.21.15.81:0/3691360033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19541037a0 0x7f19540fce10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1940004940 tx=0x7f1940009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:30.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.533+0000 7f195b154700 1 -- 172.21.15.81:0/3691360033 shutdown_connections 2024-03-22T07:12:30.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.533+0000 7f195b154700 1 --2- 172.21.15.81:0/3691360033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19541037a0 0x7f19540fce10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:30.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.533+0000 7f195b154700 1 --2- 172.21.15.81:0/3691360033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1954102e00 0x7f19541031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:30.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.533+0000 7f195b154700 1 -- 172.21.15.81:0/3691360033 >> 172.21.15.81:0/3691360033 conn(0x7f19540f89b0 msgr2=0x7f19540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:30.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.533+0000 7f195b154700 1 -- 172.21.15.81:0/3691360033 shutdown_connections 2024-03-22T07:12:30.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.533+0000 7f195b154700 1 -- 172.21.15.81:0/3691360033 wait complete. 2024-03-22T07:12:30.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.533+0000 7f195b154700 1 Processor -- start 2024-03-22T07:12:30.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.533+0000 7f195b154700 1 -- start start 2024-03-22T07:12:30.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f195b154700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1954102e00 0x7f1954113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f195b154700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19541037a0 0x7f1954113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f195b154700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541144b0 con 0x7f1954102e00 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f195b154700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195410dab0 con 0x7f19541037a0 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f1958ef0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1954102e00 0x7f1954113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f1958ef0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1954102e00 0x7f1954113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49682/0 (socket says 172.21.15.81:49682) 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f1958ef0700 1 -- 172.21.15.81:0/108259371 learned_addr learned my addr 172.21.15.81:0/108259371 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f1953fff700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19541037a0 0x7f1954113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f1958ef0700 1 -- 172.21.15.81:0/108259371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19541037a0 msgr2=0x7f1954113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f1958ef0700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19541037a0 0x7f1954113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f1958ef0700 1 -- 172.21.15.81:0/108259371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1940005040 con 0x7f1954102e00 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f1958ef0700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1954102e00 0x7f1954113920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f194800eb40 tx=0x7f194800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.534+0000 7f19517fa700 1 -- 172.21.15.81:0/108259371 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1948007cc0 con 0x7f1954102e00 2024-03-22T07:12:30.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.535+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195410dd30 con 0x7f1954102e00 2024-03-22T07:12:30.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.535+0000 7f19517fa700 1 -- 172.21.15.81:0/108259371 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f194800f040 con 0x7f1954102e00 2024-03-22T07:12:30.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.535+0000 7f19517fa700 1 -- 172.21.15.81:0/108259371 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1948009620 con 0x7f1954102e00 2024-03-22T07:12:30.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.535+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195410e240 con 0x7f1954102e00 2024-03-22T07:12:30.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.536+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19541052d0 con 0x7f1954102e00 2024-03-22T07:12:30.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.536+0000 7f19517fa700 1 -- 172.21.15.81:0/108259371 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f1948009780 con 0x7f1954102e00 2024-03-22T07:12:30.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.536+0000 7f19517fa700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f194406cc80 0x7f194406f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:30.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.537+0000 7f19517fa700 1 -- 172.21.15.81:0/108259371 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f194808bc40 con 0x7f1954102e00 2024-03-22T07:12:30.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.537+0000 7f1953fff700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f194406cc80 0x7f194406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:30.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.537+0000 7f1953fff700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f194406cc80 0x7f194406f130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1940009b30 tx=0x7f1940009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:30.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.542+0000 7f19517fa700 1 -- 172.21.15.81:0/108259371 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f1948059b00 con 0x7f1954102e00 2024-03-22T07:12:30.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.758+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f195410ef70 con 0x7f194406cc80 2024-03-22T07:12:30.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.759+0000 7f19517fa700 1 -- 172.21.15.81:0/108259371 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f195410ef70 con 0x7f194406cc80 2024-03-22T07:12:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f194406cc80 msgr2=0x7f194406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f194406cc80 0x7f194406f130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1940009b30 tx=0x7f1940009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1954102e00 msgr2=0x7f1954113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1954102e00 0x7f1954113920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f194800eb40 tx=0x7f194800b040 comp rx=0 tx=0).stop 2024-03-22T07:12:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 shutdown_connections 2024-03-22T07:12:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f194406cc80 0x7f194406f130 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1954102e00 0x7f1954113920 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:30.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 --2- 172.21.15.81:0/108259371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19541037a0 0x7f1954113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:30.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 >> 172.21.15.81:0/108259371 conn(0x7f19540f89b0 msgr2=0x7f1954102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:30.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 shutdown_connections 2024-03-22T07:12:30.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.761+0000 7f195b154700 1 -- 172.21.15.81:0/108259371 wait complete. 2024-03-22T07:12:30.772 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:12:30.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.864+0000 7fcae6a28700 1 -- 172.21.15.81:0/964794138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 msgr2=0x7fcae00fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.864+0000 7fcae6a28700 1 --2- 172.21.15.81:0/964794138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 0x7fcae00fbd30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcad4004940 tx=0x7fcad4009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.864+0000 7fcae6a28700 1 -- 172.21.15.81:0/964794138 shutdown_connections 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.864+0000 7fcae6a28700 1 --2- 172.21.15.81:0/964794138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 0x7fcae00fbd30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.864+0000 7fcae6a28700 1 --2- 172.21.15.81:0/964794138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcae0101d20 0x7fcae01020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.865+0000 7fcae6a28700 1 -- 172.21.15.81:0/964794138 >> 172.21.15.81:0/964794138 conn(0x7fcae00f7870 msgr2=0x7fcae00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.865+0000 7fcae6a28700 1 -- 172.21.15.81:0/964794138 shutdown_connections 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.865+0000 7fcae6a28700 1 -- 172.21.15.81:0/964794138 wait complete. 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.865+0000 7fcae6a28700 1 Processor -- start 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.865+0000 7fcae6a28700 1 -- start start 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.865+0000 7fcae6a28700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcae0101d20 0x7fcae0113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.865+0000 7fcae6a28700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 0x7fcae0113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:30.867 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.865+0000 7fcae6a28700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae0114500 con 0x7fcae01026c0 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.865+0000 7fcae6a28700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae010db50 con 0x7fcae0101d20 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 0x7fcae0113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 0x7fcae0113f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49694/0 (socket says 172.21.15.81:49694) 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadf7fe700 1 -- 172.21.15.81:0/809841278 learned_addr learned my addr 172.21.15.81:0/809841278 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadffff700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcae0101d20 0x7fcae0113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadf7fe700 1 -- 172.21.15.81:0/809841278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcae0101d20 msgr2=0x7fcae0113a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadf7fe700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcae0101d20 0x7fcae0113a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadf7fe700 1 -- 172.21.15.81:0/809841278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad4005040 con 0x7fcae01026c0 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadf7fe700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 0x7fcae0113f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009100 tx=0x7fcad4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadcff9700 1 -- 172.21.15.81:0/809841278 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcad400e040 con 0x7fcae01026c0 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadcff9700 1 -- 172.21.15.81:0/809841278 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcad4013420 con 0x7fcae01026c0 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae010ddd0 con 0x7fcae01026c0 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcadcff9700 1 -- 172.21.15.81:0/809841278 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcad401c630 con 0x7fcae01026c0 2024-03-22T07:12:30.868 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.866+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae010e310 con 0x7fcae01026c0 2024-03-22T07:12:30.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.867+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae00fd470 con 0x7fcae01026c0 2024-03-22T07:12:30.869 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.868+0000 7fcadcff9700 1 -- 172.21.15.81:0/809841278 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fcad4013a00 con 0x7fcae01026c0 2024-03-22T07:12:30.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.868+0000 7fcadcff9700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fcac806cc80 0x7fcac806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:30.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.868+0000 7fcadcff9700 1 -- 172.21.15.81:0/809841278 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fcad408d0f0 con 0x7fcae01026c0 2024-03-22T07:12:30.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.868+0000 7fcadffff700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fcac806cc80 0x7fcac806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:30.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.869+0000 7fcadffff700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fcac806cc80 0x7fcac806f130 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcad0007fd0 tx=0x7fcad0012040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:30.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:30.873+0000 7fcadcff9700 1 -- 172.21.15.81:0/809841278 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fcad4057450 con 0x7fcae01026c0 2024-03-22T07:12:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.095+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcae010eff0 con 0x7fcac806cc80 2024-03-22T07:12:31.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.096+0000 7fcadcff9700 1 -- 172.21.15.81:0/809841278 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fcae010eff0 con 0x7fcac806cc80 2024-03-22T07:12:31.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fcac806cc80 msgr2=0x7fcac806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.098 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fcac806cc80 0x7fcac806f130 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcad0007fd0 tx=0x7fcad0012040 comp rx=0 tx=0).stop 2024-03-22T07:12:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 msgr2=0x7fcae0113f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 0x7fcae0113f50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009100 tx=0x7fcad4009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 shutdown_connections 2024-03-22T07:12:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fcac806cc80 0x7fcac806f130 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcae0101d20 0x7fcae0113a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 --2- 172.21.15.81:0/809841278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcae01026c0 0x7fcae0113f50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 >> 172.21.15.81:0/809841278 conn(0x7fcae00f7870 msgr2=0x7fcae0066200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 shutdown_connections 2024-03-22T07:12:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.098+0000 7fcae6a28700 1 -- 172.21.15.81:0/809841278 wait complete. 2024-03-22T07:12:31.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 -- 172.21.15.81:0/1467059304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 msgr2=0x7fe4cc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 --2- 172.21.15.81:0/1467059304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 0x7fe4cc1031d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4004940 tx=0x7fe4c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:31.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 -- 172.21.15.81:0/1467059304 shutdown_connections 2024-03-22T07:12:31.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 --2- 172.21.15.81:0/1467059304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4cc1037a0 0x7fe4cc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 --2- 172.21.15.81:0/1467059304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 0x7fe4cc1031d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 -- 172.21.15.81:0/1467059304 >> 172.21.15.81:0/1467059304 conn(0x7fe4cc0f89b0 msgr2=0x7fe4cc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 -- 172.21.15.81:0/1467059304 shutdown_connections 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 -- 172.21.15.81:0/1467059304 wait complete. 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 Processor -- start 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.199+0000 7fe4d2142700 1 -- start start 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4d2142700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 0x7fe4cc115b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4d2142700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4cc1037a0 0x7fe4cc116070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4d2142700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc1166c0 con 0x7fe4cc102e00 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4d2142700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc10fce0 con 0x7fe4cc1037a0 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4cb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 0x7fe4cc115b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4cb7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 0x7fe4cc115b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49710/0 (socket says 172.21.15.81:49710) 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4cb7fe700 1 -- 172.21.15.81:0/1694158319 learned_addr learned my addr 172.21.15.81:0/1694158319 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:31.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4caffd700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4cc1037a0 0x7fe4cc116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:31.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4cb7fe700 1 -- 172.21.15.81:0/1694158319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4cc1037a0 msgr2=0x7fe4cc116070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4cb7fe700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4cc1037a0 0x7fe4cc116070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4cb7fe700 1 -- 172.21.15.81:0/1694158319 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c4005040 con 0x7fe4cc102e00 2024-03-22T07:12:31.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4caffd700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4cc1037a0 0x7fe4cc116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:12:31.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4cb7fe700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 0x7fe4cc115b30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400b3b0 tx=0x7fe4c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:31.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.200+0000 7fe4c3fff700 1 -- 172.21.15.81:0/1694158319 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c400e040 con 0x7fe4cc102e00 2024-03-22T07:12:31.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.201+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4cc10ff60 con 0x7fe4cc102e00 2024-03-22T07:12:31.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.201+0000 7fe4c3fff700 1 -- 172.21.15.81:0/1694158319 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4c400bbc0 con 0x7fe4cc102e00 2024-03-22T07:12:31.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.201+0000 7fe4c3fff700 1 -- 172.21.15.81:0/1694158319 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c401c3f0 con 0x7fe4cc102e00 2024-03-22T07:12:31.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.201+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc110520 con 0x7fe4cc102e00 2024-03-22T07:12:31.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.201+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc107300 con 0x7fe4cc102e00 2024-03-22T07:12:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.202+0000 7fe4c3fff700 1 -- 172.21.15.81:0/1694158319 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fe4c400bd30 con 0x7fe4cc102e00 2024-03-22T07:12:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.203+0000 7fe4c3fff700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe4b806cc80 0x7fe4b806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.203+0000 7fe4c3fff700 1 -- 172.21.15.81:0/1694158319 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe4c408d250 con 0x7fe4cc102e00 2024-03-22T07:12:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.203+0000 7fe4caffd700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe4b806cc80 0x7fe4b806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:31.204 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.203+0000 7fe4caffd700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe4b806cc80 0x7fe4b806f130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009fd0 tx=0x7fe4b400b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:31.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.207+0000 7fe4c3fff700 1 -- 172.21.15.81:0/1694158319 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe4c4057530 con 0x7fe4cc102e00 2024-03-22T07:12:31.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.413+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4cc109f70 con 0x7fe4b806cc80 2024-03-22T07:12:31.423 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.423+0000 7fe4c3fff700 1 -- 172.21.15.81:0/1694158319 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fe4cc109f70 con 0x7fe4b806cc80 2024-03-22T07:12:31.423 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:12:31.423 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (65s) 42s ago 11m 16.7M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:12:31.423 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (11m) 42s ago 11m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (10m) 97s ago 10m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (47s) 42s ago 11m 56.8M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (6m) 42s ago 6m 12.6M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (5m) 42s ago 5m 132M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (5m) 97s ago 5m 15.5M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (6m) 97s ago 6m 148M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (2m) 42s ago 12m 498M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (117s) 97s ago 10m 447M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (12m) 42s ago 12m 53.2M 2048M 17.2.7-726-g9c4e0c34 837124da804f 32e741b91680 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (10m) 97s ago 10m 44.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f 33803b1099d7 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (105s) 42s ago 11m 8254k - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (101s) 97s ago 10m 3640k - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (9m) 42s ago 9m 184M 1557M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:12:31.424 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (9m) 42s ago 9m 198M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:12:31.425 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (8m) 42s ago 8m 161M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:12:31.425 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (8m) 97s ago 8m 214M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:12:31.425 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (8m) 97s ago 8m 191M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:12:31.425 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (7m) 97s ago 7m 181M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:12:31.425 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (83s) 42s ago 10m 47.0M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe4b806cc80 msgr2=0x7fe4b806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe4b806cc80 0x7fe4b806f130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009fd0 tx=0x7fe4b400b040 comp rx=0 tx=0).stop 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 msgr2=0x7fe4cc115b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 0x7fe4cc115b30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400b3b0 tx=0x7fe4c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 shutdown_connections 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe4b806cc80 0x7fe4b806f130 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4cc102e00 0x7fe4cc115b30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 --2- 172.21.15.81:0/1694158319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4cc1037a0 0x7fe4cc116070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 >> 172.21.15.81:0/1694158319 conn(0x7fe4cc0f89b0 msgr2=0x7fe4cc070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 shutdown_connections 2024-03-22T07:12:31.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.425+0000 7fe4d2142700 1 -- 172.21.15.81:0/1694158319 wait complete. 2024-03-22T07:12:31.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.524+0000 7fe673ac7700 1 -- 172.21.15.81:0/191525393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c101e60 msgr2=0x7fe66c102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.524+0000 7fe673ac7700 1 --2- 172.21.15.81:0/191525393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c101e60 0x7fe66c102230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe668004940 tx=0x7fe668009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:31.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.524+0000 7fe673ac7700 1 -- 172.21.15.81:0/191525393 shutdown_connections 2024-03-22T07:12:31.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.524+0000 7fe673ac7700 1 --2- 172.21.15.81:0/191525393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe66c102800 0x7fe66c0fc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.524+0000 7fe673ac7700 1 --2- 172.21.15.81:0/191525393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c101e60 0x7fe66c102230 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.524+0000 7fe673ac7700 1 -- 172.21.15.81:0/191525393 >> 172.21.15.81:0/191525393 conn(0x7fe66c071530 msgr2=0x7fe66c071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:31.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.525+0000 7fe673ac7700 1 -- 172.21.15.81:0/191525393 shutdown_connections 2024-03-22T07:12:31.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.525+0000 7fe673ac7700 1 -- 172.21.15.81:0/191525393 wait complete. 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.525+0000 7fe673ac7700 1 Processor -- start 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.525+0000 7fe673ac7700 1 -- start start 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.525+0000 7fe673ac7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe66c101e60 0x7fe66c102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.525+0000 7fe673ac7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c102800 0x7fe66c1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.525+0000 7fe673ac7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c104bf0 con 0x7fe66c102800 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.525+0000 7fe673ac7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c103720 con 0x7fe66c101e60 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.525+0000 7fe671062700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c102800 0x7fe66c1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe671062700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c102800 0x7fe66c1031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49724/0 (socket says 172.21.15.81:49724) 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe671062700 1 -- 172.21.15.81:0/1520033705 learned_addr learned my addr 172.21.15.81:0/1520033705 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe671863700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe66c101e60 0x7fe66c102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe671062700 1 -- 172.21.15.81:0/1520033705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe66c101e60 msgr2=0x7fe66c102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe671062700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe66c101e60 0x7fe66c102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe671062700 1 -- 172.21.15.81:0/1520033705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe668005040 con 0x7fe66c102800 2024-03-22T07:12:31.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe671062700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c102800 0x7fe66c1031e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe65c002a50 tx=0x7fe65c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:31.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe6627fc700 1 -- 172.21.15.81:0/1520033705 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe65c007cc0 con 0x7fe66c102800 2024-03-22T07:12:31.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe6627fc700 1 -- 172.21.15.81:0/1520033705 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe65c00f040 con 0x7fe66c102800 2024-03-22T07:12:31.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c103a00 con 0x7fe66c102800 2024-03-22T07:12:31.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe6627fc700 1 -- 172.21.15.81:0/1520033705 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe65c00d9c0 con 0x7fe66c102800 2024-03-22T07:12:31.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.526+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c11a140 con 0x7fe66c102800 2024-03-22T07:12:31.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.527+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c1001d0 con 0x7fe66c102800 2024-03-22T07:12:31.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.528+0000 7fe6627fc700 1 -- 172.21.15.81:0/1520033705 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fe65c009730 con 0x7fe66c102800 2024-03-22T07:12:31.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.528+0000 7fe6627fc700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe65806cc80 0x7fe65806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:31.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.528+0000 7fe6627fc700 1 -- 172.21.15.81:0/1520033705 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe65c08bce0 con 0x7fe66c102800 2024-03-22T07:12:31.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.528+0000 7fe671863700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe65806cc80 0x7fe65806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:31.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.529+0000 7fe671863700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe65806cc80 0x7fe65806f130 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe668004f90 tx=0x7fe668009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:31.534 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.534+0000 7fe6627fc700 1 -- 172.21.15.81:0/1520033705 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe65c055fc0 con 0x7fe66c102800 2024-03-22T07:12:31.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.815+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe66c108f60 con 0x7fe66c102800 2024-03-22T07:12:31.816 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:31 smithi081 ceph-mon[29245]: pgmap v72: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:31.816 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:31 smithi081 ceph-mon[29245]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.816+0000 7fe6627fc700 1 -- 172.21.15.81:0/1520033705 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fe65c059680 con 0x7fe66c102800 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:12:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:12:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:12:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:12:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:12:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:12:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T07:12:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:12:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:12:31.818 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:12:31.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe65806cc80 msgr2=0x7fe65806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe65806cc80 0x7fe65806f130 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe668004f90 tx=0x7fe668009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c102800 msgr2=0x7fe66c1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c102800 0x7fe66c1031e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe65c002a50 tx=0x7fe65c004600 comp rx=0 tx=0).stop 2024-03-22T07:12:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 shutdown_connections 2024-03-22T07:12:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fe65806cc80 0x7fe65806f130 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe66c101e60 0x7fe66c102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 --2- 172.21.15.81:0/1520033705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe66c102800 0x7fe66c1031e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 >> 172.21.15.81:0/1520033705 conn(0x7fe66c071530 msgr2=0x7fe66c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 shutdown_connections 2024-03-22T07:12:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.819+0000 7fe673ac7700 1 -- 172.21.15.81:0/1520033705 wait complete. 2024-03-22T07:12:31.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.923+0000 7eff314bf700 1 -- 172.21.15.81:0/2814124263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c1037b0 msgr2=0x7eff2c0fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.923+0000 7eff314bf700 1 --2- 172.21.15.81:0/2814124263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c1037b0 0x7eff2c0fce20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7eff20004940 tx=0x7eff20009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:31.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.923+0000 7eff314bf700 1 -- 172.21.15.81:0/2814124263 shutdown_connections 2024-03-22T07:12:31.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.923+0000 7eff314bf700 1 --2- 172.21.15.81:0/2814124263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c1037b0 0x7eff2c0fce20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.923+0000 7eff314bf700 1 --2- 172.21.15.81:0/2814124263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c102e10 0x7eff2c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.923+0000 7eff314bf700 1 -- 172.21.15.81:0/2814124263 >> 172.21.15.81:0/2814124263 conn(0x7eff2c0f89a0 msgr2=0x7eff2c0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:31.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.923+0000 7eff314bf700 1 -- 172.21.15.81:0/2814124263 shutdown_connections 2024-03-22T07:12:31.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.923+0000 7eff314bf700 1 -- 172.21.15.81:0/2814124263 wait complete. 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.923+0000 7eff314bf700 1 Processor -- start 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff314bf700 1 -- start start 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff314bf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c102e10 0x7eff2c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff314bf700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c1037b0 0x7eff2c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff314bf700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c070d90 con 0x7eff2c102e10 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff314bf700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c070ed0 con 0x7eff2c1037b0 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff2affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c102e10 0x7eff2c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff2affd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c102e10 0x7eff2c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49742/0 (socket says 172.21.15.81:49742) 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff2affd700 1 -- 172.21.15.81:0/2425189531 learned_addr learned my addr 172.21.15.81:0/2425189531 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff2a7fc700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c1037b0 0x7eff2c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff2affd700 1 -- 172.21.15.81:0/2425189531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c1037b0 msgr2=0x7eff2c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:31.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff2affd700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c1037b0 0x7eff2c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:31.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff2affd700 1 -- 172.21.15.81:0/2425189531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff20005040 con 0x7eff2c102e10 2024-03-22T07:12:31.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff2a7fc700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c1037b0 0x7eff2c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:12:31.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff2affd700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c102e10 0x7eff2c0702c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7eff1c002a80 tx=0x7eff1c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:31.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff137fe700 1 -- 172.21.15.81:0/2425189531 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff1c007cc0 con 0x7eff2c102e10 2024-03-22T07:12:31.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.924+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c119de0 con 0x7eff2c102e10 2024-03-22T07:12:31.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.925+0000 7eff137fe700 1 -- 172.21.15.81:0/2425189531 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff1c00f040 con 0x7eff2c102e10 2024-03-22T07:12:31.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.925+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c11a260 con 0x7eff2c102e10 2024-03-22T07:12:31.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.925+0000 7eff137fe700 1 -- 172.21.15.81:0/2425189531 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff1c009690 con 0x7eff2c102e10 2024-03-22T07:12:31.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.925+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c1052e0 con 0x7eff2c102e10 2024-03-22T07:12:31.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.926+0000 7eff137fe700 1 -- 172.21.15.81:0/2425189531 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7eff1c0097f0 con 0x7eff2c102e10 2024-03-22T07:12:31.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.926+0000 7eff137fe700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7eff1406cc80 0x7eff1406f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:31.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.927+0000 7eff137fe700 1 -- 172.21.15.81:0/2425189531 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7eff1c08b700 con 0x7eff2c102e10 2024-03-22T07:12:31.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.927+0000 7eff2a7fc700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7eff1406cc80 0x7eff1406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:31.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.927+0000 7eff2a7fc700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7eff1406cc80 0x7eff1406f130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7eff20000c00 tx=0x7eff20009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:31.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:31.931+0000 7eff137fe700 1 -- 172.21.15.81:0/2425189531 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7eff1c0559e0 con 0x7eff2c102e10 2024-03-22T07:12:32.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:31 smithi149 ceph-mon[36068]: pgmap v72: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:32.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:31 smithi149 ceph-mon[36068]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:32.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.183+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7eff2c04bed0 con 0x7eff2c102e10 2024-03-22T07:12:32.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.184+0000 7eff137fe700 1 -- 172.21.15.81:0/2425189531 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1820 (secure 0 0 0) 0x7eff1c0590a0 con 0x7eff2c102e10 2024-03-22T07:12:32.184 INFO:teuthology.orchestra.run.smithi081.stdout:e21 2024-03-22T07:12:32.184 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:12:32.184 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:12:32.184 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:12:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:12:32.184 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:12:32.184 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:12:32.185 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 21 2024-03-22T07:12:32.185 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:12:32.185 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:12:32.185 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:12:23.482585+0000 2024-03-22T07:12:32.185 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:12:32.186 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 82 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 81 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:12:32.187 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:12:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:12:32.188 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:12:32.188 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:12:32.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7eff1406cc80 msgr2=0x7eff1406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7eff1406cc80 0x7eff1406f130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7eff20000c00 tx=0x7eff20009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c102e10 msgr2=0x7eff2c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c102e10 0x7eff2c0702c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7eff1c002a80 tx=0x7eff1c0046b0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 shutdown_connections 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7eff1406cc80 0x7eff1406f130 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff2c102e10 0x7eff2c0702c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 --2- 172.21.15.81:0/2425189531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c1037b0 0x7eff2c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 >> 172.21.15.81:0/2425189531 conn(0x7eff2c0f89a0 msgr2=0x7eff2c1021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 shutdown_connections 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.186+0000 7eff314bf700 1 -- 172.21.15.81:0/2425189531 wait complete. 2024-03-22T07:12:32.189 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 21 2024-03-22T07:12:32.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.292+0000 7fc0862fc700 1 -- 172.21.15.81:0/2276543094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 msgr2=0x7fc0801020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.295 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.292+0000 7fc0862fc700 1 --2- 172.21.15.81:0/2276543094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 0x7fc0801020f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc070004940 tx=0x7fc070009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.292+0000 7fc0862fc700 1 -- 172.21.15.81:0/2276543094 shutdown_connections 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.293+0000 7fc0862fc700 1 --2- 172.21.15.81:0/2276543094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0801026c0 0x7fc0800fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.293+0000 7fc0862fc700 1 --2- 172.21.15.81:0/2276543094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 0x7fc0801020f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.293+0000 7fc0862fc700 1 -- 172.21.15.81:0/2276543094 >> 172.21.15.81:0/2276543094 conn(0x7fc0800f7870 msgr2=0x7fc0800f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.293+0000 7fc0862fc700 1 -- 172.21.15.81:0/2276543094 shutdown_connections 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.293+0000 7fc0862fc700 1 -- 172.21.15.81:0/2276543094 wait complete. 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.293+0000 7fc0862fc700 1 Processor -- start 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.293+0000 7fc0862fc700 1 -- start start 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc0862fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 0x7fc08010c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc0862fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0801026c0 0x7fc08010ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc0862fc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc080067120 con 0x7fc080101d20 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc0862fc700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08010d3f0 con 0x7fc0801026c0 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 0x7fc08010c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 0x7fc08010c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49766/0 (socket says 172.21.15.81:49766) 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07ffff700 1 -- 172.21.15.81:0/2121510343 learned_addr learned my addr 172.21.15.81:0/2121510343 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07f7fe700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0801026c0 0x7fc08010ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07ffff700 1 -- 172.21.15.81:0/2121510343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0801026c0 msgr2=0x7fc08010ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07ffff700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0801026c0 0x7fc08010ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07ffff700 1 -- 172.21.15.81:0/2121510343 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc070005040 con 0x7fc080101d20 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07f7fe700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0801026c0 0x7fc08010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07ffff700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 0x7fc08010c970 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc070009150 tx=0x7fc070009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07cff9700 1 -- 172.21.15.81:0/2121510343 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07000e040 con 0x7fc080101d20 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08010d670 con 0x7fc080101d20 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08011a1a0 con 0x7fc080101d20 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.294+0000 7fc07cff9700 1 -- 172.21.15.81:0/2121510343 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc070013420 con 0x7fc080101d20 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.295+0000 7fc07cff9700 1 -- 172.21.15.81:0/2121510343 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07001c630 con 0x7fc080101d20 2024-03-22T07:12:32.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.295+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0800fd470 con 0x7fc080101d20 2024-03-22T07:12:32.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.296+0000 7fc07cff9700 1 -- 172.21.15.81:0/2121510343 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fc070013590 con 0x7fc080101d20 2024-03-22T07:12:32.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.296+0000 7fc07cff9700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc06806cc80 0x7fc06806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:32.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.296+0000 7fc07cff9700 1 -- 172.21.15.81:0/2121510343 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fc07008d160 con 0x7fc080101d20 2024-03-22T07:12:32.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.296+0000 7fc07f7fe700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc06806cc80 0x7fc06806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:32.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.297+0000 7fc07f7fe700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc06806cc80 0x7fc06806f130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc074009fd0 tx=0x7fc074009ee0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:32.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.301+0000 7fc07cff9700 1 -- 172.21.15.81:0/2121510343 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fc07005b090 con 0x7fc080101d20 2024-03-22T07:12:32.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.528+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 --> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0800f8f30 con 0x7fc06806cc80 2024-03-22T07:12:32.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.529+0000 7fc07cff9700 1 -- 172.21.15.81:0/2121510343 <== mgr.14688 v2:172.21.15.81:6800/2774978653 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc0800f8f30 con 0x7fc06806cc80 2024-03-22T07:12:32.529 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr" 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "2/21 daemons upgraded", 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading mon daemons", 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:12:32.530 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:12:32.531 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc06806cc80 msgr2=0x7fc06806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc06806cc80 0x7fc06806f130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc074009fd0 tx=0x7fc074009ee0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 msgr2=0x7fc08010c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 0x7fc08010c970 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc070009150 tx=0x7fc070009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 shutdown_connections 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7fc06806cc80 0x7fc06806f130 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc080101d20 0x7fc08010c970 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 --2- 172.21.15.81:0/2121510343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc0801026c0 0x7fc08010ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 >> 172.21.15.81:0/2121510343 conn(0x7fc0800f7870 msgr2=0x7fc0800f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 shutdown_connections 2024-03-22T07:12:32.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.531+0000 7fc0862fc700 1 -- 172.21.15.81:0/2121510343 wait complete. 2024-03-22T07:12:32.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.635+0000 7f316d3ad700 1 -- 172.21.15.81:0/4200527828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3168101d20 msgr2=0x7f31681020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.635+0000 7f316d3ad700 1 --2- 172.21.15.81:0/4200527828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3168101d20 0x7f31681020f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f315c004940 tx=0x7f315c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:32.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.635+0000 7f316d3ad700 1 -- 172.21.15.81:0/4200527828 shutdown_connections 2024-03-22T07:12:32.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.635+0000 7f316d3ad700 1 --2- 172.21.15.81:0/4200527828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31681026c0 0x7f31680fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.635+0000 7f316d3ad700 1 --2- 172.21.15.81:0/4200527828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3168101d20 0x7f31681020f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.635+0000 7f316d3ad700 1 -- 172.21.15.81:0/4200527828 >> 172.21.15.81:0/4200527828 conn(0x7f31680f7870 msgr2=0x7f31680f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:32.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.635+0000 7f316d3ad700 1 -- 172.21.15.81:0/4200527828 shutdown_connections 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.635+0000 7f316d3ad700 1 -- 172.21.15.81:0/4200527828 wait complete. 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f316d3ad700 1 Processor -- start 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f316d3ad700 1 -- start start 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f316d3ad700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3168101d20 0x7f31681128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f316d3ad700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31681026c0 0x7f3168112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f316d3ad700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3168113460 con 0x7f31681026c0 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f316d3ad700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316810ca60 con 0x7f3168101d20 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f31667fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31681026c0 0x7f3168112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f31667fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31681026c0 0x7f3168112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49786/0 (socket says 172.21.15.81:49786) 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f31667fc700 1 -- 172.21.15.81:0/3287933363 learned_addr learned my addr 172.21.15.81:0/3287933363 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f3166ffd700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3168101d20 0x7f31681128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f31667fc700 1 -- 172.21.15.81:0/3287933363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3168101d20 msgr2=0x7f31681128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f31667fc700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3168101d20 0x7f31681128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.638 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f31667fc700 1 -- 172.21.15.81:0/3287933363 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c005040 con 0x7f31681026c0 2024-03-22T07:12:32.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.636+0000 7f3166ffd700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3168101d20 0x7f31681128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:12:32.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.637+0000 7f31667fc700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31681026c0 0x7f3168112e10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f315800ed20 tx=0x7f3158007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:32.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.637+0000 7f314f7fe700 1 -- 172.21.15.81:0/3287933363 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f315800cd40 con 0x7f31681026c0 2024-03-22T07:12:32.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.637+0000 7f314f7fe700 1 -- 172.21.15.81:0/3287933363 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3158011940 con 0x7f31681026c0 2024-03-22T07:12:32.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.637+0000 7f314f7fe700 1 -- 172.21.15.81:0/3287933363 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3158010990 con 0x7f31681026c0 2024-03-22T07:12:32.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.637+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316810cce0 con 0x7f31681026c0 2024-03-22T07:12:32.639 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.637+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316810d2a0 con 0x7f31681026c0 2024-03-22T07:12:32.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.638+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31680fd470 con 0x7f31681026c0 2024-03-22T07:12:32.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.638+0000 7f314f7fe700 1 -- 172.21.15.81:0/3287933363 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f3158011ab0 con 0x7f31681026c0 2024-03-22T07:12:32.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.639+0000 7f314f7fe700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f315006cc80 0x7f315006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:32.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.639+0000 7f314f7fe700 1 -- 172.21.15.81:0/3287933363 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f315808e830 con 0x7f31681026c0 2024-03-22T07:12:32.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.639+0000 7f3166ffd700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f315006cc80 0x7f315006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:32.640 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.640+0000 7f3166ffd700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f315006cc80 0x7f315006f130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f315c004f90 tx=0x7f315c009e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:32.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.643+0000 7f314f7fe700 1 -- 172.21.15.81:0/3287933363 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f315805c760 con 0x7f31681026c0 2024-03-22T07:12:32.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.935+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3168108e20 con 0x7f31681026c0 2024-03-22T07:12:32.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.935+0000 7f314f7fe700 1 -- 172.21.15.81:0/3287933363 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+250 (secure 0 0 0) 0x7f315805c2f0 con 0x7f31681026c0 2024-03-22T07:12:32.936 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:12:32.936 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:12:32.936 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:12:32.936 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:12:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.937+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f315006cc80 msgr2=0x7f315006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.937+0000 7f316d3ad700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f315006cc80 0x7f315006f130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f315c004f90 tx=0x7f315c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.937+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31681026c0 msgr2=0x7f3168112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.937+0000 7f316d3ad700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31681026c0 0x7f3168112e10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f315800ed20 tx=0x7f3158007f90 comp rx=0 tx=0).stop 2024-03-22T07:12:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.937+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 shutdown_connections 2024-03-22T07:12:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.937+0000 7f316d3ad700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:6800/2774978653,v1:172.21.15.81:6801/2774978653] conn(0x7f315006cc80 0x7f315006f130 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.937+0000 7f316d3ad700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3168101d20 0x7f31681128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.937+0000 7f316d3ad700 1 --2- 172.21.15.81:0/3287933363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31681026c0 0x7f3168112e10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.937+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 >> 172.21.15.81:0/3287933363 conn(0x7f31680f7870 msgr2=0x7f31680f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:32.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.938+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 shutdown_connections 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (squid) 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: Upgrade: Setting container_image for all mgr 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T07:12:32.939 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/1520033705' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:32.940 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/2425189531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:12:32.940 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:32 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 232.625622 secs 2024-03-22T07:12:32.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:12:32.938+0000 7f316d3ad700 1 -- 172.21.15.81:0/3287933363 wait complete. 2024-03-22T07:12:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (squid) 2024-03-22T07:12:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-22T07:12:33.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:33.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:33.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: Upgrade: Setting container_image for all mgr 2024-03-22T07:12:33.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:33.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T07:12:33.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/1520033705' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:33.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/2425189531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:12:33.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:32 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 232.625622 secs 2024-03-22T07:12:33.993 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:33 smithi081 ceph-mon[29245]: pgmap v73: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:33.993 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:33 smithi081 ceph-mon[29245]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:33.993 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:33 smithi081 ceph-mon[29245]: 1 slow requests, 0 included below; oldest blocked for > 232.965410 secs 2024-03-22T07:12:33.993 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:33 smithi081 ceph-mon[29245]: from='client.? 172.21.15.81:0/3287933363' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:12:33.993 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:33 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:33.993 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:33 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:12:33.993 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:33 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:12:33.993 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:33 smithi081 ceph-mon[29245]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:33 smithi149 ceph-mon[36068]: pgmap v73: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:33 smithi149 ceph-mon[36068]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:12:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:33 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 232.965410 secs 2024-03-22T07:12:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:33 smithi149 ceph-mon[36068]: from='client.? 172.21.15.81:0/3287933363' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:12:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:33 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:34.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:33 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:12:34.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:33 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:12:34.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:33 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:35.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:34 smithi081 ceph-mon[29245]: Upgrade: Updating mon.smithi081 2024-03-22T07:12:35.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:34 smithi081 ceph-mon[29245]: Deploying daemon mon.smithi081 on smithi081 2024-03-22T07:12:35.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:34 smithi149 ceph-mon[36068]: Upgrade: Updating mon.smithi081 2024-03-22T07:12:35.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:34 smithi149 ceph-mon[36068]: Deploying daemon mon.smithi081 on smithi081 2024-03-22T07:12:35.720 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:35 smithi081 systemd[1]: Stopping Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:12:36.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:35 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081[29240]: 2024-03-22T07:12:35.719+0000 7f9a82198700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi081 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:12:36.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:35 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081[29240]: 2024-03-22T07:12:35.719+0000 7f9a82198700 -1 mon.smithi081@0(leader) e2 *** Got Signal Terminated *** 2024-03-22T07:12:36.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:35 smithi081 podman[113794]: 2024-03-22 07:12:35.902853771 +0000 UTC m=+0.294775076 container died 32e741b91680caae3d4735d27fea7aaec08f7c1cf189869ce1e6193b880f1fa2 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-22T07:12:36.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:36 smithi081 podman[113794]: 2024-03-22 07:12:36.344581453 +0000 UTC m=+0.736502765 container cleanup 32e741b91680caae3d4735d27fea7aaec08f7c1cf189869ce1e6193b880f1fa2 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=quincy-9c4e0c3, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-22T07:12:36.554 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:36 smithi081 bash[113794]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081 2024-03-22T07:12:36.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:36 smithi081 podman[113819]: 2024-03-22 07:12:36.554809925 +0000 UTC m=+0.647683309 container remove 32e741b91680caae3d4735d27fea7aaec08f7c1cf189869ce1e6193b880f1fa2 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-22T07:12:36.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:36 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service: Deactivated successfully. 2024-03-22T07:12:36.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:36 smithi081 systemd[1]: Stopped Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:12:36.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:36 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service: Consumed 12.147s CPU time. 2024-03-22T07:12:37.230 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:36 smithi081 systemd[1]: Starting Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:12:37.230 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 podman[113946]: 2024-03-22 07:12:37.099869106 +0000 UTC m=+0.019147127 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:12:37.230 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 podman[113946]: 2024-03-22 07:12:37.230974499 +0000 UTC m=+0.150252522 container create 8f07a753bc9464cdde3ad4390ee028b92c3c614c5e4229ec5f117eb439360c50 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-22T07:12:37.481 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 podman[113946]: 2024-03-22 07:12:37.442206986 +0000 UTC m=+0.361485009 container init 8f07a753bc9464cdde3ad4390ee028b92c3c614c5e4229ec5f117eb439360c50 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 podman[113946]: 2024-03-22 07:12:37.445386645 +0000 UTC m=+0.364664679 container start 8f07a753bc9464cdde3ad4390ee028b92c3c614c5e4229ec5f117eb439360c50 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 2 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: pidfile_write: ignore empty --pid-file 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: load: jerasure load: lrc 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: RocksDB version: 7.9.2 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Git sha 0 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: DB SUMMARY 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: DB Session ID: PMX1Y1P5FPOCXE145N6C 2024-03-22T07:12:37.482 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: CURRENT file: CURRENT 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1173 Bytes 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi081/store.db dir, Total Num: 1, files: 000033.sst 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi081/store.db: 000031.log size: 1232459 ; 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.error_if_exists: 0 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.create_if_missing: 0 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.paranoid_checks: 1 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.env: 0x55c182826ae0 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.info_log: 0x55c18393b760 2024-03-22T07:12:37.483 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T07:12:37.484 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.statistics: (nil) 2024-03-22T07:12:37.484 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.use_fsync: 0 2024-03-22T07:12:37.484 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_log_file_size: 0 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.allow_fallocate: 1 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.use_direct_reads: 0 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.db_log_dir: 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.wal_dir: 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T07:12:37.485 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.write_buffer_manager: 0x55c18390f400 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T07:12:37.486 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.unordered_write: 0 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.row_cache: None 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.wal_filter: None 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.two_write_queues: 0 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T07:12:37.487 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.wal_compression: 0 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.atomic_flush: 0 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.log_readahead_size: 0 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_background_jobs: 2 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_background_compactions: -1 2024-03-22T07:12:37.488 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_subcompactions: 1 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_open_files: -1 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T07:12:37.489 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_background_flushes: -1 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Compression algorithms supported: 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: kZSTD supported: 0 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: kXpressCompression supported: 0 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: kZlibCompression supported: 1 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: kSnappyCompression supported: 1 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: kLZ4Compression supported: 1 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: kBZip2Compression supported: 0 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi081/store.db/MANIFEST-000015 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T07:12:37.490 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.merge_operator: 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_filter: None 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c18393b880) 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: cache_index_and_filter_blocks: 1 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: pin_top_level_index_and_filter: 1 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: index_type: 0 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: data_block_index_type: 0 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: index_shortening: 1 2024-03-22T07:12:37.491 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: checksum: 4 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: no_block_cache: 0 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_cache: 0x55c18393f090 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_cache_name: BinnedLRUCache 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_cache_options: 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: capacity : 536870912 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: num_shard_bits : 4 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: strict_capacity_limit : 0 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: high_pri_pool_ratio: 0.000 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_cache_compressed: (nil) 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: persistent_cache: (nil) 2024-03-22T07:12:37.492 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_size: 4096 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_size_deviation: 10 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_restart_interval: 16 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: index_block_restart_interval: 1 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: metadata_block_size: 4096 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: partition_filters: 0 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: use_delta_encoding: 1 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: filter_policy: bloomfilter 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: whole_key_filtering: 1 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: verify_compression: 0 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: read_amp_bytes_per_bit: 0 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: format_version: 5 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: enable_index_compression: 1 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: block_align: 0 2024-03-22T07:12:37.493 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: max_auto_readahead_size: 262144 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: prepopulate_block_cache: 0 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: initial_auto_readahead_size: 8192 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression: NoCompression 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T07:12:37.494 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.num_levels: 7 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T07:12:37.495 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T07:12:37.496 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T07:12:37.497 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T07:12:37.497 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T07:12:37.498 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.table_properties_collectors: 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.inplace_update_support: 0 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T07:12:37.499 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.bloom_locality: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.max_successive_merges: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.ttl: 2592000 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.enable_blob_files: false 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.min_blob_size: 0 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T07:12:37.500 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi081/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12029, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6901592a-6807-4c27-9bd3-b26e6c3aea42 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091557474008, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-03-22T07:12:37.501 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091557478683, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1129285, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12030, "largest_seqno": 13105, "table_properties": {"data_size": 1124098, "index_size": 2829, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1349, "raw_key_size": 12820, "raw_average_key_size": 24, "raw_value_size": 1113280, "raw_average_value_size": 2157, "num_data_blocks": 126, "num_entries": 516, "num_filter_entries": 516, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711091557, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6901592a-6807-4c27-9bd3-b26e6c3aea42", "db_session_id": "PMX1Y1P5FPOCXE145N6C", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091557478777, "job": 1, "event": "recovery_finished"} 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi081/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c183a2e000 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: DB pointer 0x55c183a1a000 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** DB Stats ** 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:12:37.502 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** Compaction Stats [default] ** 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: L0 1/0 1.08 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 426.9 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: L6 1/0 7.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Sum 2/0 8.13 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 426.9 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 426.9 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** Compaction Stats [default] ** 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T07:12:37.503 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 426.9 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Cumulative compaction: 0.00 GB write, 157.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Interval compaction: 0.00 GB write, 157.89 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Block cache BinnedLRUCache@0x55c18393f090#2 capacity: 512.00 MB usage: 4.48 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-22T07:12:37.504 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.41 KB,0.000268221%) IndexBlock(1,3.08 KB,0.000587106%) Misc(1,0.00 KB,0%) 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: starting mon.smithi081 rank 0 at public addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] at bind addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi081 fsid a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: mon.smithi081@-1(???) e2 preinit fsid a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: mon.smithi081@-1(???).mds e21 new map 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: mon.smithi081@-1(???).mds e21 print_map 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: e21 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:12:37.505 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: legacy client fscid: 1 2024-03-22T07:12:37.506 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T07:12:37.506 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Filesystem 'cephfs' (1) 2024-03-22T07:12:37.506 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: fs_name cephfs 2024-03-22T07:12:37.506 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: epoch 21 2024-03-22T07:12:37.506 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:12:37.506 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:12:37.507 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: modified 2024-03-22T07:12:23.482585+0000 2024-03-22T07:12:37.507 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: tableserver 0 2024-03-22T07:12:37.507 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: root 0 2024-03-22T07:12:37.507 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: session_timeout 60 2024-03-22T07:12:37.507 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: session_autoclose 300 2024-03-22T07:12:37.507 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: max_file_size 1099511627776 2024-03-22T07:12:37.507 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: max_xattr_size 65536 2024-03-22T07:12:37.508 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: required_client_features {} 2024-03-22T07:12:37.508 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: last_failure 0 2024-03-22T07:12:37.508 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: last_failure_osd_epoch 45 2024-03-22T07:12:37.508 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:12:37.508 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: max_mds 2 2024-03-22T07:12:37.508 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: in 0,1 2024-03-22T07:12:37.508 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: up {0=14488,1=24251} 2024-03-22T07:12:37.508 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: failed 2024-03-22T07:12:37.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: damaged 2024-03-22T07:12:37.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: stopped 2024-03-22T07:12:37.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: data_pools [3] 2024-03-22T07:12:37.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: metadata_pool 2 2024-03-22T07:12:37.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: inline_data disabled 2024-03-22T07:12:37.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: balancer 2024-03-22T07:12:37.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: bal_rank_mask -1 2024-03-22T07:12:37.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: standby_count_wanted 1 2024-03-22T07:12:37.509 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: [mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 82 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: [mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 81 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: Standby daemons: 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: [mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout: [mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: mon.smithi081@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: mon.smithi081@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: mon.smithi081@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:12:37.510 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: mon.smithi081@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:12:37.511 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 ceph-mon[113965]: mon.smithi081@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-03-22T07:12:37.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 bash[113946]: 8f07a753bc9464cdde3ad4390ee028b92c3c614c5e4229ec5f117eb439360c50 2024-03-22T07:12:37.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:37 smithi081 systemd[1]: Started Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:12:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:12:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:12:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: pgmap v75: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 237.625717 secs 2024-03-22T07:12:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 237.965556 secs 2024-03-22T07:12:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: pgmap v76: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: mon.smithi081 calling monitor election 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: mon.smithi081 is new leader, mons smithi081,smithi149 in quorum (ranks 0,1) 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: monmap e2: 2 mons at {smithi081=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0],smithi149=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: mgrmap e33: smithi081.bcxarw(active, since 2m), standbys: smithi149.sgmtoa 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: Health detail: HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: from='mgr.14688 ' entity='' 2024-03-22T07:12:39.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:38 smithi081 ceph-mon[113965]: mgrmap e34: smithi081.bcxarw(active, since 2m), standbys: smithi149.sgmtoa 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: from='mgr.14688 172.21.15.81:0/2109080056' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: pgmap v75: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 237.625717 secs 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 237.965556 secs 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: pgmap v76: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: mon.smithi081 calling monitor election 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: mon.smithi081 is new leader, mons smithi081,smithi149 in quorum (ranks 0,1) 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: monmap e2: 2 mons at {smithi081=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0],smithi149=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:12:39.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T07:12:39.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: mgrmap e33: smithi081.bcxarw(active, since 2m), standbys: smithi149.sgmtoa 2024-03-22T07:12:39.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: Health detail: HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:12:39.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:12:39.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:12:39.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:12:39.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: from='mgr.14688 ' entity='' 2024-03-22T07:12:39.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:38 smithi149 ceph-mon[36068]: mgrmap e34: smithi081.bcxarw(active, since 2m), standbys: smithi149.sgmtoa 2024-03-22T07:12:40.233 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:39 smithi149 ceph-mon[36068]: mds.0 [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:active 2024-03-22T07:12:40.233 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:39 smithi149 ceph-mon[36068]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:active 2024-03-22T07:12:40.233 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:39 smithi149 ceph-mon[36068]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:12:40.305 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:39 smithi081 ceph-mon[113965]: mds.0 [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] up:active 2024-03-22T07:12:40.305 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:39 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] up:active 2024-03-22T07:12:40.305 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:39 smithi081 ceph-mon[113965]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:12:42.829 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:42 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 242.625804 secs 2024-03-22T07:12:42.863 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:42 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 242.625804 secs 2024-03-22T07:12:43.791 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:43 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 242.965645 secs 2024-03-22T07:12:43.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:43 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 242.965645 secs 2024-03-22T07:12:46.020 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: Active manager daemon smithi081.bcxarw restarted 2024-03-22T07:12:46.020 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: Activating manager daemon smithi081.bcxarw 2024-03-22T07:12:46.020 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T07:12:46.020 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: mgrmap e35: smithi081.bcxarw(active, starting, since 0.00518084s), standbys: smithi149.sgmtoa 2024-03-22T07:12:46.020 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:12:46.020 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:12:46.020 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi149.sgmtoa", "id": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:12:46.021 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:12:46.022 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:45 smithi081 ceph-mon[113965]: Manager daemon smithi081.bcxarw is now available 2024-03-22T07:12:46.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149[36049]: 2024-03-22T07:12:45.789+0000 7f5a8442f700 -1 mon.smithi149@1(peon).osd e49 update_from_paxos full map CRC mismatch, resetting to canonical 2024-03-22T07:12:46.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: Active manager daemon smithi081.bcxarw restarted 2024-03-22T07:12:46.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: Activating manager daemon smithi081.bcxarw 2024-03-22T07:12:46.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: mgrmap e35: smithi081.bcxarw(active, starting, since 0.00518084s), standbys: smithi149.sgmtoa 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi081.bcxarw", "id": "smithi081.bcxarw"}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr metadata", "who": "smithi149.sgmtoa", "id": "smithi149.sgmtoa"}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:12:46.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:46.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:12:46.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:12:46.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:12:46.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:45 smithi149 ceph-mon[36068]: Manager daemon smithi081.bcxarw is now available 2024-03-22T07:12:47.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: failed to encode map e49 with expected crc 2024-03-22T07:12:47.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: Standby manager daemon smithi149.sgmtoa restarted 2024-03-22T07:12:47.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: Standby manager daemon smithi149.sgmtoa started 2024-03-22T07:12:47.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: from='mgr.? 172.21.15.149:0/1400965046' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/crt"}]: dispatch 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: from='mgr.? 172.21.15.149:0/1400965046' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: from='mgr.? 172.21.15.149:0/1400965046' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/key"}]: dispatch 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: from='mgr.? 172.21.15.149:0/1400965046' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/trash_purge_schedule"}]: dispatch 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: failed to encode map e49 with expected crc 2024-03-22T07:12:47.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:46 smithi081 ceph-mon[113965]: mgrmap e36: smithi081.bcxarw(active, since 1.00983s), standbys: smithi149.sgmtoa 2024-03-22T07:12:47.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: failed to encode map e49 with expected crc 2024-03-22T07:12:47.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: Standby manager daemon smithi149.sgmtoa restarted 2024-03-22T07:12:47.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: Standby manager daemon smithi149.sgmtoa started 2024-03-22T07:12:47.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:47.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/1400965046' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/crt"}]: dispatch 2024-03-22T07:12:47.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/1400965046' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:12:47.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:47.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/1400965046' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi149.sgmtoa/key"}]: dispatch 2024-03-22T07:12:47.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: from='mgr.? 172.21.15.149:0/1400965046' entity='mgr.smithi149.sgmtoa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:12:47.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:47.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:12:47.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi081.bcxarw/trash_purge_schedule"}]: dispatch 2024-03-22T07:12:47.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: failed to encode map e49 with expected crc 2024-03-22T07:12:47.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:46 smithi149 ceph-mon[36068]: mgrmap e36: smithi081.bcxarw(active, since 1.00983s), standbys: smithi149.sgmtoa 2024-03-22T07:12:48.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:47 smithi081 ceph-mon[113965]: [22/Mar/2024:07:12:46] ENGINE Bus STARTING 2024-03-22T07:12:48.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:47 smithi081 ceph-mon[113965]: pgmap v3: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:48.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:47 smithi081 ceph-mon[113965]: [22/Mar/2024:07:12:46] ENGINE Serving on https://172.21.15.81:7150 2024-03-22T07:12:48.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:47 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 247.625929 secs 2024-03-22T07:12:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:47 smithi149 ceph-mon[36068]: [22/Mar/2024:07:12:46] ENGINE Bus STARTING 2024-03-22T07:12:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:47 smithi149 ceph-mon[36068]: pgmap v3: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:47 smithi149 ceph-mon[36068]: [22/Mar/2024:07:12:46] ENGINE Serving on https://172.21.15.81:7150 2024-03-22T07:12:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:47 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 247.625929 secs 2024-03-22T07:12:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:48 smithi149 ceph-mon[36068]: [22/Mar/2024:07:12:46] ENGINE Serving on http://172.21.15.81:8765 2024-03-22T07:12:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:48 smithi149 ceph-mon[36068]: [22/Mar/2024:07:12:46] ENGINE Bus STARTED 2024-03-22T07:12:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:48 smithi149 ceph-mon[36068]: pgmap v4: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:48 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 247.965774 secs 2024-03-22T07:12:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:48 smithi149 ceph-mon[36068]: mgrmap e37: smithi081.bcxarw(active, since 3s), standbys: smithi149.sgmtoa 2024-03-22T07:12:49.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:48 smithi081 ceph-mon[113965]: [22/Mar/2024:07:12:46] ENGINE Serving on http://172.21.15.81:8765 2024-03-22T07:12:49.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:48 smithi081 ceph-mon[113965]: [22/Mar/2024:07:12:46] ENGINE Bus STARTED 2024-03-22T07:12:49.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:48 smithi081 ceph-mon[113965]: pgmap v4: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:49.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:48 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 247.965774 secs 2024-03-22T07:12:49.170 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:48 smithi081 ceph-mon[113965]: mgrmap e37: smithi081.bcxarw(active, since 3s), standbys: smithi149.sgmtoa 2024-03-22T07:12:50.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:49 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:50.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:49 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:50.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:49 smithi081 ceph-mon[113965]: pgmap v5: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:50.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:49 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:50.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:49 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:50.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:49 smithi149 ceph-mon[36068]: pgmap v5: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:51.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:51 smithi081 ceph-mon[113965]: mgrmap e38: smithi081.bcxarw(active, since 4s), standbys: smithi149.sgmtoa 2024-03-22T07:12:51.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:51 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:51.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:51 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:51.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:51 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:12:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:51 smithi149 ceph-mon[36068]: mgrmap e38: smithi081.bcxarw(active, since 4s), standbys: smithi149.sgmtoa 2024-03-22T07:12:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:51 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:51 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:51 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:12:52.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:52 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:52.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:52 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:52.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:52 smithi081 ceph-mon[113965]: pgmap v6: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:52 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:52 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:52 smithi149 ceph-mon[36068]: pgmap v6: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:12:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:53 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:53 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:53 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 252.625997 secs 2024-03-22T07:12:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:53 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 252.965921 secs 2024-03-22T07:12:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:53 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:53 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:53 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 252.625997 secs 2024-03-22T07:12:53.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:53 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 252.965921 secs 2024-03-22T07:12:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:54 smithi081 ceph-mon[113965]: pgmap v7: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-22T07:12:54.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:54 smithi149 ceph-mon[36068]: pgmap v7: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: pgmap v8: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: Detected new or changed devices on smithi081 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: Updating smithi081:/etc/ceph/ceph.conf 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: pgmap v8: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T07:12:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: Detected new or changed devices on smithi081 2024-03-22T07:12:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: Updating smithi081:/etc/ceph/ceph.conf 2024-03-22T07:12:57.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-22T07:12:57.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:57.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:56 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:12:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:12:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:12:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:12:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:12:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:12:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T07:12:58.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:57 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 257.626107 secs 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: Updating smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: Updating smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:12:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T07:12:58.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:57 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 257.626107 secs 2024-03-22T07:12:59.260 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:58 smithi149 ceph-mon[36068]: pgmap v9: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-22T07:12:59.261 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:58 smithi149 ceph-mon[36068]: 1 slow requests, 0 included below; oldest blocked for > 257.966015 secs 2024-03-22T07:12:59.261 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:58 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:59.261 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:58 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:12:59.261 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:58 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:12:59.261 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:58 smithi149 ceph-mon[36068]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:59.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:58 smithi081 ceph-mon[113965]: pgmap v9: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-22T07:12:59.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:58 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 257.966015 secs 2024-03-22T07:12:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:12:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:12:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:12:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:00.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:59 smithi149 ceph-mon[36068]: Upgrade: Updating mon.smithi149 2024-03-22T07:13:00.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:12:59 smithi149 ceph-mon[36068]: Deploying daemon mon.smithi149 on smithi149 2024-03-22T07:13:00.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:59 smithi081 ceph-mon[113965]: Upgrade: Updating mon.smithi149 2024-03-22T07:13:00.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:12:59 smithi081 ceph-mon[113965]: Deploying daemon mon.smithi149 on smithi149 2024-03-22T07:13:00.597 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:00 smithi149 systemd[1]: Stopping Ceph mon.smithi149 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:13:00.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:00 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149[36049]: 2024-03-22T07:13:00.596+0000 7f5a89c3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi149 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:13:00.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:00 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149[36049]: 2024-03-22T07:13:00.596+0000 7f5a89c3a700 -1 mon.smithi149@1(peon) e2 *** Got Signal Terminated *** 2024-03-22T07:13:00.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:00 smithi149 podman[83286]: 2024-03-22 07:13:00.637625271 +0000 UTC m=+0.139479761 container died 33803b1099d7505a1c32007d5edf9e9fb1520777e12b5144332704b7ac53e59c (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=quincy-9c4e0c3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-22T07:13:01.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:00 smithi149 podman[83286]: 2024-03-22 07:13:00.997182171 +0000 UTC m=+0.499036679 container cleanup 33803b1099d7505a1c32007d5edf9e9fb1520777e12b5144332704b7ac53e59c (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, org.label-schema.schema-version=1.0) 2024-03-22T07:13:01.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:00 smithi149 bash[83286]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149 2024-03-22T07:13:01.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:01 smithi149 podman[83300]: 2024-03-22 07:13:01.281866598 +0000 UTC m=+0.643729083 container remove 33803b1099d7505a1c32007d5edf9e9fb1520777e12b5144332704b7ac53e59c (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-22T07:13:01.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:01 smithi149 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi149.service: Deactivated successfully. 2024-03-22T07:13:01.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:01 smithi149 systemd[1]: Stopped Ceph mon.smithi149 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:13:01.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:01 smithi149 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi149.service: Consumed 6.723s CPU time. 2024-03-22T07:13:01.967 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:01 smithi149 systemd[1]: Starting Ceph mon.smithi149 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:13:01.967 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:01 smithi149 podman[83428]: 2024-03-22 07:13:01.746377913 +0000 UTC m=+0.018373655 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:13:01.967 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:01 smithi149 podman[83428]: 2024-03-22 07:13:01.866140699 +0000 UTC m=+0.138136433 container create 2c438525329ca95fc6bea7c9dd650eb10db4efd5a7e76f4545465f8e2d5c916b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-22T07:13:02.321 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 podman[83428]: 2024-03-22 07:13:02.060519994 +0000 UTC m=+0.332515727 container init 2c438525329ca95fc6bea7c9dd650eb10db4efd5a7e76f4545465f8e2d5c916b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_BRANCH=HEAD) 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 podman[83428]: 2024-03-22 07:13:02.063562062 +0000 UTC m=+0.335557797 container start 2c438525329ca95fc6bea7c9dd650eb10db4efd5a7e76f4545465f8e2d5c916b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 2 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: pidfile_write: ignore empty --pid-file 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: load: jerasure load: lrc 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: RocksDB version: 7.9.2 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Git sha 0 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: DB SUMMARY 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: DB Session ID: YGB9CW63JQB819JSBNE2 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: CURRENT file: CURRENT 2024-03-22T07:13:02.322 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1098 Bytes 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi149/store.db dir, Total Num: 1, files: 000027.sst 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi149/store.db: 000025.log size: 4443816 ; 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.error_if_exists: 0 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.create_if_missing: 0 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.paranoid_checks: 1 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.env: 0x557e53d63ae0 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.info_log: 0x557e55763760 2024-03-22T07:13:02.323 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T07:13:02.325 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.statistics: (nil) 2024-03-22T07:13:02.325 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.use_fsync: 0 2024-03-22T07:13:02.325 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_log_file_size: 0 2024-03-22T07:13:02.325 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T07:13:02.325 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T07:13:02.325 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T07:13:02.325 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.allow_fallocate: 1 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.use_direct_reads: 0 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.db_log_dir: 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.wal_dir: 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T07:13:02.326 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.write_buffer_manager: 0x557e55737400 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.unordered_write: 0 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T07:13:02.327 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.row_cache: None 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.wal_filter: None 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.two_write_queues: 0 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.wal_compression: 0 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.atomic_flush: 0 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T07:13:02.328 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.log_readahead_size: 0 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_background_jobs: 2 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_background_compactions: -1 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_subcompactions: 1 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T07:13:02.329 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_open_files: -1 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_background_flushes: -1 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Compression algorithms supported: 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: kZSTD supported: 0 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: kXpressCompression supported: 0 2024-03-22T07:13:02.330 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: kZlibCompression supported: 1 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: kSnappyCompression supported: 1 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: kLZ4Compression supported: 1 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: kBZip2Compression supported: 0 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi149/store.db/MANIFEST-000009 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.merge_operator: 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_filter: None 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T07:13:02.331 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557e55763880) 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: cache_index_and_filter_blocks: 1 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: pin_top_level_index_and_filter: 1 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: index_type: 0 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: data_block_index_type: 0 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: index_shortening: 1 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: checksum: 4 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: no_block_cache: 0 2024-03-22T07:13:02.332 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache: 0x557e55767090 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache_name: BinnedLRUCache 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache_options: 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: capacity : 536870912 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: num_shard_bits : 4 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: strict_capacity_limit : 0 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: high_pri_pool_ratio: 0.000 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_cache_compressed: (nil) 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: persistent_cache: (nil) 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_size: 4096 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_size_deviation: 10 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_restart_interval: 16 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: index_block_restart_interval: 1 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: metadata_block_size: 4096 2024-03-22T07:13:02.333 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: partition_filters: 0 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: use_delta_encoding: 1 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: filter_policy: bloomfilter 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: whole_key_filtering: 1 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: verify_compression: 0 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: read_amp_bytes_per_bit: 0 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: format_version: 5 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: enable_index_compression: 1 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: block_align: 0 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: max_auto_readahead_size: 262144 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: prepopulate_block_cache: 0 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: initial_auto_readahead_size: 8192 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T07:13:02.334 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression: NoCompression 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.num_levels: 7 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T07:13:02.335 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T07:13:02.336 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T07:13:02.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T07:13:02.338 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.table_properties_collectors: 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.inplace_update_support: 0 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.bloom_locality: 0 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.max_successive_merges: 0 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.ttl: 2592000 2024-03-22T07:13:02.339 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.enable_blob_files: false 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.min_blob_size: 0 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T07:13:02.340 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi149/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12299, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c6bc2288-c1ae-4a79-a789-f77d1150234c 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091582092931, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091582103320, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 2758424, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12304, "largest_seqno": 14017, "table_properties": {"data_size": 2751173, "index_size": 4311, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1925, "raw_key_size": 18217, "raw_average_key_size": 24, "raw_value_size": 2735567, "raw_average_value_size": 3671, "num_data_blocks": 194, "num_entries": 745, "num_filter_entries": 745, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711091582, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c6bc2288-c1ae-4a79-a789-f77d1150234c", "db_session_id": "YGB9CW63JQB819JSBNE2", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091582103428, "job": 1, "event": "recovery_finished"} 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi149/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557e55856000 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: DB pointer 0x557e55842000 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T07:13:02.341 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** DB Stats ** 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** Compaction Stats [default] ** 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T07:13:02.342 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: L0 1/0 2.63 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 656.0 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: L6 1/0 7.05 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Sum 2/0 9.68 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 656.0 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 656.0 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** Compaction Stats [default] ** 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 656.0 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:13:02.343 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Cumulative compaction: 0.00 GB write, 199.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Interval compaction: 0.00 GB write, 199.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Block cache BinnedLRUCache@0x557e55767090#2 capacity: 512.00 MB usage: 6.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,4.58 KB,0.000873208%) Misc(1,0.00 KB,0%) 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: starting mon.smithi149 rank 1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi149 fsid a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: mon.smithi149@-1(???) e2 preinit fsid a42e8632-e819-11ee-95cd-87774f69a715 2024-03-22T07:13:02.344 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: mon.smithi149@-1(???).mds e22 new map 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: mon.smithi149@-1(???).mds e22 print_map 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: e22 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: legacy client fscid: 1 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Filesystem 'cephfs' (1) 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: fs_name cephfs 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: epoch 22 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: modified 2024-03-22T07:12:38.916315+0000 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: tableserver 0 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: root 0 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: session_timeout 60 2024-03-22T07:13:02.345 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: session_autoclose 300 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: max_file_size 1099511627776 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: max_xattr_size 65536 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: required_client_features {} 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: last_failure 0 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: last_failure_osd_epoch 45 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: max_mds 2 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: in 0,1 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: up {0=14488,1=24251} 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: failed 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: damaged 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: stopped 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: data_pools [3] 2024-03-22T07:13:02.346 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: metadata_pool 2 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: inline_data disabled 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: balancer 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: bal_rank_mask -1 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: standby_count_wanted 1 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: [mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 95 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: [mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 94 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: Standby daemons: 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: [mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout: [mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: mon.smithi149@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: mon.smithi149@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:13:02.347 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: mon.smithi149@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:13:02.348 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: mon.smithi149@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:13:02.348 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: mon.smithi149@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-03-22T07:13:02.348 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 bash[83428]: 2c438525329ca95fc6bea7c9dd650eb10db4efd5a7e76f4545465f8e2d5c916b 2024-03-22T07:13:02.348 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 systemd[1]: Started Ceph mon.smithi149 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:13:02.348 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: pgmap v10: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:13:02.348 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:02 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:02.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:02 smithi081 ceph-mon[113965]: pgmap v10: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:13:02.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:02 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:03.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.055+0000 7fbb26e7a700 1 -- 172.21.15.81:0/2286535105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 msgr2=0x7fbb200fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.055+0000 7fbb26e7a700 1 --2- 172.21.15.81:0/2286535105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 0x7fbb200fbd30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c004940 tx=0x7fbb0c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.055+0000 7fbb26e7a700 1 -- 172.21.15.81:0/2286535105 shutdown_connections 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.055+0000 7fbb26e7a700 1 --2- 172.21.15.81:0/2286535105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 0x7fbb200fbd30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.055+0000 7fbb26e7a700 1 --2- 172.21.15.81:0/2286535105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 0x7fbb201020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.055+0000 7fbb26e7a700 1 -- 172.21.15.81:0/2286535105 >> 172.21.15.81:0/2286535105 conn(0x7fbb200f78f0 msgr2=0x7fbb200f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.055+0000 7fbb26e7a700 1 -- 172.21.15.81:0/2286535105 shutdown_connections 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.055+0000 7fbb26e7a700 1 -- 172.21.15.81:0/2286535105 wait complete. 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.055+0000 7fbb26e7a700 1 Processor -- start 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb26e7a700 1 -- start start 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb26e7a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 0x7fbb20113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb26e7a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 0x7fbb20113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb26e7a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20114500 con 0x7fbb201026c0 2024-03-22T07:13:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb26e7a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2010db00 con 0x7fbb20101d20 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb1ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 0x7fbb20113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb1ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 0x7fbb20113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55064/0 (socket says 172.21.15.81:55064) 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb1ffff700 1 -- 172.21.15.81:0/836872878 learned_addr learned my addr 172.21.15.81:0/836872878 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb24c16700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 0x7fbb20113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb24c16700 1 -- 172.21.15.81:0/836872878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 msgr2=0x7fbb20113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb24c16700 1 -- 172.21.15.81:0/836872878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 msgr2=0x7fbb20113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb24c16700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 0x7fbb20113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb24c16700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 0x7fbb20113970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb1ffff700 1 -- 172.21.15.81:0/836872878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 msgr2=0x7fbb20113970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:13:03.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb1ffff700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 0x7fbb20113970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.056+0000 7fbb1ffff700 1 -- 172.21.15.81:0/836872878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c005040 con 0x7fbb201026c0 2024-03-22T07:13:03.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.057+0000 7fbb1ffff700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 0x7fbb20113eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009100 tx=0x7fbb0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:03.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.057+0000 7fbb1d7fa700 1 -- 172.21.15.81:0/836872878 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb0c00e040 con 0x7fbb201026c0 2024-03-22T07:13:03.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.057+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2010dd80 con 0x7fbb201026c0 2024-03-22T07:13:03.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.057+0000 7fbb1d7fa700 1 -- 172.21.15.81:0/836872878 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb0c004290 con 0x7fbb201026c0 2024-03-22T07:13:03.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.057+0000 7fbb1d7fa700 1 -- 172.21.15.81:0/836872878 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb0c01c3f0 con 0x7fbb201026c0 2024-03-22T07:13:03.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.057+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2010e340 con 0x7fbb201026c0 2024-03-22T07:13:03.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.058+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb200fd470 con 0x7fbb201026c0 2024-03-22T07:13:03.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.058+0000 7fbb1d7fa700 1 -- 172.21.15.81:0/836872878 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fbb0c013790 con 0x7fbb201026c0 2024-03-22T07:13:03.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.059+0000 7fbb1d7fa700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbb1006d4b0 0x7fbb1006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:03.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.059+0000 7fbb1d7fa700 1 -- 172.21.15.81:0/836872878 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbb0c08ddf0 con 0x7fbb201026c0 2024-03-22T07:13:03.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.059+0000 7fbb24c16700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbb1006d4b0 0x7fbb1006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:03.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.059+0000 7fbb24c16700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbb1006d4b0 0x7fbb1006f960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009ce0 tx=0x7fbb14009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:03.064 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.064+0000 7fbb1d7fa700 1 -- 172.21.15.81:0/836872878 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb0c05b260 con 0x7fbb201026c0 2024-03-22T07:13:03.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.275+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb20108e20 con 0x7fbb1006d4b0 2024-03-22T07:13:03.277 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.277+0000 7fbb1d7fa700 1 -- 172.21.15.81:0/836872878 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fbb20108e20 con 0x7fbb1006d4b0 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbb1006d4b0 msgr2=0x7fbb1006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbb1006d4b0 0x7fbb1006f960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009ce0 tx=0x7fbb14009f70 comp rx=0 tx=0).stop 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 msgr2=0x7fbb20113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 0x7fbb20113eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009100 tx=0x7fbb0c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 shutdown_connections 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbb1006d4b0 0x7fbb1006f960 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb20101d20 0x7fbb20113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 --2- 172.21.15.81:0/836872878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb201026c0 0x7fbb20113eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 >> 172.21.15.81:0/836872878 conn(0x7fbb200f78f0 msgr2=0x7fbb200f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 shutdown_connections 2024-03-22T07:13:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.279+0000 7fbb26e7a700 1 -- 172.21.15.81:0/836872878 wait complete. 2024-03-22T07:13:03.290 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:13:03.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.381+0000 7fc318999700 1 -- 172.21.15.81:0/3635600406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 msgr2=0x7fc3141020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.381+0000 7fc318999700 1 --2- 172.21.15.81:0/3635600406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 0x7fc3141020f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc30c004940 tx=0x7fc30c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:03.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.381+0000 7fc318999700 1 -- 172.21.15.81:0/3635600406 shutdown_connections 2024-03-22T07:13:03.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.381+0000 7fc318999700 1 --2- 172.21.15.81:0/3635600406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3141026c0 0x7fc3140fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.381+0000 7fc318999700 1 --2- 172.21.15.81:0/3635600406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 0x7fc3141020f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.381+0000 7fc318999700 1 -- 172.21.15.81:0/3635600406 >> 172.21.15.81:0/3635600406 conn(0x7fc3140f7870 msgr2=0x7fc3140f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.381+0000 7fc318999700 1 -- 172.21.15.81:0/3635600406 shutdown_connections 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.381+0000 7fc318999700 1 -- 172.21.15.81:0/3635600406 wait complete. 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc318999700 1 Processor -- start 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc318999700 1 -- start start 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc318999700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 0x7fc314102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc318999700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3141026c0 0x7fc3141030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc318999700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc314104ab0 con 0x7fc314101d20 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc318999700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc314103600 con 0x7fc3141026c0 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc31259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 0x7fc314102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:03.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc31259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 0x7fc314102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55080/0 (socket says 172.21.15.81:55080) 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc31259c700 1 -- 172.21.15.81:0/1747930965 learned_addr learned my addr 172.21.15.81:0/1747930965 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc311d9b700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3141026c0 0x7fc3141030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc31259c700 1 -- 172.21.15.81:0/1747930965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3141026c0 msgr2=0x7fc3141030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc31259c700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3141026c0 0x7fc3141030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc31259c700 1 -- 172.21.15.81:0/1747930965 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c005040 con 0x7fc314101d20 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.382+0000 7fc31259c700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 0x7fc314102b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00b3b0 tx=0x7fc30c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.383+0000 7fc302ffd700 1 -- 172.21.15.81:0/1747930965 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc30c00e040 con 0x7fc314101d20 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.383+0000 7fc302ffd700 1 -- 172.21.15.81:0/1747930965 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc30c00bbc0 con 0x7fc314101d20 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.383+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc314103880 con 0x7fc314101d20 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.383+0000 7fc302ffd700 1 -- 172.21.15.81:0/1747930965 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc30c01c3f0 con 0x7fc314101d20 2024-03-22T07:13:03.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.383+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31411a100 con 0x7fc314101d20 2024-03-22T07:13:03.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.384+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3140fd470 con 0x7fc314101d20 2024-03-22T07:13:03.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.384+0000 7fc302ffd700 1 -- 172.21.15.81:0/1747930965 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fc30c00bd30 con 0x7fc314101d20 2024-03-22T07:13:03.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.384+0000 7fc302ffd700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc2fc06d4b0 0x7fc2fc06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:03.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.385+0000 7fc311d9b700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc2fc06d4b0 0x7fc2fc06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:03.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.385+0000 7fc302ffd700 1 -- 172.21.15.81:0/1747930965 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc30c08dde0 con 0x7fc314101d20 2024-03-22T07:13:03.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.385+0000 7fc311d9b700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc2fc06d4b0 0x7fc2fc06f960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc3140f92f0 tx=0x7fc308009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:03.389 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.389+0000 7fc302ffd700 1 -- 172.21.15.81:0/1747930965 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc30c0576a0 con 0x7fc314101d20 2024-03-22T07:13:03.396 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: mon.smithi149 calling monitor election 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: mon.smithi081 calling monitor election 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: mon.smithi081 is new leader, mons smithi081,smithi149 in quorum (ranks 0,1) 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: monmap e3: 2 mons at {smithi081=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0],smithi149=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: mgrmap e38: smithi081.bcxarw(active, since 16s), standbys: smithi149.sgmtoa 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: Health detail: HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:03.397 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:03.398 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 262.626251 secs 2024-03-22T07:13:03.398 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:03 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 262.966351 secs 2024-03-22T07:13:03.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi081"}]: dispatch 2024-03-22T07:13:03.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mon metadata", "id": "smithi149"}]: dispatch 2024-03-22T07:13:03.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: mon.smithi149 calling monitor election 2024-03-22T07:13:03.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: mon.smithi081 calling monitor election 2024-03-22T07:13:03.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: mon.smithi081 is new leader, mons smithi081,smithi149 in quorum (ranks 0,1) 2024-03-22T07:13:03.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: monmap e3: 2 mons at {smithi081=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0],smithi149=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:13:03.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: fsmap cephfs:2 {0=cephfs.smithi081.zfgxzy=up:active,1=cephfs.smithi149.kdtxfw=up:active} 2 up:standby 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: mgrmap e38: smithi081.bcxarw(active, since 16s), standbys: smithi149.sgmtoa 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: Health detail: HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 262.626251 secs 2024-03-22T07:13:03.570 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:03 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 262.966351 secs 2024-03-22T07:13:03.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.601+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3140669b0 con 0x7fc2fc06d4b0 2024-03-22T07:13:03.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.602+0000 7fc302ffd700 1 -- 172.21.15.81:0/1747930965 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc3140669b0 con 0x7fc2fc06d4b0 2024-03-22T07:13:03.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.604+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc2fc06d4b0 msgr2=0x7fc2fc06f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.604+0000 7fc318999700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc2fc06d4b0 0x7fc2fc06f960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc3140f92f0 tx=0x7fc308009040 comp rx=0 tx=0).stop 2024-03-22T07:13:03.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.604+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 msgr2=0x7fc314102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.604+0000 7fc318999700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 0x7fc314102b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00b3b0 tx=0x7fc30c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:03.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.605+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 shutdown_connections 2024-03-22T07:13:03.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.605+0000 7fc318999700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc2fc06d4b0 0x7fc2fc06f960 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.605+0000 7fc318999700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc314101d20 0x7fc314102b80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.605+0000 7fc318999700 1 --2- 172.21.15.81:0/1747930965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3141026c0 0x7fc3141030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.605+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 >> 172.21.15.81:0/1747930965 conn(0x7fc3140f7870 msgr2=0x7fc3140662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:03.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.605+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 shutdown_connections 2024-03-22T07:13:03.606 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.605+0000 7fc318999700 1 -- 172.21.15.81:0/1747930965 wait complete. 2024-03-22T07:13:03.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 -- 172.21.15.81:0/1083844980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070c30 msgr2=0x7fd910109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 --2- 172.21.15.81:0/1083844980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070c30 0x7fd910109150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd900004940 tx=0x7fd900009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:03.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 -- 172.21.15.81:0/1083844980 shutdown_connections 2024-03-22T07:13:03.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 --2- 172.21.15.81:0/1083844980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070c30 0x7fd910109150 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 --2- 172.21.15.81:0/1083844980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910070320 0x7fd9100706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 -- 172.21.15.81:0/1083844980 >> 172.21.15.81:0/1083844980 conn(0x7fd9100f89b0 msgr2=0x7fd9100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:03.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 -- 172.21.15.81:0/1083844980 shutdown_connections 2024-03-22T07:13:03.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 -- 172.21.15.81:0/1083844980 wait complete. 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 Processor -- start 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.705+0000 7fd917748700 1 -- start start 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd917748700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070320 0x7fd910115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd917748700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910070c30 0x7fd9101160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd917748700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9101167b0 con 0x7fd910070320 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd917748700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91010fd00 con 0x7fd910070c30 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd9154e4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070320 0x7fd910115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd9154e4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070320 0x7fd910115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55102/0 (socket says 172.21.15.81:55102) 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd9154e4700 1 -- 172.21.15.81:0/23001269 learned_addr learned my addr 172.21.15.81:0/23001269 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd914ce3700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910070c30 0x7fd9101160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd9154e4700 1 -- 172.21.15.81:0/23001269 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910070c30 msgr2=0x7fd9101160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd9154e4700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910070c30 0x7fd9101160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd9154e4700 1 -- 172.21.15.81:0/23001269 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd900005040 con 0x7fd910070320 2024-03-22T07:13:03.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd9154e4700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070320 0x7fd910115b90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00b790 tx=0x7fd90c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:03.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd905ffb700 1 -- 172.21.15.81:0/23001269 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd90c00f770 con 0x7fd910070320 2024-03-22T07:13:03.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91010ff80 con 0x7fd910070320 2024-03-22T07:13:03.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.706+0000 7fd905ffb700 1 -- 172.21.15.81:0/23001269 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd90c00fda0 con 0x7fd910070320 2024-03-22T07:13:03.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.707+0000 7fd905ffb700 1 -- 172.21.15.81:0/23001269 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd90c00d310 con 0x7fd910070320 2024-03-22T07:13:03.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.707+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9101104c0 con 0x7fd910070320 2024-03-22T07:13:03.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.708+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd910071a60 con 0x7fd910070320 2024-03-22T07:13:03.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.708+0000 7fd905ffb700 1 -- 172.21.15.81:0/23001269 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fd90c010420 con 0x7fd910070320 2024-03-22T07:13:03.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.708+0000 7fd905ffb700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd8fc06d4b0 0x7fd8fc06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:03.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.709+0000 7fd914ce3700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd8fc06d4b0 0x7fd8fc06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:03.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.709+0000 7fd905ffb700 1 -- 172.21.15.81:0/23001269 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd90c08ca30 con 0x7fd910070320 2024-03-22T07:13:03.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.709+0000 7fd914ce3700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd8fc06d4b0 0x7fd8fc06f960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd910111430 tx=0x7fd900009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:03.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.713+0000 7fd905ffb700 1 -- 172.21.15.81:0/23001269 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd90c059fb0 con 0x7fd910070320 2024-03-22T07:13:03.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.916+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd910111280 con 0x7fd8fc06d4b0 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.927+0000 7fd905ffb700 1 -- 172.21.15.81:0/23001269 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd910111280 con 0x7fd8fc06d4b0 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (97s) 12s ago 11m 16.8M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (12m) 12s ago 12m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (11m) 14s ago 11m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (80s) 12s ago 11m 80.7M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (6m) 12s ago 6m 13.0M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (6m) 12s ago 6m 132M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (6m) 14s ago 6m 16.1M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:13:03.927 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (6m) 14s ago 6m 148M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (3m) 12s ago 13m 472M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (2m) 14s ago 11m 446M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (26s) 12s ago 13m 27.2M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 starting - - - 2048M 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (2m) 12s ago 12m 17.0M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (2m) 14s ago 11m 16.9M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (10m) 12s ago 10m 184M 1557M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (9m) 12s ago 9m 200M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (9m) 12s ago 9m 162M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (8m) 14s ago 8m 214M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (8m) 14s ago 8m 193M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (8m) 14s ago 8m 183M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:13:03.928 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (115s) 12s ago 11m 51.9M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:13:03.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd8fc06d4b0 msgr2=0x7fd8fc06f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd8fc06d4b0 0x7fd8fc06f960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd910111430 tx=0x7fd900009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:03.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070320 msgr2=0x7fd910115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:03.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070320 0x7fd910115b90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00b790 tx=0x7fd90c009250 comp rx=0 tx=0).stop 2024-03-22T07:13:03.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 shutdown_connections 2024-03-22T07:13:03.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd8fc06d4b0 0x7fd8fc06f960 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd910070320 0x7fd910115b90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 --2- 172.21.15.81:0/23001269 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd910070c30 0x7fd9101160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:03.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 >> 172.21.15.81:0/23001269 conn(0x7fd9100f89b0 msgr2=0x7fd9100f9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:03.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 shutdown_connections 2024-03-22T07:13:03.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:03.929+0000 7fd917748700 1 -- 172.21.15.81:0/23001269 wait complete. 2024-03-22T07:13:04.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.027+0000 7fd480351700 1 -- 172.21.15.81:0/3356274431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd478102e00 msgr2=0x7fd4781031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.027+0000 7fd480351700 1 --2- 172.21.15.81:0/3356274431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd478102e00 0x7fd4781031d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd474004940 tx=0x7fd474009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:04.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 -- 172.21.15.81:0/3356274431 shutdown_connections 2024-03-22T07:13:04.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 --2- 172.21.15.81:0/3356274431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4781037a0 0x7fd4780fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 --2- 172.21.15.81:0/3356274431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd478102e00 0x7fd4781031d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 -- 172.21.15.81:0/3356274431 >> 172.21.15.81:0/3356274431 conn(0x7fd4780f89b0 msgr2=0x7fd4780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 -- 172.21.15.81:0/3356274431 shutdown_connections 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 -- 172.21.15.81:0/3356274431 wait complete. 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 Processor -- start 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 -- start start 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd478102e00 0x7fd478113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4781037a0 0x7fd478113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd478114520 con 0x7fd4781037a0 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd480351700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47810db70 con 0x7fd478102e00 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.028+0000 7fd47d8ec700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4781037a0 0x7fd478113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47d8ec700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4781037a0 0x7fd478113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55118/0 (socket says 172.21.15.81:55118) 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47d8ec700 1 -- 172.21.15.81:0/3171032308 learned_addr learned my addr 172.21.15.81:0/3171032308 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47e0ed700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd478102e00 0x7fd478113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47e0ed700 1 -- 172.21.15.81:0/3171032308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd478102e00 msgr2=0x7fd478113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T07:13:04.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47e0ed700 1 -- 172.21.15.81:0/3171032308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd478102e00 msgr2=0x7fd478113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47e0ed700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd478102e00 0x7fd478113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47e0ed700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd478102e00 0x7fd478113990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47d8ec700 1 -- 172.21.15.81:0/3171032308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd478102e00 msgr2=0x7fd478113990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47d8ec700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd478102e00 0x7fd478113990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47d8ec700 1 -- 172.21.15.81:0/3171032308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd474005040 con 0x7fd4781037a0 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd47d8ec700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4781037a0 0x7fd478113ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd46800c890 tx=0x7fd4680044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd46effd700 1 -- 172.21.15.81:0/3171032308 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd46800a7d0 con 0x7fd4781037a0 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd46effd700 1 -- 172.21.15.81:0/3171032308 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd46800ae00 con 0x7fd4781037a0 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd46effd700 1 -- 172.21.15.81:0/3171032308 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4680113a0 con 0x7fd4781037a0 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47810de50 con 0x7fd4781037a0 2024-03-22T07:13:04.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.029+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47810e410 con 0x7fd4781037a0 2024-03-22T07:13:04.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.030+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd478100110 con 0x7fd4781037a0 2024-03-22T07:13:04.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.031+0000 7fd46effd700 1 -- 172.21.15.81:0/3171032308 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fd468012400 con 0x7fd4781037a0 2024-03-22T07:13:04.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.031+0000 7fd46effd700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd46406d4b0 0x7fd46406f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:04.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.031+0000 7fd46effd700 1 -- 172.21.15.81:0/3171032308 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd46808ca20 con 0x7fd4781037a0 2024-03-22T07:13:04.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.031+0000 7fd47e0ed700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd46406d4b0 0x7fd46406f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:04.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.032+0000 7fd47e0ed700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd46406d4b0 0x7fd46406f960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd474009fd0 tx=0x7fd474009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:04.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.036+0000 7fd46effd700 1 -- 172.21.15.81:0/3171032308 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd46805a040 con 0x7fd4781037a0 2024-03-22T07:13:04.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.312+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd47810efe0 con 0x7fd4781037a0 2024-03-22T07:13:04.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.313+0000 7fd46effd700 1 -- 172.21.15.81:0/3171032308 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7fd468059bd0 con 0x7fd4781037a0 2024-03-22T07:13:04.313 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:13:04.313 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:13:04.313 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:13:04.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:13:04.315 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:13:04.315 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-22T07:13:04.315 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:13:04.315 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:13:04.315 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd46406d4b0 msgr2=0x7fd46406f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd46406d4b0 0x7fd46406f960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd474009fd0 tx=0x7fd474009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4781037a0 msgr2=0x7fd478113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4781037a0 0x7fd478113ed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd46800c890 tx=0x7fd4680044c0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 shutdown_connections 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd46406d4b0 0x7fd46406f960 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd478102e00 0x7fd478113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 --2- 172.21.15.81:0/3171032308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4781037a0 0x7fd478113ed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 >> 172.21.15.81:0/3171032308 conn(0x7fd4780f89b0 msgr2=0x7fd4781073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 shutdown_connections 2024-03-22T07:13:04.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.315+0000 7fd480351700 1 -- 172.21.15.81:0/3171032308 wait complete. 2024-03-22T07:13:04.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.416+0000 7ff7048ff700 1 -- 172.21.15.81:0/632177717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 msgr2=0x7ff7000fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.416+0000 7ff7048ff700 1 --2- 172.21.15.81:0/632177717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 0x7ff7000fce10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4004940 tx=0x7ff6f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.416+0000 7ff7048ff700 1 -- 172.21.15.81:0/632177717 shutdown_connections 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.416+0000 7ff7048ff700 1 --2- 172.21.15.81:0/632177717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 0x7ff7000fce10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.416+0000 7ff7048ff700 1 --2- 172.21.15.81:0/632177717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff700102e00 0x7ff7001031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.416+0000 7ff7048ff700 1 -- 172.21.15.81:0/632177717 >> 172.21.15.81:0/632177717 conn(0x7ff7000f89b0 msgr2=0x7ff7000fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.416+0000 7ff7048ff700 1 -- 172.21.15.81:0/632177717 shutdown_connections 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.416+0000 7ff7048ff700 1 -- 172.21.15.81:0/632177717 wait complete. 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff7048ff700 1 Processor -- start 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff7048ff700 1 -- start start 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff7048ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff7048ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 0x7ff700070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff7048ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff700072210 con 0x7ff7001037a0 2024-03-22T07:13:04.419 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff7048ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff700070d40 con 0x7ff700102e00 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff6fdd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 0x7ff700070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff6fdd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 0x7ff700070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55142/0 (socket says 172.21.15.81:55142) 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff6fdd9b700 1 -- 172.21.15.81:0/10036483 learned_addr learned my addr 172.21.15.81:0/10036483 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff6fe59c700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff6fdd9b700 1 -- 172.21.15.81:0/10036483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff700102e00 msgr2=0x7ff7000702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff6fdd9b700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff6fdd9b700 1 -- 172.21.15.81:0/10036483 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f4005040 con 0x7ff7001037a0 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.417+0000 7ff6fe59c700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.418+0000 7ff6fdd9b700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 0x7ff700070800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009180 tx=0x7ff6f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.418+0000 7ff6eeffd700 1 -- 172.21.15.81:0/10036483 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6f400e040 con 0x7ff7001037a0 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.418+0000 7ff6eeffd700 1 -- 172.21.15.81:0/10036483 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6f400bbc0 con 0x7ff7001037a0 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.418+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff700070fc0 con 0x7ff7001037a0 2024-03-22T07:13:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.418+0000 7ff6eeffd700 1 -- 172.21.15.81:0/10036483 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6f401c3f0 con 0x7ff7001037a0 2024-03-22T07:13:04.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.418+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70011a0d0 con 0x7ff7001037a0 2024-03-22T07:13:04.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.419+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7001052d0 con 0x7ff7001037a0 2024-03-22T07:13:04.421 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.419+0000 7ff6eeffd700 1 -- 172.21.15.81:0/10036483 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7ff6f400bd30 con 0x7ff7001037a0 2024-03-22T07:13:04.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.419+0000 7ff6eeffd700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff6e806d4b0 0x7ff6e806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:04.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.420+0000 7ff6eeffd700 1 -- 172.21.15.81:0/10036483 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff6f408de70 con 0x7ff7001037a0 2024-03-22T07:13:04.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.420+0000 7ff6fe59c700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff6e806d4b0 0x7ff6e806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:04.422 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.420+0000 7ff6fe59c700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff6e806d4b0 0x7ff6e806f960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0007fd0 tx=0x7ff6f0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:04.424 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.424+0000 7ff6eeffd700 1 -- 172.21.15.81:0/10036483 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6f40577b0 con 0x7ff7001037a0 2024-03-22T07:13:04.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:04 smithi081 ceph-mon[113965]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:04.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:04 smithi081 ceph-mon[113965]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:04.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:04 smithi081 ceph-mon[113965]: pgmap v12: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:13:04.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:04 smithi081 ceph-mon[113965]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:04.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:04 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3171032308' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:04.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:04 smithi149 ceph-mon[83447]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:04.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:04 smithi149 ceph-mon[83447]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:04.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:04 smithi149 ceph-mon[83447]: pgmap v12: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:13:04.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:04 smithi149 ceph-mon[83447]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:04.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:04 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3171032308' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:04.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.660+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff7000fe550 con 0x7ff7001037a0 2024-03-22T07:13:04.660 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.660+0000 7ff6eeffd700 1 -- 172.21.15.81:0/10036483 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1858 (secure 0 0 0) 0x7ff7000fe550 con 0x7ff7001037a0 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout:e22 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 22 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:13:04.661 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:12:38.916315+0000 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:13:04.662 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 95 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 94 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:13:04.663 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:13:04.664 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:13:04.664 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:13:04.664 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:04.664 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff6e806d4b0 msgr2=0x7ff6e806f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff6e806d4b0 0x7ff6e806f960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0007fd0 tx=0x7ff6f0009040 comp rx=0 tx=0).stop 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 msgr2=0x7ff700070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 0x7ff700070800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009180 tx=0x7ff6f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 shutdown_connections 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff6e806d4b0 0x7ff6e806f960 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff700102e00 0x7ff7000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 --2- 172.21.15.81:0/10036483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7001037a0 0x7ff700070800 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 >> 172.21.15.81:0/10036483 conn(0x7ff7000f89b0 msgr2=0x7ff7001021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 shutdown_connections 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.662+0000 7ff7048ff700 1 -- 172.21.15.81:0/10036483 wait complete. 2024-03-22T07:13:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 22 2024-03-22T07:13:04.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.762+0000 7f918951a700 1 -- 172.21.15.81:0/721904451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 msgr2=0x7f91841020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.762+0000 7f918951a700 1 --2- 172.21.15.81:0/721904451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 0x7f91841020c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9178004940 tx=0x7f9178009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:04.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.762+0000 7f918951a700 1 -- 172.21.15.81:0/721904451 shutdown_connections 2024-03-22T07:13:04.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.762+0000 7f918951a700 1 --2- 172.21.15.81:0/721904451 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184102690 0x7f91840fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.762+0000 7f918951a700 1 --2- 172.21.15.81:0/721904451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 0x7f91841020c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.762+0000 7f918951a700 1 -- 172.21.15.81:0/721904451 >> 172.21.15.81:0/721904451 conn(0x7f91840f7860 msgr2=0x7f91840f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.762+0000 7f918951a700 1 -- 172.21.15.81:0/721904451 shutdown_connections 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.762+0000 7f918951a700 1 -- 172.21.15.81:0/721904451 wait complete. 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.763+0000 7f918951a700 1 Processor -- start 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.763+0000 7f918951a700 1 -- start start 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.763+0000 7f918951a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 0x7f918410c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.763+0000 7f918951a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184102690 0x7f918410ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.763+0000 7f918951a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918410d390 con 0x7f9184101cf0 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.763+0000 7f918951a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918410d500 con 0x7f9184102690 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.763+0000 7f9182ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 0x7f918410c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.763+0000 7f9182ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 0x7f918410c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44446/0 (socket says 172.21.15.81:44446) 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.763+0000 7f9182ffd700 1 -- 172.21.15.81:0/53832285 learned_addr learned my addr 172.21.15.81:0/53832285 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f91827fc700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184102690 0x7f918410ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:04.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f9182ffd700 1 -- 172.21.15.81:0/53832285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184102690 msgr2=0x7f918410ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f9182ffd700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184102690 0x7f918410ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f9182ffd700 1 -- 172.21.15.81:0/53832285 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9178005040 con 0x7f9184101cf0 2024-03-22T07:13:04.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f9182ffd700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 0x7f918410c910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f917800b8b0 tx=0x7f9178009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:04.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f916b7fe700 1 -- 172.21.15.81:0/53832285 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f917800e040 con 0x7f9184101cf0 2024-03-22T07:13:04.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f916b7fe700 1 -- 172.21.15.81:0/53832285 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9178004290 con 0x7f9184101cf0 2024-03-22T07:13:04.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918410d780 con 0x7f9184101cf0 2024-03-22T07:13:04.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f916b7fe700 1 -- 172.21.15.81:0/53832285 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f917801c3f0 con 0x7f9184101cf0 2024-03-22T07:13:04.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.764+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918411a300 con 0x7f9184101cf0 2024-03-22T07:13:04.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.765+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91840fd440 con 0x7f9184101cf0 2024-03-22T07:13:04.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.765+0000 7f916b7fe700 1 -- 172.21.15.81:0/53832285 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f917800baf0 con 0x7f9184101cf0 2024-03-22T07:13:04.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.766+0000 7f916b7fe700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f916c06d4a0 0x7f916c06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:04.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.766+0000 7f916b7fe700 1 -- 172.21.15.81:0/53832285 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f917808dc00 con 0x7f9184101cf0 2024-03-22T07:13:04.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.766+0000 7f91827fc700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f916c06d4a0 0x7f916c06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:04.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.766+0000 7f91827fc700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f916c06d4a0 0x7f916c06f950 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9184066890 tx=0x7f9174009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:04.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.770+0000 7f916b7fe700 1 -- 172.21.15.81:0/53832285 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9178093020 con 0x7f9184101cf0 2024-03-22T07:13:04.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.978+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f918410d780 con 0x7f916c06d4a0 2024-03-22T07:13:04.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.980+0000 7f916b7fe700 1 -- 172.21.15.81:0/53832285 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f918410d780 con 0x7f916c06d4a0 2024-03-22T07:13:04.980 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:13:04.980 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:13:04.980 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:13:04.980 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:13:04.980 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:13:04.981 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr" 2024-03-22T07:13:04.981 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:13:04.981 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "3/21 daemons upgraded", 2024-03-22T07:13:04.981 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading mon daemons", 2024-03-22T07:13:04.981 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:13:04.981 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:13:04.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.982+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f916c06d4a0 msgr2=0x7f916c06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.982+0000 7f918951a700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f916c06d4a0 0x7f916c06f950 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9184066890 tx=0x7f9174009040 comp rx=0 tx=0).stop 2024-03-22T07:13:04.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.982+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 msgr2=0x7f918410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:04.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.982+0000 7f918951a700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 0x7f918410c910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f917800b8b0 tx=0x7f9178009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:04.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.983+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 shutdown_connections 2024-03-22T07:13:04.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.983+0000 7f918951a700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f916c06d4a0 0x7f916c06f950 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.983+0000 7f918951a700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9184101cf0 0x7f918410c910 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.983+0000 7f918951a700 1 --2- 172.21.15.81:0/53832285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184102690 0x7f918410ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:04.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.983+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 >> 172.21.15.81:0/53832285 conn(0x7f91840f7860 msgr2=0x7f91840f88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:04.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.983+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 shutdown_connections 2024-03-22T07:13:04.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:04.983+0000 7f918951a700 1 -- 172.21.15.81:0/53832285 wait complete. 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 -- 172.21.15.81:0/1323848171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 msgr2=0x7f58641031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 --2- 172.21.15.81:0/1323848171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 0x7f58641031d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5854004940 tx=0x7f5854009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 -- 172.21.15.81:0/1323848171 shutdown_connections 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 --2- 172.21.15.81:0/1323848171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58641037a0 0x7f58640fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 --2- 172.21.15.81:0/1323848171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 0x7f58641031d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 -- 172.21.15.81:0/1323848171 >> 172.21.15.81:0/1323848171 conn(0x7f58640f89b0 msgr2=0x7f58640fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 -- 172.21.15.81:0/1323848171 shutdown_connections 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 -- 172.21.15.81:0/1323848171 wait complete. 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 Processor -- start 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 -- start start 2024-03-22T07:13:05.086 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 0x7f5864113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58641037a0 0x7f5864113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864114520 con 0x7f5864102e00 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.084+0000 7f5869ddb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586410db70 con 0x7f58641037a0 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f58637fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 0x7f5864113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f58637fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 0x7f5864113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44464/0 (socket says 172.21.15.81:44464) 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f58637fe700 1 -- 172.21.15.81:0/493292504 learned_addr learned my addr 172.21.15.81:0/493292504 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f585bfff700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58641037a0 0x7f5864113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f58637fe700 1 -- 172.21.15.81:0/493292504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58641037a0 msgr2=0x7f5864113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f58637fe700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58641037a0 0x7f5864113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f58637fe700 1 -- 172.21.15.81:0/493292504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5854005040 con 0x7f5864102e00 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f58637fe700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 0x7f5864113990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f585400b3b0 tx=0x7f5854009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f5860ff9700 1 -- 172.21.15.81:0/493292504 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585400e040 con 0x7f5864102e00 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f5860ff9700 1 -- 172.21.15.81:0/493292504 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f585400bbc0 con 0x7f5864102e00 2024-03-22T07:13:05.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586410ddf0 con 0x7f5864102e00 2024-03-22T07:13:05.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f5860ff9700 1 -- 172.21.15.81:0/493292504 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585401c3f0 con 0x7f5864102e00 2024-03-22T07:13:05.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.085+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586410e3b0 con 0x7f5864102e00 2024-03-22T07:13:05.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.086+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5864100110 con 0x7f5864102e00 2024-03-22T07:13:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.087+0000 7f5860ff9700 1 -- 172.21.15.81:0/493292504 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f585400bd30 con 0x7f5864102e00 2024-03-22T07:13:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.087+0000 7f5860ff9700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f585006d4b0 0x7f585006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.087+0000 7f5860ff9700 1 -- 172.21.15.81:0/493292504 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f585408ddf0 con 0x7f5864102e00 2024-03-22T07:13:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.087+0000 7f585bfff700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f585006d4b0 0x7f585006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.088+0000 7f585bfff700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f585006d4b0 0x7f585006f960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f586410ece0 tx=0x7f584c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:05.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.092+0000 7f5860ff9700 1 -- 172.21.15.81:0/493292504 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f585405b1e0 con 0x7f5864102e00 2024-03-22T07:13:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.367+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f586410efe0 con 0x7f5864102e00 2024-03-22T07:13:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.367+0000 7f5860ff9700 1 -- 172.21.15.81:0/493292504 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+250 (secure 0 0 0) 0x7f585405ad70 con 0x7f5864102e00 2024-03-22T07:13:05.368 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:13:05.368 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:13:05.368 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:13:05.368 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.369+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f585006d4b0 msgr2=0x7f585006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.369+0000 7f5869ddb700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f585006d4b0 0x7f585006f960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f586410ece0 tx=0x7f584c009ee0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.369+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 msgr2=0x7f5864113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.369+0000 7f5869ddb700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 0x7f5864113990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f585400b3b0 tx=0x7f5854009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.369+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 shutdown_connections 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.369+0000 7f5869ddb700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f585006d4b0 0x7f585006f960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.369+0000 7f5869ddb700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5864102e00 0x7f5864113990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.369+0000 7f5869ddb700 1 --2- 172.21.15.81:0/493292504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58641037a0 0x7f5864113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.369+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 >> 172.21.15.81:0/493292504 conn(0x7f58640f89b0 msgr2=0x7f58641073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:05.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.370+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 shutdown_connections 2024-03-22T07:13:05.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:05.370+0000 7f5869ddb700 1 -- 172.21.15.81:0/493292504 wait complete. 2024-03-22T07:13:05.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:05 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/10036483' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:13:05.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:05 smithi149 ceph-mon[83447]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:05.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:05 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:05.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:05 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:05.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:05 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/10036483' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:13:05.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:05 smithi081 ceph-mon[113965]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:05.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:05 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:05.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:05 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:06.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:06 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/493292504' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:13:06.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:06 smithi081 ceph-mon[113965]: pgmap v13: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:06.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:06 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/493292504' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:13:06.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:06 smithi149 ceph-mon[83447]: pgmap v13: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:07.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:07 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:07.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:07 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:07.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:07 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:07.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:07 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:08.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:08 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 267.626341 secs 2024-03-22T07:13:08.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:08 smithi149 ceph-mon[83447]: pgmap v14: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:08.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:08 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 267.966465 secs 2024-03-22T07:13:08.659 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:08 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 267.626341 secs 2024-03-22T07:13:08.659 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:08 smithi081 ceph-mon[113965]: pgmap v14: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:08.659 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:08 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 267.966465 secs 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: Detected new or changed devices on smithi149 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: pgmap v15: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: Reconfiguring mon.smithi081 (monmap changed)... 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:13:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:11.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:10 smithi081 ceph-mon[113965]: Reconfiguring daemon mon.smithi081 on smithi081 2024-03-22T07:13:11.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: Detected new or changed devices on smithi149 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: pgmap v15: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: Reconfiguring mon.smithi081 (monmap changed)... 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:10 smithi149 ceph-mon[83447]: Reconfiguring daemon mon.smithi081 on smithi081 2024-03-22T07:13:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:12 smithi081 ceph-mon[113965]: Reconfiguring mgr.smithi081.bcxarw (monmap changed)... 2024-03-22T07:13:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi081.bcxarw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:13:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:13:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:12 smithi081 ceph-mon[113965]: Reconfiguring daemon mgr.smithi081.bcxarw on smithi081 2024-03-22T07:13:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:12 smithi081 ceph-mon[113965]: pgmap v16: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:12 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 272.626404 secs 2024-03-22T07:13:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:12 smithi149 ceph-mon[83447]: Reconfiguring mgr.smithi081.bcxarw (monmap changed)... 2024-03-22T07:13:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi081.bcxarw", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:13:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:13:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:12 smithi149 ceph-mon[83447]: Reconfiguring daemon mgr.smithi081.bcxarw on smithi081 2024-03-22T07:13:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:12 smithi149 ceph-mon[83447]: pgmap v16: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:12.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:12 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 272.626404 secs 2024-03-22T07:13:13.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:13 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 272.966602 secs 2024-03-22T07:13:13.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:13 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:13.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:13 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:13.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:13 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi081", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:13:13.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:13 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:13.876 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:13 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 272.966602 secs 2024-03-22T07:13:13.876 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:13 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:13.876 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:13 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:13.876 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:13 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi081", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:13:13.876 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:13 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:14.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:14 smithi149 ceph-mon[83447]: Reconfiguring crash.smithi081 (monmap changed)... 2024-03-22T07:13:14.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:14 smithi149 ceph-mon[83447]: Reconfiguring daemon crash.smithi081 on smithi081 2024-03-22T07:13:14.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:14 smithi149 ceph-mon[83447]: pgmap v17: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:14.934 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:14 smithi081 ceph-mon[113965]: Reconfiguring crash.smithi081 (monmap changed)... 2024-03-22T07:13:14.934 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:14 smithi081 ceph-mon[113965]: Reconfiguring daemon crash.smithi081 on smithi081 2024-03-22T07:13:14.934 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:14 smithi081 ceph-mon[113965]: pgmap v17: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:15 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:15 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:15 smithi081 ceph-mon[113965]: Reconfiguring osd.0 (monmap changed)... 2024-03-22T07:13:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:15 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:13:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:15 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:15 smithi081 ceph-mon[113965]: Reconfiguring daemon osd.0 on smithi081 2024-03-22T07:13:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:15 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:15 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:15 smithi149 ceph-mon[83447]: Reconfiguring osd.0 (monmap changed)... 2024-03-22T07:13:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:15 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:13:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:15 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:15 smithi149 ceph-mon[83447]: Reconfiguring daemon osd.0 on smithi081 2024-03-22T07:13:17.115 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:16 smithi081 ceph-mon[113965]: pgmap v18: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:17.115 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:17.115 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:17.116 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:17.116 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:13:17.116 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:16 smithi149 ceph-mon[83447]: pgmap v18: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:13:17.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:18.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:17 smithi081 ceph-mon[113965]: Reconfiguring osd.1 (monmap changed)... 2024-03-22T07:13:18.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:17 smithi081 ceph-mon[113965]: Reconfiguring daemon osd.1 on smithi081 2024-03-22T07:13:18.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:17 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 277.626509 secs 2024-03-22T07:13:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:17 smithi149 ceph-mon[83447]: Reconfiguring osd.1 (monmap changed)... 2024-03-22T07:13:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:17 smithi149 ceph-mon[83447]: Reconfiguring daemon osd.1 on smithi081 2024-03-22T07:13:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:17 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 277.626509 secs 2024-03-22T07:13:19.033 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:18 smithi081 ceph-mon[113965]: pgmap v19: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:19.034 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:18 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 277.966726 secs 2024-03-22T07:13:19.034 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:19.034 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:19.034 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:13:19.034 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:19.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:18 smithi149 ceph-mon[83447]: pgmap v19: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:18 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 277.966726 secs 2024-03-22T07:13:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:13:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:19 smithi149 ceph-mon[83447]: Reconfiguring osd.2 (monmap changed)... 2024-03-22T07:13:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:19 smithi149 ceph-mon[83447]: Reconfiguring daemon osd.2 on smithi081 2024-03-22T07:13:20.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:19 smithi081 ceph-mon[113965]: Reconfiguring osd.2 (monmap changed)... 2024-03-22T07:13:20.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:19 smithi081 ceph-mon[113965]: Reconfiguring daemon osd.2 on smithi081 2024-03-22T07:13:21.038 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:20 smithi081 ceph-mon[113965]: pgmap v20: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:21.038 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:20 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:21.039 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:20 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:21.039 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:20 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.bnzfrv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:21.039 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:20 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:20 smithi149 ceph-mon[83447]: pgmap v20: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:20 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:20 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:20 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.bnzfrv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:20 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:22.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:21 smithi081 ceph-mon[113965]: Reconfiguring mds.cephfs.smithi081.bnzfrv (monmap changed)... 2024-03-22T07:13:22.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:21 smithi081 ceph-mon[113965]: Reconfiguring daemon mds.cephfs.smithi081.bnzfrv on smithi081 2024-03-22T07:13:22.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:22.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:22.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.zfgxzy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:22.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:21 smithi149 ceph-mon[83447]: Reconfiguring mds.cephfs.smithi081.bnzfrv (monmap changed)... 2024-03-22T07:13:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:21 smithi149 ceph-mon[83447]: Reconfiguring daemon mds.cephfs.smithi081.bnzfrv on smithi081 2024-03-22T07:13:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.zfgxzy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:23.216 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:22 smithi081 ceph-mon[113965]: Reconfiguring mds.cephfs.smithi081.zfgxzy (monmap changed)... 2024-03-22T07:13:23.216 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:22 smithi081 ceph-mon[113965]: Reconfiguring daemon mds.cephfs.smithi081.zfgxzy on smithi081 2024-03-22T07:13:23.216 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:22 smithi081 ceph-mon[113965]: pgmap v21: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:23.216 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:22 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 282.626581 secs 2024-03-22T07:13:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:22 smithi149 ceph-mon[83447]: Reconfiguring mds.cephfs.smithi081.zfgxzy (monmap changed)... 2024-03-22T07:13:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:22 smithi149 ceph-mon[83447]: Reconfiguring daemon mds.cephfs.smithi081.zfgxzy on smithi081 2024-03-22T07:13:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:22 smithi149 ceph-mon[83447]: pgmap v21: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:22 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 282.626581 secs 2024-03-22T07:13:24.161 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:23 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 282.966835 secs 2024-03-22T07:13:24.161 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:23 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:24.162 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:23 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:24.162 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:23 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:13:24.162 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:23 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:24.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:23 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 282.966835 secs 2024-03-22T07:13:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:23 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:23 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:23 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:13:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:23 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:24 smithi081 ceph-mon[113965]: Reconfiguring crash.smithi149 (monmap changed)... 2024-03-22T07:13:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:24 smithi081 ceph-mon[113965]: Reconfiguring daemon crash.smithi149 on smithi149 2024-03-22T07:13:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:24 smithi081 ceph-mon[113965]: pgmap v22: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:24 smithi149 ceph-mon[83447]: Reconfiguring crash.smithi149 (monmap changed)... 2024-03-22T07:13:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:24 smithi149 ceph-mon[83447]: Reconfiguring daemon crash.smithi149 on smithi149 2024-03-22T07:13:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:24 smithi149 ceph-mon[83447]: pgmap v22: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:26.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:25 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:26.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:25 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:26.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:25 smithi149 ceph-mon[83447]: Reconfiguring mgr.smithi149.sgmtoa (monmap changed)... 2024-03-22T07:13:26.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:25 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:13:26.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:25 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:13:26.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:25 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:26.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:25 smithi149 ceph-mon[83447]: Reconfiguring daemon mgr.smithi149.sgmtoa on smithi149 2024-03-22T07:13:26.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:25 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:26.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:25 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:26.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:25 smithi081 ceph-mon[113965]: Reconfiguring mgr.smithi149.sgmtoa (monmap changed)... 2024-03-22T07:13:26.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:25 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi149.sgmtoa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:13:26.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:25 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:13:26.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:25 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:26.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:25 smithi081 ceph-mon[113965]: Reconfiguring daemon mgr.smithi149.sgmtoa on smithi149 2024-03-22T07:13:27.173 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:26 smithi149 ceph-mon[83447]: pgmap v23: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:27.173 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:27.173 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:27.173 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:13:27.173 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:13:27.173 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:27.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:26 smithi081 ceph-mon[113965]: pgmap v23: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:27.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:27.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:27.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:13:27.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:13:27.221 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:28.177 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:27 smithi149 ceph-mon[83447]: Reconfiguring mon.smithi149 (monmap changed)... 2024-03-22T07:13:28.177 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:27 smithi149 ceph-mon[83447]: Reconfiguring daemon mon.smithi149 on smithi149 2024-03-22T07:13:28.177 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:27 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 287.626668 secs 2024-03-22T07:13:28.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:27 smithi081 ceph-mon[113965]: Reconfiguring mon.smithi149 (monmap changed)... 2024-03-22T07:13:28.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:27 smithi081 ceph-mon[113965]: Reconfiguring daemon mon.smithi149 on smithi149 2024-03-22T07:13:28.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:27 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 287.626668 secs 2024-03-22T07:13:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:28 smithi149 ceph-mon[83447]: pgmap v24: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:28 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 287.966907 secs 2024-03-22T07:13:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:28 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:28 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:28 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:13:29.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:28 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:28 smithi081 ceph-mon[113965]: pgmap v24: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:28 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 287.966907 secs 2024-03-22T07:13:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:28 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:28 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:28 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:13:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:28 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:30.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:29 smithi081 ceph-mon[113965]: Reconfiguring osd.3 (monmap changed)... 2024-03-22T07:13:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:29 smithi081 ceph-mon[113965]: Reconfiguring daemon osd.3 on smithi149 2024-03-22T07:13:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:29 smithi149 ceph-mon[83447]: Reconfiguring osd.3 (monmap changed)... 2024-03-22T07:13:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:29 smithi149 ceph-mon[83447]: Reconfiguring daemon osd.3 on smithi149 2024-03-22T07:13:31.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:30 smithi081 ceph-mon[113965]: pgmap v25: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:13:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:30 smithi149 ceph-mon[83447]: pgmap v25: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:13:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:32.235 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:31 smithi149 ceph-mon[83447]: Reconfiguring osd.4 (monmap changed)... 2024-03-22T07:13:32.235 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:31 smithi149 ceph-mon[83447]: Reconfiguring daemon osd.4 on smithi149 2024-03-22T07:13:32.236 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:32.236 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:32.236 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:13:32.236 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:31 smithi081 ceph-mon[113965]: Reconfiguring osd.4 (monmap changed)... 2024-03-22T07:13:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:31 smithi081 ceph-mon[113965]: Reconfiguring daemon osd.4 on smithi149 2024-03-22T07:13:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:13:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:32 smithi149 ceph-mon[83447]: Reconfiguring osd.5 (monmap changed)... 2024-03-22T07:13:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:32 smithi149 ceph-mon[83447]: Reconfiguring daemon osd.5 on smithi149 2024-03-22T07:13:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:32 smithi149 ceph-mon[83447]: pgmap v26: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:32 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 292.626762 secs 2024-03-22T07:13:33.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:32 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 292.966974 secs 2024-03-22T07:13:33.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:32 smithi081 ceph-mon[113965]: Reconfiguring osd.5 (monmap changed)... 2024-03-22T07:13:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:32 smithi081 ceph-mon[113965]: Reconfiguring daemon osd.5 on smithi149 2024-03-22T07:13:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:32 smithi081 ceph-mon[113965]: pgmap v26: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:32 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 292.626762 secs 2024-03-22T07:13:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:32 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 292.966974 secs 2024-03-22T07:13:34.422 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:34 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:34.423 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:34 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:34.423 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:34 smithi149 ceph-mon[83447]: Reconfiguring mds.cephfs.smithi149.kdtxfw (monmap changed)... 2024-03-22T07:13:34.423 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:34 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.kdtxfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:34.423 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:34 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:34.423 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:34 smithi149 ceph-mon[83447]: Reconfiguring daemon mds.cephfs.smithi149.kdtxfw on smithi149 2024-03-22T07:13:34.423 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:34 smithi149 ceph-mon[83447]: pgmap v27: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:34 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:34 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:34 smithi081 ceph-mon[113965]: Reconfiguring mds.cephfs.smithi149.kdtxfw (monmap changed)... 2024-03-22T07:13:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:34 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.kdtxfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:34 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:34 smithi081 ceph-mon[113965]: Reconfiguring daemon mds.cephfs.smithi149.kdtxfw on smithi149 2024-03-22T07:13:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:34 smithi081 ceph-mon[113965]: pgmap v27: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:35.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.484+0000 7fe70a2fa700 1 -- 172.21.15.81:0/1922773104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 msgr2=0x7fe7041031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:35.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.484+0000 7fe70a2fa700 1 --2- 172.21.15.81:0/1922773104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 0x7fe7041031b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4004940 tx=0x7fe6f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:35.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.485+0000 7fe70a2fa700 1 -- 172.21.15.81:0/1922773104 shutdown_connections 2024-03-22T07:13:35.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.485+0000 7fe70a2fa700 1 --2- 172.21.15.81:0/1922773104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe704103780 0x7fe7040fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:35.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.485+0000 7fe70a2fa700 1 --2- 172.21.15.81:0/1922773104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 0x7fe7041031b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:35.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.485+0000 7fe70a2fa700 1 -- 172.21.15.81:0/1922773104 >> 172.21.15.81:0/1922773104 conn(0x7fe7040f8950 msgr2=0x7fe7040fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:35.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.485+0000 7fe70a2fa700 1 -- 172.21.15.81:0/1922773104 shutdown_connections 2024-03-22T07:13:35.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.485+0000 7fe70a2fa700 1 -- 172.21.15.81:0/1922773104 wait complete. 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.485+0000 7fe70a2fa700 1 Processor -- start 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.485+0000 7fe70a2fa700 1 -- start start 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe70a2fa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 0x7fe7040702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe70a2fa700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe704103780 0x7fe704070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe70a2fa700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe704072210 con 0x7fe704102de0 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe70a2fa700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe704070d40 con 0x7fe704103780 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe703fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 0x7fe7040702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe703fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 0x7fe7040702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52598/0 (socket says 172.21.15.81:52598) 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe703fff700 1 -- 172.21.15.81:0/2672873020 learned_addr learned my addr 172.21.15.81:0/2672873020 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe7037fe700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe704103780 0x7fe704070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe703fff700 1 -- 172.21.15.81:0/2672873020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe704103780 msgr2=0x7fe704070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe703fff700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe704103780 0x7fe704070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe703fff700 1 -- 172.21.15.81:0/2672873020 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4005040 con 0x7fe704102de0 2024-03-22T07:13:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe703fff700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 0x7fe7040702c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009180 tx=0x7fe6f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe700ff9700 1 -- 172.21.15.81:0/2672873020 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f400e040 con 0x7fe704102de0 2024-03-22T07:13:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe70a2fa700 1 -- 172.21.15.81:0/2672873020 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe704070fc0 con 0x7fe704102de0 2024-03-22T07:13:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe700ff9700 1 -- 172.21.15.81:0/2672873020 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6f4013420 con 0x7fe704102de0 2024-03-22T07:13:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe700ff9700 1 -- 172.21.15.81:0/2672873020 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f401c630 con 0x7fe704102de0 2024-03-22T07:13:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.486+0000 7fe70a2fa700 1 -- 172.21.15.81:0/2672873020 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70411a0d0 con 0x7fe704102de0 2024-03-22T07:13:35.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.488+0000 7fe6ea7fc700 1 -- 172.21.15.81:0/2672873020 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7040fe530 con 0x7fe704102de0 2024-03-22T07:13:35.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.488+0000 7fe700ff9700 1 -- 172.21.15.81:0/2672873020 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fe6f4013590 con 0x7fe704102de0 2024-03-22T07:13:35.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.489+0000 7fe700ff9700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe6ec06d4b0 0x7fe6ec06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:35.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.489+0000 7fe7037fe700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe6ec06d4b0 0x7fe6ec06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:35.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.489+0000 7fe700ff9700 1 -- 172.21.15.81:0/2672873020 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe6f408dd50 con 0x7fe704102de0 2024-03-22T07:13:35.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.489+0000 7fe7037fe700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe6ec06d4b0 0x7fe6ec06f960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe7040719f0 tx=0x7fe6f800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:35.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.494+0000 7fe700ff9700 1 -- 172.21.15.81:0/2672873020 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe6f4057690 con 0x7fe704102de0 2024-03-22T07:13:35.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.703+0000 7fe6ea7fc700 1 -- 172.21.15.81:0/2672873020 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe70404c710 con 0x7fe6ec06d4b0 2024-03-22T07:13:35.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.704+0000 7fe700ff9700 1 -- 172.21.15.81:0/2672873020 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fe70404c710 con 0x7fe6ec06d4b0 2024-03-22T07:13:35.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 -- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe6ec06d4b0 msgr2=0x7fe6ec06f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe6ec06d4b0 0x7fe6ec06f960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe7040719f0 tx=0x7fe6f800b040 comp rx=0 tx=0).stop 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 -- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 msgr2=0x7fe7040702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 0x7fe7040702c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009180 tx=0x7fe6f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 -- 172.21.15.81:0/2672873020 shutdown_connections 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe6ec06d4b0 0x7fe6ec06f960 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe704102de0 0x7fe7040702c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 --2- 172.21.15.81:0/2672873020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe704103780 0x7fe704070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 -- 172.21.15.81:0/2672873020 >> 172.21.15.81:0/2672873020 conn(0x7fe7040f8950 msgr2=0x7fe704107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 -- 172.21.15.81:0/2672873020 shutdown_connections 2024-03-22T07:13:35.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.706+0000 7fe6ea7fc700 1 -- 172.21.15.81:0/2672873020 wait complete. 2024-03-22T07:13:35.716 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:13:35.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.809+0000 7f833d88f700 1 -- 172.21.15.81:0/2499141586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 msgr2=0x7f83380fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:35.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.809+0000 7f833d88f700 1 --2- 172.21.15.81:0/2499141586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 0x7f83380fb970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f832c004940 tx=0x7f832c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:35.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.809+0000 7f833d88f700 1 -- 172.21.15.81:0/2499141586 shutdown_connections 2024-03-22T07:13:35.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.809+0000 7f833d88f700 1 --2- 172.21.15.81:0/2499141586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 0x7f83380fb970 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:35.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.809+0000 7f833d88f700 1 --2- 172.21.15.81:0/2499141586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8338101510 0x7f83381018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:35.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.809+0000 7f833d88f700 1 -- 172.21.15.81:0/2499141586 >> 172.21.15.81:0/2499141586 conn(0x7f8338072370 msgr2=0x7f8338072770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:35.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.809+0000 7f833d88f700 1 -- 172.21.15.81:0/2499141586 shutdown_connections 2024-03-22T07:13:35.810 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.809+0000 7f833d88f700 1 -- 172.21.15.81:0/2499141586 wait complete. 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f833d88f700 1 Processor -- start 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f833d88f700 1 -- start start 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f833d88f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 0x7f8338113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f833d88f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8338101510 0x7f8338113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f833d88f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8338114620 con 0x7f83380fb500 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f833d88f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833810dbe0 con 0x7f8338101510 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f8336ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 0x7f8338113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f8336ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 0x7f8338113a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52612/0 (socket says 172.21.15.81:52612) 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f8336ffd700 1 -- 172.21.15.81:0/2171748259 learned_addr learned my addr 172.21.15.81:0/2171748259 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f83367fc700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8338101510 0x7f8338113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f8336ffd700 1 -- 172.21.15.81:0/2171748259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8338101510 msgr2=0x7f8338113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f8336ffd700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8338101510 0x7f8338113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:35.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f8336ffd700 1 -- 172.21.15.81:0/2171748259 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c005040 con 0x7f83380fb500 2024-03-22T07:13:35.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f8336ffd700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 0x7f8338113a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f832800b790 tx=0x7f8328009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:35.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.810+0000 7f831f7fe700 1 -- 172.21.15.81:0/2171748259 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f832800f770 con 0x7f83380fb500 2024-03-22T07:13:35.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.811+0000 7f831f7fe700 1 -- 172.21.15.81:0/2171748259 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f832800fda0 con 0x7f83380fb500 2024-03-22T07:13:35.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.811+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833810dec0 con 0x7f83380fb500 2024-03-22T07:13:35.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.811+0000 7f831f7fe700 1 -- 172.21.15.81:0/2171748259 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f832800d310 con 0x7f83380fb500 2024-03-22T07:13:35.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.811+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833810e400 con 0x7f83380fb500 2024-03-22T07:13:35.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.812+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8338104900 con 0x7f83380fb500 2024-03-22T07:13:35.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.812+0000 7f831f7fe700 1 -- 172.21.15.81:0/2171748259 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f8328010420 con 0x7f83380fb500 2024-03-22T07:13:35.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.812+0000 7f831f7fe700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f832006d4b0 0x7f832006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:35.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.813+0000 7f831f7fe700 1 -- 172.21.15.81:0/2171748259 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f832808c9a0 con 0x7f83380fb500 2024-03-22T07:13:35.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.813+0000 7f83367fc700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f832006d4b0 0x7f832006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:35.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.813+0000 7f83367fc700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f832006d4b0 0x7f832006f960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f833810f230 tx=0x7f832c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:35.817 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:35.817+0000 7f831f7fe700 1 -- 172.21.15.81:0/2171748259 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8328059fc0 con 0x7f83380fb500 2024-03-22T07:13:36.032 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.032+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8338108670 con 0x7f832006d4b0 2024-03-22T07:13:36.032 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:35 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:36.033 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:35 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:36.033 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:35 smithi081 ceph-mon[113965]: Reconfiguring mds.cephfs.smithi149.ffewvr (monmap changed)... 2024-03-22T07:13:36.033 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:35 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.ffewvr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:36.033 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:35 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:36.033 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:35 smithi081 ceph-mon[113965]: Reconfiguring daemon mds.cephfs.smithi149.ffewvr on smithi149 2024-03-22T07:13:36.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.033+0000 7f831f7fe700 1 -- 172.21.15.81:0/2171748259 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f8338108670 con 0x7f832006d4b0 2024-03-22T07:13:36.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.035+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f832006d4b0 msgr2=0x7f832006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.035+0000 7f833d88f700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f832006d4b0 0x7f832006f960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f833810f230 tx=0x7f832c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:36.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.035+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 msgr2=0x7f8338113a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.035+0000 7f833d88f700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 0x7f8338113a00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f832800b790 tx=0x7f8328009250 comp rx=0 tx=0).stop 2024-03-22T07:13:36.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.035+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 shutdown_connections 2024-03-22T07:13:36.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.035+0000 7f833d88f700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f832006d4b0 0x7f832006f960 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.035+0000 7f833d88f700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83380fb500 0x7f8338113a00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.035+0000 7f833d88f700 1 --2- 172.21.15.81:0/2171748259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8338101510 0x7f8338113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.035+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 >> 172.21.15.81:0/2171748259 conn(0x7f8338072370 msgr2=0x7f83380fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:36.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.036+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 shutdown_connections 2024-03-22T07:13:36.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.036+0000 7f833d88f700 1 -- 172.21.15.81:0/2171748259 wait complete. 2024-03-22T07:13:36.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:35 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:36.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:35 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:36.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:35 smithi149 ceph-mon[83447]: Reconfiguring mds.cephfs.smithi149.ffewvr (monmap changed)... 2024-03-22T07:13:36.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:35 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.ffewvr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:36.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:35 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:36.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:35 smithi149 ceph-mon[83447]: Reconfiguring daemon mds.cephfs.smithi149.ffewvr on smithi149 2024-03-22T07:13:36.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.136+0000 7f42efe77700 1 -- 172.21.15.81:0/1045749925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8102710 msgr2=0x7f42e80fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.136+0000 7f42efe77700 1 --2- 172.21.15.81:0/1045749925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8102710 0x7f42e80fbd80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f42e4004940 tx=0x7f42e4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:36.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.136+0000 7f42efe77700 1 -- 172.21.15.81:0/1045749925 shutdown_connections 2024-03-22T07:13:36.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.136+0000 7f42efe77700 1 --2- 172.21.15.81:0/1045749925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8102710 0x7f42e80fbd80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.136+0000 7f42efe77700 1 --2- 172.21.15.81:0/1045749925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e8101d70 0x7f42e8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.136+0000 7f42efe77700 1 -- 172.21.15.81:0/1045749925 >> 172.21.15.81:0/1045749925 conn(0x7f42e8072bc0 msgr2=0x7f42e8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:36.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.136+0000 7f42efe77700 1 -- 172.21.15.81:0/1045749925 shutdown_connections 2024-03-22T07:13:36.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.136+0000 7f42efe77700 1 -- 172.21.15.81:0/1045749925 wait complete. 2024-03-22T07:13:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.136+0000 7f42efe77700 1 Processor -- start 2024-03-22T07:13:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42efe77700 1 -- start start 2024-03-22T07:13:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42efe77700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8101d70 0x7f42e8102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42efe77700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e8102710 0x7f42e8103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42efe77700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81036e0 con 0x7f42e8101d70 2024-03-22T07:13:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42efe77700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e8103850 con 0x7f42e8102710 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42edc13700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8101d70 0x7f42e8102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42edc13700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8101d70 0x7f42e8102bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52630/0 (socket says 172.21.15.81:52630) 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42edc13700 1 -- 172.21.15.81:0/2219351175 learned_addr learned my addr 172.21.15.81:0/2219351175 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42ed412700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e8102710 0x7f42e8103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42edc13700 1 -- 172.21.15.81:0/2219351175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e8102710 msgr2=0x7f42e8103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42edc13700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e8102710 0x7f42e8103110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42edc13700 1 -- 172.21.15.81:0/2219351175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e4005040 con 0x7f42e8101d70 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42edc13700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8101d70 0x7f42e8102bd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42d800b790 tx=0x7f42d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42de7fc700 1 -- 172.21.15.81:0/2219351175 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d800f6f0 con 0x7f42e8101d70 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42de7fc700 1 -- 172.21.15.81:0/2219351175 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42d800fd20 con 0x7f42e8101d70 2024-03-22T07:13:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e8119e70 con 0x7f42e8101d70 2024-03-22T07:13:36.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.137+0000 7f42de7fc700 1 -- 172.21.15.81:0/2219351175 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d80103a0 con 0x7f42e8101d70 2024-03-22T07:13:36.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.138+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e811a2d0 con 0x7f42e8101d70 2024-03-22T07:13:36.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.138+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e80fd4c0 con 0x7f42e8101d70 2024-03-22T07:13:36.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.139+0000 7f42de7fc700 1 -- 172.21.15.81:0/2219351175 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f42d8011400 con 0x7f42e8101d70 2024-03-22T07:13:36.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.139+0000 7f42de7fc700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f42d406d4a0 0x7f42d406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.140+0000 7f42de7fc700 1 -- 172.21.15.81:0/2219351175 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f42d808d190 con 0x7f42e8101d70 2024-03-22T07:13:36.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.140+0000 7f42ed412700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f42d406d4a0 0x7f42d406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.140+0000 7f42ed412700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f42d406d4a0 0x7f42d406f950 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f42e80fa5c0 tx=0x7f42e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:36.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.144+0000 7f42de7fc700 1 -- 172.21.15.81:0/2219351175 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42d8056c80 con 0x7f42e8101d70 2024-03-22T07:13:36.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.350+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f42e804bf40 con 0x7f42d406d4a0 2024-03-22T07:13:36.360 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.360+0000 7f42de7fc700 1 -- 172.21.15.81:0/2219351175 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f42e804bf40 con 0x7f42d406d4a0 2024-03-22T07:13:36.360 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:13:36.360 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (2m) 45s ago 12m 16.8M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:13:36.360 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (12m) 45s ago 12m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 1013184e902c 2024-03-22T07:13:36.360 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (11m) 31s ago 11m 6665k - 17.2.7-726-g9c4e0c34 837124da804f d30ccae78504 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (112s) 45s ago 12m 80.7M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (7m) 45s ago 7m 13.0M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (7m) 45s ago 7m 132M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (7m) 31s ago 7m 16.6M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (7m) 31s ago 7m 147M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (3m) 45s ago 13m 472M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (3m) 31s ago 11m 447M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (59s) 45s ago 14m 27.2M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (34s) 31s ago 11m 21.1M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (2m) 45s ago 12m 17.0M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (2m) 31s ago 11m 17.0M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (10m) 45s ago 10m 184M 1557M 17.2.7-726-g9c4e0c34 837124da804f 703c47532b9b 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (10m) 45s ago 10m 200M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (9m) 45s ago 9m 162M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:13:36.361 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (9m) 31s ago 9m 215M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:13:36.362 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (9m) 31s ago 9m 193M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:13:36.362 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (8m) 31s ago 8m 183M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:13:36.362 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (2m) 45s ago 12m 51.9M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f42d406d4a0 msgr2=0x7f42d406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f42d406d4a0 0x7f42d406f950 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f42e80fa5c0 tx=0x7f42e4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8101d70 msgr2=0x7f42e8102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8101d70 0x7f42e8102bd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f42d800b790 tx=0x7f42d8009250 comp rx=0 tx=0).stop 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 shutdown_connections 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f42d406d4a0 0x7f42d406f950 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42e8101d70 0x7f42e8102bd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 --2- 172.21.15.81:0/2219351175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42e8102710 0x7f42e8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 >> 172.21.15.81:0/2219351175 conn(0x7f42e8072bc0 msgr2=0x7f42e8065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 shutdown_connections 2024-03-22T07:13:36.363 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.362+0000 7f42efe77700 1 -- 172.21.15.81:0/2219351175 wait complete. 2024-03-22T07:13:36.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.461+0000 7f80e8cff700 1 -- 172.21.15.81:0/2232983854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fe7c0 msgr2=0x7f80e4106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.461+0000 7f80e8cff700 1 --2- 172.21.15.81:0/2232983854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fe7c0 0x7f80e4106e00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f80d8004940 tx=0x7f80d8009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:36.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.461+0000 7f80e8cff700 1 -- 172.21.15.81:0/2232983854 shutdown_connections 2024-03-22T07:13:36.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.461+0000 7f80e8cff700 1 --2- 172.21.15.81:0/2232983854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fe7c0 0x7f80e4106e00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.461+0000 7f80e8cff700 1 --2- 172.21.15.81:0/2232983854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e40fde20 0x7f80e40fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.461+0000 7f80e8cff700 1 -- 172.21.15.81:0/2232983854 >> 172.21.15.81:0/2232983854 conn(0x7f80e40f7870 msgr2=0x7f80e40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:36.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.461+0000 7f80e8cff700 1 -- 172.21.15.81:0/2232983854 shutdown_connections 2024-03-22T07:13:36.462 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.461+0000 7f80e8cff700 1 -- 172.21.15.81:0/2232983854 wait complete. 2024-03-22T07:13:36.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e8cff700 1 Processor -- start 2024-03-22T07:13:36.463 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e8cff700 1 -- start start 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e8cff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fde20 0x7f80e41139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e8cff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e40fe7c0 0x7f80e4113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e8cff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4114530 con 0x7f80e40fde20 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e8cff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e410db30 con 0x7f80e40fe7c0 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fde20 0x7f80e41139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fde20 0x7f80e41139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52642/0 (socket says 172.21.15.81:52642) 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e259c700 1 -- 172.21.15.81:0/2461700229 learned_addr learned my addr 172.21.15.81:0/2461700229 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.462+0000 7f80e1d9b700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e40fe7c0 0x7f80e4113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.463+0000 7f80e259c700 1 -- 172.21.15.81:0/2461700229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e40fe7c0 msgr2=0x7f80e4113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.463+0000 7f80e259c700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e40fe7c0 0x7f80e4113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.463+0000 7f80e259c700 1 -- 172.21.15.81:0/2461700229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8005040 con 0x7f80e40fde20 2024-03-22T07:13:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.463+0000 7f80e259c700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fde20 0x7f80e41139a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c8c0 tx=0x7f80d40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:36.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.465+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2461700229 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d400a760 con 0x7f80e40fde20 2024-03-22T07:13:36.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.465+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e410de10 con 0x7f80e40fde20 2024-03-22T07:13:36.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.465+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2461700229 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80d400ad90 con 0x7f80e40fde20 2024-03-22T07:13:36.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.465+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e410e320 con 0x7f80e40fde20 2024-03-22T07:13:36.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.465+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2461700229 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80d40113a0 con 0x7f80e40fde20 2024-03-22T07:13:36.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.466+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c40052b0 con 0x7f80e40fde20 2024-03-22T07:13:36.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.466+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2461700229 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f80d4011500 con 0x7f80e40fde20 2024-03-22T07:13:36.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.467+0000 7f80d2ffd700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f80cc06d4b0 0x7f80cc06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.467+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2461700229 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f80d408c900 con 0x7f80e40fde20 2024-03-22T07:13:36.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.467+0000 7f80e1d9b700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f80cc06d4b0 0x7f80cc06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.467+0000 7f80e1d9b700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f80cc06d4b0 0x7f80cc06f960 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f80e410ee10 tx=0x7f80d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:36.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.471+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2461700229 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80d4059f90 con 0x7f80e40fde20 2024-03-22T07:13:36.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.759+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80c4005830 con 0x7f80e40fde20 2024-03-22T07:13:36.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.760+0000 7f80d2ffd700 1 -- 172.21.15.81:0/2461700229 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f80d4059b20 con 0x7f80e40fde20 2024-03-22T07:13:36.760 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:13:36.760 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:13:36.760 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:13:36.761 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:13:36.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f80cc06d4b0 msgr2=0x7f80cc06f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f80cc06d4b0 0x7f80cc06f960 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f80e410ee10 tx=0x7f80d8009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fde20 msgr2=0x7f80e41139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fde20 0x7f80e41139a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c8c0 tx=0x7f80d40044c0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 shutdown_connections 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f80cc06d4b0 0x7f80cc06f960 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80e40fde20 0x7f80e41139a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 --2- 172.21.15.81:0/2461700229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e40fe7c0 0x7f80e4113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 >> 172.21.15.81:0/2461700229 conn(0x7f80e40f7870 msgr2=0x7f80e40f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 shutdown_connections 2024-03-22T07:13:36.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.762+0000 7f80e8cff700 1 -- 172.21.15.81:0/2461700229 wait complete. 2024-03-22T07:13:36.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.868+0000 7fd1a057d700 1 -- 172.21.15.81:0/3698574817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 msgr2=0x7fd1980fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.868+0000 7fd1a057d700 1 --2- 172.21.15.81:0/3698574817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 0x7fd1980fbd80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd188004940 tx=0x7fd188009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.868+0000 7fd1a057d700 1 -- 172.21.15.81:0/3698574817 shutdown_connections 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.868+0000 7fd1a057d700 1 --2- 172.21.15.81:0/3698574817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 0x7fd1980fbd80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.868+0000 7fd1a057d700 1 --2- 172.21.15.81:0/3698574817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd198101d70 0x7fd198102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.868+0000 7fd1a057d700 1 -- 172.21.15.81:0/3698574817 >> 172.21.15.81:0/3698574817 conn(0x7fd198072bc0 msgr2=0x7fd198072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.868+0000 7fd1a057d700 1 -- 172.21.15.81:0/3698574817 shutdown_connections 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.868+0000 7fd1a057d700 1 -- 172.21.15.81:0/3698574817 wait complete. 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd1a057d700 1 Processor -- start 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd1a057d700 1 -- start start 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd1a057d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd198101d70 0x7fd19810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd1a057d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 0x7fd19810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd1a057d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810f6d0 con 0x7fd198102710 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd1a057d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810f810 con 0x7fd198101d70 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd19db18700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 0x7fd19810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd19db18700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 0x7fd19810f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52656/0 (socket says 172.21.15.81:52656) 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd19db18700 1 -- 172.21.15.81:0/2136542413 learned_addr learned my addr 172.21.15.81:0/2136542413 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd19e319700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd198101d70 0x7fd19810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd19db18700 1 -- 172.21.15.81:0/2136542413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd198101d70 msgr2=0x7fd19810eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd19db18700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd198101d70 0x7fd19810eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd19db18700 1 -- 172.21.15.81:0/2136542413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188005040 con 0x7fd198102710 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.869+0000 7fd19db18700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 0x7fd19810f080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd188009180 tx=0x7fd188009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.870+0000 7fd18effd700 1 -- 172.21.15.81:0/2136542413 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18800e040 con 0x7fd198102710 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.870+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd198117bb0 con 0x7fd198102710 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.870+0000 7fd18effd700 1 -- 172.21.15.81:0/2136542413 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd188013420 con 0x7fd198102710 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.870+0000 7fd18effd700 1 -- 172.21.15.81:0/2136542413 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18801c610 con 0x7fd198102710 2024-03-22T07:13:36.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.870+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1981180c0 con 0x7fd198102710 2024-03-22T07:13:36.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.871+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd198104240 con 0x7fd198102710 2024-03-22T07:13:36.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.871+0000 7fd18effd700 1 -- 172.21.15.81:0/2136542413 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fd188013a00 con 0x7fd198102710 2024-03-22T07:13:36.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.872+0000 7fd18effd700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd18406d4a0 0x7fd18406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.872+0000 7fd18effd700 1 -- 172.21.15.81:0/2136542413 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd18808dc60 con 0x7fd198102710 2024-03-22T07:13:36.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.872+0000 7fd19e319700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd18406d4a0 0x7fd18406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.872+0000 7fd19e319700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd18406d4a0 0x7fd18406f950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd194007fd0 tx=0x7fd194009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:36.881 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:36.880+0000 7fd18effd700 1 -- 172.21.15.81:0/2136542413 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd188057650 con 0x7fd198102710 2024-03-22T07:13:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: pgmap v28: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:37.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:37.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi081"}]: dispatch 2024-03-22T07:13:37.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi081"}]': finished 2024-03-22T07:13:37.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi149"}]: dispatch 2024-03-22T07:13:37.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi149"}]': finished 2024-03-22T07:13:37.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:36 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2461700229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: pgmap v28: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi081"}]: dispatch 2024-03-22T07:13:37.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi081"}]': finished 2024-03-22T07:13:37.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi149"}]: dispatch 2024-03-22T07:13:37.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi149"}]': finished 2024-03-22T07:13:37.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:36 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2461700229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:37.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.132+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd198065db0 con 0x7fd198102710 2024-03-22T07:13:37.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.133+0000 7fd18effd700 1 -- 172.21.15.81:0/2136542413 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1858 (secure 0 0 0) 0x7fd18805ad10 con 0x7fd198102710 2024-03-22T07:13:37.133 INFO:teuthology.orchestra.run.smithi081.stdout:e22 2024-03-22T07:13:37.133 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 22 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:12:38.916315+0000 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:13:37.134 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:13:37.135 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 95 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 94 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:37.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd18406d4a0 msgr2=0x7fd18406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd18406d4a0 0x7fd18406f950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd194007fd0 tx=0x7fd194009040 comp rx=0 tx=0).stop 2024-03-22T07:13:37.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 msgr2=0x7fd19810f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 0x7fd19810f080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd188009180 tx=0x7fd188009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:37.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 shutdown_connections 2024-03-22T07:13:37.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd18406d4a0 0x7fd18406f950 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd198101d70 0x7fd19810eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 --2- 172.21.15.81:0/2136542413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd198102710 0x7fd19810f080 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 >> 172.21.15.81:0/2136542413 conn(0x7fd198072bc0 msgr2=0x7fd1980fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:37.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 shutdown_connections 2024-03-22T07:13:37.138 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.136+0000 7fd1a057d700 1 -- 172.21.15.81:0/2136542413 wait complete. 2024-03-22T07:13:37.138 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 22 2024-03-22T07:13:37.240 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.240+0000 7f73d789e700 1 -- 172.21.15.81:0/1957195231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 msgr2=0x7f73d0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.240+0000 7f73d789e700 1 --2- 172.21.15.81:0/1957195231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 0x7f73d0106f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73c0004940 tx=0x7f73c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:37.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.240+0000 7f73d789e700 1 -- 172.21.15.81:0/1957195231 shutdown_connections 2024-03-22T07:13:37.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.240+0000 7f73d789e700 1 --2- 172.21.15.81:0/1957195231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 0x7f73d0106f00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.240+0000 7f73d789e700 1 --2- 172.21.15.81:0/1957195231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73d00fde90 0x7f73d00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.240+0000 7f73d789e700 1 -- 172.21.15.81:0/1957195231 >> 172.21.15.81:0/1957195231 conn(0x7f73d0072bc0 msgr2=0x7f73d0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:37.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.240+0000 7f73d789e700 1 -- 172.21.15.81:0/1957195231 shutdown_connections 2024-03-22T07:13:37.241 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.240+0000 7f73d789e700 1 -- 172.21.15.81:0/1957195231 wait complete. 2024-03-22T07:13:37.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d789e700 1 Processor -- start 2024-03-22T07:13:37.242 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d789e700 1 -- start start 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d789e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73d00fde90 0x7f73d010b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d789e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 0x7f73d010bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d789e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0112a90 con 0x7f73d00fe830 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d789e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0112bd0 con 0x7f73d00fde90 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d4e39700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 0x7f73d010bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d4e39700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 0x7f73d010bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52664/0 (socket says 172.21.15.81:52664) 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d4e39700 1 -- 172.21.15.81:0/1303794962 learned_addr learned my addr 172.21.15.81:0/1303794962 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.241+0000 7f73d563a700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73d00fde90 0x7f73d010b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.242+0000 7f73d4e39700 1 -- 172.21.15.81:0/1303794962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73d00fde90 msgr2=0x7f73d010b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.242+0000 7f73d4e39700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73d00fde90 0x7f73d010b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.242+0000 7f73d4e39700 1 -- 172.21.15.81:0/1303794962 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c0005040 con 0x7f73d00fe830 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.242+0000 7f73d4e39700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 0x7f73d010bda0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f73c0000c00 tx=0x7f73c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.242+0000 7f73c5ffb700 1 -- 172.21.15.81:0/1303794962 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c0021070 con 0x7f73d00fe830 2024-03-22T07:13:37.243 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.242+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d010c340 con 0x7f73d00fe830 2024-03-22T07:13:37.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.242+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d0110530 con 0x7f73d00fe830 2024-03-22T07:13:37.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.242+0000 7f73c5ffb700 1 -- 172.21.15.81:0/1303794962 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73c0013c00 con 0x7f73d00fe830 2024-03-22T07:13:37.244 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.242+0000 7f73c5ffb700 1 -- 172.21.15.81:0/1303794962 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c000e070 con 0x7f73d00fe830 2024-03-22T07:13:37.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.243+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b40052b0 con 0x7f73d00fe830 2024-03-22T07:13:37.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.244+0000 7f73c5ffb700 1 -- 172.21.15.81:0/1303794962 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f73c001c400 con 0x7f73d00fe830 2024-03-22T07:13:37.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.244+0000 7f73c5ffb700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f73bc06d4a0 0x7f73bc06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:37.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.244+0000 7f73c5ffb700 1 -- 172.21.15.81:0/1303794962 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f73c008dd10 con 0x7f73d00fe830 2024-03-22T07:13:37.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.244+0000 7f73d563a700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f73bc06d4a0 0x7f73bc06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:37.245 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.245+0000 7f73d563a700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f73bc06d4a0 0x7f73bc06f950 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009b80 tx=0x7f73cc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:37.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.249+0000 7f73c5ffb700 1 -- 172.21.15.81:0/1303794962 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f73c005b450 con 0x7f73d00fe830 2024-03-22T07:13:37.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.471+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73b4000bc0 con 0x7f73bc06d4a0 2024-03-22T07:13:37.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.472+0000 7f73c5ffb700 1 -- 172.21.15.81:0/1303794962 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f73b4000bc0 con 0x7f73bc06d4a0 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: "mon", 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr" 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "4/21 daemons upgraded", 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading crash daemons", 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:13:37.473 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:13:37.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f73bc06d4a0 msgr2=0x7f73bc06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f73bc06d4a0 0x7f73bc06f950 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009b80 tx=0x7f73cc00b040 comp rx=0 tx=0).stop 2024-03-22T07:13:37.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 msgr2=0x7f73d010bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 0x7f73d010bda0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f73c0000c00 tx=0x7f73c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:37.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 shutdown_connections 2024-03-22T07:13:37.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f73bc06d4a0 0x7f73bc06f950 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73d00fde90 0x7f73d010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 --2- 172.21.15.81:0/1303794962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d00fe830 0x7f73d010bda0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 >> 172.21.15.81:0/1303794962 conn(0x7f73d0072bc0 msgr2=0x7f73d00fb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:37.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 shutdown_connections 2024-03-22T07:13:37.476 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.475+0000 7f73d789e700 1 -- 172.21.15.81:0/1303794962 wait complete. 2024-03-22T07:13:37.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.575+0000 7fa5f2c72700 1 -- 172.21.15.81:0/2390627902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 msgr2=0x7fa5ec0ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.575+0000 7fa5f2c72700 1 --2- 172.21.15.81:0/2390627902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 0x7fa5ec0ff020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8004940 tx=0x7fa5d8009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:37.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.575+0000 7fa5f2c72700 1 -- 172.21.15.81:0/2390627902 shutdown_connections 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.575+0000 7fa5f2c72700 1 --2- 172.21.15.81:0/2390627902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 0x7fa5ec0ff020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.575+0000 7fa5f2c72700 1 --2- 172.21.15.81:0/2390627902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5ec105010 0x7fa5ec1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.575+0000 7fa5f2c72700 1 -- 172.21.15.81:0/2390627902 >> 172.21.15.81:0/2390627902 conn(0x7fa5ec0faba0 msgr2=0x7fa5ec0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.575+0000 7fa5f2c72700 1 -- 172.21.15.81:0/2390627902 shutdown_connections 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.575+0000 7fa5f2c72700 1 -- 172.21.15.81:0/2390627902 wait complete. 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.576+0000 7fa5f2c72700 1 Processor -- start 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.576+0000 7fa5f2c72700 1 -- start start 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.576+0000 7fa5f2c72700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5ec105010 0x7fa5ec115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.576+0000 7fa5f2c72700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 0x7fa5ec1162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.576+0000 7fa5f2c72700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec116900 con 0x7fa5ec1059b0 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.576+0000 7fa5f2c72700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec10fcd0 con 0x7fa5ec105010 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.576+0000 7fa5ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 0x7fa5ec1162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:37.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.576+0000 7fa5ebfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 0x7fa5ec1162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52668/0 (socket says 172.21.15.81:52668) 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.576+0000 7fa5ebfff700 1 -- 172.21.15.81:0/1873315180 learned_addr learned my addr 172.21.15.81:0/1873315180 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.577+0000 7fa5ebfff700 1 -- 172.21.15.81:0/1873315180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5ec105010 msgr2=0x7fa5ec115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.577+0000 7fa5ebfff700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5ec105010 0x7fa5ec115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.577+0000 7fa5ebfff700 1 -- 172.21.15.81:0/1873315180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d8005040 con 0x7fa5ec1059b0 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.577+0000 7fa5ebfff700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 0x7fa5ec1162b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009100 tx=0x7fa5d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.577+0000 7fa5e97fa700 1 -- 172.21.15.81:0/1873315180 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d800e040 con 0x7fa5ec1059b0 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.577+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec10ff50 con 0x7fa5ec1059b0 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.577+0000 7fa5e97fa700 1 -- 172.21.15.81:0/1873315180 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5d8013420 con 0x7fa5ec1059b0 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.577+0000 7fa5e97fa700 1 -- 172.21.15.81:0/1873315180 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d801c610 con 0x7fa5ec1059b0 2024-03-22T07:13:37.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.577+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec110510 con 0x7fa5ec1059b0 2024-03-22T07:13:37.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.578+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec100760 con 0x7fa5ec1059b0 2024-03-22T07:13:37.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.579+0000 7fa5e97fa700 1 -- 172.21.15.81:0/1873315180 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fa5d8013590 con 0x7fa5ec1059b0 2024-03-22T07:13:37.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.579+0000 7fa5e97fa700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa5dc06d4a0 0x7fa5dc06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:37.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.579+0000 7fa5e97fa700 1 -- 172.21.15.81:0/1873315180 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa5d808ddc0 con 0x7fa5ec1059b0 2024-03-22T07:13:37.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.579+0000 7fa5f0a0e700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa5dc06d4a0 0x7fa5dc06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:37.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.579+0000 7fa5f0a0e700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa5dc06d4a0 0x7fa5dc06f950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0004a50 tx=0x7fa5e0007cb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:37.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.584+0000 7fa5e97fa700 1 -- 172.21.15.81:0/1873315180 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5d805b230 con 0x7fa5ec1059b0 2024-03-22T07:13:37.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.869+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa5ec05f5f0 con 0x7fa5ec1059b0 2024-03-22T07:13:37.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.869+0000 7fa5e97fa700 1 -- 172.21.15.81:0/1873315180 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+250 (secure 0 0 0) 0x7fa5d805adc0 con 0x7fa5ec1059b0 2024-03-22T07:13:37.870 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:13:37.870 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:13:37.870 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:13:37.870 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:13:37.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.871+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa5dc06d4a0 msgr2=0x7fa5dc06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.871+0000 7fa5f2c72700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa5dc06d4a0 0x7fa5dc06f950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0004a50 tx=0x7fa5e0007cb0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.871+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 msgr2=0x7fa5ec1162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:37.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.871+0000 7fa5f2c72700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 0x7fa5ec1162b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009100 tx=0x7fa5d8009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:37.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.872+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 shutdown_connections 2024-03-22T07:13:37.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.872+0000 7fa5f2c72700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa5dc06d4a0 0x7fa5dc06f950 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.872+0000 7fa5f2c72700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5ec105010 0x7fa5ec115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.872+0000 7fa5f2c72700 1 --2- 172.21.15.81:0/1873315180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5ec1059b0 0x7fa5ec1162b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:37.872 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.872+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 >> 172.21.15.81:0/1873315180 conn(0x7fa5ec0faba0 msgr2=0x7fa5ec0fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:37.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.872+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 shutdown_connections 2024-03-22T07:13:37.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:13:37.872+0000 7fa5f2c72700 1 -- 172.21.15.81:0/1873315180 wait complete. 2024-03-22T07:13:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:37 smithi081 ceph-mon[113965]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:37 smithi081 ceph-mon[113965]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:37 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all mon 2024-03-22T07:13:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:37 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2136542413' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:13:38.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:37 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 297.626849 secs 2024-03-22T07:13:38.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:37 smithi149 ceph-mon[83447]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:38.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:37 smithi149 ceph-mon[83447]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:38.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:37 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all mon 2024-03-22T07:13:38.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:37 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2136542413' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:13:38.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:37 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 297.626849 secs 2024-03-22T07:13:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:38 smithi081 ceph-mon[113965]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:38 smithi081 ceph-mon[113965]: pgmap v29: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:38 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 297.967093 secs 2024-03-22T07:13:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:38 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1873315180' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:13:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi081", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:13:39.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:39.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:38 smithi149 ceph-mon[83447]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:39.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:38 smithi149 ceph-mon[83447]: pgmap v29: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:39.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:38 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 297.967093 secs 2024-03-22T07:13:39.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:38 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1873315180' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:13:39.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:39.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi081", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:13:39.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:39 smithi081 ceph-mon[113965]: Upgrade: Updating crash.smithi081 (1/2) 2024-03-22T07:13:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:39 smithi081 ceph-mon[113965]: Deploying daemon crash.smithi081 on smithi081 2024-03-22T07:13:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:39 smithi149 ceph-mon[83447]: Upgrade: Updating crash.smithi081 (1/2) 2024-03-22T07:13:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:39 smithi149 ceph-mon[83447]: Deploying daemon crash.smithi081 on smithi081 2024-03-22T07:13:40.988 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:40 smithi081 ceph-mon[113965]: pgmap v30: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:41.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:40 smithi149 ceph-mon[83447]: pgmap v30: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:43.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:42 smithi149 ceph-mon[83447]: pgmap v31: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:43.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:42 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:43.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:42 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:43.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:42 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 302.626949 secs 2024-03-22T07:13:43.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:42 smithi081 ceph-mon[113965]: pgmap v31: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:43.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:42 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:43.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:42 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:43.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:42 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 302.626949 secs 2024-03-22T07:13:44.183 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:43 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 302.967166 secs 2024-03-22T07:13:44.184 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:43 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:44.184 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:43 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:13:44.184 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:43 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:44.200 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:43 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 302.967166 secs 2024-03-22T07:13:44.200 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:43 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:44.200 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:43 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi149", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:13:44.200 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:43 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:45.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:44 smithi149 ceph-mon[83447]: Upgrade: Updating crash.smithi149 (2/2) 2024-03-22T07:13:45.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:44 smithi149 ceph-mon[83447]: Deploying daemon crash.smithi149 on smithi149 2024-03-22T07:13:45.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:44 smithi149 ceph-mon[83447]: pgmap v32: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:45.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:44 smithi081 ceph-mon[113965]: Upgrade: Updating crash.smithi149 (2/2) 2024-03-22T07:13:45.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:44 smithi081 ceph-mon[113965]: Deploying daemon crash.smithi149 on smithi149 2024-03-22T07:13:45.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:44 smithi081 ceph-mon[113965]: pgmap v32: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:46.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:45 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:46.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:47.184 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:46 smithi149 ceph-mon[83447]: pgmap v33: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:47.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:46 smithi081 ceph-mon[113965]: pgmap v33: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:48 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:48 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:48 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:48 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 307.627012 secs 2024-03-22T07:13:48.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:48 smithi149 ceph-mon[83447]: pgmap v34: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:48.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:48 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 307.967234 secs 2024-03-22T07:13:48.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:48 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:48.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:48 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:48.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:48 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:48.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:48 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 307.627012 secs 2024-03-22T07:13:48.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:48 smithi081 ceph-mon[113965]: pgmap v34: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:48.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:48 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 307.967234 secs 2024-03-22T07:13:51.187 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:50 smithi081 ceph-mon[113965]: pgmap v35: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:51.187 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:50 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:51.187 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:50 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:50 smithi149 ceph-mon[83447]: pgmap v35: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:50 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:50 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:52 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:52 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:52 smithi149 ceph-mon[83447]: pgmap v36: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:52 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:52 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:52.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:52 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:52.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:52 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:52.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:52 smithi081 ceph-mon[113965]: pgmap v36: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:52.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:52 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:52.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:52 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:53 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 312.627083 secs 2024-03-22T07:13:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:53 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 312.967331 secs 2024-03-22T07:13:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:53 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 312.627083 secs 2024-03-22T07:13:53.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:53 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 312.967331 secs 2024-03-22T07:13:54.619 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:54 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:54.619 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:54 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:54.619 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:54 smithi081 ceph-mon[113965]: pgmap v37: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:54.762 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:54 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:54.763 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:54 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:54.763 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:54 smithi149 ceph-mon[83447]: pgmap v37: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:56.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:56.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:56.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:56.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:56.921 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:56 smithi081 ceph-mon[113965]: pgmap v38: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:57.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:56 smithi149 ceph-mon[83447]: pgmap v38: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:58.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:58.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:58.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:58.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:58.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:58.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:58.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:58.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:58.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi081"}]: dispatch 2024-03-22T07:13:58.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi081"}]': finished 2024-03-22T07:13:58.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi149"}]: dispatch 2024-03-22T07:13:58.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi149"}]': finished 2024-03-22T07:13:58.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T07:13:58.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 317.627165 secs 2024-03-22T07:13:58.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:58 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 317.967453 secs 2024-03-22T07:13:58.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:58.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:58.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi081"}]: dispatch 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi081"}]': finished 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi149"}]: dispatch 2024-03-22T07:13:58.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi149"}]': finished 2024-03-22T07:13:58.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T07:13:58.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 317.627165 secs 2024-03-22T07:13:58.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:58 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 317.967453 secs 2024-03-22T07:13:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:59 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all crash 2024-03-22T07:13:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:59 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T07:13:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:59 smithi081 ceph-mon[113965]: Upgrade: osd.0 is safe to restart 2024-03-22T07:13:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:59 smithi081 ceph-mon[113965]: pgmap v39: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:59 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:59 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:13:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:13:59 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:59 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all crash 2024-03-22T07:13:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:59 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T07:13:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:59 smithi149 ceph-mon[83447]: Upgrade: osd.0 is safe to restart 2024-03-22T07:13:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:59 smithi149 ceph-mon[83447]: pgmap v39: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:13:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:59 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:13:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:59 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:13:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:13:59 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:14:00.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:00 smithi081 ceph-mon[113965]: Upgrade: Updating osd.0 2024-03-22T07:14:00.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:00 smithi081 ceph-mon[113965]: Deploying daemon osd.0 on smithi081 2024-03-22T07:14:00.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:00 smithi149 ceph-mon[83447]: Upgrade: Updating osd.0 2024-03-22T07:14:00.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:00 smithi149 ceph-mon[83447]: Deploying daemon osd.0 on smithi081 2024-03-22T07:14:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:01 smithi081 ceph-mon[113965]: pgmap v40: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:01 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:01 smithi149 ceph-mon[83447]: pgmap v40: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:01 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:02.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:02 smithi081 ceph-mon[113965]: pgmap v41: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:02.320 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:01 smithi081 systemd[1]: Stopping Ceph osd.0 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:14:02.320 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:02 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[47932]: 2024-03-22T07:14:02.223+0000 7f364c7a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:14:02.320 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:02 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[47932]: 2024-03-22T07:14:02.223+0000 7f364c7a7700 -1 osd.0 49 *** Got signal Terminated *** 2024-03-22T07:14:02.320 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:02 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[47932]: 2024-03-22T07:14:02.223+0000 7f364c7a7700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:14:02.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:02 smithi149 ceph-mon[83447]: pgmap v41: 65 pgs: 65 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:03.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:03 smithi081 ceph-mon[113965]: osd.0 marked itself down and dead 2024-03-22T07:14:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:03 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 322.627250 secs 2024-03-22T07:14:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:03 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 322.967528 secs 2024-03-22T07:14:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:03 smithi149 ceph-mon[83447]: osd.0 marked itself down and dead 2024-03-22T07:14:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:03 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 322.627250 secs 2024-03-22T07:14:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:03 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 322.967528 secs 2024-03-22T07:14:03.833 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:03 smithi081 podman[125332]: 2024-03-22 07:14:03.582779401 +0000 UTC m=+1.461021746 container died 703c47532b9b386d781394e419b3d4a71924183af5703aaac0fa9d5aacff8bee (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3) 2024-03-22T07:14:04.108 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:04 smithi081 ceph-mon[113965]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:14:04.108 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:04 smithi081 ceph-mon[113965]: failed to encode map e50 with expected crc 2024-03-22T07:14:04.108 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:04 smithi081 ceph-mon[113965]: osdmap e50: 6 total, 5 up, 6 in 2024-03-22T07:14:04.109 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:04 smithi081 ceph-mon[113965]: pgmap v43: 65 pgs: 9 stale+active+clean, 56 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:04.109 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:03 smithi081 podman[125332]: 2024-03-22 07:14:03.907576762 +0000 UTC m=+1.785819094 container cleanup 703c47532b9b386d781394e419b3d4a71924183af5703aaac0fa9d5aacff8bee (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-22T07:14:04.109 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:03 smithi081 bash[125332]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0 2024-03-22T07:14:04.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:04 smithi149 ceph-mon[83447]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:14:04.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:04 smithi149 ceph-mon[83447]: failed to encode map e50 with expected crc 2024-03-22T07:14:04.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:04 smithi149 ceph-mon[83447]: osdmap e50: 6 total, 5 up, 6 in 2024-03-22T07:14:04.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:04 smithi149 ceph-mon[83447]: pgmap v43: 65 pgs: 9 stale+active+clean, 56 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:04.518 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:04 smithi081 podman[125351]: 2024-03-22 07:14:04.109006148 +0000 UTC m=+0.521545085 container remove 703c47532b9b386d781394e419b3d4a71924183af5703aaac0fa9d5aacff8bee (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=quincy-9c4e0c3, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-22T07:14:04.779 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:04 smithi081 podman[125443]: 2024-03-22 07:14:04.422227594 +0000 UTC m=+0.018501509 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:14:04.779 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:04 smithi081 podman[125443]: 2024-03-22 07:14:04.560283373 +0000 UTC m=+0.156557333 container create 7beb93bdd2bee13742e3d53e279f2c26e162091e89e8e09aab524b874240cc03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS) 2024-03-22T07:14:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:05 smithi081 ceph-mon[113965]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:05 smithi081 ceph-mon[113965]: osdmap e51: 6 total, 5 up, 6 in 2024-03-22T07:14:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:05 smithi081 ceph-mon[113965]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:05 smithi081 ceph-mon[113965]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:05 smithi081 ceph-mon[113965]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:05 smithi081 ceph-mon[113965]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:05 smithi081 ceph-mon[113965]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.069 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:04 smithi081 podman[125443]: 2024-03-22 07:14:04.779798729 +0000 UTC m=+0.376072652 container init 7beb93bdd2bee13742e3d53e279f2c26e162091e89e8e09aab524b874240cc03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-22T07:14:05.069 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:04 smithi081 podman[125443]: 2024-03-22 07:14:04.783346178 +0000 UTC m=+0.379620103 container start 7beb93bdd2bee13742e3d53e279f2c26e162091e89e8e09aab524b874240cc03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-22T07:14:05.070 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:04 smithi081 podman[125443]: 2024-03-22 07:14:04.88787046 +0000 UTC m=+0.484144391 container attach 7beb93bdd2bee13742e3d53e279f2c26e162091e89e8e09aab524b874240cc03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-22T07:14:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:05 smithi149 ceph-mon[83447]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:05 smithi149 ceph-mon[83447]: osdmap e51: 6 total, 5 up, 6 in 2024-03-22T07:14:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:05 smithi149 ceph-mon[83447]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:05 smithi149 ceph-mon[83447]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:05 smithi149 ceph-mon[83447]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:05 smithi149 ceph-mon[83447]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:05 smithi149 ceph-mon[83447]: failed to encode map e51 with expected crc 2024-03-22T07:14:05.432 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:05 smithi081 podman[125443]: 2024-03-22 07:14:05.074416717 +0000 UTC m=+0.670690639 container died 7beb93bdd2bee13742e3d53e279f2c26e162091e89e8e09aab524b874240cc03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-22T07:14:06.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:06 smithi081 ceph-mon[113965]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:06.069 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:05 smithi081 podman[125469]: 2024-03-22 07:14:05.78386183 +0000 UTC m=+0.704737649 container remove 7beb93bdd2bee13742e3d53e279f2c26e162091e89e8e09aab524b874240cc03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-22T07:14:06.069 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:05 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-22T07:14:06.069 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:05 smithi081 systemd[1]: Stopped Ceph osd.0 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:14:06.069 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:05 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.0.service: Consumed 34.973s CPU time. 2024-03-22T07:14:06.069 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 systemd[1]: Starting Ceph osd.0 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:14:06.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:06 smithi149 ceph-mon[83447]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:06.553 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 podman[125575]: 2024-03-22 07:14:06.142694857 +0000 UTC m=+0.018037809 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:14:06.553 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 podman[125575]: 2024-03-22 07:14:06.243037438 +0000 UTC m=+0.118380432 container create b1e6fa375a9224d2e6fd2d74e1bf45c3bea6e8f4a31125c0105836318be0955f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-22T07:14:06.554 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 podman[125575]: 2024-03-22 07:14:06.462345074 +0000 UTC m=+0.337688030 container init b1e6fa375a9224d2e6fd2d74e1bf45c3bea6e8f4a31125c0105836318be0955f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-03-22T07:14:06.554 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 podman[125575]: 2024-03-22 07:14:06.465423009 +0000 UTC m=+0.340765960 container start b1e6fa375a9224d2e6fd2d74e1bf45c3bea6e8f4a31125c0105836318be0955f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-22T07:14:06.554 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 podman[125575]: 2024-03-22 07:14:06.554003237 +0000 UTC m=+0.429346182 container attach b1e6fa375a9224d2e6fd2d74e1bf45c3bea6e8f4a31125c0105836318be0955f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-22T07:14:07.246 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate[125590]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T07:14:07.246 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate[125590]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T07:14:07.246 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 bash[125575]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T07:14:07.246 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 bash[125575]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T07:14:07.246 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 bash[125575]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T07:14:07.246 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 bash[125575]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T07:14:07.246 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 bash[125575]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-22T07:14:07.247 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 bash[125575]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T07:14:07.247 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 bash[125575]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-22T07:14:07.247 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate[125590]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T07:14:07.247 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate[125590]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T07:14:07.247 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate[125590]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-22T07:14:07.247 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate[125590]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T07:14:07.247 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate[125590]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-22T07:14:07.247 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:06 smithi081 podman[125575]: 2024-03-22 07:14:06.930084343 +0000 UTC m=+0.805427291 container died b1e6fa375a9224d2e6fd2d74e1bf45c3bea6e8f4a31125c0105836318be0955f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-22T07:14:07.886 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:07 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 327.627395 secs 2024-03-22T07:14:07.887 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:07 smithi081 podman[125729]: 2024-03-22 07:14:07.667215155 +0000 UTC m=+0.731921739 container remove b1e6fa375a9224d2e6fd2d74e1bf45c3bea6e8f4a31125c0105836318be0955f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-22T07:14:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.978+0000 7fa879160700 1 -- 172.21.15.81:0/4141354547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 msgr2=0x7fa874102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.978+0000 7fa879160700 1 --2- 172.21.15.81:0/4141354547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 0x7fa874102140 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa864004940 tx=0x7fa864009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.978+0000 7fa879160700 1 -- 172.21.15.81:0/4141354547 shutdown_connections 2024-03-22T07:14:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.978+0000 7fa879160700 1 --2- 172.21.15.81:0/4141354547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa874102710 0x7fa8740fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.978+0000 7fa879160700 1 --2- 172.21.15.81:0/4141354547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 0x7fa874102140 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.978+0000 7fa879160700 1 -- 172.21.15.81:0/4141354547 >> 172.21.15.81:0/4141354547 conn(0x7fa874072bc0 msgr2=0x7fa874072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa879160700 1 -- 172.21.15.81:0/4141354547 shutdown_connections 2024-03-22T07:14:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa879160700 1 -- 172.21.15.81:0/4141354547 wait complete. 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa879160700 1 Processor -- start 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa879160700 1 -- start start 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa879160700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 0x7fa87410ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa879160700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa874102710 0x7fa87410f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa879160700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87410f700 con 0x7fa874101d70 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa879160700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87410f840 con 0x7fa874102710 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa872d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 0x7fa87410ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa872d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 0x7fa87410ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60364/0 (socket says 172.21.15.81:60364) 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.979+0000 7fa872d9d700 1 -- 172.21.15.81:0/1166051225 learned_addr learned my addr 172.21.15.81:0/1166051225 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa872d9d700 1 -- 172.21.15.81:0/1166051225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa874102710 msgr2=0x7fa87410f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa87259c700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa874102710 0x7fa87410f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa872d9d700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa874102710 0x7fa87410f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa872d9d700 1 -- 172.21.15.81:0/1166051225 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864005040 con 0x7fa874101d70 2024-03-22T07:14:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa87259c700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa874102710 0x7fa87410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:14:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa872d9d700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 0x7fa87410ec80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b8b0 tx=0x7fa864009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa85b7fe700 1 -- 172.21.15.81:0/1166051225 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa86400e040 con 0x7fa874101d70 2024-03-22T07:14:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa85b7fe700 1 -- 172.21.15.81:0/1166051225 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa864004290 con 0x7fa874101d70 2024-03-22T07:14:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa874117b30 con 0x7fa874101d70 2024-03-22T07:14:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa85b7fe700 1 -- 172.21.15.81:0/1166051225 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa86401c3f0 con 0x7fa874101d70 2024-03-22T07:14:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.980+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa874118040 con 0x7fa874101d70 2024-03-22T07:14:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.981+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8740ff080 con 0x7fa874101d70 2024-03-22T07:14:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.982+0000 7fa85b7fe700 1 -- 172.21.15.81:0/1166051225 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fa8640137b0 con 0x7fa874101d70 2024-03-22T07:14:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.982+0000 7fa85b7fe700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa85c06d4a0 0x7fa85c06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.982+0000 7fa85b7fe700 1 -- 172.21.15.81:0/1166051225 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa86408dda0 con 0x7fa874101d70 2024-03-22T07:14:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.982+0000 7fa87259c700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa85c06d4a0 0x7fa85c06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.983+0000 7fa87259c700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa85c06d4a0 0x7fa85c06f950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa868007fd0 tx=0x7fa868009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:07.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:07.987+0000 7fa85b7fe700 1 -- 172.21.15.81:0/1166051225 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa864057780 con 0x7fa874101d70 2024-03-22T07:14:08.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:07 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 327.627395 secs 2024-03-22T07:14:08.166 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:07 smithi081 podman[125792]: 2024-03-22 07:14:07.790171095 +0000 UTC m=+0.018445892 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:14:08.166 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:07 smithi081 podman[125792]: 2024-03-22 07:14:07.91819281 +0000 UTC m=+0.146467597 container create 033c080308b6a70f9f5613a0cee5a3a56425efcaff2a19743b2be785bf6ebb4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-22T07:14:08.167 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:08 smithi081 podman[125792]: 2024-03-22 07:14:08.167156224 +0000 UTC m=+0.395431011 container init 033c080308b6a70f9f5613a0cee5a3a56425efcaff2a19743b2be785bf6ebb4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b) 2024-03-22T07:14:08.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.232+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa87404bed0 con 0x7fa85c06d4a0 2024-03-22T07:14:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.233+0000 7fa85b7fe700 1 -- 172.21.15.81:0/1166051225 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa87404bed0 con 0x7fa85c06d4a0 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa85c06d4a0 msgr2=0x7fa85c06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa85c06d4a0 0x7fa85c06f950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa868007fd0 tx=0x7fa868009040 comp rx=0 tx=0).stop 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 msgr2=0x7fa87410ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 0x7fa87410ec80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa86400b8b0 tx=0x7fa864009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 shutdown_connections 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa85c06d4a0 0x7fa85c06f950 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa874101d70 0x7fa87410ec80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 --2- 172.21.15.81:0/1166051225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa874102710 0x7fa87410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 >> 172.21.15.81:0/1166051225 conn(0x7fa874072bc0 msgr2=0x7fa8740fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 shutdown_connections 2024-03-22T07:14:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.235+0000 7fa879160700 1 -- 172.21.15.81:0/1166051225 wait complete. 2024-03-22T07:14:08.245 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:14:08.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.347+0000 7fc8e668c700 1 -- 172.21.15.81:0/1614874366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 msgr2=0x7fc8e00fca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.347+0000 7fc8e668c700 1 --2- 172.21.15.81:0/1614874366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 0x7fc8e00fca50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4004940 tx=0x7fc8d4009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:08.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.347+0000 7fc8e668c700 1 -- 172.21.15.81:0/1614874366 shutdown_connections 2024-03-22T07:14:08.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.347+0000 7fc8e668c700 1 --2- 172.21.15.81:0/1614874366 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 0x7fc8e00fca50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.347 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.347+0000 7fc8e668c700 1 --2- 172.21.15.81:0/1614874366 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e0101f00 0x7fc8e01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.347+0000 7fc8e668c700 1 -- 172.21.15.81:0/1614874366 >> 172.21.15.81:0/1614874366 conn(0x7fc8e00714c0 msgr2=0x7fc8e00718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:08.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.348+0000 7fc8e668c700 1 -- 172.21.15.81:0/1614874366 shutdown_connections 2024-03-22T07:14:08.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.348+0000 7fc8e668c700 1 -- 172.21.15.81:0/1614874366 wait complete. 2024-03-22T07:14:08.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.348+0000 7fc8e668c700 1 Processor -- start 2024-03-22T07:14:08.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.348+0000 7fc8e668c700 1 -- start start 2024-03-22T07:14:08.349 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.348+0000 7fc8e668c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 0x7fc8e0112950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.348+0000 7fc8e668c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e0101f00 0x7fc8e0112e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.348+0000 7fc8e668c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0113570 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.348+0000 7fc8e668c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e010cae0 con 0x7fc8e0101f00 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.348+0000 7fc8dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 0x7fc8e0112950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8dffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 0x7fc8e0112950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60388/0 (socket says 172.21.15.81:60388) 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8dffff700 1 -- 172.21.15.81:0/2822376485 learned_addr learned my addr 172.21.15.81:0/2822376485 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8df7fe700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e0101f00 0x7fc8e0112e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8dffff700 1 -- 172.21.15.81:0/2822376485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e0101f00 msgr2=0x7fc8e0112e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8dffff700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e0101f00 0x7fc8e0112e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8dffff700 1 -- 172.21.15.81:0/2822376485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d4005040 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8df7fe700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e0101f00 0x7fc8e0112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8dffff700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 0x7fc8e0112950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000b790 tx=0x7fc8d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:08.350 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8dcff9700 1 -- 172.21.15.81:0/2822376485 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d000f770 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8dcff9700 1 -- 172.21.15.81:0/2822376485 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8d000fda0 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e010cdc0 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8dcff9700 1 -- 172.21.15.81:0/2822376485 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d000d310 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.349+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e010d300 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.351 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.350+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e0104340 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.351+0000 7fc8dcff9700 1 -- 172.21.15.81:0/2822376485 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fc8d0010420 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.351+0000 7fc8dcff9700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc8c806d4b0 0x7fc8c806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:08.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.351+0000 7fc8df7fe700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc8c806d4b0 0x7fc8c806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:08.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.351+0000 7fc8dcff9700 1 -- 172.21.15.81:0/2822376485 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc8d008c990 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.352 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.352+0000 7fc8df7fe700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc8c806d4b0 0x7fc8c806f960 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4000c00 tx=0x7fc8d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:08.355 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.356+0000 7fc8dcff9700 1 -- 172.21.15.81:0/2822376485 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc8d0059fb0 con 0x7fc8e00fc5e0 2024-03-22T07:14:08.569 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:08 smithi081 podman[125792]: 2024-03-22 07:14:08.170404918 +0000 UTC m=+0.398679713 container start 033c080308b6a70f9f5613a0cee5a3a56425efcaff2a19743b2be785bf6ebb4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-22T07:14:08.569 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:08 smithi081 ceph-osd[125834]: -- 172.21.15.81:0/2468086510 <== mon.0 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559de35451e0 con 0x559de3516c00 2024-03-22T07:14:08.569 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:08 smithi081 bash[125792]: 033c080308b6a70f9f5613a0cee5a3a56425efcaff2a19743b2be785bf6ebb4e 2024-03-22T07:14:08.569 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:08 smithi081 systemd[1]: Started Ceph osd.0 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:14:08.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.589+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8e010dfa0 con 0x7fc8c806d4b0 2024-03-22T07:14:08.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.590+0000 7fc8dcff9700 1 -- 172.21.15.81:0/2822376485 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc8e010dfa0 con 0x7fc8c806d4b0 2024-03-22T07:14:08.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc8c806d4b0 msgr2=0x7fc8c806f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc8c806d4b0 0x7fc8c806f960 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4000c00 tx=0x7fc8d4009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:08.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 msgr2=0x7fc8e0112950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 0x7fc8e0112950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000b790 tx=0x7fc8d0009250 comp rx=0 tx=0).stop 2024-03-22T07:14:08.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 shutdown_connections 2024-03-22T07:14:08.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc8c806d4b0 0x7fc8c806f960 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8e00fc5e0 0x7fc8e0112950 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 --2- 172.21.15.81:0/2822376485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8e0101f00 0x7fc8e0112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 >> 172.21.15.81:0/2822376485 conn(0x7fc8e00714c0 msgr2=0x7fc8e00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:08.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 shutdown_connections 2024-03-22T07:14:08.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.594+0000 7fc8e668c700 1 -- 172.21.15.81:0/2822376485 wait complete. 2024-03-22T07:14:08.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.700+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/3063011493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 msgr2=0x7fdbd41020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.700+0000 7fdbd8ffb700 1 --2- 172.21.15.81:0/3063011493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 0x7fdbd41020f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4004940 tx=0x7fdbc4009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:08.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.700+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/3063011493 shutdown_connections 2024-03-22T07:14:08.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 --2- 172.21.15.81:0/3063011493 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd41026c0 0x7fdbd40fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 --2- 172.21.15.81:0/3063011493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 0x7fdbd41020f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/3063011493 >> 172.21.15.81:0/3063011493 conn(0x7fdbd40f78d0 msgr2=0x7fdbd40f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:08.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/3063011493 shutdown_connections 2024-03-22T07:14:08.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/3063011493 wait complete. 2024-03-22T07:14:08.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 Processor -- start 2024-03-22T07:14:08.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 -- start start 2024-03-22T07:14:08.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 0x7fdbd4112ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd41026c0 0x7fdbd4113020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd4113670 con 0x7fdbd4101d20 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.701+0000 7fdbd8ffb700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd410ca60 con 0x7fdbd41026c0 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 0x7fdbd4112ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 0x7fdbd4112ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60396/0 (socket says 172.21.15.81:60396) 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd259c700 1 -- 172.21.15.81:0/1120391258 learned_addr learned my addr 172.21.15.81:0/1120391258 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd1d9b700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd41026c0 0x7fdbd4113020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd259c700 1 -- 172.21.15.81:0/1120391258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd41026c0 msgr2=0x7fdbd4113020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd259c700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd41026c0 0x7fdbd4113020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd259c700 1 -- 172.21.15.81:0/1120391258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc4005040 con 0x7fdbd4101d20 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd259c700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 0x7fdbd4112ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400b3b0 tx=0x7fdbc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbc2ffd700 1 -- 172.21.15.81:0/1120391258 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbc400e040 con 0x7fdbd4101d20 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbc2ffd700 1 -- 172.21.15.81:0/1120391258 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbc400bdd0 con 0x7fdbd4101d20 2024-03-22T07:14:08.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd410cce0 con 0x7fdbd4101d20 2024-03-22T07:14:08.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbc2ffd700 1 -- 172.21.15.81:0/1120391258 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbc4013660 con 0x7fdbd4101d20 2024-03-22T07:14:08.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.702+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd410d240 con 0x7fdbd4101d20 2024-03-22T07:14:08.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.703+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd40fd470 con 0x7fdbd4101d20 2024-03-22T07:14:08.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.704+0000 7fdbc2ffd700 1 -- 172.21.15.81:0/1120391258 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fdbc401e420 con 0x7fdbd4101d20 2024-03-22T07:14:08.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.704+0000 7fdbc2ffd700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdbbc06d4a0 0x7fdbbc06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:08.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.704+0000 7fdbc2ffd700 1 -- 172.21.15.81:0/1120391258 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdbc4090930 con 0x7fdbd4101d20 2024-03-22T07:14:08.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.705+0000 7fdbd1d9b700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdbbc06d4a0 0x7fdbbc06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:08.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.705+0000 7fdbd1d9b700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdbbc06d4a0 0x7fdbbc06f950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbd410dc20 tx=0x7fdbc8009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:08.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.709+0000 7fdbc2ffd700 1 -- 172.21.15.81:0/1120391258 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdbc405dd20 con 0x7fdbd4101d20 2024-03-22T07:14:08.846 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:08 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[125830]: 2024-03-22T07:14:08.712+0000 7f3a9a8557c0 -1 Falling back to public interface 2024-03-22T07:14:08.846 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:08 smithi081 ceph-mon[113965]: pgmap v46: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:08.846 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:08 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 327.967689 secs 2024-03-22T07:14:08.846 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:08 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:08.846 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:08 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:08.846 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:08 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:08.935 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.936+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbd4108e20 con 0x7fdbbc06d4a0 2024-03-22T07:14:08.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.947+0000 7fdbc2ffd700 1 -- 172.21.15.81:0/1120391258 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdbd4108e20 con 0x7fdbbc06d4a0 2024-03-22T07:14:08.947 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:14:08.947 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (2m) 16s ago 13m 26.0M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:14:08.947 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (27s) 16s ago 13m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:14:08.947 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (22s) 18s ago 12m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:14:08.947 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (2m) 16s ago 12m 80.4M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:14:08.948 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (7m) 16s ago 7m 13.3M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:14:08.948 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (7m) 16s ago 7m 131M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:14:08.948 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (7m) 18s ago 7m 16.8M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:14:08.948 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (7m) 18s ago 7m 147M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:14:08.948 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (4m) 16s ago 14m 496M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:14:08.948 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (3m) 18s ago 12m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:14:08.949 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (91s) 16s ago 14m 39.9M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:14:08.949 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (67s) 18s ago 12m 31.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:14:08.949 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (3m) 16s ago 13m 16.6M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:14:08.949 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (3m) 18s ago 12m 16.7M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:14:08.949 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 starting - - - 1557M 2024-03-22T07:14:08.949 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (10m) 16s ago 10m 200M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:14:08.950 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (10m) 16s ago 10m 163M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:14:08.950 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (10m) 18s ago 10m 215M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:14:08.950 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (9m) 18s ago 9m 194M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:14:08.950 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (9m) 18s ago 9m 184M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:14:08.950 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (3m) 16s ago 12m 59.1M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:14:08.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdbbc06d4a0 msgr2=0x7fdbbc06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdbbc06d4a0 0x7fdbbc06f950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbd410dc20 tx=0x7fdbc8009ee0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 msgr2=0x7fdbd4112ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 0x7fdbd4112ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400b3b0 tx=0x7fdbc4009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:08.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 shutdown_connections 2024-03-22T07:14:08.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdbbc06d4a0 0x7fdbbc06f950 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdbd4101d20 0x7fdbd4112ae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 --2- 172.21.15.81:0/1120391258 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdbd41026c0 0x7fdbd4113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 >> 172.21.15.81:0/1120391258 conn(0x7fdbd40f78d0 msgr2=0x7fdbd40f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:08.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 shutdown_connections 2024-03-22T07:14:08.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:08.949+0000 7fdbd8ffb700 1 -- 172.21.15.81:0/1120391258 wait complete. 2024-03-22T07:14:09.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.057+0000 7ff128bb2700 1 -- 172.21.15.81:0/1264948242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 msgr2=0x7ff124102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.057+0000 7ff128bb2700 1 --2- 172.21.15.81:0/1264948242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 0x7ff124102100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff114004940 tx=0x7ff114009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:09.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.057+0000 7ff128bb2700 1 -- 172.21.15.81:0/1264948242 shutdown_connections 2024-03-22T07:14:09.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.057+0000 7ff128bb2700 1 --2- 172.21.15.81:0/1264948242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1241026d0 0x7ff1240fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.057+0000 7ff128bb2700 1 --2- 172.21.15.81:0/1264948242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 0x7ff124102100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.057+0000 7ff128bb2700 1 -- 172.21.15.81:0/1264948242 >> 172.21.15.81:0/1264948242 conn(0x7ff1240f7860 msgr2=0x7ff1240f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.057+0000 7ff128bb2700 1 -- 172.21.15.81:0/1264948242 shutdown_connections 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.057+0000 7ff128bb2700 1 -- 172.21.15.81:0/1264948242 wait complete. 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff128bb2700 1 Processor -- start 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff128bb2700 1 -- start start 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff128bb2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 0x7ff1241128f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff128bb2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1241026d0 0x7ff124112e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff128bb2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff124113510 con 0x7ff124101d30 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff128bb2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12410ca80 con 0x7ff1241026d0 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff12259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 0x7ff1241128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff12259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 0x7ff1241128f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60410/0 (socket says 172.21.15.81:60410) 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff12259c700 1 -- 172.21.15.81:0/811698035 learned_addr learned my addr 172.21.15.81:0/811698035 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:09.061 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.058+0000 7ff121d9b700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1241026d0 0x7ff124112e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.059+0000 7ff12259c700 1 -- 172.21.15.81:0/811698035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1241026d0 msgr2=0x7ff124112e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.059+0000 7ff12259c700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1241026d0 0x7ff124112e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.059+0000 7ff12259c700 1 -- 172.21.15.81:0/811698035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff114005040 con 0x7ff124101d30 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.059+0000 7ff12259c700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 0x7ff1241128f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff11400b3b0 tx=0x7ff114009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.059+0000 7ff112ffd700 1 -- 172.21.15.81:0/811698035 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11400e040 con 0x7ff124101d30 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.059+0000 7ff112ffd700 1 -- 172.21.15.81:0/811698035 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff11400bbc0 con 0x7ff124101d30 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.059+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12410cd00 con 0x7ff124101d30 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.059+0000 7ff112ffd700 1 -- 172.21.15.81:0/811698035 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11401c3f0 con 0x7ff124101d30 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.059+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12410d2c0 con 0x7ff124101d30 2024-03-22T07:14:09.062 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.060+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1240fd480 con 0x7ff124101d30 2024-03-22T07:14:09.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.061+0000 7ff112ffd700 1 -- 172.21.15.81:0/811698035 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7ff11400bd30 con 0x7ff124101d30 2024-03-22T07:14:09.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.061+0000 7ff112ffd700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff10c06d4b0 0x7ff10c06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:09.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.061+0000 7ff121d9b700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff10c06d4b0 0x7ff10c06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:09.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.061+0000 7ff112ffd700 1 -- 172.21.15.81:0/811698035 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff11408dce0 con 0x7ff124101d30 2024-03-22T07:14:09.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.061+0000 7ff121d9b700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff10c06d4b0 0x7ff10c06f960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff118006fd0 tx=0x7ff118008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:09.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.066+0000 7ff112ffd700 1 -- 172.21.15.81:0/811698035 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff11405b150 con 0x7ff124101d30 2024-03-22T07:14:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:08 smithi149 ceph-mon[83447]: pgmap v46: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:08 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 327.967689 secs 2024-03-22T07:14:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:08 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:08 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:08 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:09.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.353+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff12410dfb0 con 0x7ff124101d30 2024-03-22T07:14:09.353 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.353+0000 7ff112ffd700 1 -- 172.21.15.81:0/811698035 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7ff12410dfb0 con 0x7ff124101d30 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:14:09.354 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:14:09.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:14:09.355 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:14:09.355 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 9, 2024-03-22T07:14:09.355 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:14:09.355 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:14:09.355 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.355+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff10c06d4b0 msgr2=0x7ff10c06f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.355+0000 7ff128bb2700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff10c06d4b0 0x7ff10c06f960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff118006fd0 tx=0x7ff118008040 comp rx=0 tx=0).stop 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.355+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 msgr2=0x7ff1241128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.355+0000 7ff128bb2700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 0x7ff1241128f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff11400b3b0 tx=0x7ff114009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.355+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 shutdown_connections 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.355+0000 7ff128bb2700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff10c06d4b0 0x7ff10c06f960 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.355+0000 7ff128bb2700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff124101d30 0x7ff1241128f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.355+0000 7ff128bb2700 1 --2- 172.21.15.81:0/811698035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1241026d0 0x7ff124112e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.355+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 >> 172.21.15.81:0/811698035 conn(0x7ff1240f7860 msgr2=0x7ff1240f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.356+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 shutdown_connections 2024-03-22T07:14:09.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.356+0000 7ff128bb2700 1 -- 172.21.15.81:0/811698035 wait complete. 2024-03-22T07:14:09.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 -- 172.21.15.81:0/440655769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80101d70 msgr2=0x7f5d80102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 --2- 172.21.15.81:0/440655769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80101d70 0x7f5d80102140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5d70004940 tx=0x7f5d70009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:09.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 -- 172.21.15.81:0/440655769 shutdown_connections 2024-03-22T07:14:09.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 --2- 172.21.15.81:0/440655769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d80102710 0x7f5d800fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 --2- 172.21.15.81:0/440655769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80101d70 0x7f5d80102140 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 -- 172.21.15.81:0/440655769 >> 172.21.15.81:0/440655769 conn(0x7f5d80072bc0 msgr2=0x7f5d80072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:09.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 -- 172.21.15.81:0/440655769 shutdown_connections 2024-03-22T07:14:09.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 -- 172.21.15.81:0/440655769 wait complete. 2024-03-22T07:14:09.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 Processor -- start 2024-03-22T07:14:09.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.463+0000 7f5d85c24700 1 -- start start 2024-03-22T07:14:09.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d85c24700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d80101d70 0x7f5d80112a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:09.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d85c24700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80102710 0x7f5d80112fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:09.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d85c24700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801135f0 con 0x7f5d80102710 2024-03-22T07:14:09.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d85c24700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8010c9e0 con 0x7f5d80101d70 2024-03-22T07:14:09.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d77fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80102710 0x7f5d80112fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:09.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d77fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80102710 0x7f5d80112fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60424/0 (socket says 172.21.15.81:60424) 2024-03-22T07:14:09.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d77fff700 1 -- 172.21.15.81:0/3246774590 learned_addr learned my addr 172.21.15.81:0/3246774590 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:09.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d7f7fe700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d80101d70 0x7f5d80112a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:09.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d77fff700 1 -- 172.21.15.81:0/3246774590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d80101d70 msgr2=0x7f5d80112a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d77fff700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d80101d70 0x7f5d80112a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d77fff700 1 -- 172.21.15.81:0/3246774590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70005040 con 0x7f5d80102710 2024-03-22T07:14:09.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d7f7fe700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d80101d70 0x7f5d80112a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:14:09.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.464+0000 7f5d77fff700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80102710 0x7f5d80112fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d68002a80 tx=0x7f5d68004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:09.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.465+0000 7f5d7cff9700 1 -- 172.21.15.81:0/3246774590 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d68007cc0 con 0x7f5d80102710 2024-03-22T07:14:09.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.465+0000 7f5d7cff9700 1 -- 172.21.15.81:0/3246774590 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d6800f040 con 0x7f5d80102710 2024-03-22T07:14:09.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.465+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8010ccc0 con 0x7f5d80102710 2024-03-22T07:14:09.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.465+0000 7f5d7cff9700 1 -- 172.21.15.81:0/3246774590 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d6800daf0 con 0x7f5d80102710 2024-03-22T07:14:09.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.465+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8010d200 con 0x7f5d80102710 2024-03-22T07:14:09.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.466+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d800fd4c0 con 0x7f5d80102710 2024-03-22T07:14:09.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.466+0000 7f5d7cff9700 1 -- 172.21.15.81:0/3246774590 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f5d6800dc90 con 0x7f5d80102710 2024-03-22T07:14:09.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.467+0000 7f5d7cff9700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5d6c06d4a0 0x7f5d6c06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:09.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.467+0000 7f5d7cff9700 1 -- 172.21.15.81:0/3246774590 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5d6808c190 con 0x7f5d80102710 2024-03-22T07:14:09.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.467+0000 7f5d7f7fe700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5d6c06d4a0 0x7f5d6c06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:09.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.467+0000 7f5d7f7fe700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5d6c06d4a0 0x7f5d6c06f950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5d70004560 tx=0x7f5d70009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:09.474 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.471+0000 7f5d7cff9700 1 -- 172.21.15.81:0/3246774590 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d68059600 con 0x7f5d80102710 2024-03-22T07:14:09.717 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.714+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5d8005f5f0 con 0x7f5d80102710 2024-03-22T07:14:09.718 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:09 smithi081 ceph-mon[113965]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:09.718 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:09 smithi081 ceph-mon[113965]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:09.718 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:09 smithi081 ceph-mon[113965]: Health check failed: Degraded data redundancy: 4872/32856 objects degraded (14.828%), 33 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:09.718 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:09 smithi081 ceph-mon[113965]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:09.718 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:09 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/811698035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:09.718 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.715+0000 7f5d7cff9700 1 -- 172.21.15.81:0/3246774590 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f5d68059190 con 0x7f5d80102710 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:e22 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 22 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:12:38.916315+0000 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:14:09.719 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:14:09.720 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 95 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 94 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:14:09.721 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:14:09.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5d6c06d4a0 msgr2=0x7f5d6c06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5d6c06d4a0 0x7f5d6c06f950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5d70004560 tx=0x7f5d70009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:09.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80102710 msgr2=0x7f5d80112fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80102710 0x7f5d80112fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d68002a80 tx=0x7f5d68004630 comp rx=0 tx=0).stop 2024-03-22T07:14:09.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 shutdown_connections 2024-03-22T07:14:09.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5d6c06d4a0 0x7f5d6c06f950 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d80101d70 0x7f5d80112a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 --2- 172.21.15.81:0/3246774590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5d80102710 0x7f5d80112fa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 >> 172.21.15.81:0/3246774590 conn(0x7f5d80072bc0 msgr2=0x7f5d800fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:09.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 shutdown_connections 2024-03-22T07:14:09.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.717+0000 7f5d85c24700 1 -- 172.21.15.81:0/3246774590 wait complete. 2024-03-22T07:14:09.723 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 22 2024-03-22T07:14:09.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.830+0000 7f9430717700 1 -- 172.21.15.81:0/129767273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 msgr2=0x7f9428106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.830+0000 7f9430717700 1 --2- 172.21.15.81:0/129767273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 0x7f9428106e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9424004940 tx=0x7f9424009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:09.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.830+0000 7f9430717700 1 -- 172.21.15.81:0/129767273 shutdown_connections 2024-03-22T07:14:09.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.830+0000 7f9430717700 1 --2- 172.21.15.81:0/129767273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 0x7f9428106e60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.830+0000 7f9430717700 1 --2- 172.21.15.81:0/129767273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94280fde80 0x7f94280fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.830+0000 7f9430717700 1 -- 172.21.15.81:0/129767273 >> 172.21.15.81:0/129767273 conn(0x7f94280f78d0 msgr2=0x7f94280f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:09.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.830+0000 7f9430717700 1 -- 172.21.15.81:0/129767273 shutdown_connections 2024-03-22T07:14:09.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f9430717700 1 -- 172.21.15.81:0/129767273 wait complete. 2024-03-22T07:14:09.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f9430717700 1 Processor -- start 2024-03-22T07:14:09.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f9430717700 1 -- start start 2024-03-22T07:14:09.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f9430717700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94280fde80 0x7f9428113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:09.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f9430717700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 0x7f9428113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:09.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f9430717700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9428114640 con 0x7f94280fe820 2024-03-22T07:14:09.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f9430717700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942810da30 con 0x7f94280fde80 2024-03-22T07:14:09.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f942dcb2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 0x7f9428113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:09.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f942dcb2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 0x7f9428113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60440/0 (socket says 172.21.15.81:60440) 2024-03-22T07:14:09.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.831+0000 7f942dcb2700 1 -- 172.21.15.81:0/2845604678 learned_addr learned my addr 172.21.15.81:0/2845604678 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:09.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f942e4b3700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94280fde80 0x7f9428113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:09.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f942dcb2700 1 -- 172.21.15.81:0/2845604678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94280fde80 msgr2=0x7f9428113ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:09.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f942dcb2700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94280fde80 0x7f9428113ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:09.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f942dcb2700 1 -- 172.21.15.81:0/2845604678 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424005040 con 0x7f94280fe820 2024-03-22T07:14:09.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f942dcb2700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 0x7f9428113ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9424009180 tx=0x7f9424009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:09.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f941effd700 1 -- 172.21.15.81:0/2845604678 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f942400e040 con 0x7f94280fe820 2024-03-22T07:14:09.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f941effd700 1 -- 172.21.15.81:0/2845604678 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9424013420 con 0x7f94280fe820 2024-03-22T07:14:09.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942810dd10 con 0x7f94280fe820 2024-03-22T07:14:09.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f941effd700 1 -- 172.21.15.81:0/2845604678 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f942401c630 con 0x7f94280fe820 2024-03-22T07:14:09.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.832+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942810e250 con 0x7f94280fe820 2024-03-22T07:14:09.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.833+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9428066710 con 0x7f94280fe820 2024-03-22T07:14:09.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.834+0000 7f941effd700 1 -- 172.21.15.81:0/2845604678 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f9424013590 con 0x7f94280fe820 2024-03-22T07:14:09.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.834+0000 7f941effd700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f941406d4a0 0x7f941406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:09.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.834+0000 7f941effd700 1 -- 172.21.15.81:0/2845604678 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f942408dee0 con 0x7f94280fe820 2024-03-22T07:14:09.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.834+0000 7f942e4b3700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f941406d4a0 0x7f941406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:09.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.835+0000 7f942e4b3700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f941406d4a0 0x7f941406f950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9418009fd0 tx=0x7f9418009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:09.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:09.840+0000 7f941effd700 1 -- 172.21.15.81:0/2845604678 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f942405b350 con 0x7f94280fe820 2024-03-22T07:14:10.067 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.067+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9428113780 con 0x7f941406d4a0 2024-03-22T07:14:10.068 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.068+0000 7f941effd700 1 -- 172.21.15.81:0/2845604678 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9428113780 con 0x7f941406d4a0 2024-03-22T07:14:10.068 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:14:10.069 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:14:10.069 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:14:10.069 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:14:10.069 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:14:10.069 INFO:teuthology.orchestra.run.smithi081.stdout: "mon", 2024-03-22T07:14:10.069 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr", 2024-03-22T07:14:10.070 INFO:teuthology.orchestra.run.smithi081.stdout: "crash" 2024-03-22T07:14:10.070 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:14:10.070 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "6/21 daemons upgraded", 2024-03-22T07:14:10.070 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:14:10.070 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:14:10.071 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:14:10.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f941406d4a0 msgr2=0x7f941406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:10.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f941406d4a0 0x7f941406f950 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9418009fd0 tx=0x7f9418009f60 comp rx=0 tx=0).stop 2024-03-22T07:14:10.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 msgr2=0x7f9428113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:10.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 0x7f9428113ff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9424009180 tx=0x7f9424009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:10.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 shutdown_connections 2024-03-22T07:14:10.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f941406d4a0 0x7f941406f950 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:10.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94280fde80 0x7f9428113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:10.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 --2- 172.21.15.81:0/2845604678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94280fe820 0x7f9428113ff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:10.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 >> 172.21.15.81:0/2845604678 conn(0x7f94280f78d0 msgr2=0x7f94280f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:10.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 shutdown_connections 2024-03-22T07:14:10.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.070+0000 7f9430717700 1 -- 172.21.15.81:0/2845604678 wait complete. 2024-03-22T07:14:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:09 smithi149 ceph-mon[83447]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:09 smithi149 ceph-mon[83447]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:09 smithi149 ceph-mon[83447]: Health check failed: Degraded data redundancy: 4872/32856 objects degraded (14.828%), 33 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:09 smithi149 ceph-mon[83447]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:10.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:09 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/811698035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:10.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.176+0000 7f8615733700 1 -- 172.21.15.81:0/446380724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 msgr2=0x7f86100fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:10.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.176+0000 7f8615733700 1 --2- 172.21.15.81:0/446380724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 0x7f86100fbd80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8604004940 tx=0x7f8604009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:10.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.176+0000 7f8615733700 1 -- 172.21.15.81:0/446380724 shutdown_connections 2024-03-22T07:14:10.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.176+0000 7f8615733700 1 --2- 172.21.15.81:0/446380724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 0x7f86100fbd80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:10.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.176+0000 7f8615733700 1 --2- 172.21.15.81:0/446380724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8610101d70 0x7f8610102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:10.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.176+0000 7f8615733700 1 -- 172.21.15.81:0/446380724 >> 172.21.15.81:0/446380724 conn(0x7f8610072bc0 msgr2=0x7f8610072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:10.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.176+0000 7f8615733700 1 -- 172.21.15.81:0/446380724 shutdown_connections 2024-03-22T07:14:10.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.176+0000 7f8615733700 1 -- 172.21.15.81:0/446380724 wait complete. 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f8615733700 1 Processor -- start 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f8615733700 1 -- start start 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f8615733700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8610101d70 0x7f8610113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f8615733700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 0x7f86101140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f8615733700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8610114730 con 0x7f8610102710 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f8615733700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861010db20 con 0x7f8610101d70 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f860e7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 0x7f86101140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f860e7fc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 0x7f86101140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60450/0 (socket says 172.21.15.81:60450) 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f860e7fc700 1 -- 172.21.15.81:0/576516804 learned_addr learned my addr 172.21.15.81:0/576516804 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:10.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f860effd700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8610101d70 0x7f8610113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f860e7fc700 1 -- 172.21.15.81:0/576516804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8610101d70 msgr2=0x7f8610113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f860e7fc700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8610101d70 0x7f8610113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.177+0000 7f860e7fc700 1 -- 172.21.15.81:0/576516804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604005040 con 0x7f8610102710 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.178+0000 7f860e7fc700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 0x7f86101140e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8604009100 tx=0x7f8604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.178+0000 7f85f77fe700 1 -- 172.21.15.81:0/576516804 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f860400e040 con 0x7f8610102710 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.178+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861010dda0 con 0x7f8610102710 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.178+0000 7f85f77fe700 1 -- 172.21.15.81:0/576516804 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8604013420 con 0x7f8610102710 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.178+0000 7f85f77fe700 1 -- 172.21.15.81:0/576516804 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f860401c610 con 0x7f8610102710 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.178+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861010e360 con 0x7f8610102710 2024-03-22T07:14:10.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.179+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86100fd4c0 con 0x7f8610102710 2024-03-22T07:14:10.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.179+0000 7f85f77fe700 1 -- 172.21.15.81:0/576516804 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 38) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f8604013590 con 0x7f8610102710 2024-03-22T07:14:10.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.180+0000 7f85f77fe700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f85f806d4a0 0x7f85f806f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:10.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.180+0000 7f85f77fe700 1 -- 172.21.15.81:0/576516804 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f860408ddc0 con 0x7f8610102710 2024-03-22T07:14:10.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.180+0000 7f860effd700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f85f806d4a0 0x7f85f806f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:10.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.180+0000 7f860effd700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f85f806d4a0 0x7f85f806f950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8600009fd0 tx=0x7f8600009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:10.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.184+0000 7f85f77fe700 1 -- 172.21.15.81:0/576516804 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f860405b230 con 0x7f8610102710 2024-03-22T07:14:10.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.465+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8610108e50 con 0x7f8610102710 2024-03-22T07:14:10.466 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.466+0000 7f85f77fe700 1 -- 172.21.15.81:0/576516804 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+2353 (secure 0 0 0) 0x7f860405adc0 con 0x7f8610102710 2024-03-22T07:14:10.466 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_WARN 2 MDSs report slow requests; 1 osds down; Degraded data redundancy: 4872/32856 objects degraded (14.828%), 33 pgs degraded 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: osd.0 (root=default,host=smithi081) is down 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 4872/32856 objects degraded (14.828%), 33 pgs degraded 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-03-22T07:14:10.467 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.1 is active+undersized+degraded, acting [4,2] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.3 is active+undersized+degraded, acting [4,3] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.6 is active+undersized+degraded, acting [1,4] 2024-03-22T07:14:10.468 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.b is active+undersized+degraded, acting [1,4] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.c is active+undersized+degraded, acting [5,3] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.f is active+undersized+degraded, acting [5,3] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.10 is active+undersized+degraded, acting [5,1] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.11 is active+undersized+degraded, acting [3,4] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.12 is active+undersized+degraded, acting [3,1] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.15 is active+undersized+degraded, acting [3,4] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.17 is active+undersized+degraded, acting [5,2] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.18 is active+undersized+degraded, acting [2,1] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.1b is active+undersized+degraded, acting [4,3] 2024-03-22T07:14:10.469 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.1f is active+undersized+degraded, acting [3,2] 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f85f806d4a0 msgr2=0x7f85f806f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f85f806d4a0 0x7f85f806f950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8600009fd0 tx=0x7f8600009e50 comp rx=0 tx=0).stop 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 msgr2=0x7f86101140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 0x7f86101140e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8604009100 tx=0x7f8604009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 shutdown_connections 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f85f806d4a0 0x7f85f806f950 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8610101d70 0x7f8610113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 --2- 172.21.15.81:0/576516804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8610102710 0x7f86101140e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 >> 172.21.15.81:0/576516804 conn(0x7f8610072bc0 msgr2=0x7f8610100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 shutdown_connections 2024-03-22T07:14:10.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:10.468+0000 7f8615733700 1 -- 172.21.15.81:0/576516804 wait complete. 2024-03-22T07:14:10.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:10 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3246774590' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:14:10.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:10 smithi081 ceph-mon[113965]: pgmap v47: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:10.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:10 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/576516804' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:14:11.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:10 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3246774590' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:14:11.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:10 smithi149 ceph-mon[83447]: pgmap v47: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:11.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:10 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/576516804' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:14:11.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:11 smithi081 ceph-mon[113965]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:12.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:11 smithi149 ceph-mon[83447]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:12.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:12 smithi081 ceph-mon[113965]: pgmap v48: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:12 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 332.627464 secs 2024-03-22T07:14:12.819 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:12 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[125830]: 2024-03-22T07:14:12.579+0000 7f3a9a8557c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-22T07:14:13.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:12 smithi149 ceph-mon[83447]: pgmap v48: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:13.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:12 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 332.627464 secs 2024-03-22T07:14:13.318 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:12 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[125830]: 2024-03-22T07:14:12.979+0000 7f3a9a8557c0 -1 osd.0 49 log_to_monitors true 2024-03-22T07:14:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:13 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 332.967804 secs 2024-03-22T07:14:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:13 smithi081 ceph-mon[113965]: from='osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T07:14:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:13 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:14.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:13 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:14.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:13 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 332.967804 secs 2024-03-22T07:14:14.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:13 smithi149 ceph-mon[83447]: from='osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T07:14:14.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:13 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:14.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:13 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:14.819 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:14:14 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[125830]: 2024-03-22T07:14:14.514+0000 7f3a87ca5700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:14:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:14 smithi081 ceph-mon[113965]: from='osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T07:14:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:14 smithi081 ceph-mon[113965]: failed to encode map e52 with expected crc 2024-03-22T07:14:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:14 smithi081 ceph-mon[113965]: osdmap e52: 6 total, 5 up, 6 in 2024-03-22T07:14:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:14 smithi081 ceph-mon[113965]: from='osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:14:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:14 smithi081 ceph-mon[113965]: pgmap v50: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:14 smithi081 ceph-mon[113965]: failed to encode map e52 with expected crc 2024-03-22T07:14:14.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:14 smithi081 ceph-mon[113965]: from='osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858]' entity='osd.0' 2024-03-22T07:14:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:14 smithi149 ceph-mon[83447]: from='osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T07:14:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:14 smithi149 ceph-mon[83447]: failed to encode map e52 with expected crc 2024-03-22T07:14:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:14 smithi149 ceph-mon[83447]: osdmap e52: 6 total, 5 up, 6 in 2024-03-22T07:14:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:14 smithi149 ceph-mon[83447]: from='osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:14:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:14 smithi149 ceph-mon[83447]: pgmap v50: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:14 smithi149 ceph-mon[83447]: failed to encode map e52 with expected crc 2024-03-22T07:14:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:14 smithi149 ceph-mon[83447]: from='osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858]' entity='osd.0' 2024-03-22T07:14:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:15 smithi081 ceph-mon[113965]: failed to encode map e52 with expected crc 2024-03-22T07:14:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:15 smithi081 ceph-mon[113965]: failed to encode map e52 with expected crc 2024-03-22T07:14:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:15 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:15 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:15 smithi081 ceph-mon[113965]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:14:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:15 smithi081 ceph-mon[113965]: osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858] boot 2024-03-22T07:14:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:15 smithi081 ceph-mon[113965]: osdmap e53: 6 total, 6 up, 6 in 2024-03-22T07:14:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:15 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:14:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:15 smithi149 ceph-mon[83447]: failed to encode map e52 with expected crc 2024-03-22T07:14:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:15 smithi149 ceph-mon[83447]: failed to encode map e52 with expected crc 2024-03-22T07:14:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:15 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:15 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:16.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:15 smithi149 ceph-mon[83447]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:14:16.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:15 smithi149 ceph-mon[83447]: osd.0 [v2:172.21.15.81:6802/2753302858,v1:172.21.15.81:6803/2753302858] boot 2024-03-22T07:14:16.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:15 smithi149 ceph-mon[83447]: osdmap e53: 6 total, 6 up, 6 in 2024-03-22T07:14:16.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:15 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:14:17.065 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:16 smithi081 ceph-mon[113965]: failed to encode map e53 with expected crc 2024-03-22T07:14:17.066 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:16 smithi081 ceph-mon[113965]: pgmap v52: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:17.066 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:17.066 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:17.066 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:16 smithi081 ceph-mon[113965]: osdmap e54: 6 total, 6 up, 6 in 2024-03-22T07:14:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:16 smithi149 ceph-mon[83447]: failed to encode map e53 with expected crc 2024-03-22T07:14:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:16 smithi149 ceph-mon[83447]: pgmap v52: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 4872/32856 objects degraded (14.828%) 2024-03-22T07:14:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:16 smithi149 ceph-mon[83447]: osdmap e54: 6 total, 6 up, 6 in 2024-03-22T07:14:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:17 smithi081 ceph-mon[113965]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:17 smithi081 ceph-mon[113965]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:17 smithi081 ceph-mon[113965]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:17 smithi081 ceph-mon[113965]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:17 smithi081 ceph-mon[113965]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:17 smithi081 ceph-mon[113965]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:17 smithi081 ceph-mon[113965]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:17 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 337.627528 secs 2024-03-22T07:14:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:17 smithi149 ceph-mon[83447]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:17 smithi149 ceph-mon[83447]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:17 smithi149 ceph-mon[83447]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:17 smithi149 ceph-mon[83447]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:17 smithi149 ceph-mon[83447]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:17 smithi149 ceph-mon[83447]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:17 smithi149 ceph-mon[83447]: failed to encode map e54 with expected crc 2024-03-22T07:14:18.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:17 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 337.627528 secs 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: pgmap v54: 65 pgs: 9 active+undersized+degraded, 56 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 853 B/s wr, 0 op/s; 2049/32856 objects degraded (6.236%) 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 337.967944 secs 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: Health check update: Degraded data redundancy: 2049/32856 objects degraded (6.236%), 9 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:19.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:19.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: pgmap v54: 65 pgs: 9 active+undersized+degraded, 56 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 853 B/s wr, 0 op/s; 2049/32856 objects degraded (6.236%) 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 337.967944 secs 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: Health check update: Degraded data redundancy: 2049/32856 objects degraded (6.236%), 9 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:19.255 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:14:20.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:19 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:14:20.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:19 smithi081 ceph-mon[113965]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-03-22T07:14:20.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:19 smithi081 ceph-mon[113965]: mgrmap e39: smithi081.bcxarw(active, since 93s), standbys: smithi149.sgmtoa 2024-03-22T07:14:20.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:19 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:14:20.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:19 smithi149 ceph-mon[83447]: Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-03-22T07:14:20.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:19 smithi149 ceph-mon[83447]: mgrmap e39: smithi081.bcxarw(active, since 93s), standbys: smithi149.sgmtoa 2024-03-22T07:14:21.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:20 smithi081 ceph-mon[113965]: pgmap v55: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 827 B/s wr, 0 op/s 2024-03-22T07:14:21.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:20 smithi081 ceph-mon[113965]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2049/32856 objects degraded (6.236%), 9 pgs degraded) 2024-03-22T07:14:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:20 smithi149 ceph-mon[83447]: pgmap v55: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 827 B/s wr, 0 op/s 2024-03-22T07:14:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:20 smithi149 ceph-mon[83447]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2049/32856 objects degraded (6.236%), 9 pgs degraded) 2024-03-22T07:14:23.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:22 smithi081 ceph-mon[113965]: pgmap v56: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 639 B/s wr, 0 op/s 2024-03-22T07:14:23.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:22 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 342.627657 secs 2024-03-22T07:14:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:22 smithi149 ceph-mon[83447]: pgmap v56: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 639 B/s wr, 0 op/s 2024-03-22T07:14:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:22 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 342.627657 secs 2024-03-22T07:14:24.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:23 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 342.968047 secs 2024-03-22T07:14:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:23 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 342.968047 secs 2024-03-22T07:14:25.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:24 smithi081 ceph-mon[113965]: pgmap v57: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 616 B/s wr, 0 op/s 2024-03-22T07:14:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:24 smithi149 ceph-mon[83447]: pgmap v57: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 616 B/s wr, 0 op/s 2024-03-22T07:14:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:26 smithi081 ceph-mon[113965]: pgmap v58: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:14:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:26 smithi149 ceph-mon[83447]: pgmap v58: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-03-22T07:14:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:28.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:27 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 347.627771 secs 2024-03-22T07:14:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:27 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 347.627771 secs 2024-03-22T07:14:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:28 smithi081 ceph-mon[113965]: pgmap v59: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 452 B/s wr, 0 op/s 2024-03-22T07:14:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:28 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 347.968165 secs 2024-03-22T07:14:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:28 smithi149 ceph-mon[83447]: pgmap v59: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 452 B/s wr, 0 op/s 2024-03-22T07:14:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:28 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 347.968165 secs 2024-03-22T07:14:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:30 smithi081 ceph-mon[113965]: pgmap v60: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:30 smithi149 ceph-mon[83447]: pgmap v60: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:32 smithi081 ceph-mon[113965]: pgmap v61: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:32 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 352.627890 secs 2024-03-22T07:14:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:32 smithi149 ceph-mon[83447]: pgmap v61: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:32 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 352.627890 secs 2024-03-22T07:14:34.238 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:33 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 352.968281 secs 2024-03-22T07:14:34.238 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:33 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:14:34.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:33 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 352.968281 secs 2024-03-22T07:14:34.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:33 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:14:35.287 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:34 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:14:35.287 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:34 smithi081 ceph-mon[113965]: Upgrade: osd.1 is safe to restart 2024-03-22T07:14:35.287 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:34 smithi081 ceph-mon[113965]: pgmap v62: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:35.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:34 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:14:35.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:34 smithi149 ceph-mon[83447]: Upgrade: osd.1 is safe to restart 2024-03-22T07:14:35.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:34 smithi149 ceph-mon[83447]: pgmap v62: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:36.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:36 smithi081 ceph-mon[113965]: Upgrade: Updating osd.1 2024-03-22T07:14:36.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:36.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:14:36.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:36 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:14:36.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:36 smithi081 ceph-mon[113965]: Deploying daemon osd.1 on smithi081 2024-03-22T07:14:36.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:36 smithi081 ceph-mon[113965]: pgmap v63: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:36.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:36 smithi149 ceph-mon[83447]: Upgrade: Updating osd.1 2024-03-22T07:14:36.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:36.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:14:36.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:36 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:14:36.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:36 smithi149 ceph-mon[83447]: Deploying daemon osd.1 on smithi081 2024-03-22T07:14:36.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:36 smithi149 ceph-mon[83447]: pgmap v63: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:37.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:37 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 357.627951 secs 2024-03-22T07:14:38.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:37 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 357.627951 secs 2024-03-22T07:14:38.798 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:38 smithi081 ceph-mon[113965]: pgmap v64: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:38.799 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:38 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 357.968401 secs 2024-03-22T07:14:39.057 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:38 smithi081 systemd[1]: Stopping Ceph osd.1 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:14:39.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:38 smithi149 ceph-mon[83447]: pgmap v64: 65 pgs: 65 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:39.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:38 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 357.968401 secs 2024-03-22T07:14:39.319 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:39 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[55116]: 2024-03-22T07:14:39.056+0000 7f2ee3cbc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:14:39.319 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:39 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[55116]: 2024-03-22T07:14:39.056+0000 7f2ee3cbc700 -1 osd.1 54 *** Got signal Terminated *** 2024-03-22T07:14:39.319 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:39 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[55116]: 2024-03-22T07:14:39.056+0000 7f2ee3cbc700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:14:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:39 smithi081 ceph-mon[113965]: osd.1 marked itself down and dead 2024-03-22T07:14:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:39 smithi149 ceph-mon[83447]: osd.1 marked itself down and dead 2024-03-22T07:14:40.350 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:40 smithi081 podman[131173]: 2024-03-22 07:14:40.099500869 +0000 UTC m=+1.134535062 container died c29105d56e37ee608ec6448d596f3b04db6581ca8174b9d1afcf9ddf8fc4b467 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:14:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.580+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/1871595511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c18102fd0 msgr2=0x7f8c181033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.580+0000 7f8c1fc8a700 1 --2- 172.21.15.81:0/1871595511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c18102fd0 0x7f8c181033a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c10004940 tx=0x7f8c10009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.580+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/1871595511 shutdown_connections 2024-03-22T07:14:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.580+0000 7f8c1fc8a700 1 --2- 172.21.15.81:0/1871595511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c180fb540 0x7f8c180fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.580+0000 7f8c1fc8a700 1 --2- 172.21.15.81:0/1871595511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c18102fd0 0x7f8c181033a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.580+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/1871595511 >> 172.21.15.81:0/1871595511 conn(0x7f8c18071530 msgr2=0x7f8c18071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.580+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/1871595511 shutdown_connections 2024-03-22T07:14:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.580+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/1871595511 wait complete. 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.581+0000 7f8c1fc8a700 1 Processor -- start 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.581+0000 7f8c1fc8a700 1 -- start start 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.581+0000 7f8c1fc8a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c180fb540 0x7f8c181139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.581+0000 7f8c1fc8a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c18102fd0 0x7f8c18113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.581+0000 7f8c1fc8a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c181145c0 con 0x7f8c180fb540 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.581+0000 7f8c1fc8a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1810dba0 con 0x7f8c18102fd0 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.581+0000 7f8c1da26700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c180fb540 0x7f8c181139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.581+0000 7f8c1da26700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c180fb540 0x7f8c181139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56198/0 (socket says 172.21.15.81:56198) 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.581+0000 7f8c1da26700 1 -- 172.21.15.81:0/189080693 learned_addr learned my addr 172.21.15.81:0/189080693 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.582+0000 7f8c1da26700 1 -- 172.21.15.81:0/189080693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c18102fd0 msgr2=0x7f8c18113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.582+0000 7f8c1da26700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c18102fd0 0x7f8c18113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.582+0000 7f8c1da26700 1 -- 172.21.15.81:0/189080693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c10005040 con 0x7f8c180fb540 2024-03-22T07:14:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.582+0000 7f8c1da26700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c180fb540 0x7f8c181139a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b8b0 tx=0x7f8c10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.582+0000 7f8c167fc700 1 -- 172.21.15.81:0/189080693 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1000e040 con 0x7f8c180fb540 2024-03-22T07:14:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.582+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1810de20 con 0x7f8c180fb540 2024-03-22T07:14:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.582+0000 7f8c167fc700 1 -- 172.21.15.81:0/189080693 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c10004290 con 0x7f8c180fb540 2024-03-22T07:14:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.582+0000 7f8c167fc700 1 -- 172.21.15.81:0/189080693 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1001c3f0 con 0x7f8c180fb540 2024-03-22T07:14:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.582+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1810e360 con 0x7f8c180fb540 2024-03-22T07:14:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.583+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c18100270 con 0x7f8c180fb540 2024-03-22T07:14:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.584+0000 7f8c167fc700 1 -- 172.21.15.81:0/189080693 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f8c100137b0 con 0x7f8c180fb540 2024-03-22T07:14:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.584+0000 7f8c167fc700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8c0006d380 0x7f8c0006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.584+0000 7f8c167fc700 1 -- 172.21.15.81:0/189080693 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8c1008e630 con 0x7f8c180fb540 2024-03-22T07:14:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.584+0000 7f8c1d225700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8c0006d380 0x7f8c0006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.585+0000 7f8c1d225700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8c0006d380 0x7f8c0006f830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c006fd0 tx=0x7f8c0c006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:40.588 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.588+0000 7f8c167fc700 1 -- 172.21.15.81:0/189080693 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c10058230 con 0x7f8c180fb540 2024-03-22T07:14:40.618 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:40 smithi081 podman[131173]: 2024-03-22 07:14:40.440709097 +0000 UTC m=+1.475743291 container cleanup c29105d56e37ee608ec6448d596f3b04db6581ca8174b9d1afcf9ddf8fc4b467 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-22T07:14:40.618 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:40 smithi081 bash[131173]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1 2024-03-22T07:14:40.850 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.849+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c1810a090 con 0x7f8c0006d380 2024-03-22T07:14:40.851 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.851+0000 7f8c167fc700 1 -- 172.21.15.81:0/189080693 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8c1810a090 con 0x7f8c0006d380 2024-03-22T07:14:40.853 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8c0006d380 msgr2=0x7f8c0006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8c0006d380 0x7f8c0006f830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c006fd0 tx=0x7f8c0c006db0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c180fb540 msgr2=0x7f8c181139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c180fb540 0x7f8c181139a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b8b0 tx=0x7f8c10009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 shutdown_connections 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8c0006d380 0x7f8c0006f830 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c180fb540 0x7f8c181139a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 --2- 172.21.15.81:0/189080693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c18102fd0 0x7f8c18113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 >> 172.21.15.81:0/189080693 conn(0x7f8c18071530 msgr2=0x7f8c180fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 shutdown_connections 2024-03-22T07:14:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.853+0000 7f8c1fc8a700 1 -- 172.21.15.81:0/189080693 wait complete. 2024-03-22T07:14:40.864 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:14:40.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.972+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/1725133933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 msgr2=0x7fd0d01031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:40.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.972+0000 7fd0d4e4e700 1 --2- 172.21.15.81:0/1725133933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 0x7fd0d01031e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0004940 tx=0x7fd0c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:40.973 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:40 smithi081 ceph-mon[113965]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:14:40.974 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:40 smithi081 ceph-mon[113965]: osdmap e55: 6 total, 5 up, 6 in 2024-03-22T07:14:40.974 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:40 smithi081 ceph-mon[113965]: failed to encode map e55 with expected crc 2024-03-22T07:14:40.974 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:40 smithi081 ceph-mon[113965]: failed to encode map e55 with expected crc 2024-03-22T07:14:40.974 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:40 smithi081 ceph-mon[113965]: failed to encode map e55 with expected crc 2024-03-22T07:14:40.974 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:40 smithi081 ceph-mon[113965]: failed to encode map e55 with expected crc 2024-03-22T07:14:40.974 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:40 smithi081 ceph-mon[113965]: pgmap v66: 65 pgs: 12 stale+active+clean, 53 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:40.974 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:40 smithi081 podman[131190]: 2024-03-22 07:14:40.6574803 +0000 UTC m=+0.553952397 container remove c29105d56e37ee608ec6448d596f3b04db6581ca8174b9d1afcf9ddf8fc4b467 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=quincy-9c4e0c3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-22T07:14:40.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/1725133933 shutdown_connections 2024-03-22T07:14:40.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 --2- 172.21.15.81:0/1725133933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d01037b0 0x7fd0d00fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 --2- 172.21.15.81:0/1725133933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 0x7fd0d01031e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/1725133933 >> 172.21.15.81:0/1725133933 conn(0x7fd0d00f8980 msgr2=0x7fd0d00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:40.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/1725133933 shutdown_connections 2024-03-22T07:14:40.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/1725133933 wait complete. 2024-03-22T07:14:40.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 Processor -- start 2024-03-22T07:14:40.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 -- start start 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 0x7fd0d00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d01037b0 0x7fd0d0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0070e20 con 0x7fd0d0102e10 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.973+0000 7fd0d4e4e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0070f60 con 0x7fd0d01037b0 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.974+0000 7fd0ce59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 0x7fd0d00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.974+0000 7fd0ce59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 0x7fd0d00702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56224/0 (socket says 172.21.15.81:56224) 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.974+0000 7fd0ce59c700 1 -- 172.21.15.81:0/2114868983 learned_addr learned my addr 172.21.15.81:0/2114868983 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.974+0000 7fd0cdd9b700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d01037b0 0x7fd0d0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.974+0000 7fd0ce59c700 1 -- 172.21.15.81:0/2114868983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d01037b0 msgr2=0x7fd0d0070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.974+0000 7fd0ce59c700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d01037b0 0x7fd0d0070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.974+0000 7fd0ce59c700 1 -- 172.21.15.81:0/2114868983 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c0005040 con 0x7fd0d0102e10 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.975+0000 7fd0ce59c700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 0x7fd0d00702c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000b3b0 tx=0x7fd0c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.975+0000 7fd0beffd700 1 -- 172.21.15.81:0/2114868983 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c000e040 con 0x7fd0d0102e10 2024-03-22T07:14:40.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.975+0000 7fd0beffd700 1 -- 172.21.15.81:0/2114868983 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0c0013420 con 0x7fd0d0102e10 2024-03-22T07:14:40.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.975+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d0119eb0 con 0x7fd0d0102e10 2024-03-22T07:14:40.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.975+0000 7fd0beffd700 1 -- 172.21.15.81:0/2114868983 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0c001c630 con 0x7fd0d0102e10 2024-03-22T07:14:40.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.975+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d011a2d0 con 0x7fd0d0102e10 2024-03-22T07:14:40.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.976+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d00fe560 con 0x7fd0d0102e10 2024-03-22T07:14:40.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.977+0000 7fd0beffd700 1 -- 172.21.15.81:0/2114868983 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fd0c000baf0 con 0x7fd0d0102e10 2024-03-22T07:14:40.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.977+0000 7fd0beffd700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd0b806d380 0x7fd0b806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:40.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.977+0000 7fd0beffd700 1 -- 172.21.15.81:0/2114868983 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd0c008d9e0 con 0x7fd0d0102e10 2024-03-22T07:14:40.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.978+0000 7fd0cdd9b700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd0b806d380 0x7fd0b806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:40.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.978+0000 7fd0cdd9b700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd0b806d380 0x7fd0b806f830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0071c00 tx=0x7fd0c40091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:40.983 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:40.983+0000 7fd0beffd700 1 -- 172.21.15.81:0/2114868983 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd0c00575e0 con 0x7fd0d0102e10 2024-03-22T07:14:41.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:40 smithi149 ceph-mon[83447]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:14:41.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:40 smithi149 ceph-mon[83447]: osdmap e55: 6 total, 5 up, 6 in 2024-03-22T07:14:41.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:40 smithi149 ceph-mon[83447]: failed to encode map e55 with expected crc 2024-03-22T07:14:41.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:40 smithi149 ceph-mon[83447]: failed to encode map e55 with expected crc 2024-03-22T07:14:41.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:40 smithi149 ceph-mon[83447]: failed to encode map e55 with expected crc 2024-03-22T07:14:41.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:40 smithi149 ceph-mon[83447]: failed to encode map e55 with expected crc 2024-03-22T07:14:41.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:40 smithi149 ceph-mon[83447]: pgmap v66: 65 pgs: 12 stale+active+clean, 53 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail 2024-03-22T07:14:41.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.202+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0d00719b0 con 0x7fd0b806d380 2024-03-22T07:14:41.203 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.203+0000 7fd0beffd700 1 -- 172.21.15.81:0/2114868983 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd0d00719b0 con 0x7fd0b806d380 2024-03-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.205+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd0b806d380 msgr2=0x7fd0b806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.205+0000 7fd0d4e4e700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd0b806d380 0x7fd0b806f830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0071c00 tx=0x7fd0c40091a0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.205+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 msgr2=0x7fd0d00702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.205+0000 7fd0d4e4e700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 0x7fd0d00702c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000b3b0 tx=0x7fd0c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.205+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 shutdown_connections 2024-03-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.205+0000 7fd0d4e4e700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd0b806d380 0x7fd0b806f830 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.205+0000 7fd0d4e4e700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d0102e10 0x7fd0d00702c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.205+0000 7fd0d4e4e700 1 --2- 172.21.15.81:0/2114868983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d01037b0 0x7fd0d0070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.205+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 >> 172.21.15.81:0/2114868983 conn(0x7fd0d00f8980 msgr2=0x7fd0d00f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.206+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 shutdown_connections 2024-03-22T07:14:41.207 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.206+0000 7fd0d4e4e700 1 -- 172.21.15.81:0/2114868983 wait complete. 2024-03-22T07:14:41.256 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:41 smithi081 podman[131327]: 2024-03-22 07:14:41.006144931 +0000 UTC m=+0.018484685 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:14:41.257 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:41 smithi081 podman[131327]: 2024-03-22 07:14:41.126924247 +0000 UTC m=+0.139263999 container create 4ba797cfb595435879aa7b2b0d5e620e65d0f8a6f58ee656c947f7423b44449c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:14:41.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.309+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/2688156717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 msgr2=0x7f2e24106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.309+0000 7f2e2c0d9700 1 --2- 172.21.15.81:0/2688156717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 0x7f2e24106e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e14004940 tx=0x7f2e14009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:41.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.309+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/2688156717 shutdown_connections 2024-03-22T07:14:41.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.309+0000 7f2e2c0d9700 1 --2- 172.21.15.81:0/2688156717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 0x7f2e24106e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.309+0000 7f2e2c0d9700 1 --2- 172.21.15.81:0/2688156717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e240fde70 0x7f2e240fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.309+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/2688156717 >> 172.21.15.81:0/2688156717 conn(0x7f2e240f7880 msgr2=0x7f2e240f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:41.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.309+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/2688156717 shutdown_connections 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.309+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/2688156717 wait complete. 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.309+0000 7f2e2c0d9700 1 Processor -- start 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e2c0d9700 1 -- start start 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e2c0d9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e240fde70 0x7f2e24113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e2c0d9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 0x7f2e24114370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e2c0d9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241149c0 con 0x7f2e240fe810 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e2c0d9700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2410dd90 con 0x7f2e240fde70 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e29674700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 0x7f2e24114370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e29674700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 0x7f2e24114370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56240/0 (socket says 172.21.15.81:56240) 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e29674700 1 -- 172.21.15.81:0/1606998758 learned_addr learned my addr 172.21.15.81:0/1606998758 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e29e75700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e240fde70 0x7f2e24113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e29674700 1 -- 172.21.15.81:0/1606998758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e240fde70 msgr2=0x7f2e24113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e29674700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e240fde70 0x7f2e24113e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e29674700 1 -- 172.21.15.81:0/1606998758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e14005040 con 0x7f2e240fe810 2024-03-22T07:14:41.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.310+0000 7f2e29674700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 0x7f2e24114370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009380 tx=0x7f2e14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:41.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.311+0000 7f2e1a7fc700 1 -- 172.21.15.81:0/1606998758 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1400e040 con 0x7f2e240fe810 2024-03-22T07:14:41.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.311+0000 7f2e1a7fc700 1 -- 172.21.15.81:0/1606998758 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e14009af0 con 0x7f2e240fe810 2024-03-22T07:14:41.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.311+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2410e070 con 0x7f2e240fe810 2024-03-22T07:14:41.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.311+0000 7f2e1a7fc700 1 -- 172.21.15.81:0/1606998758 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1401c630 con 0x7f2e240fe810 2024-03-22T07:14:41.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.311+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2410e5b0 con 0x7f2e240fe810 2024-03-22T07:14:41.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.312+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24066710 con 0x7f2e240fe810 2024-03-22T07:14:41.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.312+0000 7f2e1a7fc700 1 -- 172.21.15.81:0/1606998758 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f2e1400bbc0 con 0x7f2e240fe810 2024-03-22T07:14:41.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.313+0000 7f2e1a7fc700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2e1006d380 0x7f2e1006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:41.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.313+0000 7f2e1a7fc700 1 -- 172.21.15.81:0/1606998758 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2e1408dab0 con 0x7f2e240fe810 2024-03-22T07:14:41.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.313+0000 7f2e29e75700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2e1006d380 0x7f2e1006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:41.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.313+0000 7f2e29e75700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2e1006d380 0x7f2e1006f830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c004990 tx=0x7f2e1c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:41.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.317+0000 7f2e1a7fc700 1 -- 172.21.15.81:0/1606998758 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e1405b1e0 con 0x7f2e240fe810 2024-03-22T07:14:41.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.526+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e24109e30 con 0x7f2e1006d380 2024-03-22T07:14:41.527 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:41 smithi081 podman[131327]: 2024-03-22 07:14:41.389601084 +0000 UTC m=+0.401940842 container init 4ba797cfb595435879aa7b2b0d5e620e65d0f8a6f58ee656c947f7423b44449c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-03-22T07:14:41.527 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:41 smithi081 podman[131327]: 2024-03-22 07:14:41.394539352 +0000 UTC m=+0.406879115 container start 4ba797cfb595435879aa7b2b0d5e620e65d0f8a6f58ee656c947f7423b44449c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:14:41.527 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:41 smithi081 podman[131327]: 2024-03-22 07:14:41.480132878 +0000 UTC m=+0.492472630 container attach 4ba797cfb595435879aa7b2b0d5e620e65d0f8a6f58ee656c947f7423b44449c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-03-22T07:14:41.537 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.537+0000 7f2e1a7fc700 1 -- 172.21.15.81:0/1606998758 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2e24109e30 con 0x7f2e1006d380 2024-03-22T07:14:41.537 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:14:41.537 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (3m) 27s ago 13m 26.0M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:14:41.537 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (60s) 27s ago 13m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:14:41.537 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (54s) 51s ago 12m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (2m) 27s ago 13m 80.5M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (8m) 27s ago 8m 13.4M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (8m) 27s ago 8m 131M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (8m) 51s ago 8m 16.8M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (8m) 51s ago 8m 147M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (4m) 27s ago 14m 499M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (4m) 51s ago 12m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (2m) 27s ago 15m 41.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (99s) 51s ago 12m 31.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (3m) 27s ago 13m 16.4M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (3m) 51s ago 12m 16.7M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (33s) 27s ago 11m 12.7M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 033c080308b6 2024-03-22T07:14:41.538 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (11m) 27s ago 11m 200M 1557M 17.2.7-726-g9c4e0c34 837124da804f c29105d56e37 2024-03-22T07:14:41.539 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (10m) 27s ago 10m 163M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:14:41.539 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (10m) 51s ago 10m 215M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:14:41.539 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (10m) 51s ago 10m 194M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:14:41.539 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (9m) 51s ago 9m 184M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:14:41.539 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (3m) 27s ago 13m 59.1M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.539+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2e1006d380 msgr2=0x7f2e1006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.539+0000 7f2e2c0d9700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2e1006d380 0x7f2e1006f830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c004990 tx=0x7f2e1c009ee0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.539+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 msgr2=0x7f2e24114370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.539+0000 7f2e2c0d9700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 0x7f2e24114370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009380 tx=0x7f2e14009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.539+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 shutdown_connections 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.539+0000 7f2e2c0d9700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2e1006d380 0x7f2e1006f830 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.539+0000 7f2e2c0d9700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e240fde70 0x7f2e24113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.539+0000 7f2e2c0d9700 1 --2- 172.21.15.81:0/1606998758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240fe810 0x7f2e24114370 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.539+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 >> 172.21.15.81:0/1606998758 conn(0x7f2e240f7880 msgr2=0x7f2e240f8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.540+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 shutdown_connections 2024-03-22T07:14:41.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.540+0000 7f2e2c0d9700 1 -- 172.21.15.81:0/1606998758 wait complete. 2024-03-22T07:14:41.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.643+0000 7f77141c8700 1 -- 172.21.15.81:0/437583421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 msgr2=0x7f770c0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.643+0000 7f77141c8700 1 --2- 172.21.15.81:0/437583421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 0x7f770c0fbd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f76fc004940 tx=0x7f76fc009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:41.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.643+0000 7f77141c8700 1 -- 172.21.15.81:0/437583421 shutdown_connections 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.643+0000 7f77141c8700 1 --2- 172.21.15.81:0/437583421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 0x7f770c0fbd30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.643+0000 7f77141c8700 1 --2- 172.21.15.81:0/437583421 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c101d20 0x7f770c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.643+0000 7f77141c8700 1 -- 172.21.15.81:0/437583421 >> 172.21.15.81:0/437583421 conn(0x7f770c0f78d0 msgr2=0x7f770c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.643+0000 7f77141c8700 1 -- 172.21.15.81:0/437583421 shutdown_connections 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.643+0000 7f77141c8700 1 -- 172.21.15.81:0/437583421 wait complete. 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f77141c8700 1 Processor -- start 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f77141c8700 1 -- start start 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f77141c8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c101d20 0x7f770c113c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f77141c8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 0x7f770c114140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f77141c8700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c114790 con 0x7f770c1026c0 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f77141c8700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c10db80 con 0x7f770c101d20 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f7711763700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 0x7f770c114140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:41.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f7711763700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 0x7f770c114140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56258/0 (socket says 172.21.15.81:56258) 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f7711763700 1 -- 172.21.15.81:0/3747862518 learned_addr learned my addr 172.21.15.81:0/3747862518 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f7711763700 1 -- 172.21.15.81:0/3747862518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c101d20 msgr2=0x7f770c113c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f7711763700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c101d20 0x7f770c113c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.644+0000 7f7711763700 1 -- 172.21.15.81:0/3747862518 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc005040 con 0x7f770c1026c0 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.645+0000 7f7711763700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 0x7f770c114140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009100 tx=0x7f76fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.645+0000 7f77027fc700 1 -- 172.21.15.81:0/3747862518 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76fc00e040 con 0x7f770c1026c0 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.645+0000 7f77027fc700 1 -- 172.21.15.81:0/3747862518 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76fc013420 con 0x7f770c1026c0 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.645+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c10de00 con 0x7f770c1026c0 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.645+0000 7f77027fc700 1 -- 172.21.15.81:0/3747862518 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76fc01c630 con 0x7f770c1026c0 2024-03-22T07:14:41.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.645+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c10e3c0 con 0x7f770c1026c0 2024-03-22T07:14:41.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.646+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c0fd470 con 0x7f770c1026c0 2024-03-22T07:14:41.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.646+0000 7f77027fc700 1 -- 172.21.15.81:0/3747862518 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f76fc013590 con 0x7f770c1026c0 2024-03-22T07:14:41.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.647+0000 7f77027fc700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f76f806d390 0x7f76f806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:41.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.647+0000 7f77027fc700 1 -- 172.21.15.81:0/3747862518 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f76fc08dab0 con 0x7f770c1026c0 2024-03-22T07:14:41.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.647+0000 7f7711f64700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f76f806d390 0x7f76f806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:41.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.647+0000 7f7711f64700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f76f806d390 0x7f76f806f840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7708004b70 tx=0x7f7708007cc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:41.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.651+0000 7f77027fc700 1 -- 172.21.15.81:0/3747862518 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f76fc08fe60 con 0x7f770c1026c0 2024-03-22T07:14:41.818 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:41 smithi081 podman[131327]: 2024-03-22 07:14:41.694429864 +0000 UTC m=+0.706769623 container died 4ba797cfb595435879aa7b2b0d5e620e65d0f8a6f58ee656c947f7423b44449c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:14:41.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:41 smithi081 ceph-mon[113965]: failed to encode map e56 with expected crc 2024-03-22T07:14:41.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:41 smithi081 ceph-mon[113965]: osdmap e56: 6 total, 5 up, 6 in 2024-03-22T07:14:41.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:41 smithi081 ceph-mon[113965]: failed to encode map e56 with expected crc 2024-03-22T07:14:41.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:41 smithi081 ceph-mon[113965]: failed to encode map e56 with expected crc 2024-03-22T07:14:41.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:41 smithi081 ceph-mon[113965]: failed to encode map e56 with expected crc 2024-03-22T07:14:41.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:41 smithi081 ceph-mon[113965]: failed to encode map e56 with expected crc 2024-03-22T07:14:41.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:41 smithi081 ceph-mon[113965]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:41.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.949+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f770c108e00 con 0x7f770c1026c0 2024-03-22T07:14:41.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.950+0000 7f77027fc700 1 -- 172.21.15.81:0/3747862518 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f76fc05b120 con 0x7f770c1026c0 2024-03-22T07:14:41.950 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:14:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:14:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:14:41.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 8, 2024-03-22T07:14:41.951 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 5 2024-03-22T07:14:41.952 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:14:41.952 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:14:41.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f76f806d390 msgr2=0x7f76f806f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f76f806d390 0x7f76f806f840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7708004b70 tx=0x7f7708007cc0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 msgr2=0x7f770c114140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:41.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 0x7f770c114140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009100 tx=0x7f76fc009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:41.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 shutdown_connections 2024-03-22T07:14:41.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f76f806d390 0x7f76f806f840 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c101d20 0x7f770c113c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 --2- 172.21.15.81:0/3747862518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f770c1026c0 0x7f770c114140 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:41.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 >> 172.21.15.81:0/3747862518 conn(0x7f770c0f78d0 msgr2=0x7f770c065ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:41.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 shutdown_connections 2024-03-22T07:14:41.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:41.952+0000 7f77141c8700 1 -- 172.21.15.81:0/3747862518 wait complete. 2024-03-22T07:14:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.055+0000 7fdb5df5d700 1 -- 172.21.15.81:0/263600653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 msgr2=0x7fdb58102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.055+0000 7fdb5df5d700 1 --2- 172.21.15.81:0/263600653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 0x7fdb58102140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb48004940 tx=0x7fdb48009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.055+0000 7fdb5df5d700 1 -- 172.21.15.81:0/263600653 shutdown_connections 2024-03-22T07:14:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.055+0000 7fdb5df5d700 1 --2- 172.21.15.81:0/263600653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb58102710 0x7fdb580fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.057 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.055+0000 7fdb5df5d700 1 --2- 172.21.15.81:0/263600653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 0x7fdb58102140 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.055+0000 7fdb5df5d700 1 -- 172.21.15.81:0/263600653 >> 172.21.15.81:0/263600653 conn(0x7fdb58072bc0 msgr2=0x7fdb58072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.055+0000 7fdb5df5d700 1 -- 172.21.15.81:0/263600653 shutdown_connections 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.055+0000 7fdb5df5d700 1 -- 172.21.15.81:0/263600653 wait complete. 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb5df5d700 1 Processor -- start 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb5df5d700 1 -- start start 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb5df5d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 0x7fdb58112a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb5df5d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb58102710 0x7fdb58112f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb5df5d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb58113630 con 0x7fdb58101d70 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb5df5d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5810c990 con 0x7fdb58102710 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 0x7fdb58112a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:42.058 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 0x7fdb58112a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56276/0 (socket says 172.21.15.81:56276) 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb577fe700 1 -- 172.21.15.81:0/1112432972 learned_addr learned my addr 172.21.15.81:0/1112432972 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb577fe700 1 -- 172.21.15.81:0/1112432972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb58102710 msgr2=0x7fdb58112f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb577fe700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb58102710 0x7fdb58112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.056+0000 7fdb577fe700 1 -- 172.21.15.81:0/1112432972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48005040 con 0x7fdb58101d70 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.057+0000 7fdb577fe700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 0x7fdb58112a10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b3b0 tx=0x7fdb48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.057+0000 7fdb54ff9700 1 -- 172.21.15.81:0/1112432972 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb4800e040 con 0x7fdb58101d70 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.057+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5810cc10 con 0x7fdb58101d70 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.057+0000 7fdb54ff9700 1 -- 172.21.15.81:0/1112432972 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb4800bbc0 con 0x7fdb58101d70 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.057+0000 7fdb54ff9700 1 -- 172.21.15.81:0/1112432972 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb4801c3f0 con 0x7fdb58101d70 2024-03-22T07:14:42.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.057+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5810d1d0 con 0x7fdb58101d70 2024-03-22T07:14:42.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.058+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb580ff080 con 0x7fdb58101d70 2024-03-22T07:14:42.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.058+0000 7fdb54ff9700 1 -- 172.21.15.81:0/1112432972 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fdb4800bd30 con 0x7fdb58101d70 2024-03-22T07:14:42.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.059+0000 7fdb54ff9700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb4406d380 0x7fdb4406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:42.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.059+0000 7fdb54ff9700 1 -- 172.21.15.81:0/1112432972 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdb4808db60 con 0x7fdb58101d70 2024-03-22T07:14:42.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.059+0000 7fdb4effd700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb4406d380 0x7fdb4406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:42.060 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.059+0000 7fdb4effd700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb4406d380 0x7fdb4406f830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdb40006fd0 tx=0x7fdb40006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:42.063 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.063+0000 7fdb54ff9700 1 -- 172.21.15.81:0/1112432972 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb4805b170 con 0x7fdb58101d70 2024-03-22T07:14:42.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:41 smithi149 ceph-mon[83447]: failed to encode map e56 with expected crc 2024-03-22T07:14:42.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:41 smithi149 ceph-mon[83447]: osdmap e56: 6 total, 5 up, 6 in 2024-03-22T07:14:42.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:41 smithi149 ceph-mon[83447]: failed to encode map e56 with expected crc 2024-03-22T07:14:42.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:41 smithi149 ceph-mon[83447]: failed to encode map e56 with expected crc 2024-03-22T07:14:42.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:41 smithi149 ceph-mon[83447]: failed to encode map e56 with expected crc 2024-03-22T07:14:42.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:41 smithi149 ceph-mon[83447]: failed to encode map e56 with expected crc 2024-03-22T07:14:42.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:41 smithi149 ceph-mon[83447]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:42.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.302+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdb5810e090 con 0x7fdb58101d70 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.302+0000 7fdb54ff9700 1 -- 172.21.15.81:0/1112432972 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1858 (secure 0 0 0) 0x7fdb4805ad00 con 0x7fdb58101d70 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stdout:e22 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 22 2024-03-22T07:14:42.303 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:12:38.916315+0000 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:14:42.304 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 95 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 94 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:14:42.305 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:14:42.306 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:14:42.306 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:14:42.306 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:14:42.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb4406d380 msgr2=0x7fdb4406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb4406d380 0x7fdb4406f830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdb40006fd0 tx=0x7fdb40006db0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 msgr2=0x7fdb58112a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 0x7fdb58112a10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800b3b0 tx=0x7fdb48009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:42.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 shutdown_connections 2024-03-22T07:14:42.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb4406d380 0x7fdb4406f830 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb58101d70 0x7fdb58112a10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 --2- 172.21.15.81:0/1112432972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb58102710 0x7fdb58112f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 >> 172.21.15.81:0/1112432972 conn(0x7fdb58072bc0 msgr2=0x7fdb580fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:42.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 shutdown_connections 2024-03-22T07:14:42.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.305+0000 7fdb5df5d700 1 -- 172.21.15.81:0/1112432972 wait complete. 2024-03-22T07:14:42.308 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 22 2024-03-22T07:14:42.409 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.409+0000 7fc88dfae700 1 -- 172.21.15.81:0/292240793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 msgr2=0x7fc888103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.409+0000 7fc88dfae700 1 --2- 172.21.15.81:0/292240793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 0x7fc888103240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc880009ce0 tx=0x7fc880004d10 comp rx=0 tx=0).stop 2024-03-22T07:14:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.409+0000 7fc88dfae700 1 -- 172.21.15.81:0/292240793 shutdown_connections 2024-03-22T07:14:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.409+0000 7fc88dfae700 1 --2- 172.21.15.81:0/292240793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8880fb4a0 0x7fc8880fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.409+0000 7fc88dfae700 1 --2- 172.21.15.81:0/292240793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 0x7fc888103240 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.409+0000 7fc88dfae700 1 -- 172.21.15.81:0/292240793 >> 172.21.15.81:0/292240793 conn(0x7fc8880714c0 msgr2=0x7fc8880718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.409+0000 7fc88dfae700 1 -- 172.21.15.81:0/292240793 shutdown_connections 2024-03-22T07:14:42.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.409+0000 7fc88dfae700 1 -- 172.21.15.81:0/292240793 wait complete. 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.410+0000 7fc88dfae700 1 Processor -- start 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.410+0000 7fc88dfae700 1 -- start start 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.410+0000 7fc88dfae700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8880fb4a0 0x7fc888113cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.410+0000 7fc88dfae700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 0x7fc8881141f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.410+0000 7fc88dfae700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc888114840 con 0x7fc888102e70 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.410+0000 7fc88dfae700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88810dc10 con 0x7fc8880fb4a0 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.410+0000 7fc886ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 0x7fc8881141f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.410+0000 7fc886ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 0x7fc8881141f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56290/0 (socket says 172.21.15.81:56290) 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.410+0000 7fc886ffd700 1 -- 172.21.15.81:0/1664880520 learned_addr learned my addr 172.21.15.81:0/1664880520 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:42.412 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.411+0000 7fc886ffd700 1 -- 172.21.15.81:0/1664880520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8880fb4a0 msgr2=0x7fc888113cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.411+0000 7fc886ffd700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8880fb4a0 0x7fc888113cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.411+0000 7fc886ffd700 1 -- 172.21.15.81:0/1664880520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc880009940 con 0x7fc888102e70 2024-03-22T07:14:42.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.411+0000 7fc886ffd700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 0x7fc8881141f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc870002a50 tx=0x7fc8700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:42.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.411+0000 7fc87ffff700 1 -- 172.21.15.81:0/1664880520 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc870007cc0 con 0x7fc888102e70 2024-03-22T07:14:42.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.411+0000 7fc87ffff700 1 -- 172.21.15.81:0/1664880520 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc87000f040 con 0x7fc888102e70 2024-03-22T07:14:42.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.411+0000 7fc87ffff700 1 -- 172.21.15.81:0/1664880520 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc870009640 con 0x7fc888102e70 2024-03-22T07:14:42.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.411+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88810def0 con 0x7fc888102e70 2024-03-22T07:14:42.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.411+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88810e4b0 con 0x7fc888102e70 2024-03-22T07:14:42.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.412+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc888100110 con 0x7fc888102e70 2024-03-22T07:14:42.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.413+0000 7fc87ffff700 1 -- 172.21.15.81:0/1664880520 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fc8700097a0 con 0x7fc888102e70 2024-03-22T07:14:42.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.413+0000 7fc87ffff700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc87406d380 0x7fc87406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:42.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.413+0000 7fc87ffff700 1 -- 172.21.15.81:0/1664880520 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc87008c670 con 0x7fc888102e70 2024-03-22T07:14:42.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.413+0000 7fc8877fe700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc87406d380 0x7fc87406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:42.415 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.413+0000 7fc8877fe700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc87406d380 0x7fc87406f830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc880009cb0 tx=0x7fc880004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:42.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.418+0000 7fc87ffff700 1 -- 172.21.15.81:0/1664880520 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc870059c80 con 0x7fc888102e70 2024-03-22T07:14:42.569 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:42 smithi081 podman[131399]: 2024-03-22 07:14:42.469089289 +0000 UTC m=+0.769405900 container remove 4ba797cfb595435879aa7b2b0d5e620e65d0f8a6f58ee656c947f7423b44449c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-db0330b, ceph=True) 2024-03-22T07:14:42.569 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:42 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-22T07:14:42.569 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:42 smithi081 systemd[1]: Stopped Ceph osd.1 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:14:42.569 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:42 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.1.service: Consumed 38.186s CPU time. 2024-03-22T07:14:42.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.650+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc88810f370 con 0x7fc87406d380 2024-03-22T07:14:42.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.651+0000 7fc87ffff700 1 -- 172.21.15.81:0/1664880520 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc88810f370 con 0x7fc87406d380 2024-03-22T07:14:42.651 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "mon", 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr", 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "crash" 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "7/21 daemons upgraded", 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:14:42.652 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:14:42.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc87406d380 msgr2=0x7fc87406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc87406d380 0x7fc87406f830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc880009cb0 tx=0x7fc880004d10 comp rx=0 tx=0).stop 2024-03-22T07:14:42.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 msgr2=0x7fc8881141f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 0x7fc8881141f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc870002a50 tx=0x7fc8700046b0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 shutdown_connections 2024-03-22T07:14:42.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc87406d380 0x7fc87406f830 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8880fb4a0 0x7fc888113cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 --2- 172.21.15.81:0/1664880520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc888102e70 0x7fc8881141f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 >> 172.21.15.81:0/1664880520 conn(0x7fc8880714c0 msgr2=0x7fc8880fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:42.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 shutdown_connections 2024-03-22T07:14:42.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.654+0000 7fc88dfae700 1 -- 172.21.15.81:0/1664880520 wait complete. 2024-03-22T07:14:42.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3220069083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fbbf0 msgr2=0x7f7af00fbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 --2- 172.21.15.81:0/3220069083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fbbf0 0x7f7af00fbfc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4004940 tx=0x7f7ae4009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:42.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3220069083 shutdown_connections 2024-03-22T07:14:42.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 --2- 172.21.15.81:0/3220069083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7af00fc500 0x7f7af0109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 --2- 172.21.15.81:0/3220069083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fbbf0 0x7f7af00fbfc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3220069083 >> 172.21.15.81:0/3220069083 conn(0x7f7af00f7860 msgr2=0x7f7af00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3220069083 shutdown_connections 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3220069083 wait complete. 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 Processor -- start 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.764+0000 7f7af6a9c700 1 -- start start 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7af6a9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7af00fbbf0 0x7f7af0115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7af6a9c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fc500 0x7f7af0116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7af6a9c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01166e0 con 0x7f7af00fc500 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7af6a9c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af010fcc0 con 0x7f7af00fbbf0 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fc500 0x7f7af0116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fc500 0x7f7af0116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56300/0 (socket says 172.21.15.81:56300) 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7aeffff700 1 -- 172.21.15.81:0/3351604708 learned_addr learned my addr 172.21.15.81:0/3351604708 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:14:42.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7af4838700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7af00fbbf0 0x7f7af0115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:42.767 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7aeffff700 1 -- 172.21.15.81:0/3351604708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7af00fbbf0 msgr2=0x7f7af0115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7aeffff700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7af00fbbf0 0x7f7af0115b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.765+0000 7f7aeffff700 1 -- 172.21.15.81:0/3351604708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae4005040 con 0x7f7af00fc500 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.766+0000 7f7aeffff700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fc500 0x7f7af0116090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7adc002a50 tx=0x7f7adc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.766+0000 7f7aed7fa700 1 -- 172.21.15.81:0/3351604708 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7adc007cc0 con 0x7f7af00fc500 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.766+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af010ffa0 con 0x7f7af00fc500 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.766+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af0110560 con 0x7f7af00fc500 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.766+0000 7f7aed7fa700 1 -- 172.21.15.81:0/3351604708 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7adc00f040 con 0x7f7af00fc500 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.766+0000 7f7aed7fa700 1 -- 172.21.15.81:0/3351604708 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7adc00da60 con 0x7f7af00fc500 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.767+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af0066a70 con 0x7f7af00fc500 2024-03-22T07:14:42.768 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.768+0000 7f7aed7fa700 1 -- 172.21.15.81:0/3351604708 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f7adc00dbc0 con 0x7f7af00fc500 2024-03-22T07:14:42.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.768+0000 7f7aed7fa700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7ae006d380 0x7f7ae006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:42.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.768+0000 7f7aed7fa700 1 -- 172.21.15.81:0/3351604708 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7adc08c530 con 0x7f7af00fc500 2024-03-22T07:14:42.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.768+0000 7f7af4838700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7ae006d380 0x7f7ae006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:42.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.769+0000 7f7af4838700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7ae006d380 0x7f7ae006f830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400b3b0 tx=0x7f7ae4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:42.773 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:42.773+0000 7f7aed7fa700 1 -- 172.21.15.81:0/3351604708 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7adc059bc0 con 0x7f7af00fc500 2024-03-22T07:14:42.947 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:42 smithi081 systemd[1]: Starting Ceph osd.1 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:14:42.947 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:42 smithi081 ceph-mon[113965]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:42.947 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:42 smithi081 ceph-mon[113965]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:42.947 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:42 smithi081 ceph-mon[113965]: pgmap v68: 65 pgs: 29 active+undersized+degraded, 3 stale+active+clean, 33 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 5110/32856 objects degraded (15.553%) 2024-03-22T07:14:42.947 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:42 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3747862518' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:42.947 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:42 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1112432972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:14:42.947 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:42 smithi081 ceph-mon[113965]: 1 slow requests, 1 included below; oldest blocked for > 362.628044 secs 2024-03-22T07:14:42.947 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:42 smithi081 ceph-mon[113965]: slow request 362.628044 seconds old, received at 2024-03-22T07:08:39.885715+0000: client_request(client.14534:13454 rename #0x100000034b7/d56 #0x100000033f3/d2e 2024-03-22T07:08:39.884274+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently acquired locks 2024-03-22T07:14:43.072 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.071+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7af010c0a0 con 0x7f7af00fc500 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.072+0000 7f7aed7fa700 1 -- 172.21.15.81:0/3351604708 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+2129 (secure 0 0 0) 0x7f7adc05ee60 con 0x7f7af00fc500 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_WARN 2 MDSs report slow requests; 1 osds down; Degraded data redundancy: 5110/32856 objects degraded (15.553%), 29 pgs degraded 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stdout: osd.1 (root=default,host=smithi081) is down 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 5110/32856 objects degraded (15.553%), 29 pgs degraded 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-22T07:14:43.073 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-22T07:14:43.074 INFO:teuthology.orchestra.run.smithi081.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.0 is active+undersized+degraded, acting [2,4] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.2 is active+undersized+degraded, acting [5,3] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.4 is active+undersized+degraded, acting [2,3] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.7 is active+undersized+degraded, acting [2,4] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.8 is active+undersized+degraded, acting [2,5] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.a is active+undersized+degraded, acting [3,4] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.e is active+undersized+degraded, acting [5,4] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.10 is active+undersized+degraded, acting [5,0] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.16 is active+undersized+degraded, acting [5,3] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.18 is active+undersized+degraded, acting [2,0] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.19 is active+undersized+degraded, acting [4,3] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.1a is active+undersized+degraded, acting [4,2] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.1c is active+undersized+degraded, acting [5,4] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.1d is active+undersized+degraded, acting [5,4] 2024-03-22T07:14:43.075 INFO:teuthology.orchestra.run.smithi081.stdout: pg 3.1e is active+undersized+degraded, acting [2,3] 2024-03-22T07:14:43.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7ae006d380 msgr2=0x7f7ae006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:43.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7ae006d380 0x7f7ae006f830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400b3b0 tx=0x7f7ae4009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:43.076 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fc500 msgr2=0x7f7af0116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:43.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fc500 0x7f7af0116090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7adc002a50 tx=0x7f7adc0046b0 comp rx=0 tx=0).stop 2024-03-22T07:14:43.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 shutdown_connections 2024-03-22T07:14:43.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7ae006d380 0x7f7ae006f830 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:43.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7af00fbbf0 0x7f7af0115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:43.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 --2- 172.21.15.81:0/3351604708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af00fc500 0x7f7af0116090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:43.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 >> 172.21.15.81:0/3351604708 conn(0x7f7af00f7860 msgr2=0x7f7af00f7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:43.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 shutdown_connections 2024-03-22T07:14:43.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:14:43.074+0000 7f7af6a9c700 1 -- 172.21.15.81:0/3351604708 wait complete. 2024-03-22T07:14:43.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:42 smithi149 ceph-mon[83447]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:43.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:42 smithi149 ceph-mon[83447]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:43.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:42 smithi149 ceph-mon[83447]: pgmap v68: 65 pgs: 29 active+undersized+degraded, 3 stale+active+clean, 33 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 5110/32856 objects degraded (15.553%) 2024-03-22T07:14:43.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:42 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3747862518' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:43.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:42 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1112432972' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:14:43.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:42 smithi149 ceph-mon[83447]: 1 slow requests, 1 included below; oldest blocked for > 362.628044 secs 2024-03-22T07:14:43.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:42 smithi149 ceph-mon[83447]: slow request 362.628044 seconds old, received at 2024-03-22T07:08:39.885715+0000: client_request(client.14534:13454 rename #0x100000034b7/d56 #0x100000033f3/d2e 2024-03-22T07:08:39.884274+0000 caller_uid=1000, caller_gid=1301{6,36,1000,1301,}) currently acquired locks 2024-03-22T07:14:43.272 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:42 smithi081 podman[131572]: 2024-03-22 07:14:42.849548745 +0000 UTC m=+0.022442062 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:14:43.272 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:42 smithi081 podman[131572]: 2024-03-22 07:14:42.974501845 +0000 UTC m=+0.147395153 container create a2ccb649c80c1a4616493271548b9ef352ae2d94745dd3b29c400e1264d7a5ee (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-22T07:14:43.272 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 podman[131572]: 2024-03-22 07:14:43.181215115 +0000 UTC m=+0.354108432 container init a2ccb649c80c1a4616493271548b9ef352ae2d94745dd3b29c400e1264d7a5ee (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-22T07:14:43.272 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 podman[131572]: 2024-03-22 07:14:43.185079018 +0000 UTC m=+0.357972326 container start a2ccb649c80c1a4616493271548b9ef352ae2d94745dd3b29c400e1264d7a5ee (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:14:43.272 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 podman[131572]: 2024-03-22 07:14:43.272721395 +0000 UTC m=+0.445614712 container attach a2ccb649c80c1a4616493271548b9ef352ae2d94745dd3b29c400e1264d7a5ee (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:14:43.978 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-mon[113965]: Health check failed: Degraded data redundancy: 5110/32856 objects degraded (15.553%), 29 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:43.979 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-mon[113965]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:43.979 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-mon[113965]: 1 slow requests, 1 included below; oldest blocked for > 362.968527 secs 2024-03-22T07:14:43.979 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-mon[113965]: slow request 362.968527 seconds old, received at 2024-03-22T07:08:39.886131+0000: peer_request:client.14534:13454 currently acquired locks 2024-03-22T07:14:43.979 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3351604708' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:14:43.979 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate[131587]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T07:14:43.979 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate[131587]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T07:14:43.979 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 bash[131572]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 bash[131572]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 bash[131572]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 bash[131572]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 bash[131572]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 bash[131572]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 bash[131572]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate[131587]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate[131587]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate[131587]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate[131587]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate[131587]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-22T07:14:43.980 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:43 smithi081 podman[131572]: 2024-03-22 07:14:43.651359988 +0000 UTC m=+0.824253298 container died a2ccb649c80c1a4616493271548b9ef352ae2d94745dd3b29c400e1264d7a5ee (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:14:44.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:43 smithi149 ceph-mon[83447]: Health check failed: Degraded data redundancy: 5110/32856 objects degraded (15.553%), 29 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:44.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:43 smithi149 ceph-mon[83447]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:44.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:43 smithi149 ceph-mon[83447]: 1 slow requests, 1 included below; oldest blocked for > 362.968527 secs 2024-03-22T07:14:44.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:43 smithi149 ceph-mon[83447]: slow request 362.968527 seconds old, received at 2024-03-22T07:08:39.886131+0000: peer_request:client.14534:13454 currently acquired locks 2024-03-22T07:14:44.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:43 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3351604708' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:14:44.641 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:44 smithi081 podman[131714]: 2024-03-22 07:14:44.369299978 +0000 UTC m=+0.712872074 container remove a2ccb649c80c1a4616493271548b9ef352ae2d94745dd3b29c400e1264d7a5ee (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:14:44.642 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:44 smithi081 podman[131781]: 2024-03-22 07:14:44.487786601 +0000 UTC m=+0.018833903 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:14:44.642 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:44 smithi081 podman[131781]: 2024-03-22 07:14:44.595266534 +0000 UTC m=+0.126313813 container create 8341fe32f3fa1503ef948553b33fec7a5086c6493786664abf074eb09dc4ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-22T07:14:44.891 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:44 smithi081 ceph-mon[113965]: pgmap v69: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:44.892 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:44 smithi081 podman[131781]: 2024-03-22 07:14:44.797745574 +0000 UTC m=+0.328792869 container init 8341fe32f3fa1503ef948553b33fec7a5086c6493786664abf074eb09dc4ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:14:44.892 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:44 smithi081 podman[131781]: 2024-03-22 07:14:44.800968683 +0000 UTC m=+0.332015971 container start 8341fe32f3fa1503ef948553b33fec7a5086c6493786664abf074eb09dc4ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-22T07:14:44.892 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:44 smithi081 ceph-osd[131800]: -- 172.21.15.81:0/4258506679 <== mon.0 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ca08091e0 con 0x561ca07db000 2024-03-22T07:14:44.892 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:44 smithi081 bash[131781]: 8341fe32f3fa1503ef948553b33fec7a5086c6493786664abf074eb09dc4ec7e 2024-03-22T07:14:44.892 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:44 smithi081 systemd[1]: Started Ceph osd.1 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:14:45.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:44 smithi149 ceph-mon[83447]: pgmap v69: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:45.727 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:45 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[131796]: 2024-03-22T07:14:45.343+0000 7f064dd057c0 -1 Falling back to public interface 2024-03-22T07:14:46.000 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:46.000 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:46.000 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:46.000 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:46.000 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:45 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:45 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:45 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:45 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:45 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:47.314 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:46 smithi081 ceph-mon[113965]: pgmap v70: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:47.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:46 smithi149 ceph-mon[83447]: pgmap v70: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:48.245 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:47 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 367.628126 secs 2024-03-22T07:14:48.246 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:47 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 367.968647 secs 2024-03-22T07:14:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:47 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 367.628126 secs 2024-03-22T07:14:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:47 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 367.968647 secs 2024-03-22T07:14:49.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:48 smithi081 ceph-mon[113965]: pgmap v71: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:49.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:48 smithi149 ceph-mon[83447]: pgmap v71: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:50.068 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:49 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[131796]: 2024-03-22T07:14:49.702+0000 7f064dd057c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-22T07:14:50.568 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:50 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[131796]: 2024-03-22T07:14:50.193+0000 7f064dd057c0 -1 osd.1 54 log_to_monitors true 2024-03-22T07:14:51.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:50 smithi081 ceph-mon[113965]: pgmap v72: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:51.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:50 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:51.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:50 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:51.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:50 smithi081 ceph-mon[113965]: from='osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T07:14:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:50 smithi149 ceph-mon[83447]: pgmap v72: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:50 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:50 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:50 smithi149 ceph-mon[83447]: from='osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T07:14:52.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:52 smithi081 ceph-mon[113965]: from='osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T07:14:52.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:52 smithi081 ceph-mon[113965]: osdmap e57: 6 total, 5 up, 6 in 2024-03-22T07:14:52.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:52 smithi081 ceph-mon[113965]: from='osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:14:52.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:52 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:52.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:52 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:52.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:52 smithi081 ceph-mon[113965]: pgmap v74: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:52.237 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:52 smithi081 ceph-mon[113965]: from='osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948]' entity='osd.1' 2024-03-22T07:14:52.237 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:14:52 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[131796]: 2024-03-22T07:14:52.054+0000 7f063b155700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:14:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:52 smithi149 ceph-mon[83447]: from='osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T07:14:52.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:52 smithi149 ceph-mon[83447]: osdmap e57: 6 total, 5 up, 6 in 2024-03-22T07:14:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:52 smithi149 ceph-mon[83447]: from='osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:14:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:52 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:52 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:52 smithi149 ceph-mon[83447]: pgmap v74: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 6169/32856 objects degraded (18.776%) 2024-03-22T07:14:52.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:52 smithi149 ceph-mon[83447]: from='osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948]' entity='osd.1' 2024-03-22T07:14:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:53 smithi081 ceph-mon[113965]: failed to encode map e57 with expected crc 2024-03-22T07:14:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:53 smithi081 ceph-mon[113965]: Health check update: Degraded data redundancy: 6169/32856 objects degraded (18.776%), 34 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:53 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 372.628255 secs 2024-03-22T07:14:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:53 smithi081 ceph-mon[113965]: failed to encode map e57 with expected crc 2024-03-22T07:14:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:53 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 372.968770 secs 2024-03-22T07:14:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:53 smithi081 ceph-mon[113965]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:14:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:53 smithi081 ceph-mon[113965]: osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948] boot 2024-03-22T07:14:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:53 smithi081 ceph-mon[113965]: osdmap e58: 6 total, 6 up, 6 in 2024-03-22T07:14:53.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:53 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:14:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:53 smithi149 ceph-mon[83447]: failed to encode map e57 with expected crc 2024-03-22T07:14:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:53 smithi149 ceph-mon[83447]: Health check update: Degraded data redundancy: 6169/32856 objects degraded (18.776%), 34 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:53 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 372.628255 secs 2024-03-22T07:14:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:53 smithi149 ceph-mon[83447]: failed to encode map e57 with expected crc 2024-03-22T07:14:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:53 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 372.968770 secs 2024-03-22T07:14:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:53 smithi149 ceph-mon[83447]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:14:53.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:53 smithi149 ceph-mon[83447]: osd.1 [v2:172.21.15.81:6810/356075948,v1:172.21.15.81:6811/356075948] boot 2024-03-22T07:14:53.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:53 smithi149 ceph-mon[83447]: osdmap e58: 6 total, 6 up, 6 in 2024-03-22T07:14:53.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:53 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:14:54.453 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:54 smithi081 ceph-mon[113965]: failed to encode map e57 with expected crc 2024-03-22T07:14:54.453 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:54 smithi081 ceph-mon[113965]: pgmap v76: 65 pgs: 2 peering, 32 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 5457/32856 objects degraded (16.609%) 2024-03-22T07:14:54.453 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:54 smithi081 ceph-mon[113965]: failed to encode map e59 with expected crc 2024-03-22T07:14:54.453 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:54 smithi081 ceph-mon[113965]: failed to encode map e59 with expected crc 2024-03-22T07:14:54.453 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:54 smithi081 ceph-mon[113965]: osdmap e59: 6 total, 6 up, 6 in 2024-03-22T07:14:54.453 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:54 smithi081 ceph-mon[113965]: failed to encode map e59 with expected crc 2024-03-22T07:14:54.453 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:54 smithi081 ceph-mon[113965]: failed to encode map e59 with expected crc 2024-03-22T07:14:54.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:54 smithi149 ceph-mon[83447]: failed to encode map e57 with expected crc 2024-03-22T07:14:54.606 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:54 smithi149 ceph-mon[83447]: pgmap v76: 65 pgs: 2 peering, 32 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 5457/32856 objects degraded (16.609%) 2024-03-22T07:14:54.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:54 smithi149 ceph-mon[83447]: failed to encode map e59 with expected crc 2024-03-22T07:14:54.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:54 smithi149 ceph-mon[83447]: failed to encode map e59 with expected crc 2024-03-22T07:14:54.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:54 smithi149 ceph-mon[83447]: osdmap e59: 6 total, 6 up, 6 in 2024-03-22T07:14:54.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:54 smithi149 ceph-mon[83447]: failed to encode map e59 with expected crc 2024-03-22T07:14:54.607 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:54 smithi149 ceph-mon[83447]: failed to encode map e59 with expected crc 2024-03-22T07:14:56.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: failed to encode map e59 with expected crc 2024-03-22T07:14:56.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: failed to encode map e59 with expected crc 2024-03-22T07:14:56.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: failed to encode map e59 with expected crc 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: failed to encode map e59 with expected crc 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:14:56.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:55 smithi149 ceph-mon[83447]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: failed to encode map e59 with expected crc 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: failed to encode map e59 with expected crc 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: failed to encode map e59 with expected crc 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: failed to encode map e59 with expected crc 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:14:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:56.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:56.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:56.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:56.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:14:56.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:14:56.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:55 smithi081 ceph-mon[113965]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-22T07:14:57.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:56 smithi149 ceph-mon[83447]: pgmap v78: 65 pgs: 2 peering, 32 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 5457/32856 objects degraded (16.609%) 2024-03-22T07:14:57.219 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:56 smithi081 ceph-mon[113965]: pgmap v78: 65 pgs: 2 peering, 32 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 5457/32856 objects degraded (16.609%) 2024-03-22T07:14:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:57 smithi081 ceph-mon[113965]: Health check update: Degraded data redundancy: 5457/32856 objects degraded (16.609%), 32 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:57 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 377.628374 secs 2024-03-22T07:14:58.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:57 smithi149 ceph-mon[83447]: Health check update: Degraded data redundancy: 5457/32856 objects degraded (16.609%), 32 pgs degraded (PG_DEGRADED) 2024-03-22T07:14:58.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:57 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 377.628374 secs 2024-03-22T07:14:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:58 smithi081 ceph-mon[113965]: pgmap v79: 65 pgs: 2 peering, 63 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:14:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:58 smithi081 ceph-mon[113965]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5457/32856 objects degraded (16.609%), 32 pgs degraded) 2024-03-22T07:14:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:14:58 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 377.968867 secs 2024-03-22T07:14:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:58 smithi149 ceph-mon[83447]: pgmap v79: 65 pgs: 2 peering, 63 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:14:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:58 smithi149 ceph-mon[83447]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5457/32856 objects degraded (16.609%), 32 pgs degraded) 2024-03-22T07:14:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:14:58 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 377.968867 secs 2024-03-22T07:15:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:00 smithi081 ceph-mon[113965]: pgmap v80: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:00 smithi149 ceph-mon[83447]: pgmap v80: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:02.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:01 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:02.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:01 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:02.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:01 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:02.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:01 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:03.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:02 smithi081 ceph-mon[113965]: pgmap v81: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:02 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 382.628488 secs 2024-03-22T07:15:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:02 smithi149 ceph-mon[83447]: pgmap v81: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:02 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 382.628488 secs 2024-03-22T07:15:04.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:03 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 382.968983 secs 2024-03-22T07:15:04.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:03 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 382.968983 secs 2024-03-22T07:15:05.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:04 smithi081 ceph-mon[113965]: pgmap v82: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:04 smithi149 ceph-mon[83447]: pgmap v82: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:07.272 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:06 smithi081 ceph-mon[113965]: pgmap v83: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:07.272 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:06 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:07.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:06 smithi149 ceph-mon[83447]: pgmap v83: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:07.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:06 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:07 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 387.628592 secs 2024-03-22T07:15:08.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:07 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 387.628592 secs 2024-03-22T07:15:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:08 smithi081 ceph-mon[113965]: pgmap v84: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:08 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 387.969089 secs 2024-03-22T07:15:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:08 smithi149 ceph-mon[83447]: pgmap v84: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:08 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 387.969089 secs 2024-03-22T07:15:11.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:10 smithi081 ceph-mon[113965]: pgmap v85: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:15:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:10 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:15:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:10 smithi081 ceph-mon[113965]: Upgrade: osd.2 is safe to restart 2024-03-22T07:15:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:10 smithi149 ceph-mon[83447]: pgmap v85: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:15:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:10 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:15:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:10 smithi149 ceph-mon[83447]: Upgrade: osd.2 is safe to restart 2024-03-22T07:15:12.818 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:12 smithi081 ceph-mon[113965]: Upgrade: Updating osd.2 2024-03-22T07:15:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:15:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:15:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:12 smithi081 ceph-mon[113965]: Deploying daemon osd.2 on smithi081 2024-03-22T07:15:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:12 smithi081 ceph-mon[113965]: pgmap v86: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:12.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:12 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 392.628700 secs 2024-03-22T07:15:13.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:12 smithi149 ceph-mon[83447]: Upgrade: Updating osd.2 2024-03-22T07:15:13.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:13.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:15:13.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:15:13.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:12 smithi149 ceph-mon[83447]: Deploying daemon osd.2 on smithi081 2024-03-22T07:15:13.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:12 smithi149 ceph-mon[83447]: pgmap v86: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:13.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:12 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 392.628700 secs 2024-03-22T07:15:13.184 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.183+0000 7f938ae49700 1 -- 172.21.15.81:0/1290457026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 msgr2=0x7f93840fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.183+0000 7f938ae49700 1 --2- 172.21.15.81:0/1290457026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 0x7f93840fb940 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9370004940 tx=0x7f9370009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.183+0000 7f938ae49700 1 -- 172.21.15.81:0/1290457026 shutdown_connections 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.183+0000 7f938ae49700 1 --2- 172.21.15.81:0/1290457026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 0x7f93840fb940 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.183+0000 7f938ae49700 1 --2- 172.21.15.81:0/1290457026 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9384102f50 0x7f9384103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.183+0000 7f938ae49700 1 -- 172.21.15.81:0/1290457026 >> 172.21.15.81:0/1290457026 conn(0x7f93840714c0 msgr2=0x7f93840718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.183+0000 7f938ae49700 1 -- 172.21.15.81:0/1290457026 shutdown_connections 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.183+0000 7f938ae49700 1 -- 172.21.15.81:0/1290457026 wait complete. 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f938ae49700 1 Processor -- start 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f938ae49700 1 -- start start 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f938ae49700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 0x7f93841139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f938ae49700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9384102f50 0x7f9384113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:13.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f938ae49700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384114530 con 0x7f93840fb4d0 2024-03-22T07:15:13.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f938ae49700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938410db50 con 0x7f9384102f50 2024-03-22T07:15:13.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f9388be5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 0x7f93841139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:13.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f9388be5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 0x7f93841139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43298/0 (socket says 172.21.15.81:43298) 2024-03-22T07:15:13.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f9388be5700 1 -- 172.21.15.81:0/2954813898 learned_addr learned my addr 172.21.15.81:0/2954813898 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:13.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.184+0000 7f9383fff700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9384102f50 0x7f9384113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:13.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.185+0000 7f9388be5700 1 -- 172.21.15.81:0/2954813898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9384102f50 msgr2=0x7f9384113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.185+0000 7f9388be5700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9384102f50 0x7f9384113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.185+0000 7f9388be5700 1 -- 172.21.15.81:0/2954813898 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370005040 con 0x7f93840fb4d0 2024-03-22T07:15:13.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.185+0000 7f9388be5700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 0x7f93841139a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9378002a80 tx=0x7f9378004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:13.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.185+0000 7f93817fa700 1 -- 172.21.15.81:0/2954813898 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9378007cc0 con 0x7f93840fb4d0 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.185+0000 7f93817fa700 1 -- 172.21.15.81:0/2954813898 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f937800f040 con 0x7f93840fb4d0 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.185+0000 7f93817fa700 1 -- 172.21.15.81:0/2954813898 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f937800da30 con 0x7f93840fb4d0 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.185+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938410de30 con 0x7f93840fb4d0 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.185+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938410e370 con 0x7f93840fb4d0 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.186+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9384105330 con 0x7f93840fb4d0 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.186+0000 7f93817fa700 1 -- 172.21.15.81:0/2954813898 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f937800db90 con 0x7f93840fb4d0 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.187+0000 7f93817fa700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f937406d380 0x7f937406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.187+0000 7f93817fa700 1 -- 172.21.15.81:0/2954813898 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f937808c570 con 0x7f93840fb4d0 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.187+0000 7f9383fff700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f937406d380 0x7f937406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:13.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.187+0000 7f9383fff700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f937406d380 0x7f937406f830 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f938410f240 tx=0x7f9370009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:13.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.192+0000 7f93817fa700 1 -- 172.21.15.81:0/2954813898 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f93780560d0 con 0x7f93840fb4d0 2024-03-22T07:15:13.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.413+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f938410a110 con 0x7f937406d380 2024-03-22T07:15:13.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.414+0000 7f93817fa700 1 -- 172.21.15.81:0/2954813898 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f938410a110 con 0x7f937406d380 2024-03-22T07:15:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f937406d380 msgr2=0x7f937406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f937406d380 0x7f937406f830 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f938410f240 tx=0x7f9370009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 msgr2=0x7f93841139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 0x7f93841139a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9378002a80 tx=0x7f9378004600 comp rx=0 tx=0).stop 2024-03-22T07:15:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 shutdown_connections 2024-03-22T07:15:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f937406d380 0x7f937406f830 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93840fb4d0 0x7f93841139a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 --2- 172.21.15.81:0/2954813898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9384102f50 0x7f9384113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 >> 172.21.15.81:0/2954813898 conn(0x7f93840714c0 msgr2=0x7f93840fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 shutdown_connections 2024-03-22T07:15:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.416+0000 7f938ae49700 1 -- 172.21.15.81:0/2954813898 wait complete. 2024-03-22T07:15:13.427 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:15:13.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 -- 172.21.15.81:0/2215168760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 msgr2=0x7f51481031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 --2- 172.21.15.81:0/2215168760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 0x7f51481031d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5144004940 tx=0x7f5144009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:13.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 -- 172.21.15.81:0/2215168760 shutdown_connections 2024-03-22T07:15:13.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 --2- 172.21.15.81:0/2215168760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51481037a0 0x7f51480fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 --2- 172.21.15.81:0/2215168760 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 0x7f51481031d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 -- 172.21.15.81:0/2215168760 >> 172.21.15.81:0/2215168760 conn(0x7f51480f89b0 msgr2=0x7f51480fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 -- 172.21.15.81:0/2215168760 shutdown_connections 2024-03-22T07:15:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 -- 172.21.15.81:0/2215168760 wait complete. 2024-03-22T07:15:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 Processor -- start 2024-03-22T07:15:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 -- start start 2024-03-22T07:15:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 0x7f5148113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:13.524 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51481037a0 0x7f5148113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:13.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148114520 con 0x7f5148102e00 2024-03-22T07:15:13.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.520+0000 7f514fa7f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514810db70 con 0x7f51481037a0 2024-03-22T07:15:13.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514d81b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 0x7f5148113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:13.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514d81b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 0x7f5148113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43316/0 (socket says 172.21.15.81:43316) 2024-03-22T07:15:13.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514d81b700 1 -- 172.21.15.81:0/702087530 learned_addr learned my addr 172.21.15.81:0/702087530 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:13.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514d01a700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51481037a0 0x7f5148113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:13.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514d81b700 1 -- 172.21.15.81:0/702087530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51481037a0 msgr2=0x7f5148113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514d81b700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51481037a0 0x7f5148113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514d81b700 1 -- 172.21.15.81:0/702087530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144005040 con 0x7f5148102e00 2024-03-22T07:15:13.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514d81b700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 0x7f5148113990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f514400b3b0 tx=0x7f5144009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:13.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f513e7fc700 1 -- 172.21.15.81:0/702087530 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f514400e040 con 0x7f5148102e00 2024-03-22T07:15:13.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f513e7fc700 1 -- 172.21.15.81:0/702087530 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f514400bbc0 con 0x7f5148102e00 2024-03-22T07:15:13.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514810ddf0 con 0x7f5148102e00 2024-03-22T07:15:13.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f513e7fc700 1 -- 172.21.15.81:0/702087530 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f514401c3f0 con 0x7f5148102e00 2024-03-22T07:15:13.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.521+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514810e3b0 con 0x7f5148102e00 2024-03-22T07:15:13.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.522+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5148100110 con 0x7f5148102e00 2024-03-22T07:15:13.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.523+0000 7f513e7fc700 1 -- 172.21.15.81:0/702087530 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f514400bd30 con 0x7f5148102e00 2024-03-22T07:15:13.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.523+0000 7f513e7fc700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f513406d380 0x7f513406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:13.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.523+0000 7f513e7fc700 1 -- 172.21.15.81:0/702087530 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f514408daa0 con 0x7f5148102e00 2024-03-22T07:15:13.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.523+0000 7f514d01a700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f513406d380 0x7f513406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:13.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.523+0000 7f514d01a700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f513406d380 0x7f513406f830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f514810ece0 tx=0x7f513800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:13.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.528+0000 7f513e7fc700 1 -- 172.21.15.81:0/702087530 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f514405b130 con 0x7f5148102e00 2024-03-22T07:15:13.730 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:13 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 392.969204 secs 2024-03-22T07:15:13.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.748+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f514810f050 con 0x7f513406d380 2024-03-22T07:15:13.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.749+0000 7f513e7fc700 1 -- 172.21.15.81:0/702087530 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f514810f050 con 0x7f513406d380 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f513406d380 msgr2=0x7f513406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f513406d380 0x7f513406f830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f514810ece0 tx=0x7f513800b040 comp rx=0 tx=0).stop 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 msgr2=0x7f5148113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 0x7f5148113990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f514400b3b0 tx=0x7f5144009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 shutdown_connections 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f513406d380 0x7f513406f830 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5148102e00 0x7f5148113990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 --2- 172.21.15.81:0/702087530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51481037a0 0x7f5148113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 >> 172.21.15.81:0/702087530 conn(0x7f51480f89b0 msgr2=0x7f51481073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 shutdown_connections 2024-03-22T07:15:13.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.751+0000 7f514fa7f700 1 -- 172.21.15.81:0/702087530 wait complete. 2024-03-22T07:15:13.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.856+0000 7f48deac5700 1 -- 172.21.15.81:0/3823626400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8102710 msgr2=0x7f48d80fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.856+0000 7f48deac5700 1 --2- 172.21.15.81:0/3823626400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8102710 0x7f48d80fbd80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f48c4004940 tx=0x7f48c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:13.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.856+0000 7f48deac5700 1 -- 172.21.15.81:0/3823626400 shutdown_connections 2024-03-22T07:15:13.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.856+0000 7f48deac5700 1 --2- 172.21.15.81:0/3823626400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8102710 0x7f48d80fbd80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.856+0000 7f48deac5700 1 --2- 172.21.15.81:0/3823626400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d8101d70 0x7f48d8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.856+0000 7f48deac5700 1 -- 172.21.15.81:0/3823626400 >> 172.21.15.81:0/3823626400 conn(0x7f48d8072bc0 msgr2=0x7f48d8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:13.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.856+0000 7f48deac5700 1 -- 172.21.15.81:0/3823626400 shutdown_connections 2024-03-22T07:15:13.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.856+0000 7f48deac5700 1 -- 172.21.15.81:0/3823626400 wait complete. 2024-03-22T07:15:13.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.856+0000 7f48deac5700 1 Processor -- start 2024-03-22T07:15:13.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48deac5700 1 -- start start 2024-03-22T07:15:13.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48deac5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8101d70 0x7f48d810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:13.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48deac5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d8102710 0x7f48d810f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:13.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48deac5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d810f6d0 con 0x7f48d8101d70 2024-03-22T07:15:13.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48deac5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d810f810 con 0x7f48d8102710 2024-03-22T07:15:13.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48dc861700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8101d70 0x7f48d810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:13.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48dc861700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8101d70 0x7f48d810eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43340/0 (socket says 172.21.15.81:43340) 2024-03-22T07:15:13.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48dc861700 1 -- 172.21.15.81:0/3784080967 learned_addr learned my addr 172.21.15.81:0/3784080967 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:13.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48d7fff700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d8102710 0x7f48d810f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:13.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48dc861700 1 -- 172.21.15.81:0/3784080967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d8102710 msgr2=0x7f48d810f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:13.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48dc861700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d8102710 0x7f48d810f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:13.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.857+0000 7f48dc861700 1 -- 172.21.15.81:0/3784080967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c4005040 con 0x7f48d8101d70 2024-03-22T07:15:13.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.858+0000 7f48dc861700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8101d70 0x7f48d810eb40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00c8c0 tx=0x7f48cc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:13.861 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.858+0000 7f48d57fa700 1 -- 172.21.15.81:0/3784080967 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48cc00a760 con 0x7f48d8101d70 2024-03-22T07:15:13.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.858+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d8117bb0 con 0x7f48d8101d70 2024-03-22T07:15:13.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.858+0000 7f48d57fa700 1 -- 172.21.15.81:0/3784080967 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48cc00ad90 con 0x7f48d8101d70 2024-03-22T07:15:13.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.858+0000 7f48d57fa700 1 -- 172.21.15.81:0/3784080967 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48cc0113a0 con 0x7f48d8101d70 2024-03-22T07:15:13.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.858+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d8118110 con 0x7f48d8101d70 2024-03-22T07:15:13.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.859+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d8104240 con 0x7f48d8101d70 2024-03-22T07:15:13.863 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.859+0000 7f48d57fa700 1 -- 172.21.15.81:0/3784080967 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f48cc012400 con 0x7f48d8101d70 2024-03-22T07:15:13.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.860+0000 7f48d57fa700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f48c806d380 0x7f48c806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:13.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.860+0000 7f48d57fa700 1 -- 172.21.15.81:0/3784080967 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f48cc08c810 con 0x7f48d8101d70 2024-03-22T07:15:13.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.860+0000 7f48d7fff700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f48c806d380 0x7f48c806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:13.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.860+0000 7f48d7fff700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f48c806d380 0x7f48c806f830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48d8066000 tx=0x7f48c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:13.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:13.864+0000 7f48d57fa700 1 -- 172.21.15.81:0/3784080967 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f48cc056410 con 0x7f48d8101d70 2024-03-22T07:15:14.077 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.077+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f48d804bed0 con 0x7f48c806d380 2024-03-22T07:15:14.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.087+0000 7f48d57fa700 1 -- 172.21.15.81:0/3784080967 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f48d804bed0 con 0x7f48c806d380 2024-03-22T07:15:14.087 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:15:14.087 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (3m) 23s ago 14m 26.0M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:15:14.087 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (92s) 23s ago 14m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:15:14.087 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (87s) 83s ago 13m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:15:14.087 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (3m) 23s ago 13m 80.6M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (8m) 23s ago 8m 13.6M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (8m) 23s ago 8m 131M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (8m) 83s ago 8m 16.8M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (8m) 83s ago 8m 147M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (5m) 23s ago 15m 501M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (4m) 83s ago 13m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (2m) 23s ago 15m 44.2M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (2m) 83s ago 13m 31.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (4m) 23s ago 14m 16.9M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (4m) 83s ago 13m 16.7M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (66s) 23s ago 12m 93.1M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 033c080308b6 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (29s) 23s ago 12m 15.1M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 8341fe32f3fa 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (11m) 23s ago 11m 162M 1557M 17.2.7-726-g9c4e0c34 837124da804f 594f8bd2800f 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (11m) 83s ago 11m 215M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:15:14.088 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (10m) 83s ago 10m 194M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:15:14.089 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (10m) 83s ago 10m 184M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:15:14.089 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (4m) 23s ago 13m 59.1M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:15:14.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f48c806d380 msgr2=0x7f48c806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f48c806d380 0x7f48c806f830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f48d8066000 tx=0x7f48c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:14.089 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8101d70 msgr2=0x7f48d810eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8101d70 0x7f48d810eb40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00c8c0 tx=0x7f48cc0044c0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 shutdown_connections 2024-03-22T07:15:14.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f48c806d380 0x7f48c806f830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48d8101d70 0x7f48d810eb40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 --2- 172.21.15.81:0/3784080967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48d8102710 0x7f48d810f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 >> 172.21.15.81:0/3784080967 conn(0x7f48d8072bc0 msgr2=0x7f48d80fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:14.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 shutdown_connections 2024-03-22T07:15:14.090 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.089+0000 7f48deac5700 1 -- 172.21.15.81:0/3784080967 wait complete. 2024-03-22T07:15:14.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:13 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 392.969204 secs 2024-03-22T07:15:14.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.188+0000 7fb8b3d52700 1 -- 172.21.15.81:0/2804358806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 msgr2=0x7fb8ac106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.188+0000 7fb8b3d52700 1 --2- 172.21.15.81:0/2804358806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 0x7fb8ac106ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb89c004970 tx=0x7fb89c009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:14.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 -- 172.21.15.81:0/2804358806 shutdown_connections 2024-03-22T07:15:14.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 --2- 172.21.15.81:0/2804358806 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 0x7fb8ac106ea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 --2- 172.21.15.81:0/2804358806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8ac0fde30 0x7fb8ac0fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 -- 172.21.15.81:0/2804358806 >> 172.21.15.81:0/2804358806 conn(0x7fb8ac0f7870 msgr2=0x7fb8ac0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:14.190 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 -- 172.21.15.81:0/2804358806 shutdown_connections 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 -- 172.21.15.81:0/2804358806 wait complete. 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 Processor -- start 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 -- start start 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8ac0fde30 0x7fb8ac113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 0x7fb8ac113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac1144e0 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.189+0000 7fb8b3d52700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac10dae0 con 0x7fb8ac0fde30 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b12ed700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 0x7fb8ac113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b12ed700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 0x7fb8ac113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43364/0 (socket says 172.21.15.81:43364) 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b12ed700 1 -- 172.21.15.81:0/146938406 learned_addr learned my addr 172.21.15.81:0/146938406 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b1aee700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8ac0fde30 0x7fb8ac113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b12ed700 1 -- 172.21.15.81:0/146938406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8ac0fde30 msgr2=0x7fb8ac113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b12ed700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8ac0fde30 0x7fb8ac113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b12ed700 1 -- 172.21.15.81:0/146938406 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c005040 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b12ed700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 0x7fb8ac113e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb89c005010 tx=0x7fb89c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8a27fc700 1 -- 172.21.15.81:0/146938406 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89c00e040 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8a27fc700 1 -- 172.21.15.81:0/146938406 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb89c009850 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8a27fc700 1 -- 172.21.15.81:0/146938406 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89c01c3f0 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ac10dd60 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.190+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac10e2a0 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.191+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ac101840 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.192+0000 7fb8a27fc700 1 -- 172.21.15.81:0/146938406 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fb89c013420 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.192+0000 7fb8a27fc700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb89806d380 0x7fb89806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.192+0000 7fb8a27fc700 1 -- 172.21.15.81:0/146938406 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb89c08da70 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.192+0000 7fb8b1aee700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb89806d380 0x7fb89806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:14.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.193+0000 7fb8b1aee700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb89806d380 0x7fb89806f830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009b80 tx=0x7fb8a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:14.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.197+0000 7fb8a27fc700 1 -- 172.21.15.81:0/146938406 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb89c05b100 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.483+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb8ac10efa0 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.484+0000 7fb8a27fc700 1 -- 172.21.15.81:0/146938406 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fb89c05ac90 con 0x7fb8ac0fe7d0 2024-03-22T07:15:14.484 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:15:14.484 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:15:14.484 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:15:14.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 8, 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:15:14.485 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:15:14.486 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:15:14.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb89806d380 msgr2=0x7fb89806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb89806d380 0x7fb89806f830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009b80 tx=0x7fb8a800b040 comp rx=0 tx=0).stop 2024-03-22T07:15:14.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 msgr2=0x7fb8ac113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 0x7fb8ac113e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb89c005010 tx=0x7fb89c009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:14.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 shutdown_connections 2024-03-22T07:15:14.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb89806d380 0x7fb89806f830 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8ac0fde30 0x7fb8ac113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 --2- 172.21.15.81:0/146938406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8ac0fe7d0 0x7fb8ac113e90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 >> 172.21.15.81:0/146938406 conn(0x7fb8ac0f7870 msgr2=0x7fb8ac0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:14.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 shutdown_connections 2024-03-22T07:15:14.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.486+0000 7fb8b3d52700 1 -- 172.21.15.81:0/146938406 wait complete. 2024-03-22T07:15:14.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.589+0000 7f32ee455700 1 -- 172.21.15.81:0/1180559542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fde20 msgr2=0x7f32e80fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.589+0000 7f32ee455700 1 --2- 172.21.15.81:0/1180559542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fde20 0x7f32e80fe1f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f32d8004940 tx=0x7f32d8009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:14.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.589+0000 7f32ee455700 1 -- 172.21.15.81:0/1180559542 shutdown_connections 2024-03-22T07:15:14.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.589+0000 7f32ee455700 1 --2- 172.21.15.81:0/1180559542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80fe7c0 0x7f32e8106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.589+0000 7f32ee455700 1 --2- 172.21.15.81:0/1180559542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fde20 0x7f32e80fe1f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.589+0000 7f32ee455700 1 -- 172.21.15.81:0/1180559542 >> 172.21.15.81:0/1180559542 conn(0x7f32e80f7870 msgr2=0x7f32e80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:14.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.589+0000 7f32ee455700 1 -- 172.21.15.81:0/1180559542 shutdown_connections 2024-03-22T07:15:14.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.589+0000 7f32ee455700 1 -- 172.21.15.81:0/1180559542 wait complete. 2024-03-22T07:15:14.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32ee455700 1 Processor -- start 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32ee455700 1 -- start start 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32ee455700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80fde20 0x7f32e8113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32ee455700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fe7c0 0x7f32e8113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32ee455700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e81144a0 con 0x7f32e80fe7c0 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32ee455700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e810daa0 con 0x7f32e80fde20 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32e77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fe7c0 0x7f32e8113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32e77fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fe7c0 0x7f32e8113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43380/0 (socket says 172.21.15.81:43380) 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32e77fe700 1 -- 172.21.15.81:0/3419052134 learned_addr learned my addr 172.21.15.81:0/3419052134 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.590+0000 7f32e7fff700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80fde20 0x7f32e8113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.591+0000 7f32e77fe700 1 -- 172.21.15.81:0/3419052134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80fde20 msgr2=0x7f32e8113910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.591+0000 7f32e77fe700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80fde20 0x7f32e8113910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.591+0000 7f32e77fe700 1 -- 172.21.15.81:0/3419052134 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d8005040 con 0x7f32e80fe7c0 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.591+0000 7f32e77fe700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fe7c0 0x7f32e8113e50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00eb70 tx=0x7f32dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:14.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.591+0000 7f32e4ff9700 1 -- 172.21.15.81:0/3419052134 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32dc007cc0 con 0x7f32e80fe7c0 2024-03-22T07:15:14.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.591+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e810dd20 con 0x7f32e80fe7c0 2024-03-22T07:15:14.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.591+0000 7f32e4ff9700 1 -- 172.21.15.81:0/3419052134 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32dc00f040 con 0x7f32e80fe7c0 2024-03-22T07:15:14.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.591+0000 7f32e4ff9700 1 -- 172.21.15.81:0/3419052134 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32dc00ca90 con 0x7f32e80fe7c0 2024-03-22T07:15:14.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.591+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e810e260 con 0x7f32e80fe7c0 2024-03-22T07:15:14.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.592+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e8066b50 con 0x7f32e80fe7c0 2024-03-22T07:15:14.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.592+0000 7f32e4ff9700 1 -- 172.21.15.81:0/3419052134 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f32dc00cbf0 con 0x7f32e80fe7c0 2024-03-22T07:15:14.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.593+0000 7f32e4ff9700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32d006d380 0x7f32d006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:14.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.593+0000 7f32e4ff9700 1 -- 172.21.15.81:0/3419052134 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f32dc08c340 con 0x7f32e80fe7c0 2024-03-22T07:15:14.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.593+0000 7f32e7fff700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32d006d380 0x7f32d006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:14.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.593+0000 7f32e7fff700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32d006d380 0x7f32d006f830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32d800b0d0 tx=0x7f32d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:14.597 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.597+0000 7f32e4ff9700 1 -- 172.21.15.81:0/3419052134 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32dc059a00 con 0x7f32e80fe7c0 2024-03-22T07:15:14.844 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:14 smithi081 ceph-mon[113965]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:14.844 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:14 smithi081 ceph-mon[113965]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:14.844 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:14 smithi081 ceph-mon[113965]: pgmap v87: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:14.844 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:14 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/146938406' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:14.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.843+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f32e810ef60 con 0x7f32e80fe7c0 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:e22 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 22 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:15:14.845 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:12:38.916315+0000 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:15:14.846 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 95 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 94 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:15:14.847 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:15:14.848 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:15:14.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.843+0000 7f32e4ff9700 1 -- 172.21.15.81:0/3419052134 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f32dc059590 con 0x7f32e80fe7c0 2024-03-22T07:15:14.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.846+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32d006d380 msgr2=0x7f32d006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.846+0000 7f32ee455700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32d006d380 0x7f32d006f830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f32d800b0d0 tx=0x7f32d8009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:14.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.846+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fe7c0 msgr2=0x7f32e8113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.846+0000 7f32ee455700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fe7c0 0x7f32e8113e50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00eb70 tx=0x7f32dc00b040 comp rx=0 tx=0).stop 2024-03-22T07:15:14.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.847+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 shutdown_connections 2024-03-22T07:15:14.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.847+0000 7f32ee455700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32d006d380 0x7f32d006f830 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.847+0000 7f32ee455700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32e80fde20 0x7f32e8113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.847+0000 7f32ee455700 1 --2- 172.21.15.81:0/3419052134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32e80fe7c0 0x7f32e8113e50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.847+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 >> 172.21.15.81:0/3419052134 conn(0x7f32e80f7870 msgr2=0x7f32e80f8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:14.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.847+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 shutdown_connections 2024-03-22T07:15:14.849 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.847+0000 7f32ee455700 1 -- 172.21.15.81:0/3419052134 wait complete. 2024-03-22T07:15:14.849 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 22 2024-03-22T07:15:14.958 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.956+0000 7fb50a72b700 1 -- 172.21.15.81:0/3386264123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 msgr2=0x7fb5040fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.956+0000 7fb50a72b700 1 --2- 172.21.15.81:0/3386264123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 0x7fb5040fc5b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4004940 tx=0x7fb4f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.956+0000 7fb50a72b700 1 -- 172.21.15.81:0/3386264123 shutdown_connections 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.956+0000 7fb50a72b700 1 --2- 172.21.15.81:0/3386264123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 0x7fb5040fc5b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.956+0000 7fb50a72b700 1 --2- 172.21.15.81:0/3386264123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb504101d50 0x7fb504102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.956+0000 7fb50a72b700 1 -- 172.21.15.81:0/3386264123 >> 172.21.15.81:0/3386264123 conn(0x7fb5040714c0 msgr2=0x7fb5040718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.956+0000 7fb50a72b700 1 -- 172.21.15.81:0/3386264123 shutdown_connections 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.956+0000 7fb50a72b700 1 -- 172.21.15.81:0/3386264123 wait complete. 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.957+0000 7fb50a72b700 1 Processor -- start 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.957+0000 7fb50a72b700 1 -- start start 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.957+0000 7fb50a72b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb504101d50 0x7fb504112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.957+0000 7fb50a72b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 0x7fb504113050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.957+0000 7fb50a72b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5041136a0 con 0x7fb504102660 2024-03-22T07:15:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.957+0000 7fb50a72b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50410ca90 con 0x7fb504101d50 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.957+0000 7fb5037fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 0x7fb504113050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.957+0000 7fb5037fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 0x7fb504113050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42848/0 (socket says 172.21.15.81:42848) 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.957+0000 7fb5037fe700 1 -- 172.21.15.81:0/3499609710 learned_addr learned my addr 172.21.15.81:0/3499609710 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb5037fe700 1 -- 172.21.15.81:0/3499609710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb504101d50 msgr2=0x7fb504112b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb503fff700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb504101d50 0x7fb504112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb5037fe700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb504101d50 0x7fb504112b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb5037fe700 1 -- 172.21.15.81:0/3499609710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4005040 con 0x7fb504102660 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb503fff700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb504101d50 0x7fb504112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb5037fe700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 0x7fb504113050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400b3b0 tx=0x7fb4f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb500ff9700 1 -- 172.21.15.81:0/3499609710 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4f400e040 con 0x7fb504102660 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50410cd10 con 0x7fb504102660 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb500ff9700 1 -- 172.21.15.81:0/3499609710 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4f400bbc0 con 0x7fb504102660 2024-03-22T07:15:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50410d250 con 0x7fb504102660 2024-03-22T07:15:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.958+0000 7fb500ff9700 1 -- 172.21.15.81:0/3499609710 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4f4025400 con 0x7fb504102660 2024-03-22T07:15:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.959+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5040fd5c0 con 0x7fb504102660 2024-03-22T07:15:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.960+0000 7fb500ff9700 1 -- 172.21.15.81:0/3499609710 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fb4f400bd30 con 0x7fb504102660 2024-03-22T07:15:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.960+0000 7fb500ff9700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb4ec06d380 0x7fb4ec06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.960+0000 7fb500ff9700 1 -- 172.21.15.81:0/3499609710 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb4f408dbd0 con 0x7fb504102660 2024-03-22T07:15:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.960+0000 7fb503fff700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb4ec06d380 0x7fb4ec06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.961+0000 7fb503fff700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb4ec06d380 0x7fb4ec06f830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009fd0 tx=0x7fb4f800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:14.965+0000 7fb500ff9700 1 -- 172.21.15.81:0/3499609710 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb4f405b1e0 con 0x7fb504102660 2024-03-22T07:15:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:14 smithi149 ceph-mon[83447]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:15.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:14 smithi149 ceph-mon[83447]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:15.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:14 smithi149 ceph-mon[83447]: pgmap v87: 65 pgs: 65 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:15.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:14 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/146938406' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:15.195 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:15 smithi081 systemd[1]: Stopping Ceph osd.2 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:15:15.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.194+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb504108e00 con 0x7fb4ec06d380 2024-03-22T07:15:15.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.195+0000 7fb500ff9700 1 -- 172.21.15.81:0/3499609710 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb504108e00 con 0x7fb4ec06d380 2024-03-22T07:15:15.196 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:15:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:15:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:15:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:15:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:15:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: "mon", 2024-03-22T07:15:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr", 2024-03-22T07:15:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: "crash" 2024-03-22T07:15:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:15:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "8/21 daemons upgraded", 2024-03-22T07:15:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:15:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:15:15.198 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:15:15.198 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb4ec06d380 msgr2=0x7fb4ec06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb4ec06d380 0x7fb4ec06f830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009fd0 tx=0x7fb4f800b040 comp rx=0 tx=0).stop 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 msgr2=0x7fb504113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 0x7fb504113050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400b3b0 tx=0x7fb4f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 shutdown_connections 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb4ec06d380 0x7fb4ec06f830 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb504101d50 0x7fb504112b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 --2- 172.21.15.81:0/3499609710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb504102660 0x7fb504113050 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 >> 172.21.15.81:0/3499609710 conn(0x7fb5040714c0 msgr2=0x7fb504104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 shutdown_connections 2024-03-22T07:15:15.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.198+0000 7fb50a72b700 1 -- 172.21.15.81:0/3499609710 wait complete. 2024-03-22T07:15:15.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.300+0000 7fd61068e700 1 -- 172.21.15.81:0/755763918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 msgr2=0x7fd6081020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.300+0000 7fd61068e700 1 --2- 172.21.15.81:0/755763918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 0x7fd6081020f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd604004940 tx=0x7fd604009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 -- 172.21.15.81:0/755763918 shutdown_connections 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 --2- 172.21.15.81:0/755763918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6081026c0 0x7fd6080fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 --2- 172.21.15.81:0/755763918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 0x7fd6081020f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 -- 172.21.15.81:0/755763918 >> 172.21.15.81:0/755763918 conn(0x7fd6080f7870 msgr2=0x7fd6080f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 -- 172.21.15.81:0/755763918 shutdown_connections 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 -- 172.21.15.81:0/755763918 wait complete. 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 Processor -- start 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 -- start start 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 0x7fd60810c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6081026c0 0x7fd60810ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd608067100 con 0x7fd608101d20 2024-03-22T07:15:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd61068e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd60810d3f0 con 0x7fd6081026c0 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd60e42a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 0x7fd60810c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd60e42a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 0x7fd60810c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42870/0 (socket says 172.21.15.81:42870) 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.301+0000 7fd60e42a700 1 -- 172.21.15.81:0/2165018616 learned_addr learned my addr 172.21.15.81:0/2165018616 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.302+0000 7fd60e42a700 1 -- 172.21.15.81:0/2165018616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6081026c0 msgr2=0x7fd60810ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.302+0000 7fd60e42a700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6081026c0 0x7fd60810ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.302+0000 7fd60e42a700 1 -- 172.21.15.81:0/2165018616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd604005040 con 0x7fd608101d20 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.302+0000 7fd60e42a700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 0x7fd60810c970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd60400b3b0 tx=0x7fd604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.302+0000 7fd5feffd700 1 -- 172.21.15.81:0/2165018616 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60400e040 con 0x7fd608101d20 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.302+0000 7fd5feffd700 1 -- 172.21.15.81:0/2165018616 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd60400bbc0 con 0x7fd608101d20 2024-03-22T07:15:15.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.302+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd60810d670 con 0x7fd608101d20 2024-03-22T07:15:15.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.302+0000 7fd5feffd700 1 -- 172.21.15.81:0/2165018616 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60401c3f0 con 0x7fd608101d20 2024-03-22T07:15:15.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.302+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60811a1a0 con 0x7fd608101d20 2024-03-22T07:15:15.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.303+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6080fd470 con 0x7fd608101d20 2024-03-22T07:15:15.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.304+0000 7fd5feffd700 1 -- 172.21.15.81:0/2165018616 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fd60400bd30 con 0x7fd608101d20 2024-03-22T07:15:15.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.304+0000 7fd5feffd700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd5f406d380 0x7fd5f406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:15.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.304+0000 7fd60dc29700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd5f406d380 0x7fd5f406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:15.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.304+0000 7fd5feffd700 1 -- 172.21.15.81:0/2165018616 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd60408da20 con 0x7fd608101d20 2024-03-22T07:15:15.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.305+0000 7fd60dc29700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd5f406d380 0x7fd5f406f830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd608066a60 tx=0x7fd5f8009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:15.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.309+0000 7fd5feffd700 1 -- 172.21.15.81:0/2165018616 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd60405b0b0 con 0x7fd608101d20 2024-03-22T07:15:15.569 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:15 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[62641]: 2024-03-22T07:15:15.431+0000 7faa83473700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:15:15.569 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:15 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[62641]: 2024-03-22T07:15:15.431+0000 7faa83473700 -1 osd.2 59 *** Got signal Terminated *** 2024-03-22T07:15:15.569 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:15 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[62641]: 2024-03-22T07:15:15.431+0000 7faa83473700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:15:15.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.589+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd608108eb0 con 0x7fd608101d20 2024-03-22T07:15:15.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.589+0000 7fd5feffd700 1 -- 172.21.15.81:0/2165018616 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+250 (secure 0 0 0) 0x7fd60405ac40 con 0x7fd608101d20 2024-03-22T07:15:15.590 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:15:15.590 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:15:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:15:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd5f406d380 msgr2=0x7fd5f406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd5f406d380 0x7fd5f406f830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd608066a60 tx=0x7fd5f8009ce0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 msgr2=0x7fd60810c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 0x7fd60810c970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd60400b3b0 tx=0x7fd604009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 shutdown_connections 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd5f406d380 0x7fd5f406f830 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd608101d20 0x7fd60810c970 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 --2- 172.21.15.81:0/2165018616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd6081026c0 0x7fd60810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 >> 172.21.15.81:0/2165018616 conn(0x7fd6080f7870 msgr2=0x7fd6080f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:15.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 shutdown_connections 2024-03-22T07:15:15.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:15.591+0000 7fd61068e700 1 -- 172.21.15.81:0/2165018616 wait complete. 2024-03-22T07:15:16.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:15 smithi081 ceph-mon[113965]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:15 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3419052134' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:15:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:15 smithi081 ceph-mon[113965]: osd.2 marked itself down and dead 2024-03-22T07:15:16.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:15 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2165018616' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:15:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:15 smithi149 ceph-mon[83447]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:15 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3419052134' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:15:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:15 smithi149 ceph-mon[83447]: osd.2 marked itself down and dead 2024-03-22T07:15:16.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:15 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2165018616' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:15:16.399 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:16 smithi081 podman[136682]: 2024-03-22 07:15:16.073500082 +0000 UTC m=+0.898020213 container died 594f8bd2800f0be87670ed96c3f4e13994c56b9870cd2b3f8cd2e784a0a1c971 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3) 2024-03-22T07:15:16.659 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:16 smithi081 podman[136682]: 2024-03-22 07:15:16.50720804 +0000 UTC m=+1.331728170 container cleanup 594f8bd2800f0be87670ed96c3f4e13994c56b9870cd2b3f8cd2e784a0a1c971 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-22T07:15:16.659 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:16 smithi081 bash[136682]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2 2024-03-22T07:15:17.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:16 smithi081 podman[136728]: 2024-03-22 07:15:16.708676862 +0000 UTC m=+0.634074910 container remove 594f8bd2800f0be87670ed96c3f4e13994c56b9870cd2b3f8cd2e784a0a1c971 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-03-22T07:15:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:15:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: osdmap e60: 6 total, 5 up, 6 in 2024-03-22T07:15:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: failed to encode map e60 with expected crc 2024-03-22T07:15:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: failed to encode map e60 with expected crc 2024-03-22T07:15:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: failed to encode map e60 with expected crc 2024-03-22T07:15:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: failed to encode map e60 with expected crc 2024-03-22T07:15:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: pgmap v89: 65 pgs: 9 stale+active+clean, 56 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:15:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: osdmap e60: 6 total, 5 up, 6 in 2024-03-22T07:15:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: failed to encode map e60 with expected crc 2024-03-22T07:15:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: failed to encode map e60 with expected crc 2024-03-22T07:15:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: failed to encode map e60 with expected crc 2024-03-22T07:15:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: failed to encode map e60 with expected crc 2024-03-22T07:15:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: pgmap v89: 65 pgs: 9 stale+active+clean, 56 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:17.421 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:17 smithi081 podman[136819]: 2024-03-22 07:15:17.029302615 +0000 UTC m=+0.023228303 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:15:17.421 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:17 smithi081 podman[136819]: 2024-03-22 07:15:17.155904847 +0000 UTC m=+0.149830524 container create 8131631ec0ee4e92d499eaa7e6395b11d6a6de2cf047c454f317c536cf6a88c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-22T07:15:17.421 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:17 smithi081 podman[136819]: 2024-03-22 07:15:17.421161045 +0000 UTC m=+0.415086732 container init 8131631ec0ee4e92d499eaa7e6395b11d6a6de2cf047c454f317c536cf6a88c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-03-22T07:15:17.720 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:17 smithi081 ceph-mon[113965]: osdmap e61: 6 total, 5 up, 6 in 2024-03-22T07:15:17.720 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:17 smithi081 ceph-mon[113965]: failed to encode map e61 with expected crc 2024-03-22T07:15:17.720 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:17 smithi081 ceph-mon[113965]: failed to encode map e61 with expected crc 2024-03-22T07:15:17.720 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:17 smithi081 ceph-mon[113965]: failed to encode map e61 with expected crc 2024-03-22T07:15:17.720 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:17 smithi081 ceph-mon[113965]: failed to encode map e61 with expected crc 2024-03-22T07:15:17.721 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:17 smithi081 ceph-mon[113965]: failed to encode map e61 with expected crc 2024-03-22T07:15:17.721 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:17 smithi081 ceph-mon[113965]: failed to encode map e61 with expected crc 2024-03-22T07:15:17.721 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:17 smithi081 ceph-mon[113965]: failed to encode map e61 with expected crc 2024-03-22T07:15:17.721 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:17 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 397.628784 secs 2024-03-22T07:15:17.721 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:17 smithi081 podman[136819]: 2024-03-22 07:15:17.424987939 +0000 UTC m=+0.418913618 container start 8131631ec0ee4e92d499eaa7e6395b11d6a6de2cf047c454f317c536cf6a88c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:15:17.721 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:17 smithi081 podman[136819]: 2024-03-22 07:15:17.546162422 +0000 UTC m=+0.540088121 container attach 8131631ec0ee4e92d499eaa7e6395b11d6a6de2cf047c454f317c536cf6a88c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:15:18.068 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:17 smithi081 podman[136819]: 2024-03-22 07:15:17.720615875 +0000 UTC m=+0.714541559 container died 8131631ec0ee4e92d499eaa7e6395b11d6a6de2cf047c454f317c536cf6a88c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-22T07:15:18.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:17 smithi149 ceph-mon[83447]: osdmap e61: 6 total, 5 up, 6 in 2024-03-22T07:15:18.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:17 smithi149 ceph-mon[83447]: failed to encode map e61 with expected crc 2024-03-22T07:15:18.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:17 smithi149 ceph-mon[83447]: failed to encode map e61 with expected crc 2024-03-22T07:15:18.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:17 smithi149 ceph-mon[83447]: failed to encode map e61 with expected crc 2024-03-22T07:15:18.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:17 smithi149 ceph-mon[83447]: failed to encode map e61 with expected crc 2024-03-22T07:15:18.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:17 smithi149 ceph-mon[83447]: failed to encode map e61 with expected crc 2024-03-22T07:15:18.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:17 smithi149 ceph-mon[83447]: failed to encode map e61 with expected crc 2024-03-22T07:15:18.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:17 smithi149 ceph-mon[83447]: failed to encode map e61 with expected crc 2024-03-22T07:15:18.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:17 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 397.628784 secs 2024-03-22T07:15:18.410 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:18 smithi081 podman[136845]: 2024-03-22 07:15:18.375690246 +0000 UTC m=+0.649886956 container remove 8131631ec0ee4e92d499eaa7e6395b11d6a6de2cf047c454f317c536cf6a88c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0) 2024-03-22T07:15:18.410 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:18 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-22T07:15:18.410 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:18 smithi081 systemd[1]: Stopped Ceph osd.2 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:15:18.410 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:18 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.2.service: Consumed 31.945s CPU time. 2024-03-22T07:15:18.667 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:18 smithi081 systemd[1]: Starting Ceph osd.2 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:15:18.968 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:18 smithi081 ceph-mon[113965]: pgmap v91: 65 pgs: 11 active+undersized+degraded, 7 peering, 3 stale+active+clean, 1 active+undersized, 43 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 2050/32856 objects degraded (6.239%) 2024-03-22T07:15:18.968 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:18 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 397.969326 secs 2024-03-22T07:15:18.968 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:18 smithi081 podman[136953]: 2024-03-22 07:15:18.732823081 +0000 UTC m=+0.019158954 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:15:18.968 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:18 smithi081 podman[136953]: 2024-03-22 07:15:18.843388054 +0000 UTC m=+0.129723920 container create 939402cfbd47e1a4faa67dbc552da13f075f8f1eb9c0ae96fd58d1648e050ad6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-03-22T07:15:19.003 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:18 smithi149 ceph-mon[83447]: pgmap v91: 65 pgs: 11 active+undersized+degraded, 7 peering, 3 stale+active+clean, 1 active+undersized, 43 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 2050/32856 objects degraded (6.239%) 2024-03-22T07:15:19.003 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:18 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 397.969326 secs 2024-03-22T07:15:19.319 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 podman[136953]: 2024-03-22 07:15:19.087700645 +0000 UTC m=+0.374036521 container init 939402cfbd47e1a4faa67dbc552da13f075f8f1eb9c0ae96fd58d1648e050ad6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-22T07:15:19.319 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 podman[136953]: 2024-03-22 07:15:19.091426724 +0000 UTC m=+0.377762593 container start 939402cfbd47e1a4faa67dbc552da13f075f8f1eb9c0ae96fd58d1648e050ad6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-22T07:15:19.319 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 podman[136953]: 2024-03-22 07:15:19.170951496 +0000 UTC m=+0.457287363 container attach 939402cfbd47e1a4faa67dbc552da13f075f8f1eb9c0ae96fd58d1648e050ad6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-22T07:15:19.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:19 smithi081 ceph-mon[113965]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-22T07:15:19.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:19 smithi081 ceph-mon[113965]: Health check failed: Degraded data redundancy: 2050/32856 objects degraded (6.239%), 11 pgs degraded (PG_DEGRADED) 2024-03-22T07:15:19.819 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate[136976]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T07:15:19.819 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate[136976]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T07:15:19.819 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 bash[136953]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T07:15:19.819 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 bash[136953]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 bash[136953]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 bash[136953]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 bash[136953]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 bash[136953]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 bash[136953]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate[136976]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate[136976]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate[136976]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate[136976]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate[136976]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-22T07:15:19.820 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:19 smithi081 podman[136953]: 2024-03-22 07:15:19.558379687 +0000 UTC m=+0.844715563 container died 939402cfbd47e1a4faa67dbc552da13f075f8f1eb9c0ae96fd58d1648e050ad6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-22T07:15:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:19 smithi149 ceph-mon[83447]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-03-22T07:15:20.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:19 smithi149 ceph-mon[83447]: Health check failed: Degraded data redundancy: 2050/32856 objects degraded (6.239%), 11 pgs degraded (PG_DEGRADED) 2024-03-22T07:15:20.569 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:20 smithi081 podman[137117]: 2024-03-22 07:15:20.292491324 +0000 UTC m=+0.728936392 container remove 939402cfbd47e1a4faa67dbc552da13f075f8f1eb9c0ae96fd58d1648e050ad6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-22T07:15:20.569 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:20 smithi081 podman[137185]: 2024-03-22 07:15:20.412713339 +0000 UTC m=+0.017795971 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:15:20.569 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:20 smithi081 podman[137185]: 2024-03-22 07:15:20.518230308 +0000 UTC m=+0.123312934 container create 0b8c514dced1e190bfbeaaf930ab6d853a291e717877eef6e79af60cfa22fadf (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-22T07:15:21.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:20 smithi081 ceph-mon[113965]: pgmap v92: 65 pgs: 19 active+undersized+degraded, 7 peering, 1 active+undersized, 38 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 3384/32856 objects degraded (10.299%) 2024-03-22T07:15:21.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:20 smithi081 podman[137185]: 2024-03-22 07:15:20.787892627 +0000 UTC m=+0.392975257 container init 0b8c514dced1e190bfbeaaf930ab6d853a291e717877eef6e79af60cfa22fadf (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-22T07:15:21.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:20 smithi081 podman[137185]: 2024-03-22 07:15:20.791211545 +0000 UTC m=+0.396294173 container start 0b8c514dced1e190bfbeaaf930ab6d853a291e717877eef6e79af60cfa22fadf (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-22T07:15:21.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:20 smithi081 ceph-osd[137204]: -- 172.21.15.81:0/1897883419 <== mon.0 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562183a691e0 con 0x562183a3b000 2024-03-22T07:15:21.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:20 smithi081 bash[137185]: 0b8c514dced1e190bfbeaaf930ab6d853a291e717877eef6e79af60cfa22fadf 2024-03-22T07:15:21.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:20 smithi081 systemd[1]: Started Ceph osd.2 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:15:21.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:20 smithi149 ceph-mon[83447]: pgmap v92: 65 pgs: 19 active+undersized+degraded, 7 peering, 1 active+undersized, 38 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 3384/32856 objects degraded (10.299%) 2024-03-22T07:15:21.742 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:21 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[137200]: 2024-03-22T07:15:21.342+0000 7fcc2eee67c0 -1 Falling back to public interface 2024-03-22T07:15:22.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:22.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:22.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:15:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:15:23.205 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:22 smithi081 ceph-mon[113965]: pgmap v93: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 4739/32856 objects degraded (14.424%) 2024-03-22T07:15:23.205 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:22 smithi081 ceph-mon[113965]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-22T07:15:23.205 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:22 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 402.628851 secs 2024-03-22T07:15:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:22 smithi149 ceph-mon[83447]: pgmap v93: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 4739/32856 objects degraded (14.424%) 2024-03-22T07:15:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:22 smithi149 ceph-mon[83447]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-03-22T07:15:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:22 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 402.628851 secs 2024-03-22T07:15:24.251 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:23 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 402.969460 secs 2024-03-22T07:15:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:23 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 402.969460 secs 2024-03-22T07:15:25.013 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:24 smithi081 ceph-mon[113965]: pgmap v94: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 4739/32856 objects degraded (14.424%) 2024-03-22T07:15:25.318 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:25 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[137200]: 2024-03-22T07:15:25.172+0000 7fcc2eee67c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-22T07:15:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:24 smithi149 ceph-mon[83447]: pgmap v94: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 4739/32856 objects degraded (14.424%) 2024-03-22T07:15:25.799 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:25 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[137200]: 2024-03-22T07:15:25.544+0000 7fcc2eee67c0 -1 osd.2 59 log_to_monitors true 2024-03-22T07:15:26.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:25 smithi081 ceph-mon[113965]: from='osd.2 [v2:172.21.15.81:6818/3743751332,v1:172.21.15.81:6819/3743751332]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T07:15:26.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:15:25 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[137200]: 2024-03-22T07:15:25.975+0000 7fcc2033e700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:15:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:25 smithi149 ceph-mon[83447]: from='osd.2 [v2:172.21.15.81:6818/3743751332,v1:172.21.15.81:6819/3743751332]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T07:15:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:26 smithi081 ceph-mon[113965]: pgmap v95: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 4739/32856 objects degraded (14.424%) 2024-03-22T07:15:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:26 smithi081 ceph-mon[113965]: from='osd.2 [v2:172.21.15.81:6818/3743751332,v1:172.21.15.81:6819/3743751332]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T07:15:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:26 smithi081 ceph-mon[113965]: osdmap e62: 6 total, 5 up, 6 in 2024-03-22T07:15:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:26 smithi081 ceph-mon[113965]: from='osd.2 [v2:172.21.15.81:6818/3743751332,v1:172.21.15.81:6819/3743751332]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:15:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:26 smithi081 ceph-mon[113965]: failed to encode map e62 with expected crc 2024-03-22T07:15:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:26 smithi149 ceph-mon[83447]: pgmap v95: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 882 MiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 4739/32856 objects degraded (14.424%) 2024-03-22T07:15:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:26 smithi149 ceph-mon[83447]: from='osd.2 [v2:172.21.15.81:6818/3743751332,v1:172.21.15.81:6819/3743751332]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T07:15:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:26 smithi149 ceph-mon[83447]: osdmap e62: 6 total, 5 up, 6 in 2024-03-22T07:15:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:26 smithi149 ceph-mon[83447]: from='osd.2 [v2:172.21.15.81:6818/3743751332,v1:172.21.15.81:6819/3743751332]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-03-22T07:15:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:26 smithi149 ceph-mon[83447]: failed to encode map e62 with expected crc 2024-03-22T07:15:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:28.219 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: failed to encode map e62 with expected crc 2024-03-22T07:15:28.219 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: failed to encode map e62 with expected crc 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: failed to encode map e63 with expected crc 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: osd.2 [v2:172.21.15.81:6818/3743751332,v1:172.21.15.81:6819/3743751332] boot 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: osdmap e63: 6 total, 6 up, 6 in 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: failed to encode map e63 with expected crc 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: failed to encode map e63 with expected crc 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: Health check update: Degraded data redundancy: 4739/32856 objects degraded (14.424%), 26 pgs degraded (PG_DEGRADED) 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 407.628922 secs 2024-03-22T07:15:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:27 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 407.969588 secs 2024-03-22T07:15:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: failed to encode map e62 with expected crc 2024-03-22T07:15:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: failed to encode map e62 with expected crc 2024-03-22T07:15:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:15:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: failed to encode map e63 with expected crc 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: osd.2 [v2:172.21.15.81:6818/3743751332,v1:172.21.15.81:6819/3743751332] boot 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: osdmap e63: 6 total, 6 up, 6 in 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: failed to encode map e63 with expected crc 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: failed to encode map e63 with expected crc 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: Health check update: Degraded data redundancy: 4739/32856 objects degraded (14.424%), 26 pgs degraded (PG_DEGRADED) 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 407.628922 secs 2024-03-22T07:15:28.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:27 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 407.969588 secs 2024-03-22T07:15:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:28 smithi081 ceph-mon[113965]: pgmap v98: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 4739/32856 objects degraded (14.424%) 2024-03-22T07:15:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:28 smithi081 ceph-mon[113965]: failed to encode map e64 with expected crc 2024-03-22T07:15:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:28 smithi081 ceph-mon[113965]: osdmap e64: 6 total, 6 up, 6 in 2024-03-22T07:15:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:28 smithi081 ceph-mon[113965]: failed to encode map e64 with expected crc 2024-03-22T07:15:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:28 smithi081 ceph-mon[113965]: failed to encode map e64 with expected crc 2024-03-22T07:15:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:28 smithi081 ceph-mon[113965]: failed to encode map e64 with expected crc 2024-03-22T07:15:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:28 smithi149 ceph-mon[83447]: pgmap v98: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 4739/32856 objects degraded (14.424%) 2024-03-22T07:15:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:28 smithi149 ceph-mon[83447]: failed to encode map e64 with expected crc 2024-03-22T07:15:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:28 smithi149 ceph-mon[83447]: osdmap e64: 6 total, 6 up, 6 in 2024-03-22T07:15:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:28 smithi149 ceph-mon[83447]: failed to encode map e64 with expected crc 2024-03-22T07:15:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:28 smithi149 ceph-mon[83447]: failed to encode map e64 with expected crc 2024-03-22T07:15:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:28 smithi149 ceph-mon[83447]: failed to encode map e64 with expected crc 2024-03-22T07:15:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:30 smithi081 ceph-mon[113965]: pgmap v100: 65 pgs: 21 active+undersized+degraded, 1 active+undersized, 43 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 3746/32856 objects degraded (11.401%) 2024-03-22T07:15:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:30 smithi149 ceph-mon[83447]: pgmap v100: 65 pgs: 21 active+undersized+degraded, 1 active+undersized, 43 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 3746/32856 objects degraded (11.401%) 2024-03-22T07:15:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:15:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:15:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:15:32.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:32.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:32.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:32.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:15:32.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:15:32.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-03-22T07:15:32.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:32 smithi081 ceph-mon[113965]: pgmap v101: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:15:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:15:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:15:32.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:32.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:32.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:32.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:15:32.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:15:32.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-03-22T07:15:32.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:32 smithi149 ceph-mon[83447]: pgmap v101: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:33.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:33 smithi081 ceph-mon[113965]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3746/32856 objects degraded (11.401%), 21 pgs degraded) 2024-03-22T07:15:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:33 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 412.629033 secs 2024-03-22T07:15:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:33 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 412.969719 secs 2024-03-22T07:15:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:33 smithi149 ceph-mon[83447]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3746/32856 objects degraded (11.401%), 21 pgs degraded) 2024-03-22T07:15:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:33 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 412.629033 secs 2024-03-22T07:15:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:33 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 412.969719 secs 2024-03-22T07:15:34.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:34 smithi149 ceph-mon[83447]: pgmap v102: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:34.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:34 smithi081 ceph-mon[113965]: pgmap v102: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:36 smithi081 ceph-mon[113965]: pgmap v103: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:37.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:36 smithi149 ceph-mon[83447]: pgmap v103: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:38.316 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:37 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 417.629144 secs 2024-03-22T07:15:38.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:37 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 417.629144 secs 2024-03-22T07:15:39.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:38 smithi081 ceph-mon[113965]: pgmap v104: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:39.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:38 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 417.969833 secs 2024-03-22T07:15:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:38 smithi149 ceph-mon[83447]: pgmap v104: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:38 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 417.969833 secs 2024-03-22T07:15:41.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:40 smithi081 ceph-mon[113965]: pgmap v105: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:41.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:40 smithi149 ceph-mon[83447]: pgmap v105: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:42.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:41 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:41 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:43.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:42 smithi081 ceph-mon[113965]: pgmap v106: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:43.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:42 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 422.629262 secs 2024-03-22T07:15:43.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:42 smithi149 ceph-mon[83447]: pgmap v106: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:43.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:42 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 422.629262 secs 2024-03-22T07:15:44.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:43 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 422.969947 secs 2024-03-22T07:15:44.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:43 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 422.969947 secs 2024-03-22T07:15:45.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:44 smithi081 ceph-mon[113965]: pgmap v107: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:44 smithi149 ceph-mon[83447]: pgmap v107: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:45.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.706+0000 7f4fae5f7700 1 -- 172.21.15.81:0/4051921326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 msgr2=0x7f4fa81020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:45.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.706+0000 7f4fae5f7700 1 --2- 172.21.15.81:0/4051921326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 0x7f4fa81020f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4f98004940 tx=0x7f4f98009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:45.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.706+0000 7f4fae5f7700 1 -- 172.21.15.81:0/4051921326 shutdown_connections 2024-03-22T07:15:45.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.706+0000 7f4fae5f7700 1 --2- 172.21.15.81:0/4051921326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa81026c0 0x7f4fa80fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:45.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.706+0000 7f4fae5f7700 1 --2- 172.21.15.81:0/4051921326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 0x7f4fa81020f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.706+0000 7f4fae5f7700 1 -- 172.21.15.81:0/4051921326 >> 172.21.15.81:0/4051921326 conn(0x7f4fa80f7870 msgr2=0x7f4fa80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.706+0000 7f4fae5f7700 1 -- 172.21.15.81:0/4051921326 shutdown_connections 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.706+0000 7f4fae5f7700 1 -- 172.21.15.81:0/4051921326 wait complete. 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.707+0000 7f4fae5f7700 1 Processor -- start 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.707+0000 7f4fae5f7700 1 -- start start 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.707+0000 7f4fae5f7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 0x7f4fa81128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.707+0000 7f4fae5f7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa81026c0 0x7f4fa8112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.707+0000 7f4fae5f7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8113460 con 0x7f4fa8101d20 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.707+0000 7f4fae5f7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa810ca60 con 0x7f4fa81026c0 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.707+0000 7f4fa7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 0x7f4fa81128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.707+0000 7f4fa7fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 0x7f4fa81128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59592/0 (socket says 172.21.15.81:59592) 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.707+0000 7f4fa7fff700 1 -- 172.21.15.81:0/993601514 learned_addr learned my addr 172.21.15.81:0/993601514 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:45.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fa77fe700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa81026c0 0x7f4fa8112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fa7fff700 1 -- 172.21.15.81:0/993601514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa81026c0 msgr2=0x7f4fa8112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fa7fff700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa81026c0 0x7f4fa8112e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fa7fff700 1 -- 172.21.15.81:0/993601514 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f98005040 con 0x7f4fa8101d20 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fa77fe700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa81026c0 0x7f4fa8112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fa7fff700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 0x7f4fa81128d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800b3b0 tx=0x7f4f98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fa4ff9700 1 -- 172.21.15.81:0/993601514 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f9800e040 con 0x7f4fa8101d20 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa810cce0 con 0x7f4fa8101d20 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fa4ff9700 1 -- 172.21.15.81:0/993601514 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f9800bbc0 con 0x7f4fa8101d20 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fa4ff9700 1 -- 172.21.15.81:0/993601514 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f9801c3f0 con 0x7f4fa8101d20 2024-03-22T07:15:45.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.708+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa810d2a0 con 0x7f4fa8101d20 2024-03-22T07:15:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.709+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa80fd470 con 0x7f4fa8101d20 2024-03-22T07:15:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.710+0000 7f4fa4ff9700 1 -- 172.21.15.81:0/993601514 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f4f98013420 con 0x7f4fa8101d20 2024-03-22T07:15:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.710+0000 7f4fa4ff9700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4f9006d380 0x7f4f9006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.710+0000 7f4fa4ff9700 1 -- 172.21.15.81:0/993601514 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4f9808dba0 con 0x7f4fa8101d20 2024-03-22T07:15:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.710+0000 7f4fa77fe700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4f9006d380 0x7f4f9006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:45.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.711+0000 7f4fa77fe700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4f9006d380 0x7f4f9006f830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009fd0 tx=0x7f4f9c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:45.715 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.715+0000 7f4fa4ff9700 1 -- 172.21.15.81:0/993601514 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f9805b1b0 con 0x7f4fa8101d20 2024-03-22T07:15:45.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.928+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fa80fd470 con 0x7f4f9006d380 2024-03-22T07:15:45.929 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.929+0000 7f4fa4ff9700 1 -- 172.21.15.81:0/993601514 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4fa80fd470 con 0x7f4f9006d380 2024-03-22T07:15:45.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4f9006d380 msgr2=0x7f4f9006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:45.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4f9006d380 0x7f4f9006f830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009fd0 tx=0x7f4f9c00b040 comp rx=0 tx=0).stop 2024-03-22T07:15:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 msgr2=0x7f4fa81128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 0x7f4fa81128d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800b3b0 tx=0x7f4f98009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 shutdown_connections 2024-03-22T07:15:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4f9006d380 0x7f4f9006f830 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fa8101d20 0x7f4fa81128d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 --2- 172.21.15.81:0/993601514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fa81026c0 0x7f4fa8112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 >> 172.21.15.81:0/993601514 conn(0x7f4fa80f7870 msgr2=0x7f4fa80f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 shutdown_connections 2024-03-22T07:15:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:45.931+0000 7f4fae5f7700 1 -- 172.21.15.81:0/993601514 wait complete. 2024-03-22T07:15:45.942 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:15:46.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.034+0000 7fbef9f0b700 1 -- 172.21.15.81:0/704595826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 msgr2=0x7fbef41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.034+0000 7fbef9f0b700 1 --2- 172.21.15.81:0/704595826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 0x7fbef41031f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbee4004940 tx=0x7fbee4009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:46.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.034+0000 7fbef9f0b700 1 -- 172.21.15.81:0/704595826 shutdown_connections 2024-03-22T07:15:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.034+0000 7fbef9f0b700 1 --2- 172.21.15.81:0/704595826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbef41037c0 0x7fbef40fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.034+0000 7fbef9f0b700 1 --2- 172.21.15.81:0/704595826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 0x7fbef41031f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.034+0000 7fbef9f0b700 1 -- 172.21.15.81:0/704595826 >> 172.21.15.81:0/704595826 conn(0x7fbef40f89d0 msgr2=0x7fbef40fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.034+0000 7fbef9f0b700 1 -- 172.21.15.81:0/704595826 shutdown_connections 2024-03-22T07:15:46.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.034+0000 7fbef9f0b700 1 -- 172.21.15.81:0/704595826 wait complete. 2024-03-22T07:15:46.036 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef9f0b700 1 Processor -- start 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef9f0b700 1 -- start start 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef9f0b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 0x7fbef40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef9f0b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbef41037c0 0x7fbef4070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef9f0b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4070e20 con 0x7fbef4102e20 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef9f0b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4070f60 con 0x7fbef41037c0 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 0x7fbef40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef37fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 0x7fbef40702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59600/0 (socket says 172.21.15.81:59600) 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef37fe700 1 -- 172.21.15.81:0/3362857062 learned_addr learned my addr 172.21.15.81:0/3362857062 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef2ffd700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbef41037c0 0x7fbef4070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef37fe700 1 -- 172.21.15.81:0/3362857062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbef41037c0 msgr2=0x7fbef4070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef37fe700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbef41037c0 0x7fbef4070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.035+0000 7fbef37fe700 1 -- 172.21.15.81:0/3362857062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee4005040 con 0x7fbef4102e20 2024-03-22T07:15:46.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.036+0000 7fbef37fe700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 0x7fbef40702c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009150 tx=0x7fbee4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:46.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.036+0000 7fbedbfff700 1 -- 172.21.15.81:0/3362857062 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee400e040 con 0x7fbef4102e20 2024-03-22T07:15:46.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.036+0000 7fbedbfff700 1 -- 172.21.15.81:0/3362857062 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbee4013420 con 0x7fbef4102e20 2024-03-22T07:15:46.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.036+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef4119ec0 con 0x7fbef4102e20 2024-03-22T07:15:46.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.036+0000 7fbedbfff700 1 -- 172.21.15.81:0/3362857062 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee401c610 con 0x7fbef4102e20 2024-03-22T07:15:46.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.036+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef411a3a0 con 0x7fbef4102e20 2024-03-22T07:15:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.037+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef40fe570 con 0x7fbef4102e20 2024-03-22T07:15:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.038+0000 7fbedbfff700 1 -- 172.21.15.81:0/3362857062 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbee400baf0 con 0x7fbef4102e20 2024-03-22T07:15:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.038+0000 7fbedbfff700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbedc06d390 0x7fbedc06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.038+0000 7fbedbfff700 1 -- 172.21.15.81:0/3362857062 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbee408dab0 con 0x7fbef4102e20 2024-03-22T07:15:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.038+0000 7fbef2ffd700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbedc06d390 0x7fbedc06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:46.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.039+0000 7fbef2ffd700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbedc06d390 0x7fbedc06f840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbef4071c00 tx=0x7fbee80091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:46.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.043+0000 7fbedbfff700 1 -- 172.21.15.81:0/3362857062 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbee4057610 con 0x7fbef4102e20 2024-03-22T07:15:46.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.257+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbef404bf70 con 0x7fbedc06d390 2024-03-22T07:15:46.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.258+0000 7fbedbfff700 1 -- 172.21.15.81:0/3362857062 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbee800e6b0 con 0x7fbedc06d390 2024-03-22T07:15:46.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.260+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbedc06d390 msgr2=0x7fbedc06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.260+0000 7fbef9f0b700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbedc06d390 0x7fbedc06f840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbef4071c00 tx=0x7fbee80091a0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.261+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 msgr2=0x7fbef40702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.261+0000 7fbef9f0b700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 0x7fbef40702c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009150 tx=0x7fbee4009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:46.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.261+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 shutdown_connections 2024-03-22T07:15:46.261 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.261+0000 7fbef9f0b700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbedc06d390 0x7fbedc06f840 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.261+0000 7fbef9f0b700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbef4102e20 0x7fbef40702c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.261+0000 7fbef9f0b700 1 --2- 172.21.15.81:0/3362857062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbef41037c0 0x7fbef4070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.261+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 >> 172.21.15.81:0/3362857062 conn(0x7fbef40f89d0 msgr2=0x7fbef40f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:46.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.261+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 shutdown_connections 2024-03-22T07:15:46.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.261+0000 7fbef9f0b700 1 -- 172.21.15.81:0/3362857062 wait complete. 2024-03-22T07:15:46.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.364+0000 7f344473d700 1 -- 172.21.15.81:0/3362263705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 msgr2=0x7f343c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.364+0000 7f344473d700 1 --2- 172.21.15.81:0/3362263705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 0x7f343c0fbd80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f342c004940 tx=0x7f342c009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:46.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.364+0000 7f344473d700 1 -- 172.21.15.81:0/3362263705 shutdown_connections 2024-03-22T07:15:46.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.364+0000 7f344473d700 1 --2- 172.21.15.81:0/3362263705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 0x7f343c0fbd80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.364+0000 7f344473d700 1 --2- 172.21.15.81:0/3362263705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c101d70 0x7f343c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.364+0000 7f344473d700 1 -- 172.21.15.81:0/3362263705 >> 172.21.15.81:0/3362263705 conn(0x7f343c072bc0 msgr2=0x7f343c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:46.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.364+0000 7f344473d700 1 -- 172.21.15.81:0/3362263705 shutdown_connections 2024-03-22T07:15:46.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.364+0000 7f344473d700 1 -- 172.21.15.81:0/3362263705 wait complete. 2024-03-22T07:15:46.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f344473d700 1 Processor -- start 2024-03-22T07:15:46.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f344473d700 1 -- start start 2024-03-22T07:15:46.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f344473d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c101d70 0x7f343c112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:46.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f344473d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 0x7f343c112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:46.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f344473d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c113630 con 0x7f343c102710 2024-03-22T07:15:46.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f344473d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c10ca20 con 0x7f343c101d70 2024-03-22T07:15:46.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f3441cd8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 0x7f343c112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:46.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f3441cd8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 0x7f343c112fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59624/0 (socket says 172.21.15.81:59624) 2024-03-22T07:15:46.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f3441cd8700 1 -- 172.21.15.81:0/3982240398 learned_addr learned my addr 172.21.15.81:0/3982240398 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:46.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f34424d9700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c101d70 0x7f343c112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:46.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f3441cd8700 1 -- 172.21.15.81:0/3982240398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c101d70 msgr2=0x7f343c112aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f3441cd8700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c101d70 0x7f343c112aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.372 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.365+0000 7f3441cd8700 1 -- 172.21.15.81:0/3982240398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c005040 con 0x7f343c102710 2024-03-22T07:15:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.366+0000 7f3441cd8700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 0x7f343c112fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f342c00b3b0 tx=0x7f342c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.366+0000 7f3432ffd700 1 -- 172.21.15.81:0/3982240398 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342c00e040 con 0x7f343c102710 2024-03-22T07:15:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.366+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c10cca0 con 0x7f343c102710 2024-03-22T07:15:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.366+0000 7f3432ffd700 1 -- 172.21.15.81:0/3982240398 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f342c00bd00 con 0x7f343c102710 2024-03-22T07:15:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.366+0000 7f3432ffd700 1 -- 172.21.15.81:0/3982240398 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342c013660 con 0x7f343c102710 2024-03-22T07:15:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.366+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c10d200 con 0x7f343c102710 2024-03-22T07:15:46.373 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.367+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c0fd4c0 con 0x7f343c102710 2024-03-22T07:15:46.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.367+0000 7f3432ffd700 1 -- 172.21.15.81:0/3982240398 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f342c01e420 con 0x7f343c102710 2024-03-22T07:15:46.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.367+0000 7f3432ffd700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f342806d380 0x7f342806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:46.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.368+0000 7f3432ffd700 1 -- 172.21.15.81:0/3982240398 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f342c0907d0 con 0x7f343c102710 2024-03-22T07:15:46.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.368+0000 7f34424d9700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f342806d380 0x7f342806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:46.374 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.368+0000 7f34424d9700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f342806d380 0x7f342806f830 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3434004f40 tx=0x7f343400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:46.375 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.372+0000 7f3432ffd700 1 -- 172.21.15.81:0/3982240398 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f342c05dde0 con 0x7f343c102710 2024-03-22T07:15:46.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.575+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f343c112770 con 0x7f342806d380 2024-03-22T07:15:46.586 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.586+0000 7f3432ffd700 1 -- 172.21.15.81:0/3982240398 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f343c112770 con 0x7f342806d380 2024-03-22T07:15:46.586 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:15:46.586 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (4m) 20s ago 14m 26.0M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:15:46.586 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (2m) 20s ago 14m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:15:46.586 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (119s) 116s ago 13m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:15:46.587 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (4m) 20s ago 14m 84.5M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:15:46.587 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (9m) 20s ago 9m 13.7M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:15:46.587 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (9m) 20s ago 9m 131M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:15:46.587 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (9m) 116s ago 9m 16.8M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:15:46.587 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (9m) 116s ago 9m 147M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:15:46.587 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (5m) 20s ago 16m 502M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:15:46.587 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (5m) 116s ago 13m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:15:46.587 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (3m) 20s ago 16m 44.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:15:46.587 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (2m) 116s ago 13m 31.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:15:46.588 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (5m) 20s ago 14m 16.2M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:15:46.588 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (4m) 116s ago 13m 16.7M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:15:46.588 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (98s) 20s ago 12m 94.5M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 033c080308b6 2024-03-22T07:15:46.588 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (62s) 20s ago 12m 105M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 8341fe32f3fa 2024-03-22T07:15:46.588 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (26s) 20s ago 12m 12.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 0b8c514dced1 2024-03-22T07:15:46.588 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (11m) 116s ago 11m 215M 4096M 17.2.7-726-g9c4e0c34 837124da804f f6fa79d4d665 2024-03-22T07:15:46.588 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (11m) 116s ago 11m 194M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:15:46.589 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (10m) 116s ago 10m 184M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:15:46.589 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (4m) 20s ago 14m 59.1M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:15:46.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f342806d380 msgr2=0x7f342806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f342806d380 0x7f342806f830 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3434004f40 tx=0x7f343400b040 comp rx=0 tx=0).stop 2024-03-22T07:15:46.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 msgr2=0x7f343c112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 0x7f343c112fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f342c00b3b0 tx=0x7f342c009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:46.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 shutdown_connections 2024-03-22T07:15:46.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f342806d380 0x7f342806f830 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f343c101d70 0x7f343c112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 --2- 172.21.15.81:0/3982240398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f343c102710 0x7f343c112fe0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 >> 172.21.15.81:0/3982240398 conn(0x7f343c072bc0 msgr2=0x7f343c0fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:46.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 shutdown_connections 2024-03-22T07:15:46.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.588+0000 7f344473d700 1 -- 172.21.15.81:0/3982240398 wait complete. 2024-03-22T07:15:46.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.688+0000 7f5c674dc700 1 -- 172.21.15.81:0/499355375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 msgr2=0x7f5c600fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.688+0000 7f5c674dc700 1 --2- 172.21.15.81:0/499355375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 0x7f5c600fbd80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5c54004940 tx=0x7f5c54009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:46.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.688+0000 7f5c674dc700 1 -- 172.21.15.81:0/499355375 shutdown_connections 2024-03-22T07:15:46.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.688+0000 7f5c674dc700 1 --2- 172.21.15.81:0/499355375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 0x7f5c600fbd80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.688+0000 7f5c674dc700 1 --2- 172.21.15.81:0/499355375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c60101d70 0x7f5c60102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.688+0000 7f5c674dc700 1 -- 172.21.15.81:0/499355375 >> 172.21.15.81:0/499355375 conn(0x7f5c60072bc0 msgr2=0x7f5c60072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.688+0000 7f5c674dc700 1 -- 172.21.15.81:0/499355375 shutdown_connections 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.688+0000 7f5c674dc700 1 -- 172.21.15.81:0/499355375 wait complete. 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c674dc700 1 Processor -- start 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c674dc700 1 -- start start 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c674dc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c60101d70 0x7f5c6010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c674dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 0x7f5c6010ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c674dc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6010d3b0 con 0x7f5c60102710 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c674dc700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6010d520 con 0x7f5c60101d70 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c64a77700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 0x7f5c6010ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c64a77700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 0x7f5c6010ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59650/0 (socket says 172.21.15.81:59650) 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c64a77700 1 -- 172.21.15.81:0/1085522912 learned_addr learned my addr 172.21.15.81:0/1085522912 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:46.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c65278700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c60101d70 0x7f5c6010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:46.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c64a77700 1 -- 172.21.15.81:0/1085522912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c60101d70 msgr2=0x7f5c6010c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c64a77700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c60101d70 0x7f5c6010c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.689+0000 7f5c64a77700 1 -- 172.21.15.81:0/1085522912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c54005040 con 0x7f5c60102710 2024-03-22T07:15:46.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.690+0000 7f5c64a77700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 0x7f5c6010ce70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009100 tx=0x7f5c54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:46.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.690+0000 7f5c5dffb700 1 -- 172.21.15.81:0/1085522912 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c5400e040 con 0x7f5c60102710 2024-03-22T07:15:46.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.690+0000 7f5c5dffb700 1 -- 172.21.15.81:0/1085522912 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c54009af0 con 0x7f5c60102710 2024-03-22T07:15:46.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.690+0000 7f5c5dffb700 1 -- 172.21.15.81:0/1085522912 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c5401c3f0 con 0x7f5c60102710 2024-03-22T07:15:46.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.690+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6010d7a0 con 0x7f5c60102710 2024-03-22T07:15:46.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.690+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6011a320 con 0x7f5c60102710 2024-03-22T07:15:46.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.691+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c60104240 con 0x7f5c60102710 2024-03-22T07:15:46.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.692+0000 7f5c5dffb700 1 -- 172.21.15.81:0/1085522912 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f5c5400baf0 con 0x7f5c60102710 2024-03-22T07:15:46.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.692+0000 7f5c5dffb700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5c4806d380 0x7f5c4806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:46.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.692+0000 7f5c5dffb700 1 -- 172.21.15.81:0/1085522912 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5c5408d7e0 con 0x7f5c60102710 2024-03-22T07:15:46.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.692+0000 7f5c65278700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5c4806d380 0x7f5c4806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:46.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.693+0000 7f5c65278700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5c4806d380 0x7f5c4806f830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c58007970 tx=0x7f5c58009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:46.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.697+0000 7f5c5dffb700 1 -- 172.21.15.81:0/1085522912 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c5405af10 con 0x7f5c60102710 2024-03-22T07:15:46.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.974+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5c600fd4c0 con 0x7f5c60102710 2024-03-22T07:15:46.975 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:46 smithi081 ceph-mon[113965]: pgmap v108: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:46.975 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:46.975 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:46.975 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:46 smithi081 ceph-mon[113965]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:46.975 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:15:46.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.974+0000 7f5c5dffb700 1 -- 172.21.15.81:0/1085522912 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f5c5405aaa0 con 0x7f5c60102710 2024-03-22T07:15:46.976 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:15:46.977 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:15:46.977 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:15:46.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:15:46.977 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:15:46.977 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:15:46.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:15:46.977 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:15:46.978 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-22T07:15:46.978 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-22T07:15:46.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:15:46.978 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:15:46.978 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:15:46.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:15:46.979 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:15:46.979 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 7, 2024-03-22T07:15:46.979 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 7 2024-03-22T07:15:46.979 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:15:46.979 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:15:46.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.976+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5c4806d380 msgr2=0x7f5c4806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.976+0000 7f5c674dc700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5c4806d380 0x7f5c4806f830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c58007970 tx=0x7f5c58009da0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.976+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 msgr2=0x7f5c6010ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:46.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.976+0000 7f5c674dc700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 0x7f5c6010ce70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009100 tx=0x7f5c54009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:46.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.977+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 shutdown_connections 2024-03-22T07:15:46.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.977+0000 7f5c674dc700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5c4806d380 0x7f5c4806f830 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.977+0000 7f5c674dc700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c60101d70 0x7f5c6010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.977+0000 7f5c674dc700 1 --2- 172.21.15.81:0/1085522912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c60102710 0x7f5c6010ce70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:46.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.977+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 >> 172.21.15.81:0/1085522912 conn(0x7f5c60072bc0 msgr2=0x7f5c600fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:46.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.977+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 shutdown_connections 2024-03-22T07:15:46.981 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:46.977+0000 7f5c674dc700 1 -- 172.21.15.81:0/1085522912 wait complete. 2024-03-22T07:15:47.063 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:46 smithi149 ceph-mon[83447]: pgmap v108: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:47.063 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:47.063 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:47.063 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:46 smithi149 ceph-mon[83447]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:47.064 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.078+0000 7f4b9034c700 1 -- 172.21.15.81:0/2127521345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b880fb3b0 msgr2=0x7f4b880fb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.078+0000 7f4b9034c700 1 --2- 172.21.15.81:0/2127521345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b880fb3b0 0x7f4b880fb820 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b78004940 tx=0x7f4b78009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.078+0000 7f4b9034c700 1 -- 172.21.15.81:0/2127521345 shutdown_connections 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 --2- 172.21.15.81:0/2127521345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b880fb3b0 0x7f4b880fb820 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 --2- 172.21.15.81:0/2127521345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b88102f70 0x7f4b88103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 -- 172.21.15.81:0/2127521345 >> 172.21.15.81:0/2127521345 conn(0x7f4b88072310 msgr2=0x7f4b88072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 -- 172.21.15.81:0/2127521345 shutdown_connections 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 -- 172.21.15.81:0/2127521345 wait complete. 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 Processor -- start 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 -- start start 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b880fb3b0 0x7f4b88113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:47.081 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b88102f70 0x7f4b88113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b881144f0 con 0x7f4b88102f70 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b9034c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8810daf0 con 0x7f4b880fb3b0 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b8d8e7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b88102f70 0x7f4b88113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b8d8e7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b88102f70 0x7f4b88113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59680/0 (socket says 172.21.15.81:59680) 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b8e0e8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b880fb3b0 0x7f4b88113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.079+0000 7f4b8d8e7700 1 -- 172.21.15.81:0/3982132190 learned_addr learned my addr 172.21.15.81:0/3982132190 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.080+0000 7f4b8d8e7700 1 -- 172.21.15.81:0/3982132190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b880fb3b0 msgr2=0x7f4b88113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.080+0000 7f4b8d8e7700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b880fb3b0 0x7f4b88113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.080+0000 7f4b8d8e7700 1 -- 172.21.15.81:0/3982132190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b78005040 con 0x7f4b88102f70 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.080+0000 7f4b8d8e7700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b88102f70 0x7f4b88113ea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800b3b0 tx=0x7f4b78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.080+0000 7f4b7effd700 1 -- 172.21.15.81:0/3982132190 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b7800e040 con 0x7f4b88102f70 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.080+0000 7f4b7effd700 1 -- 172.21.15.81:0/3982132190 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b78013420 con 0x7f4b88102f70 2024-03-22T07:15:47.082 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.080+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8810dd70 con 0x7f4b88102f70 2024-03-22T07:15:47.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.080+0000 7f4b7effd700 1 -- 172.21.15.81:0/3982132190 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b7801c630 con 0x7f4b88102f70 2024-03-22T07:15:47.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.080+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8810e330 con 0x7f4b88102f70 2024-03-22T07:15:47.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.081+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b88105350 con 0x7f4b88102f70 2024-03-22T07:15:47.083 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.082+0000 7f4b7effd700 1 -- 172.21.15.81:0/3982132190 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f4b78013590 con 0x7f4b88102f70 2024-03-22T07:15:47.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.082+0000 7f4b7effd700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b7406d380 0x7f4b7406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:47.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.082+0000 7f4b7effd700 1 -- 172.21.15.81:0/3982132190 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4b7808dac0 con 0x7f4b88102f70 2024-03-22T07:15:47.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.082+0000 7f4b8e0e8700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b7406d380 0x7f4b7406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:47.084 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.082+0000 7f4b8e0e8700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b7406d380 0x7f4b7406f830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b84004f40 tx=0x7f4b8400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:47.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.086+0000 7f4b7effd700 1 -- 172.21.15.81:0/3982132190 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b7805b150 con 0x7f4b88102f70 2024-03-22T07:15:47.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.326+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b8810efb0 con 0x7f4b88102f70 2024-03-22T07:15:47.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.326+0000 7f4b7effd700 1 -- 172.21.15.81:0/3982132190 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f4b7805ace0 con 0x7f4b88102f70 2024-03-22T07:15:47.327 INFO:teuthology.orchestra.run.smithi081.stdout:e22 2024-03-22T07:15:47.327 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:15:47.327 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:15:47.327 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:15:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:15:47.327 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 22 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:12:38.916315+0000 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:15:47.328 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:15:47.329 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 95 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 94 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:15:47.330 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:15:47.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b7406d380 msgr2=0x7f4b7406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b7406d380 0x7f4b7406f830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4b84004f40 tx=0x7f4b8400b040 comp rx=0 tx=0).stop 2024-03-22T07:15:47.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b88102f70 msgr2=0x7f4b88113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b88102f70 0x7f4b88113ea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800b3b0 tx=0x7f4b78009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:47.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 shutdown_connections 2024-03-22T07:15:47.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b7406d380 0x7f4b7406f830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b880fb3b0 0x7f4b88113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 --2- 172.21.15.81:0/3982132190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b88102f70 0x7f4b88113ea0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 >> 172.21.15.81:0/3982132190 conn(0x7f4b88072310 msgr2=0x7f4b880f9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:47.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 shutdown_connections 2024-03-22T07:15:47.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.329+0000 7f4b9034c700 1 -- 172.21.15.81:0/3982132190 wait complete. 2024-03-22T07:15:47.332 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 22 2024-03-22T07:15:47.432 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.430+0000 7f50ac9d4700 1 -- 172.21.15.81:0/4224736908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a8102e00 msgr2=0x7f50a81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.430+0000 7f50ac9d4700 1 --2- 172.21.15.81:0/4224736908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a8102e00 0x7f50a81031d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5098004940 tx=0x7f5098009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 -- 172.21.15.81:0/4224736908 shutdown_connections 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 --2- 172.21.15.81:0/4224736908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a81037a0 0x7f50a80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 --2- 172.21.15.81:0/4224736908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a8102e00 0x7f50a81031d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 -- 172.21.15.81:0/4224736908 >> 172.21.15.81:0/4224736908 conn(0x7f50a80f89b0 msgr2=0x7f50a80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 -- 172.21.15.81:0/4224736908 shutdown_connections 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 -- 172.21.15.81:0/4224736908 wait complete. 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 Processor -- start 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 -- start start 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a8102e00 0x7f50a8113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a81037a0 0x7f50a8113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a8114510 con 0x7f50a81037a0 2024-03-22T07:15:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50ac9d4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a810db30 con 0x7f50a8102e00 2024-03-22T07:15:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50a5d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a81037a0 0x7f50a8113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50a5d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a81037a0 0x7f50a8113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59694/0 (socket says 172.21.15.81:59694) 2024-03-22T07:15:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50a5d9b700 1 -- 172.21.15.81:0/3733111572 learned_addr learned my addr 172.21.15.81:0/3733111572 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.431+0000 7f50a659c700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a8102e00 0x7f50a8113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:47.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.432+0000 7f50a5d9b700 1 -- 172.21.15.81:0/3733111572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a8102e00 msgr2=0x7f50a8113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.432+0000 7f50a5d9b700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a8102e00 0x7f50a8113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.432+0000 7f50a5d9b700 1 -- 172.21.15.81:0/3733111572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098005040 con 0x7f50a81037a0 2024-03-22T07:15:47.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.432+0000 7f50a5d9b700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a81037a0 0x7f50a8113ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f509c002a80 tx=0x7f509c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:47.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.432+0000 7f5096ffd700 1 -- 172.21.15.81:0/3733111572 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f509c004e80 con 0x7f50a81037a0 2024-03-22T07:15:47.435 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.432+0000 7f5096ffd700 1 -- 172.21.15.81:0/3733111572 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f509c00f040 con 0x7f50a81037a0 2024-03-22T07:15:47.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.432+0000 7f5096ffd700 1 -- 172.21.15.81:0/3733111572 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f509c00daa0 con 0x7f50a81037a0 2024-03-22T07:15:47.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.432+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a810de10 con 0x7f50a81037a0 2024-03-22T07:15:47.436 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.432+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a810e3d0 con 0x7f50a81037a0 2024-03-22T07:15:47.437 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.433+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a8100110 con 0x7f50a81037a0 2024-03-22T07:15:47.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.434+0000 7f5096ffd700 1 -- 172.21.15.81:0/3733111572 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f509c007cc0 con 0x7f50a81037a0 2024-03-22T07:15:47.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.434+0000 7f5096ffd700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f509006d380 0x7f509006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:47.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.434+0000 7f5096ffd700 1 -- 172.21.15.81:0/3733111572 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f509c08bf10 con 0x7f50a81037a0 2024-03-22T07:15:47.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.434+0000 7f50a659c700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f509006d380 0x7f509006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:47.438 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.434+0000 7f50a659c700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f509006d380 0x7f509006f830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5098004f90 tx=0x7f5098009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:47.439 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.439+0000 7f5096ffd700 1 -- 172.21.15.81:0/3733111572 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f509c0559f0 con 0x7f50a81037a0 2024-03-22T07:15:47.651 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.651+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50a810efd0 con 0x7f509006d380 2024-03-22T07:15:47.652 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.652+0000 7f5096ffd700 1 -- 172.21.15.81:0/3733111572 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f50a810efd0 con 0x7f509006d380 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: "mon", 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr", 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: "crash" 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "9/21 daemons upgraded", 2024-03-22T07:15:47.653 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:15:47.654 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:15:47.654 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.654+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f509006d380 msgr2=0x7f509006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.654+0000 7f50ac9d4700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f509006d380 0x7f509006f830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5098004f90 tx=0x7f5098009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.654+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a81037a0 msgr2=0x7f50a8113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.654+0000 7f50ac9d4700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a81037a0 0x7f50a8113ec0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f509c002a80 tx=0x7f509c004630 comp rx=0 tx=0).stop 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.654+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 shutdown_connections 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.654+0000 7f50ac9d4700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f509006d380 0x7f509006f830 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.655+0000 7f50ac9d4700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a8102e00 0x7f50a8113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.655+0000 7f50ac9d4700 1 --2- 172.21.15.81:0/3733111572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50a81037a0 0x7f50a8113ec0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.655+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 >> 172.21.15.81:0/3733111572 conn(0x7f50a80f89b0 msgr2=0x7f50a81073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.655+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 shutdown_connections 2024-03-22T07:15:47.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.655+0000 7f50ac9d4700 1 -- 172.21.15.81:0/3733111572 wait complete. 2024-03-22T07:15:47.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 -- 172.21.15.81:0/1901860290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c101d70 msgr2=0x7f121c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 --2- 172.21.15.81:0/1901860290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c101d70 0x7f121c102140 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1218004940 tx=0x7f1218009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 -- 172.21.15.81:0/1901860290 shutdown_connections 2024-03-22T07:15:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 --2- 172.21.15.81:0/1901860290 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f121c102710 0x7f121c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 --2- 172.21.15.81:0/1901860290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c101d70 0x7f121c102140 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 -- 172.21.15.81:0/1901860290 >> 172.21.15.81:0/1901860290 conn(0x7f121c072bc0 msgr2=0x7f121c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 -- 172.21.15.81:0/1901860290 shutdown_connections 2024-03-22T07:15:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 -- 172.21.15.81:0/1901860290 wait complete. 2024-03-22T07:15:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 Processor -- start 2024-03-22T07:15:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.755+0000 7f1222b38700 1 -- start start 2024-03-22T07:15:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1222b38700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f121c101d70 0x7f121c112a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1222b38700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c102710 0x7f121c112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1222b38700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c1135d0 con 0x7f121c102710 2024-03-22T07:15:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1222b38700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c10c9c0 con 0x7f121c101d70 2024-03-22T07:15:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1213fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c102710 0x7f121c112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1213fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c102710 0x7f121c112f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59710/0 (socket says 172.21.15.81:59710) 2024-03-22T07:15:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1213fff700 1 -- 172.21.15.81:0/3568596851 learned_addr learned my addr 172.21.15.81:0/3568596851 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:15:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f12208d4700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f121c101d70 0x7f121c112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1213fff700 1 -- 172.21.15.81:0/3568596851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f121c101d70 msgr2=0x7f121c112a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1213fff700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f121c101d70 0x7f121c112a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1213fff700 1 -- 172.21.15.81:0/3568596851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218005040 con 0x7f121c102710 2024-03-22T07:15:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.756+0000 7f1213fff700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c102710 0x7f121c112f80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f120c002a50 tx=0x7f120c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:47.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.757+0000 7f12117fa700 1 -- 172.21.15.81:0/3568596851 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120c007cc0 con 0x7f121c102710 2024-03-22T07:15:47.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.757+0000 7f12117fa700 1 -- 172.21.15.81:0/3568596851 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f120c00f040 con 0x7f121c102710 2024-03-22T07:15:47.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.757+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121c10cca0 con 0x7f121c102710 2024-03-22T07:15:47.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.757+0000 7f12117fa700 1 -- 172.21.15.81:0/3568596851 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120c00da40 con 0x7f121c102710 2024-03-22T07:15:47.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.757+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121c10d260 con 0x7f121c102710 2024-03-22T07:15:47.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.758+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121c0ff080 con 0x7f121c102710 2024-03-22T07:15:47.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.758+0000 7f12117fa700 1 -- 172.21.15.81:0/3568596851 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f120c0097c0 con 0x7f121c102710 2024-03-22T07:15:47.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.758+0000 7f12117fa700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f120406d390 0x7f120406f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:47.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.759+0000 7f12117fa700 1 -- 172.21.15.81:0/3568596851 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f120c08c6d0 con 0x7f121c102710 2024-03-22T07:15:47.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.759+0000 7f12208d4700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f120406d390 0x7f120406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:47.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.759+0000 7f12208d4700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f120406d390 0x7f120406f840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f121800b3b0 tx=0x7f1218009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:47.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:47.763+0000 7f12117fa700 1 -- 172.21.15.81:0/3568596851 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f120c059a50 con 0x7f121c102710 2024-03-22T07:15:48.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.040+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f121c0fd4c0 con 0x7f121c102710 2024-03-22T07:15:48.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.040+0000 7f12117fa700 1 -- 172.21.15.81:0/3568596851 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+250 (secure 0 0 0) 0x7f120c059870 con 0x7f121c102710 2024-03-22T07:15:48.041 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:15:48.041 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: Upgrade: osd.3 is safe to restart 2024-03-22T07:15:48.041 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:48.041 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:48.041 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1085522912' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:48.041 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3982132190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:15:48.041 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 427.629347 secs 2024-03-22T07:15:48.041 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:48.041 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:15:48.042 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:47 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:15:48.042 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:15:48.042 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:15:48.042 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:15:48.042 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.042+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f120406d390 msgr2=0x7f120406f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.042+0000 7f1222b38700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f120406d390 0x7f120406f840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f121800b3b0 tx=0x7f1218009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.042+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c102710 msgr2=0x7f121c112f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.042+0000 7f1222b38700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c102710 0x7f121c112f80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f120c002a50 tx=0x7f120c004600 comp rx=0 tx=0).stop 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.043+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 shutdown_connections 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.043+0000 7f1222b38700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f120406d390 0x7f120406f840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.043+0000 7f1222b38700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f121c101d70 0x7f121c112a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.043+0000 7f1222b38700 1 --2- 172.21.15.81:0/3568596851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f121c102710 0x7f121c112f80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.043+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 >> 172.21.15.81:0/3568596851 conn(0x7f121c072bc0 msgr2=0x7f121c0fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.043+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 shutdown_connections 2024-03-22T07:15:48.043 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:15:48.043+0000 7f1222b38700 1 -- 172.21.15.81:0/3568596851 wait complete. 2024-03-22T07:15:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:15:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: Upgrade: osd.3 is safe to restart 2024-03-22T07:15:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1085522912' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:48.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3982132190' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:15:48.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 427.629347 secs 2024-03-22T07:15:48.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:48.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:15:48.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:47 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:15:49.064 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:48 smithi149 ceph-mon[83447]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:49.065 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:48 smithi149 ceph-mon[83447]: pgmap v109: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:49.065 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:48 smithi149 ceph-mon[83447]: Upgrade: Updating osd.3 2024-03-22T07:15:49.065 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:48 smithi149 ceph-mon[83447]: Deploying daemon osd.3 on smithi149 2024-03-22T07:15:49.065 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:48 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 427.970026 secs 2024-03-22T07:15:49.065 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:48 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3568596851' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:15:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:48 smithi081 ceph-mon[113965]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:48 smithi081 ceph-mon[113965]: pgmap v109: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:48 smithi081 ceph-mon[113965]: Upgrade: Updating osd.3 2024-03-22T07:15:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:48 smithi081 ceph-mon[113965]: Deploying daemon osd.3 on smithi149 2024-03-22T07:15:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:48 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 427.970026 secs 2024-03-22T07:15:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:48 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3568596851' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:15:51.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:50 smithi149 ceph-mon[83447]: pgmap v110: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:51.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:50 smithi081 ceph-mon[113965]: pgmap v110: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:51.422 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:51 smithi149 systemd[1]: Stopping Ceph osd.3 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:15:51.852 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:51 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[41639]: 2024-03-22T07:15:51.418+0000 7fc32b10f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:15:51.852 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:51 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[41639]: 2024-03-22T07:15:51.418+0000 7fc32b10f700 -1 osd.3 64 *** Got signal Terminated *** 2024-03-22T07:15:51.852 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:51 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[41639]: 2024-03-22T07:15:51.418+0000 7fc32b10f700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:15:52.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:51 smithi081 ceph-mon[113965]: osd.3 marked itself down and dead 2024-03-22T07:15:52.337 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:51 smithi149 ceph-mon[83447]: osd.3 marked itself down and dead 2024-03-22T07:15:52.602 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:52 smithi149 podman[92346]: 2024-03-22 07:15:52.338361112 +0000 UTC m=+1.034693864 container died f6fa79d4d66571a7e5473cc148f7715d3db637f8204748d9137c82ce3ee60d31 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3, RELEASE=quincy-9c4e0c3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-22T07:15:52.898 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:52 smithi149 podman[92346]: 2024-03-22 07:15:52.702602917 +0000 UTC m=+1.398935675 container cleanup f6fa79d4d66571a7e5473cc148f7715d3db637f8204748d9137c82ce3ee60d31 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-22T07:15:52.898 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:52 smithi149 bash[92346]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3 2024-03-22T07:15:53.276 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:52 smithi149 podman[92364]: 2024-03-22 07:15:52.962202199 +0000 UTC m=+0.621784421 container remove f6fa79d4d66571a7e5473cc148f7715d3db637f8204748d9137c82ce3ee60d31 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3, RELEASE=quincy-9c4e0c3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-22T07:15:53.276 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:52 smithi149 ceph-mon[83447]: pgmap v111: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:53.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:52 smithi149 ceph-mon[83447]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:15:53.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:52 smithi149 ceph-mon[83447]: failed to encode map e65 with expected crc 2024-03-22T07:15:53.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:52 smithi149 ceph-mon[83447]: osdmap e65: 6 total, 5 up, 6 in 2024-03-22T07:15:53.277 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:52 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 432.629468 secs 2024-03-22T07:15:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:52 smithi081 ceph-mon[113965]: pgmap v111: 65 pgs: 65 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:15:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:52 smithi081 ceph-mon[113965]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:15:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:52 smithi081 ceph-mon[113965]: failed to encode map e65 with expected crc 2024-03-22T07:15:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:52 smithi081 ceph-mon[113965]: osdmap e65: 6 total, 5 up, 6 in 2024-03-22T07:15:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:52 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 432.629468 secs 2024-03-22T07:15:53.581 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:53 smithi149 podman[92452]: 2024-03-22 07:15:53.17965417 +0000 UTC m=+0.017932343 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:15:53.582 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:53 smithi149 podman[92452]: 2024-03-22 07:15:53.299647819 +0000 UTC m=+0.137925981 container create e7907589a21d734471f00327c690f4a284dcd075007e3780c92b330cbafc8e90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-22T07:15:53.582 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:53 smithi149 podman[92452]: 2024-03-22 07:15:53.507250937 +0000 UTC m=+0.345529112 container init e7907589a21d734471f00327c690f4a284dcd075007e3780c92b330cbafc8e90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-22T07:15:53.582 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:53 smithi149 podman[92452]: 2024-03-22 07:15:53.511007325 +0000 UTC m=+0.349285494 container start e7907589a21d734471f00327c690f4a284dcd075007e3780c92b330cbafc8e90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-22T07:15:53.582 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:53 smithi149 podman[92452]: 2024-03-22 07:15:53.58220064 +0000 UTC m=+0.420478804 container attach e7907589a21d734471f00327c690f4a284dcd075007e3780c92b330cbafc8e90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True) 2024-03-22T07:15:53.852 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:53 smithi149 podman[92452]: 2024-03-22 07:15:53.799767567 +0000 UTC m=+0.638045732 container died e7907589a21d734471f00327c690f4a284dcd075007e3780c92b330cbafc8e90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0) 2024-03-22T07:15:54.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:53 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 432.970137 secs 2024-03-22T07:15:54.470 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:53 smithi081 ceph-mon[113965]: failed to encode map e66 with expected crc 2024-03-22T07:15:54.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:53 smithi081 ceph-mon[113965]: osdmap e66: 6 total, 5 up, 6 in 2024-03-22T07:15:54.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:53 smithi081 ceph-mon[113965]: failed to encode map e66 with expected crc 2024-03-22T07:15:54.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:53 smithi081 ceph-mon[113965]: failed to encode map e66 with expected crc 2024-03-22T07:15:54.471 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:53 smithi081 ceph-mon[113965]: failed to encode map e66 with expected crc 2024-03-22T07:15:54.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:53 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 432.970137 secs 2024-03-22T07:15:54.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:53 smithi149 ceph-mon[83447]: failed to encode map e66 with expected crc 2024-03-22T07:15:54.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:53 smithi149 ceph-mon[83447]: osdmap e66: 6 total, 5 up, 6 in 2024-03-22T07:15:54.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:53 smithi149 ceph-mon[83447]: failed to encode map e66 with expected crc 2024-03-22T07:15:54.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:53 smithi149 ceph-mon[83447]: failed to encode map e66 with expected crc 2024-03-22T07:15:54.472 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:53 smithi149 ceph-mon[83447]: failed to encode map e66 with expected crc 2024-03-22T07:15:54.803 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:54 smithi149 podman[92478]: 2024-03-22 07:15:54.38288915 +0000 UTC m=+0.578383564 container remove e7907589a21d734471f00327c690f4a284dcd075007e3780c92b330cbafc8e90 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-22T07:15:54.804 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:54 smithi149 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-22T07:15:54.804 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:54 smithi149 systemd[1]: Stopped Ceph osd.3 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:15:54.804 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:54 smithi149 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.3.service: Consumed 46.891s CPU time. 2024-03-22T07:15:54.804 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:54 smithi149 systemd[1]: Starting Ceph osd.3 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:15:55.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:54 smithi149 ceph-mon[83447]: pgmap v114: 65 pgs: 7 active+undersized+degraded, 13 stale+active+clean, 45 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 1312/32856 objects degraded (3.993%) 2024-03-22T07:15:55.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:54 smithi149 ceph-mon[83447]: Health check failed: Degraded data redundancy: 1312/32856 objects degraded (3.993%), 7 pgs degraded (PG_DEGRADED) 2024-03-22T07:15:55.102 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:54 smithi149 podman[92578]: 2024-03-22 07:15:54.803836158 +0000 UTC m=+0.101754465 container create f4ddc682684f7a6d88bfb6aa0bd711f370e031f86362e81c80cfa9fda24a9519 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-22T07:15:55.102 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:54 smithi149 podman[92578]: 2024-03-22 07:15:54.720785909 +0000 UTC m=+0.018704162 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:15:55.102 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 podman[92578]: 2024-03-22 07:15:55.04816614 +0000 UTC m=+0.346084398 container init f4ddc682684f7a6d88bfb6aa0bd711f370e031f86362e81c80cfa9fda24a9519 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-22T07:15:55.103 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 podman[92578]: 2024-03-22 07:15:55.051230891 +0000 UTC m=+0.349149140 container start f4ddc682684f7a6d88bfb6aa0bd711f370e031f86362e81c80cfa9fda24a9519 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-22T07:15:55.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:54 smithi081 ceph-mon[113965]: pgmap v114: 65 pgs: 7 active+undersized+degraded, 13 stale+active+clean, 45 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 1312/32856 objects degraded (3.993%) 2024-03-22T07:15:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:54 smithi081 ceph-mon[113965]: Health check failed: Degraded data redundancy: 1312/32856 objects degraded (3.993%), 7 pgs degraded (PG_DEGRADED) 2024-03-22T07:15:55.495 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 podman[92578]: 2024-03-22 07:15:55.13283882 +0000 UTC m=+0.430757078 container attach f4ddc682684f7a6d88bfb6aa0bd711f370e031f86362e81c80cfa9fda24a9519 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-22T07:15:55.811 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate[92593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T07:15:55.811 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate[92593]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T07:15:55.811 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 bash[92578]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T07:15:55.811 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 bash[92578]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T07:15:55.811 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 bash[92578]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 bash[92578]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 bash[92578]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 bash[92578]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 bash[92578]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate[92593]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate[92593]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate[92593]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate[92593]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate[92593]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-22T07:15:55.812 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:55 smithi149 podman[92578]: 2024-03-22 07:15:55.520495777 +0000 UTC m=+0.818414018 container died f4ddc682684f7a6d88bfb6aa0bd711f370e031f86362e81c80cfa9fda24a9519 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-22T07:15:56.394 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:56 smithi149 podman[92717]: 2024-03-22 07:15:56.127316141 +0000 UTC m=+0.601540704 container remove f4ddc682684f7a6d88bfb6aa0bd711f370e031f86362e81c80cfa9fda24a9519 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-22T07:15:56.394 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:56 smithi149 podman[92775]: 2024-03-22 07:15:56.243526363 +0000 UTC m=+0.017895397 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:15:56.394 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:56 smithi149 podman[92775]: 2024-03-22 07:15:56.394817134 +0000 UTC m=+0.169186148 container create c8234f7ed003772449ea65125060011d609cf556e3fc36906eecbefd1409a54f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-22T07:15:56.705 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:56 smithi149 podman[92775]: 2024-03-22 07:15:56.607092645 +0000 UTC m=+0.381461659 container init c8234f7ed003772449ea65125060011d609cf556e3fc36906eecbefd1409a54f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-22T07:15:56.705 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:56 smithi149 podman[92775]: 2024-03-22 07:15:56.610322161 +0000 UTC m=+0.384691172 container start c8234f7ed003772449ea65125060011d609cf556e3fc36906eecbefd1409a54f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-22T07:15:56.705 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:56 smithi149 ceph-osd[92793]: -- 172.21.15.149:0/2191744433 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6c1f6f1e0 con 0x55b6c1f41000 2024-03-22T07:15:56.705 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:56 smithi149 bash[92775]: c8234f7ed003772449ea65125060011d609cf556e3fc36906eecbefd1409a54f 2024-03-22T07:15:57.102 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:56 smithi149 systemd[1]: Started Ceph osd.3 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:15:57.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:56 smithi149 ceph-mon[83447]: pgmap v115: 65 pgs: 7 active+undersized+degraded, 13 stale+active+clean, 45 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 1312/32856 objects degraded (3.993%) 2024-03-22T07:15:57.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:56 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:57.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:56 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:57.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:56 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:15:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:56 smithi081 ceph-mon[113965]: pgmap v115: 65 pgs: 7 active+undersized+degraded, 13 stale+active+clean, 45 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 1312/32856 objects degraded (3.993%) 2024-03-22T07:15:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:15:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:15:57.543 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:15:57 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[92789]: 2024-03-22T07:15:57.159+0000 7fb1f59037c0 -1 Falling back to public interface 2024-03-22T07:15:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:57 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 437.629530 secs 2024-03-22T07:15:58.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:57 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 437.629530 secs 2024-03-22T07:15:59.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:58 smithi149 ceph-mon[83447]: pgmap v116: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:15:59.171 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:15:58 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 437.970244 secs 2024-03-22T07:15:59.236 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:58 smithi081 ceph-mon[113965]: pgmap v116: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:15:59.236 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:15:58 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 437.970244 secs 2024-03-22T07:16:01.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:01.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:01.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:00 smithi081 ceph-mon[113965]: pgmap v117: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:16:01.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:01.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:01.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:00 smithi149 ceph-mon[83447]: pgmap v117: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:16:01.744 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:16:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[92789]: 2024-03-22T07:16:01.620+0000 7fb1f59037c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-22T07:16:02.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:01 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:02.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:01 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:02.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:01 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:02.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:01 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:02.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:01 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:02.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:01 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:02.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:01 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:02.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:01 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:02.511 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:16:02 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[92789]: 2024-03-22T07:16:02.135+0000 7fb1f59037c0 -1 osd.3 64 log_to_monitors true 2024-03-22T07:16:03.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:02 smithi149 ceph-mon[83447]: pgmap v118: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:16:03.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:02 smithi149 ceph-mon[83447]: from='osd.3 [v2:172.21.15.149:6800/1325806457,v1:172.21.15.149:6801/1325806457]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:16:03.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:02 smithi149 ceph-mon[83447]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:16:03.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:02 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 442.629607 secs 2024-03-22T07:16:03.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:02 smithi149 ceph-mon[83447]: Health check update: Degraded data redundancy: 6151/32856 objects degraded (18.721%), 37 pgs degraded (PG_DEGRADED) 2024-03-22T07:16:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:02 smithi081 ceph-mon[113965]: pgmap v118: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:16:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:02 smithi081 ceph-mon[113965]: from='osd.3 [v2:172.21.15.149:6800/1325806457,v1:172.21.15.149:6801/1325806457]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:16:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:02 smithi081 ceph-mon[113965]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:16:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:02 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 442.629607 secs 2024-03-22T07:16:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:02 smithi081 ceph-mon[113965]: Health check update: Degraded data redundancy: 6151/32856 objects degraded (18.721%), 37 pgs degraded (PG_DEGRADED) 2024-03-22T07:16:04.102 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:16:03 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[92789]: 2024-03-22T07:16:03.756+0000 7fb1e2d53700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:16:04.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:03 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 442.970324 secs 2024-03-22T07:16:04.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:03 smithi149 ceph-mon[83447]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T07:16:04.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:03 smithi149 ceph-mon[83447]: from='osd.3 [v2:172.21.15.149:6800/1325806457,v1:172.21.15.149:6801/1325806457]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:16:04.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:03 smithi149 ceph-mon[83447]: osdmap e67: 6 total, 5 up, 6 in 2024-03-22T07:16:04.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:03 smithi149 ceph-mon[83447]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:16:04.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:03 smithi149 ceph-mon[83447]: from='osd.3 ' entity='osd.3' 2024-03-22T07:16:04.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:03 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 442.970324 secs 2024-03-22T07:16:04.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:03 smithi081 ceph-mon[113965]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T07:16:04.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:03 smithi081 ceph-mon[113965]: from='osd.3 [v2:172.21.15.149:6800/1325806457,v1:172.21.15.149:6801/1325806457]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:16:04.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:03 smithi081 ceph-mon[113965]: osdmap e67: 6 total, 5 up, 6 in 2024-03-22T07:16:04.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:03 smithi081 ceph-mon[113965]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:16:04.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:03 smithi081 ceph-mon[113965]: from='osd.3 ' entity='osd.3' 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: failed to encode map e67 with expected crc 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: failed to encode map e67 with expected crc 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: pgmap v120: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:05.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:05.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:05.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:05.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:16:05.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:16:05.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: osd.3 [v2:172.21.15.149:6800/1325806457,v1:172.21.15.149:6801/1325806457] boot 2024-03-22T07:16:05.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: osdmap e68: 6 total, 6 up, 6 in 2024-03-22T07:16:05.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:04 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:16:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: failed to encode map e67 with expected crc 2024-03-22T07:16:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: failed to encode map e67 with expected crc 2024-03-22T07:16:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: pgmap v120: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:16:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:16:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:05.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:05.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:05.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:05.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:05.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:16:05.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:16:05.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: osd.3 [v2:172.21.15.149:6800/1325806457,v1:172.21.15.149:6801/1325806457] boot 2024-03-22T07:16:05.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: osdmap e68: 6 total, 6 up, 6 in 2024-03-22T07:16:05.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:04 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:16:06.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:05 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:16:06.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:05 smithi081 ceph-mon[113965]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-22T07:16:06.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:05 smithi081 ceph-mon[113965]: failed to encode map e68 with expected crc 2024-03-22T07:16:06.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:05 smithi081 ceph-mon[113965]: osdmap e69: 6 total, 6 up, 6 in 2024-03-22T07:16:06.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:05 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:16:06.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:05 smithi149 ceph-mon[83447]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-22T07:16:06.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:05 smithi149 ceph-mon[83447]: failed to encode map e68 with expected crc 2024-03-22T07:16:06.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:05 smithi149 ceph-mon[83447]: osdmap e69: 6 total, 6 up, 6 in 2024-03-22T07:16:07.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:06 smithi081 ceph-mon[113965]: failed to encode map e69 with expected crc 2024-03-22T07:16:07.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:06 smithi081 ceph-mon[113965]: failed to encode map e69 with expected crc 2024-03-22T07:16:07.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:06 smithi081 ceph-mon[113965]: failed to encode map e69 with expected crc 2024-03-22T07:16:07.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:06 smithi081 ceph-mon[113965]: pgmap v123: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:16:07.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:06 smithi149 ceph-mon[83447]: failed to encode map e69 with expected crc 2024-03-22T07:16:07.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:06 smithi149 ceph-mon[83447]: failed to encode map e69 with expected crc 2024-03-22T07:16:07.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:06 smithi149 ceph-mon[83447]: failed to encode map e69 with expected crc 2024-03-22T07:16:07.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:06 smithi149 ceph-mon[83447]: pgmap v123: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6151/32856 objects degraded (18.721%) 2024-03-22T07:16:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:07 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 447.629728 secs 2024-03-22T07:16:08.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:07 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 447.629728 secs 2024-03-22T07:16:09.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:08 smithi081 ceph-mon[113965]: pgmap v124: 65 pgs: 20 active+undersized+degraded, 45 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 3061/32856 objects degraded (9.316%) 2024-03-22T07:16:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:08 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 447.970443 secs 2024-03-22T07:16:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:08 smithi081 ceph-mon[113965]: Health check update: Degraded data redundancy: 3061/32856 objects degraded (9.316%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T07:16:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:08 smithi149 ceph-mon[83447]: pgmap v124: 65 pgs: 20 active+undersized+degraded, 45 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 3061/32856 objects degraded (9.316%) 2024-03-22T07:16:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:08 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 447.970443 secs 2024-03-22T07:16:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:08 smithi149 ceph-mon[83447]: Health check update: Degraded data redundancy: 3061/32856 objects degraded (9.316%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T07:16:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:09 smithi081 ceph-mon[113965]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3061/32856 objects degraded (9.316%), 20 pgs degraded) 2024-03-22T07:16:10.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:09 smithi149 ceph-mon[83447]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3061/32856 objects degraded (9.316%), 20 pgs degraded) 2024-03-22T07:16:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:10 smithi081 ceph-mon[113965]: pgmap v125: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:10 smithi149 ceph-mon[83447]: pgmap v125: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:13.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:12 smithi081 ceph-mon[113965]: pgmap v126: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:13.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:12 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 452.629908 secs 2024-03-22T07:16:13.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:12 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 452.970514 secs 2024-03-22T07:16:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:12 smithi149 ceph-mon[83447]: pgmap v126: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:12 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 452.629908 secs 2024-03-22T07:16:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:12 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 452.970514 secs 2024-03-22T07:16:15.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:14 smithi081 ceph-mon[113965]: pgmap v127: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:15.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:14 smithi149 ceph-mon[83447]: pgmap v127: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:17.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:16 smithi081 ceph-mon[113965]: pgmap v128: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:17.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:17.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:17.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:16 smithi149 ceph-mon[83447]: pgmap v128: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:18.157 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:17 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 457.630041 secs 2024-03-22T07:16:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.156+0000 7f3dfbca6700 1 -- 172.21.15.81:0/3121235683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 msgr2=0x7f3df4070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.156+0000 7f3dfbca6700 1 --2- 172.21.15.81:0/3121235683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 0x7f3df4070730 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3df0004970 tx=0x7f3df0009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.157+0000 7f3dfbca6700 1 -- 172.21.15.81:0/3121235683 shutdown_connections 2024-03-22T07:16:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.157+0000 7f3dfbca6700 1 --2- 172.21.15.81:0/3121235683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3df4070c70 0x7f3df41090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.157+0000 7f3dfbca6700 1 --2- 172.21.15.81:0/3121235683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 0x7f3df4070730 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.157+0000 7f3dfbca6700 1 -- 172.21.15.81:0/3121235683 >> 172.21.15.81:0/3121235683 conn(0x7f3df40f8950 msgr2=0x7f3df40fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.157+0000 7f3dfbca6700 1 -- 172.21.15.81:0/3121235683 shutdown_connections 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.157+0000 7f3dfbca6700 1 -- 172.21.15.81:0/3121235683 wait complete. 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.157+0000 7f3dfbca6700 1 Processor -- start 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.157+0000 7f3dfbca6700 1 -- start start 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3dfbca6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 0x7f3df4115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3dfbca6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3df4070c70 0x7f3df4116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3dfbca6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4116710 con 0x7f3df4070360 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3dfbca6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df410fd10 con 0x7f3df4070c70 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3df9a42700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 0x7f3df4115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3df9a42700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 0x7f3df4115af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39442/0 (socket says 172.21.15.81:39442) 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3df9a42700 1 -- 172.21.15.81:0/777691661 learned_addr learned my addr 172.21.15.81:0/777691661 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3df9241700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3df4070c70 0x7f3df4116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3df9a42700 1 -- 172.21.15.81:0/777691661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3df4070c70 msgr2=0x7f3df4116030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3df9a42700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3df4070c70 0x7f3df4116030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3df9a42700 1 -- 172.21.15.81:0/777691661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df0005040 con 0x7f3df4070360 2024-03-22T07:16:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3df9a42700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 0x7f3df4115af0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3df0004f50 tx=0x7f3df0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3dea7fc700 1 -- 172.21.15.81:0/777691661 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df000e040 con 0x7f3df4070360 2024-03-22T07:16:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3dea7fc700 1 -- 172.21.15.81:0/777691661 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3df0009400 con 0x7f3df4070360 2024-03-22T07:16:18.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df410ff90 con 0x7f3df4070360 2024-03-22T07:16:18.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3dea7fc700 1 -- 172.21.15.81:0/777691661 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df001c3f0 con 0x7f3df4070360 2024-03-22T07:16:18.162 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.158+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df41104d0 con 0x7f3df4070360 2024-03-22T07:16:18.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.159+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df4071aa0 con 0x7f3df4070360 2024-03-22T07:16:18.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.160+0000 7f3dea7fc700 1 -- 172.21.15.81:0/777691661 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f3df0013420 con 0x7f3df4070360 2024-03-22T07:16:18.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.160+0000 7f3dea7fc700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3de006d380 0x7f3de006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:18.164 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.161+0000 7f3dea7fc700 1 -- 172.21.15.81:0/777691661 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3df008db00 con 0x7f3df4070360 2024-03-22T07:16:18.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.161+0000 7f3df9241700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3de006d380 0x7f3de006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:18.165 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.161+0000 7f3df9241700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3de006d380 0x7f3de006f830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3df41049e0 tx=0x7f3de40091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:18.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.165+0000 7f3dea7fc700 1 -- 172.21.15.81:0/777691661 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3df0057660 con 0x7f3df4070360 2024-03-22T07:16:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:17 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 457.630041 secs 2024-03-22T07:16:18.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.377+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3df410c150 con 0x7f3de006d380 2024-03-22T07:16:18.378 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.378+0000 7f3dea7fc700 1 -- 172.21.15.81:0/777691661 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f3df410c150 con 0x7f3de006d380 2024-03-22T07:16:18.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.380+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3de006d380 msgr2=0x7f3de006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.380+0000 7f3dfbca6700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3de006d380 0x7f3de006f830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3df41049e0 tx=0x7f3de40091a0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.380+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 msgr2=0x7f3df4115af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.380+0000 7f3dfbca6700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 0x7f3df4115af0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3df0004f50 tx=0x7f3df0009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:18.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.381+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 shutdown_connections 2024-03-22T07:16:18.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.381+0000 7f3dfbca6700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3de006d380 0x7f3de006f830 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.381+0000 7f3dfbca6700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3df4070360 0x7f3df4115af0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.381+0000 7f3dfbca6700 1 --2- 172.21.15.81:0/777691661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3df4070c70 0x7f3df4116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.381+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 >> 172.21.15.81:0/777691661 conn(0x7f3df40f8950 msgr2=0x7f3df40f9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:18.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.381+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 shutdown_connections 2024-03-22T07:16:18.382 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.381+0000 7f3dfbca6700 1 -- 172.21.15.81:0/777691661 wait complete. 2024-03-22T07:16:18.392 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:16:18.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.482+0000 7f3a009a3700 1 -- 172.21.15.81:0/1976884011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 msgr2=0x7f39fc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.482+0000 7f3a009a3700 1 --2- 172.21.15.81:0/1976884011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 0x7f39fc1031d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009ce0 tx=0x7f39f4004d10 comp rx=0 tx=0).stop 2024-03-22T07:16:18.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.482+0000 7f3a009a3700 1 -- 172.21.15.81:0/1976884011 shutdown_connections 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.482+0000 7f3a009a3700 1 --2- 172.21.15.81:0/1976884011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc1037a0 0x7f39fc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.482+0000 7f3a009a3700 1 --2- 172.21.15.81:0/1976884011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 0x7f39fc1031d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.482+0000 7f3a009a3700 1 -- 172.21.15.81:0/1976884011 >> 172.21.15.81:0/1976884011 conn(0x7f39fc0f89b0 msgr2=0x7f39fc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.482+0000 7f3a009a3700 1 -- 172.21.15.81:0/1976884011 shutdown_connections 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.482+0000 7f3a009a3700 1 -- 172.21.15.81:0/1976884011 wait complete. 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f3a009a3700 1 Processor -- start 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f3a009a3700 1 -- start start 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f3a009a3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 0x7f39fc113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f3a009a3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc1037a0 0x7f39fc113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f3a009a3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc1146c0 con 0x7f39fc102e00 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f3a009a3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc10dc10 con 0x7f39fc1037a0 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f39fa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 0x7f39fc113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f39fa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 0x7f39fc113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39470/0 (socket says 172.21.15.81:39470) 2024-03-22T07:16:18.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f39fa59c700 1 -- 172.21.15.81:0/1111991495 learned_addr learned my addr 172.21.15.81:0/1111991495 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.483+0000 7f39f9d9b700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc1037a0 0x7f39fc113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f39fa59c700 1 -- 172.21.15.81:0/1111991495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc1037a0 msgr2=0x7f39fc113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f39fa59c700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc1037a0 0x7f39fc113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f39fa59c700 1 -- 172.21.15.81:0/1111991495 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f4009940 con 0x7f39fc102e00 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f39f9d9b700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc1037a0 0x7f39fc113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f39fa59c700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 0x7f39fc113aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39f400be90 tx=0x7f39f4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f39eaffd700 1 -- 172.21.15.81:0/1111991495 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39f400e040 con 0x7f39fc102e00 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f39eaffd700 1 -- 172.21.15.81:0/1111991495 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39f4008ea0 con 0x7f39fc102e00 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc10de30 con 0x7f39fc102e00 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f39eaffd700 1 -- 172.21.15.81:0/1111991495 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39f401c810 con 0x7f39fc102e00 2024-03-22T07:16:18.486 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.484+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc10e3f0 con 0x7f39fc102e00 2024-03-22T07:16:18.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.485+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc0fe550 con 0x7f39fc102e00 2024-03-22T07:16:18.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.485+0000 7f39eaffd700 1 -- 172.21.15.81:0/1111991495 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f39f4012c00 con 0x7f39fc102e00 2024-03-22T07:16:18.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.486+0000 7f39eaffd700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f39e406d380 0x7f39e406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:18.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.486+0000 7f39eaffd700 1 -- 172.21.15.81:0/1111991495 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f39f408e4d0 con 0x7f39fc102e00 2024-03-22T07:16:18.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.486+0000 7f39f9d9b700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f39e406d380 0x7f39e406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:18.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.486+0000 7f39f9d9b700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f39e406d380 0x7f39e406f830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39f0006fd0 tx=0x7f39f0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:18.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.490+0000 7f39eaffd700 1 -- 172.21.15.81:0/1111991495 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f39f405bae0 con 0x7f39fc102e00 2024-03-22T07:16:18.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.703+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f39fc10f190 con 0x7f39e406d380 2024-03-22T07:16:18.704 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.704+0000 7f39eaffd700 1 -- 172.21.15.81:0/1111991495 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f39fc10f190 con 0x7f39e406d380 2024-03-22T07:16:18.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f39e406d380 msgr2=0x7f39e406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f39e406d380 0x7f39e406f830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f39f0006fd0 tx=0x7f39f0008040 comp rx=0 tx=0).stop 2024-03-22T07:16:18.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 msgr2=0x7f39fc113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 0x7f39fc113aa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39f400be90 tx=0x7f39f4004d10 comp rx=0 tx=0).stop 2024-03-22T07:16:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 shutdown_connections 2024-03-22T07:16:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f39e406d380 0x7f39e406f830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39fc102e00 0x7f39fc113aa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 --2- 172.21.15.81:0/1111991495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39fc1037a0 0x7f39fc113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 >> 172.21.15.81:0/1111991495 conn(0x7f39fc0f89b0 msgr2=0x7f39fc0f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 shutdown_connections 2024-03-22T07:16:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.706+0000 7f3a009a3700 1 -- 172.21.15.81:0/1111991495 wait complete. 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.807+0000 7fef9106e700 1 -- 172.21.15.81:0/1796701072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 msgr2=0x7fef8c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.807+0000 7fef9106e700 1 --2- 172.21.15.81:0/1796701072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 0x7fef8c102140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fef80004940 tx=0x7fef80009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.807+0000 7fef9106e700 1 -- 172.21.15.81:0/1796701072 shutdown_connections 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.807+0000 7fef9106e700 1 --2- 172.21.15.81:0/1796701072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef8c102710 0x7fef8c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.807+0000 7fef9106e700 1 --2- 172.21.15.81:0/1796701072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 0x7fef8c102140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.807+0000 7fef9106e700 1 -- 172.21.15.81:0/1796701072 >> 172.21.15.81:0/1796701072 conn(0x7fef8c072bc0 msgr2=0x7fef8c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.807+0000 7fef9106e700 1 -- 172.21.15.81:0/1796701072 shutdown_connections 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.807+0000 7fef9106e700 1 -- 172.21.15.81:0/1796701072 wait complete. 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef9106e700 1 Processor -- start 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef9106e700 1 -- start start 2024-03-22T07:16:18.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef9106e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 0x7fef8c1032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef9106e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef8c102710 0x7fef8c103810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef9106e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c103da0 con 0x7fef8c101d70 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef9106e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c103ee0 con 0x7fef8c102710 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef8ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 0x7fef8c1032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef8ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 0x7fef8c1032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39490/0 (socket says 172.21.15.81:39490) 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef8ad9d700 1 -- 172.21.15.81:0/134522167 learned_addr learned my addr 172.21.15.81:0/134522167 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef8a59c700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef8c102710 0x7fef8c103810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef8ad9d700 1 -- 172.21.15.81:0/134522167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef8c102710 msgr2=0x7fef8c103810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef8ad9d700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef8c102710 0x7fef8c103810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.808+0000 7fef8ad9d700 1 -- 172.21.15.81:0/134522167 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef80005040 con 0x7fef8c101d70 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.809+0000 7fef8ad9d700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 0x7fef8c1032d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fef8000b3b0 tx=0x7fef80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.809+0000 7fef737fe700 1 -- 172.21.15.81:0/134522167 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8000e040 con 0x7fef8c101d70 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.809+0000 7fef737fe700 1 -- 172.21.15.81:0/134522167 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef80013420 con 0x7fef8c101d70 2024-03-22T07:16:18.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.809+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c104130 con 0x7fef8c101d70 2024-03-22T07:16:18.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.809+0000 7fef737fe700 1 -- 172.21.15.81:0/134522167 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8001c630 con 0x7fef8c101d70 2024-03-22T07:16:18.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.809+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c11a250 con 0x7fef8c101d70 2024-03-22T07:16:18.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.810+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8c0fd4c0 con 0x7fef8c101d70 2024-03-22T07:16:18.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.811+0000 7fef737fe700 1 -- 172.21.15.81:0/134522167 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fef80013590 con 0x7fef8c101d70 2024-03-22T07:16:18.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.811+0000 7fef737fe700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fef7406d380 0x7fef7406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:18.814 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.811+0000 7fef8a59c700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fef7406d380 0x7fef7406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:18.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.811+0000 7fef737fe700 1 -- 172.21.15.81:0/134522167 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fef8008d9a0 con 0x7fef8c101d70 2024-03-22T07:16:18.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.812+0000 7fef8a59c700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fef7406d380 0x7fef7406f830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef8c104c10 tx=0x7fef7c0043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:18.819 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:18.816+0000 7fef737fe700 1 -- 172.21.15.81:0/134522167 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef800575a0 con 0x7fef8c101d70 2024-03-22T07:16:19.020 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.020+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fef8c1049c0 con 0x7fef7406d380 2024-03-22T07:16:19.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.030+0000 7fef737fe700 1 -- 172.21.15.81:0/134522167 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fef8c1049c0 con 0x7fef7406d380 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (4m) 52s ago 15m 26.0M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (2m) 52s ago 15m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (2m) 19s ago 14m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (4m) 52s ago 14m 84.5M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (9m) 52s ago 9m 13.7M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (9m) 52s ago 9m 131M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (9m) 19s ago 9m 17.3M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (9m) 19s ago 9m 147M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:16:19.031 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (6m) 52s ago 16m 502M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (5m) 19s ago 14m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (3m) 52s ago 16m 44.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (3m) 19s ago 14m 34.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (5m) 52s ago 15m 16.2M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (5m) 19s ago 14m 16.4M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (2m) 52s ago 13m 94.5M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 033c080308b6 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (94s) 52s ago 13m 105M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 8341fe32f3fa 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (58s) 52s ago 12m 12.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 0b8c514dced1 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (22s) 19s ago 12m 12.6M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c8234f7ed003 2024-03-22T07:16:19.032 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (11m) 19s ago 11m 193M 4096M 17.2.7-726-g9c4e0c34 837124da804f d37c642ce9cb 2024-03-22T07:16:19.033 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (11m) 19s ago 11m 183M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:16:19.033 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (5m) 52s ago 14m 59.1M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:16:19.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fef7406d380 msgr2=0x7fef7406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.033 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fef7406d380 0x7fef7406f830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef8c104c10 tx=0x7fef7c0043c0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 msgr2=0x7fef8c1032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 0x7fef8c1032d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fef8000b3b0 tx=0x7fef80009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:19.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 shutdown_connections 2024-03-22T07:16:19.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fef7406d380 0x7fef7406f830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fef8c101d70 0x7fef8c1032d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 --2- 172.21.15.81:0/134522167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef8c102710 0x7fef8c103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.034 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 >> 172.21.15.81:0/134522167 conn(0x7fef8c072bc0 msgr2=0x7fef8c105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:19.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 shutdown_connections 2024-03-22T07:16:19.035 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.032+0000 7fef9106e700 1 -- 172.21.15.81:0/134522167 wait complete. 2024-03-22T07:16:19.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:18 smithi081 ceph-mon[113965]: pgmap v129: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:19.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:18 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 457.970647 secs 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.132+0000 7ff1c3c18700 1 -- 172.21.15.81:0/1805903045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 msgr2=0x7ff1bc0fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.132+0000 7ff1c3c18700 1 --2- 172.21.15.81:0/1805903045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 0x7ff1bc0fe260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8004940 tx=0x7ff1b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 -- 172.21.15.81:0/1805903045 shutdown_connections 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 --2- 172.21.15.81:0/1805903045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1bc0fe830 0x7ff1bc106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 --2- 172.21.15.81:0/1805903045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 0x7ff1bc0fe260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 -- 172.21.15.81:0/1805903045 >> 172.21.15.81:0/1805903045 conn(0x7ff1bc072bc0 msgr2=0x7ff1bc072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 -- 172.21.15.81:0/1805903045 shutdown_connections 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 -- 172.21.15.81:0/1805903045 wait complete. 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 Processor -- start 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 -- start start 2024-03-22T07:16:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 0x7ff1bc113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.133+0000 7ff1c3c18700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1bc0fe830 0x7ff1bc1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c3c18700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc114720 con 0x7ff1bc0fde90 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c3c18700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc10db30 con 0x7ff1bc0fe830 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c19b4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 0x7ff1bc113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c19b4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 0x7ff1bc113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39504/0 (socket says 172.21.15.81:39504) 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c19b4700 1 -- 172.21.15.81:0/3694901537 learned_addr learned my addr 172.21.15.81:0/3694901537 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c11b3700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1bc0fe830 0x7ff1bc1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c19b4700 1 -- 172.21.15.81:0/3694901537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1bc0fe830 msgr2=0x7ff1bc1140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c19b4700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1bc0fe830 0x7ff1bc1140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c19b4700 1 -- 172.21.15.81:0/3694901537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8005040 con 0x7ff1bc0fde90 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c19b4700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 0x7ff1bc113b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009360 tx=0x7ff1b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1b27fc700 1 -- 172.21.15.81:0/3694901537 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1b800e040 con 0x7ff1bc0fde90 2024-03-22T07:16:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1b27fc700 1 -- 172.21.15.81:0/3694901537 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1b8013420 con 0x7ff1bc0fde90 2024-03-22T07:16:19.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc10ddb0 con 0x7ff1bc0fde90 2024-03-22T07:16:19.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1b27fc700 1 -- 172.21.15.81:0/3694901537 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1b801c630 con 0x7ff1bc0fde90 2024-03-22T07:16:19.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.134+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc10e2f0 con 0x7ff1bc0fde90 2024-03-22T07:16:19.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.135+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc101980 con 0x7ff1bc0fde90 2024-03-22T07:16:19.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.136+0000 7ff1b27fc700 1 -- 172.21.15.81:0/3694901537 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7ff1b8013a90 con 0x7ff1bc0fde90 2024-03-22T07:16:19.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.136+0000 7ff1b27fc700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff1a806d380 0x7ff1a806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:19.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.136+0000 7ff1b27fc700 1 -- 172.21.15.81:0/3694901537 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff1b808d9f0 con 0x7ff1bc0fde90 2024-03-22T07:16:19.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.136+0000 7ff1c11b3700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff1a806d380 0x7ff1a806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:19.139 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.136+0000 7ff1c11b3700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff1a806d380 0x7ff1a806f830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac007fd0 tx=0x7ff1ac009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:19.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.141+0000 7ff1b27fc700 1 -- 172.21.15.81:0/3694901537 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff1b808fe60 con 0x7ff1bc0fde90 2024-03-22T07:16:19.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:18 smithi149 ceph-mon[83447]: pgmap v129: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:19.254 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:18 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 457.970647 secs 2024-03-22T07:16:19.413 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.413+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff1bc05f5f0 con 0x7ff1bc0fde90 2024-03-22T07:16:19.414 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.413+0000 7ff1b27fc700 1 -- 172.21.15.81:0/3694901537 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7ff1b80575f0 con 0x7ff1bc0fde90 2024-03-22T07:16:19.414 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:16:19.414 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:16:19.414 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:16:19.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:16:19.414 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:16:19.414 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2, 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6, 2024-03-22T07:16:19.415 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 8 2024-03-22T07:16:19.416 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:16:19.416 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:16:19.416 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff1a806d380 msgr2=0x7ff1a806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff1a806d380 0x7ff1a806f830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac007fd0 tx=0x7ff1ac009040 comp rx=0 tx=0).stop 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 msgr2=0x7ff1bc113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 0x7ff1bc113b90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009360 tx=0x7ff1b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 shutdown_connections 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff1a806d380 0x7ff1a806f830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1bc0fde90 0x7ff1bc113b90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 --2- 172.21.15.81:0/3694901537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1bc0fe830 0x7ff1bc1140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 >> 172.21.15.81:0/3694901537 conn(0x7ff1bc072bc0 msgr2=0x7ff1bc0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 shutdown_connections 2024-03-22T07:16:19.417 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.416+0000 7ff1c3c18700 1 -- 172.21.15.81:0/3694901537 wait complete. 2024-03-22T07:16:19.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.516+0000 7ff4d8a67700 1 -- 172.21.15.81:0/1494176926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 msgr2=0x7ff4d40fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.516+0000 7ff4d8a67700 1 --2- 172.21.15.81:0/1494176926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 0x7ff4d40fb910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8004940 tx=0x7ff4c8009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:19.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.516+0000 7ff4d8a67700 1 -- 172.21.15.81:0/1494176926 shutdown_connections 2024-03-22T07:16:19.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.516+0000 7ff4d8a67700 1 --2- 172.21.15.81:0/1494176926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 0x7ff4d40fb910 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.516+0000 7ff4d8a67700 1 --2- 172.21.15.81:0/1494176926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d4102e70 0x7ff4d4103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.518 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.516+0000 7ff4d8a67700 1 -- 172.21.15.81:0/1494176926 >> 172.21.15.81:0/1494176926 conn(0x7ff4d40714c0 msgr2=0x7ff4d40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.516+0000 7ff4d8a67700 1 -- 172.21.15.81:0/1494176926 shutdown_connections 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.516+0000 7ff4d8a67700 1 -- 172.21.15.81:0/1494176926 wait complete. 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d8a67700 1 Processor -- start 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d8a67700 1 -- start start 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d8a67700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 0x7ff4d4113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d8a67700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d4102e70 0x7ff4d41140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d8a67700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d4114700 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d8a67700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d410daf0 con 0x7ff4d4102e70 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 0x7ff4d4113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 0x7ff4d4113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39524/0 (socket says 172.21.15.81:39524) 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d259c700 1 -- 172.21.15.81:0/3160506427 learned_addr learned my addr 172.21.15.81:0/3160506427 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d1d9b700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d4102e70 0x7ff4d41140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:19.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d259c700 1 -- 172.21.15.81:0/3160506427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d4102e70 msgr2=0x7ff4d41140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d259c700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d4102e70 0x7ff4d41140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d259c700 1 -- 172.21.15.81:0/3160506427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c8005040 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4d259c700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 0x7ff4d4113b70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4002a50 tx=0x7ff4c40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:19.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.517+0000 7ff4c2ffd700 1 -- 172.21.15.81:0/3160506427 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c4007cc0 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.518+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d410ddd0 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.518+0000 7ff4c2ffd700 1 -- 172.21.15.81:0/3160506427 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4c400f040 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.518+0000 7ff4c2ffd700 1 -- 172.21.15.81:0/3160506427 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c400da30 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.518+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d410e360 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.519+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d4100110 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.519+0000 7ff4c2ffd700 1 -- 172.21.15.81:0/3160506427 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7ff4c400db90 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.519+0000 7ff4c2ffd700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff4bc06d380 0x7ff4bc06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:19.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.520+0000 7ff4c2ffd700 1 -- 172.21.15.81:0/3160506427 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff4c408c650 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.520+0000 7ff4d1d9b700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff4bc06d380 0x7ff4bc06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:19.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.520+0000 7ff4d1d9b700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff4bc06d380 0x7ff4bc06f830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff4d410edd0 tx=0x7ff4c8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:19.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.524+0000 7ff4c2ffd700 1 -- 172.21.15.81:0/3160506427 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4c4059c60 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.761+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff4d410f1c0 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.761+0000 7ff4c2ffd700 1 -- 172.21.15.81:0/3160506427 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1858 (secure 0 0 0) 0x7ff4c40597f0 con 0x7ff4d40fb4a0 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:e22 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 22 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:12:38.916315+0000 2024-03-22T07:16:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:16:19.763 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 95 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 94 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:16:19.764 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:16:19.765 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:16:19.765 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:16:19.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff4bc06d380 msgr2=0x7ff4bc06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff4bc06d380 0x7ff4bc06f830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff4d410edd0 tx=0x7ff4c8009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 msgr2=0x7ff4d4113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 0x7ff4d4113b70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4002a50 tx=0x7ff4c40046b0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 shutdown_connections 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff4bc06d380 0x7ff4bc06f830 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4d40fb4a0 0x7ff4d4113b70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 --2- 172.21.15.81:0/3160506427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4d4102e70 0x7ff4d41140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 >> 172.21.15.81:0/3160506427 conn(0x7ff4d40714c0 msgr2=0x7ff4d40fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 shutdown_connections 2024-03-22T07:16:19.766 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.764+0000 7ff4d8a67700 1 -- 172.21.15.81:0/3160506427 wait complete. 2024-03-22T07:16:19.767 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 22 2024-03-22T07:16:19.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/2946059625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8101d70 msgr2=0x7f9dd8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 --2- 172.21.15.81:0/2946059625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8101d70 0x7f9dd8102140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc004940 tx=0x7f9dcc009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:19.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/2946059625 shutdown_connections 2024-03-22T07:16:19.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 --2- 172.21.15.81:0/2946059625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dd8102710 0x7f9dd80fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 --2- 172.21.15.81:0/2946059625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8101d70 0x7f9dd8102140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/2946059625 >> 172.21.15.81:0/2946059625 conn(0x7f9dd8072bc0 msgr2=0x7f9dd8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:19.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/2946059625 shutdown_connections 2024-03-22T07:16:19.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/2946059625 wait complete. 2024-03-22T07:16:19.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 Processor -- start 2024-03-22T07:16:19.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.866+0000 7f9ddd1a4700 1 -- start start 2024-03-22T07:16:19.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9ddd1a4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dd8101d70 0x7f9dd810c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:19.912 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9ddd1a4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8102710 0x7f9dd810ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:19.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9ddd1a4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd810d370 con 0x7f9dd8102710 2024-03-22T07:16:19.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9ddd1a4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd810d4e0 con 0x7f9dd8101d70 2024-03-22T07:16:19.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dd659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8102710 0x7f9dd810ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:19.913 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dd659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8102710 0x7f9dd810ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39540/0 (socket says 172.21.15.81:39540) 2024-03-22T07:16:19.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dd659c700 1 -- 172.21.15.81:0/1235724918 learned_addr learned my addr 172.21.15.81:0/1235724918 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:19.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dd6d9d700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dd8101d70 0x7f9dd810c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:19.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dd659c700 1 -- 172.21.15.81:0/1235724918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dd8101d70 msgr2=0x7f9dd810c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:19.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dd659c700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dd8101d70 0x7f9dd810c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:19.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dd659c700 1 -- 172.21.15.81:0/1235724918 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc005040 con 0x7f9dd8102710 2024-03-22T07:16:19.914 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dd659c700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8102710 0x7f9dd810ce30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8002a80 tx=0x7f9dc8004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:19.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dbf7fe700 1 -- 172.21.15.81:0/1235724918 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc8004e80 con 0x7f9dd8102710 2024-03-22T07:16:19.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dbf7fe700 1 -- 172.21.15.81:0/1235724918 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dc800f040 con 0x7f9dd8102710 2024-03-22T07:16:19.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd8119f60 con 0x7f9dd8102710 2024-03-22T07:16:19.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9dbf7fe700 1 -- 172.21.15.81:0/1235724918 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc800daa0 con 0x7f9dd8102710 2024-03-22T07:16:19.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.867+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd811a330 con 0x7f9dd8102710 2024-03-22T07:16:19.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.868+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd80ff080 con 0x7f9dd8102710 2024-03-22T07:16:19.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.869+0000 7f9dbf7fe700 1 -- 172.21.15.81:0/1235724918 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f9dc800dc00 con 0x7f9dd8102710 2024-03-22T07:16:19.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.869+0000 7f9dbf7fe700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9dc006d380 0x7f9dc006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:19.916 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.869+0000 7f9dbf7fe700 1 -- 172.21.15.81:0/1235724918 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9dc808be60 con 0x7f9dd8102710 2024-03-22T07:16:19.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.869+0000 7f9dd6d9d700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9dc006d380 0x7f9dc006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:19.917 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.870+0000 7f9dd6d9d700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9dc006d380 0x7f9dc006f830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc004f90 tx=0x7f9dcc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:19.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:19.874+0000 7f9dbf7fe700 1 -- 172.21.15.81:0/1235724918 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9dc8059470 con 0x7f9dd8102710 2024-03-22T07:16:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:19 smithi081 ceph-mon[113965]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:19 smithi081 ceph-mon[113965]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:19 smithi081 ceph-mon[113965]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:19 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3694901537' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:19 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:16:20.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:19 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3160506427' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:16:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.087+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9dd8108e90 con 0x7f9dc006d380 2024-03-22T07:16:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.088+0000 7f9dbf7fe700 1 -- 172.21.15.81:0/1235724918 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9dd8108e90 con 0x7f9dc006d380 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: "mon", 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr", 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: "crash" 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "10/21 daemons upgraded", 2024-03-22T07:16:20.089 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:16:20.090 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:16:20.090 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:16:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.090+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9dc006d380 msgr2=0x7f9dc006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.090+0000 7f9ddd1a4700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9dc006d380 0x7f9dc006f830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc004f90 tx=0x7f9dcc009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.090+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8102710 msgr2=0x7f9dd810ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.090+0000 7f9ddd1a4700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8102710 0x7f9dd810ce30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8002a80 tx=0x7f9dc8004630 comp rx=0 tx=0).stop 2024-03-22T07:16:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.090+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 shutdown_connections 2024-03-22T07:16:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.090+0000 7f9ddd1a4700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9dc006d380 0x7f9dc006f830 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.090+0000 7f9ddd1a4700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dd8101d70 0x7f9dd810c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.090+0000 7f9ddd1a4700 1 --2- 172.21.15.81:0/1235724918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9dd8102710 0x7f9dd810ce30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.090+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 >> 172.21.15.81:0/1235724918 conn(0x7f9dd8072bc0 msgr2=0x7f9dd80fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.091+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 shutdown_connections 2024-03-22T07:16:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.091+0000 7f9ddd1a4700 1 -- 172.21.15.81:0/1235724918 wait complete. 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.191+0000 7f4a9a582700 1 -- 172.21.15.81:0/1206309481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 msgr2=0x7f4a941033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.191+0000 7f4a9a582700 1 --2- 172.21.15.81:0/1206309481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 0x7f4a941033a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a84004940 tx=0x7f4a84009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.191+0000 7f4a9a582700 1 -- 172.21.15.81:0/1206309481 shutdown_connections 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.191+0000 7f4a9a582700 1 --2- 172.21.15.81:0/1206309481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a940fb540 0x7f4a940fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.191+0000 7f4a9a582700 1 --2- 172.21.15.81:0/1206309481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 0x7f4a941033a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.191+0000 7f4a9a582700 1 -- 172.21.15.81:0/1206309481 >> 172.21.15.81:0/1206309481 conn(0x7f4a94071530 msgr2=0x7f4a94071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.191+0000 7f4a9a582700 1 -- 172.21.15.81:0/1206309481 shutdown_connections 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.191+0000 7f4a9a582700 1 -- 172.21.15.81:0/1206309481 wait complete. 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a9a582700 1 Processor -- start 2024-03-22T07:16:20.193 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a9a582700 1 -- start start 2024-03-22T07:16:20.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a9a582700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a940fb540 0x7f4a94113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:20.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a9a582700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 0x7f4a94113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:20.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a9a582700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a941144b0 con 0x7f4a94102fd0 2024-03-22T07:16:20.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a9a582700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9410dab0 con 0x7f4a940fb540 2024-03-22T07:16:20.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a937fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 0x7f4a94113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:20.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a937fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 0x7f4a94113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39560/0 (socket says 172.21.15.81:39560) 2024-03-22T07:16:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a937fe700 1 -- 172.21.15.81:0/3177102862 learned_addr learned my addr 172.21.15.81:0/3177102862 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a93fff700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a940fb540 0x7f4a94113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a937fe700 1 -- 172.21.15.81:0/3177102862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a940fb540 msgr2=0x7f4a94113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a937fe700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a940fb540 0x7f4a94113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.192+0000 7f4a937fe700 1 -- 172.21.15.81:0/3177102862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a84005040 con 0x7f4a94102fd0 2024-03-22T07:16:20.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.193+0000 7f4a937fe700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 0x7f4a94113e60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a88002a50 tx=0x7f4a880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.193+0000 7f4a90ff9700 1 -- 172.21.15.81:0/3177102862 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a88007cc0 con 0x7f4a94102fd0 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.193+0000 7f4a90ff9700 1 -- 172.21.15.81:0/3177102862 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a8800f040 con 0x7f4a94102fd0 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.193+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9410dd90 con 0x7f4a94102fd0 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.193+0000 7f4a90ff9700 1 -- 172.21.15.81:0/3177102862 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a8800da30 con 0x7f4a94102fd0 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.193+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9410e350 con 0x7f4a94102fd0 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.194+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a94100270 con 0x7f4a94102fd0 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.194+0000 7f4a90ff9700 1 -- 172.21.15.81:0/3177102862 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f4a8800db90 con 0x7f4a94102fd0 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.195+0000 7f4a90ff9700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4a7c06d380 0x7f4a7c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.195+0000 7f4a90ff9700 1 -- 172.21.15.81:0/3177102862 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4a8808c570 con 0x7f4a94102fd0 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.195+0000 7f4a93fff700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4a7c06d380 0x7f4a7c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.195+0000 7f4a93fff700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4a7c06d380 0x7f4a7c06f830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a84004f90 tx=0x7f4a84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:20.199 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.199+0000 7f4a90ff9700 1 -- 172.21.15.81:0/3177102862 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a88059c00 con 0x7f4a94102fd0 2024-03-22T07:16:20.288 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:19 smithi149 ceph-mon[83447]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:20.288 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:19 smithi149 ceph-mon[83447]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:20.288 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:19 smithi149 ceph-mon[83447]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:20.288 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:19 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3694901537' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:20.288 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:19 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:16:20.288 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:19 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3160506427' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:16:20.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.470+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4a9410ef70 con 0x7f4a94102fd0 2024-03-22T07:16:20.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.470+0000 7f4a90ff9700 1 -- 172.21.15.81:0/3177102862 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+250 (secure 0 0 0) 0x7f4a88059790 con 0x7f4a94102fd0 2024-03-22T07:16:20.471 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:16:20.471 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:16:20.471 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:16:20.471 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.473+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4a7c06d380 msgr2=0x7f4a7c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.473+0000 7f4a9a582700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4a7c06d380 0x7f4a7c06f830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a84004f90 tx=0x7f4a84009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.473+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 msgr2=0x7f4a94113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.473+0000 7f4a9a582700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 0x7f4a94113e60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a88002a50 tx=0x7f4a880046b0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.473+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 shutdown_connections 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.473+0000 7f4a9a582700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4a7c06d380 0x7f4a7c06f830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.473+0000 7f4a9a582700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a940fb540 0x7f4a94113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.473+0000 7f4a9a582700 1 --2- 172.21.15.81:0/3177102862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a94102fd0 0x7f4a94113e60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.473+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 >> 172.21.15.81:0/3177102862 conn(0x7f4a94071530 msgr2=0x7f4a940fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.474+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 shutdown_connections 2024-03-22T07:16:20.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:20.474+0000 7f4a9a582700 1 -- 172.21.15.81:0/3177102862 wait complete. 2024-03-22T07:16:21.203 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:20 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:16:21.203 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:20 smithi149 ceph-mon[83447]: Upgrade: osd.4 is safe to restart 2024-03-22T07:16:21.203 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:20 smithi149 ceph-mon[83447]: pgmap v130: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:21.203 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:20 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3177102862' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:16:21.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:20 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:16:21.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:20 smithi081 ceph-mon[113965]: Upgrade: osd.4 is safe to restart 2024-03-22T07:16:21.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:20 smithi081 ceph-mon[113965]: pgmap v130: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:21.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:20 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3177102862' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:16:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:21 smithi149 ceph-mon[83447]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:16:22.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:22.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:21 smithi081 ceph-mon[113965]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:22.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:22.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:16:22.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:23.239 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:22 smithi149 ceph-mon[83447]: Upgrade: Updating osd.4 2024-03-22T07:16:23.239 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:22 smithi149 ceph-mon[83447]: Deploying daemon osd.4 on smithi149 2024-03-22T07:16:23.239 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:22 smithi149 ceph-mon[83447]: pgmap v131: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:23.239 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:22 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 462.630162 secs 2024-03-22T07:16:23.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:22 smithi081 ceph-mon[113965]: Upgrade: Updating osd.4 2024-03-22T07:16:23.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:22 smithi081 ceph-mon[113965]: Deploying daemon osd.4 on smithi149 2024-03-22T07:16:23.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:22 smithi081 ceph-mon[113965]: pgmap v131: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:23.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:22 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 462.630162 secs 2024-03-22T07:16:24.209 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:23 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 462.970689 secs 2024-03-22T07:16:24.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:23 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 462.970689 secs 2024-03-22T07:16:24.814 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:24 smithi149 systemd[1]: Stopping Ceph osd.4 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:16:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:24 smithi149 ceph-mon[83447]: pgmap v132: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:25.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:24 smithi149 ceph-mon[83447]: osd.4 marked itself down and dead 2024-03-22T07:16:25.102 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:24 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[48471]: 2024-03-22T07:16:24.813+0000 7fc604eb3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:16:25.103 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:24 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[48471]: 2024-03-22T07:16:24.813+0000 7fc604eb3700 -1 osd.4 69 *** Got signal Terminated *** 2024-03-22T07:16:25.103 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:24 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[48471]: 2024-03-22T07:16:24.813+0000 7fc604eb3700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:16:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:24 smithi081 ceph-mon[113965]: pgmap v132: 65 pgs: 65 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:24 smithi081 ceph-mon[113965]: osd.4 marked itself down and dead 2024-03-22T07:16:25.539 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:25 smithi149 podman[97473]: 2024-03-22 07:16:25.283523451 +0000 UTC m=+0.570624931 container died d37c642ce9cb3a16d43aa6453cec533460c7fe759519fec9528fff973cb9b5a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-03-22T07:16:25.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:25 smithi149 podman[97473]: 2024-03-22 07:16:25.638557062 +0000 UTC m=+0.925658534 container cleanup d37c642ce9cb3a16d43aa6453cec533460c7fe759519fec9528fff973cb9b5a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-22T07:16:25.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:25 smithi149 bash[97473]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4 2024-03-22T07:16:26.203 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:25 smithi149 ceph-mon[83447]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:16:26.203 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:25 smithi149 ceph-mon[83447]: failed to encode map e70 with expected crc 2024-03-22T07:16:26.204 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:25 smithi149 ceph-mon[83447]: osdmap e70: 6 total, 5 up, 6 in 2024-03-22T07:16:26.204 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:25 smithi149 podman[97490]: 2024-03-22 07:16:25.881777117 +0000 UTC m=+0.600130419 container remove d37c642ce9cb3a16d43aa6453cec533460c7fe759519fec9528fff973cb9b5a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:16:26.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:25 smithi081 ceph-mon[113965]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:16:26.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:25 smithi081 ceph-mon[113965]: failed to encode map e70 with expected crc 2024-03-22T07:16:26.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:25 smithi081 ceph-mon[113965]: osdmap e70: 6 total, 5 up, 6 in 2024-03-22T07:16:26.484 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:26 smithi149 podman[97577]: 2024-03-22 07:16:26.107356963 +0000 UTC m=+0.018352116 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:16:26.485 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:26 smithi149 podman[97577]: 2024-03-22 07:16:26.23596349 +0000 UTC m=+0.146958643 container create 97ec8a969bc56ff0382bd7a0bea284f9fe9190875d96f986b57e1fed7f357d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:16:26.485 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:26 smithi149 podman[97577]: 2024-03-22 07:16:26.485296099 +0000 UTC m=+0.396291253 container init 97ec8a969bc56ff0382bd7a0bea284f9fe9190875d96f986b57e1fed7f357d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:16:26.776 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:26 smithi149 podman[97577]: 2024-03-22 07:16:26.488491078 +0000 UTC m=+0.399486231 container start 97ec8a969bc56ff0382bd7a0bea284f9fe9190875d96f986b57e1fed7f357d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-22T07:16:26.776 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:26 smithi149 podman[97577]: 2024-03-22 07:16:26.56853172 +0000 UTC m=+0.479526886 container attach 97ec8a969bc56ff0382bd7a0bea284f9fe9190875d96f986b57e1fed7f357d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-22T07:16:26.778 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:26 smithi149 podman[97577]: 2024-03-22 07:16:26.777396835 +0000 UTC m=+0.688391991 container died 97ec8a969bc56ff0382bd7a0bea284f9fe9190875d96f986b57e1fed7f357d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-22T07:16:27.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:26 smithi149 ceph-mon[83447]: pgmap v134: 65 pgs: 6 stale+active+clean, 59 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:27.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:26 smithi149 ceph-mon[83447]: osdmap e71: 6 total, 5 up, 6 in 2024-03-22T07:16:27.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:26 smithi149 ceph-mon[83447]: failed to encode map e71 with expected crc 2024-03-22T07:16:27.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:26 smithi149 ceph-mon[83447]: failed to encode map e71 with expected crc 2024-03-22T07:16:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:26 smithi081 ceph-mon[113965]: pgmap v134: 65 pgs: 6 stale+active+clean, 59 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:26 smithi081 ceph-mon[113965]: osdmap e71: 6 total, 5 up, 6 in 2024-03-22T07:16:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:26 smithi081 ceph-mon[113965]: failed to encode map e71 with expected crc 2024-03-22T07:16:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:26 smithi081 ceph-mon[113965]: failed to encode map e71 with expected crc 2024-03-22T07:16:27.517 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:27 smithi149 podman[97603]: 2024-03-22 07:16:27.296950814 +0000 UTC m=+0.513847031 container remove 97ec8a969bc56ff0382bd7a0bea284f9fe9190875d96f986b57e1fed7f357d81 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-22T07:16:27.518 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:27 smithi149 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-22T07:16:27.518 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:27 smithi149 systemd[1]: Stopped Ceph osd.4 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:16:27.518 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:27 smithi149 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.4.service: Consumed 40.276s CPU time. 2024-03-22T07:16:27.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:27 smithi149 systemd[1]: Starting Ceph osd.4 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:16:27.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:27 smithi149 podman[97709]: 2024-03-22 07:16:27.639554156 +0000 UTC m=+0.018362794 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:16:27.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:27 smithi149 podman[97709]: 2024-03-22 07:16:27.74866763 +0000 UTC m=+0.127476272 container create e507cc2129887dc14db87e2c1012a336eac663825ae43eb6f116d41e4a3eceb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-22T07:16:28.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:27 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 467.630296 secs 2024-03-22T07:16:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:27 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 467.630296 secs 2024-03-22T07:16:28.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 podman[97709]: 2024-03-22 07:16:28.026301926 +0000 UTC m=+0.405110564 container init e507cc2129887dc14db87e2c1012a336eac663825ae43eb6f116d41e4a3eceb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS) 2024-03-22T07:16:28.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 podman[97709]: 2024-03-22 07:16:28.029697095 +0000 UTC m=+0.408505740 container start e507cc2129887dc14db87e2c1012a336eac663825ae43eb6f116d41e4a3eceb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:16:28.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 podman[97709]: 2024-03-22 07:16:28.092706657 +0000 UTC m=+0.471515304 container attach e507cc2129887dc14db87e2c1012a336eac663825ae43eb6f116d41e4a3eceb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-22T07:16:28.804 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate[97724]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate[97724]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 bash[97709]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 bash[97709]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 bash[97709]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 bash[97709]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 bash[97709]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 bash[97709]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 bash[97709]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate[97724]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate[97724]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate[97724]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate[97724]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate[97724]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-22T07:16:28.805 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:28 smithi149 podman[97709]: 2024-03-22 07:16:28.500051665 +0000 UTC m=+0.878860304 container died e507cc2129887dc14db87e2c1012a336eac663825ae43eb6f116d41e4a3eceb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate, GIT_CLEAN=True, RELEASE=squid-db0330b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-22T07:16:29.062 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-mon[83447]: pgmap v136: 65 pgs: 15 active+undersized+degraded, 3 stale+active+clean, 47 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 2988/32856 objects degraded (9.094%) 2024-03-22T07:16:29.062 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 467.970766 secs 2024-03-22T07:16:29.062 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:28 smithi149 ceph-mon[83447]: Health check failed: Degraded data redundancy: 2988/32856 objects degraded (9.094%), 15 pgs degraded (PG_DEGRADED) 2024-03-22T07:16:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:28 smithi081 ceph-mon[113965]: pgmap v136: 65 pgs: 15 active+undersized+degraded, 3 stale+active+clean, 47 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 2988/32856 objects degraded (9.094%) 2024-03-22T07:16:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:28 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 467.970766 secs 2024-03-22T07:16:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:28 smithi081 ceph-mon[113965]: Health check failed: Degraded data redundancy: 2988/32856 objects degraded (9.094%), 15 pgs degraded (PG_DEGRADED) 2024-03-22T07:16:29.353 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:29 smithi149 podman[97858]: 2024-03-22 07:16:29.06313883 +0000 UTC m=+0.554344155 container remove e507cc2129887dc14db87e2c1012a336eac663825ae43eb6f116d41e4a3eceb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-22T07:16:29.353 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:29 smithi149 podman[97918]: 2024-03-22 07:16:29.177630533 +0000 UTC m=+0.018380179 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:16:29.353 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:29 smithi149 podman[97918]: 2024-03-22 07:16:29.289145798 +0000 UTC m=+0.129895440 container create 437455ba97c044a7e40a8e75a20fb38f262d46d8d10617f36528e5078880d990 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-22T07:16:29.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:29 smithi149 podman[97918]: 2024-03-22 07:16:29.525459197 +0000 UTC m=+0.366208852 container init 437455ba97c044a7e40a8e75a20fb38f262d46d8d10617f36528e5078880d990 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-22T07:16:29.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:29 smithi149 podman[97918]: 2024-03-22 07:16:29.528588711 +0000 UTC m=+0.369338350 container start 437455ba97c044a7e40a8e75a20fb38f262d46d8d10617f36528e5078880d990 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-db0330b) 2024-03-22T07:16:29.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:29 smithi149 ceph-osd[97937]: -- 172.21.15.149:0/3935815293 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5569db8d51e0 con 0x5569db8a7000 2024-03-22T07:16:29.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:29 smithi149 bash[97918]: 437455ba97c044a7e40a8e75a20fb38f262d46d8d10617f36528e5078880d990 2024-03-22T07:16:29.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:29 smithi149 systemd[1]: Started Ceph osd.4 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:16:30.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:30 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:16:30.072+0000 7f5f8e96b7c0 -1 Falling back to public interface 2024-03-22T07:16:30.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:30.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:30.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:30.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:30 smithi149 ceph-mon[83447]: pgmap v137: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:30 smithi081 ceph-mon[113965]: pgmap v137: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:32.224 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:32.225 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:32.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:32 smithi081 ceph-mon[113965]: pgmap v138: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:32 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 472.630340 secs 2024-03-22T07:16:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:32 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:32 smithi149 ceph-mon[83447]: pgmap v138: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:32 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 472.630340 secs 2024-03-22T07:16:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:33.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:32 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:34.279 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:33 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 472.970853 secs 2024-03-22T07:16:34.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:33 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:34.280 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:33 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:34.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:33 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 472.970853 secs 2024-03-22T07:16:34.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:33 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:34.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:33 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:34.549 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:34 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:16:34.278+0000 7f5f8e96b7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-22T07:16:35.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:34 smithi081 ceph-mon[113965]: pgmap v139: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:35.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:34 smithi149 ceph-mon[83447]: pgmap v139: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:35.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:34 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:16:34.909+0000 7f5f8e96b7c0 -1 osd.4 69 log_to_monitors true 2024-03-22T07:16:36.003 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:35 smithi149 ceph-mon[83447]: from='osd.4 [v2:172.21.15.149:6808/1122192339,v1:172.21.15.149:6809/1122192339]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:16:36.003 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:35 smithi149 ceph-mon[83447]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:16:36.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:35 smithi081 ceph-mon[113965]: from='osd.4 [v2:172.21.15.149:6808/1122192339,v1:172.21.15.149:6809/1122192339]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:16:36.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:35 smithi081 ceph-mon[113965]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:16:37.102 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:16:36 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:16:36.798+0000 7f5f7bdbb700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:16:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:36 smithi149 ceph-mon[83447]: pgmap v140: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:36 smithi149 ceph-mon[83447]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T07:16:37.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:36 smithi149 ceph-mon[83447]: failed to encode map e72 with expected crc 2024-03-22T07:16:37.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:36 smithi149 ceph-mon[83447]: osdmap e72: 6 total, 5 up, 6 in 2024-03-22T07:16:37.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:36 smithi149 ceph-mon[83447]: from='osd.4 [v2:172.21.15.149:6808/1122192339,v1:172.21.15.149:6809/1122192339]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:16:37.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:36 smithi149 ceph-mon[83447]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:16:37.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:36 smithi149 ceph-mon[83447]: from='osd.4 ' entity='osd.4' 2024-03-22T07:16:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:36 smithi081 ceph-mon[113965]: pgmap v140: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.3 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:36 smithi081 ceph-mon[113965]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T07:16:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:36 smithi081 ceph-mon[113965]: failed to encode map e72 with expected crc 2024-03-22T07:16:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:36 smithi081 ceph-mon[113965]: osdmap e72: 6 total, 5 up, 6 in 2024-03-22T07:16:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:36 smithi081 ceph-mon[113965]: from='osd.4 [v2:172.21.15.149:6808/1122192339,v1:172.21.15.149:6809/1122192339]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:16:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:36 smithi081 ceph-mon[113965]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:16:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:36 smithi081 ceph-mon[113965]: from='osd.4 ' entity='osd.4' 2024-03-22T07:16:38.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:38.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:38.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:38.819 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 477.630366 secs 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: Health check update: Degraded data redundancy: 6130/32856 objects degraded (18.657%), 34 pgs degraded (PG_DEGRADED) 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: failed to encode map e73 with expected crc 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: osd.4 [v2:172.21.15.149:6808/1122192339,v1:172.21.15.149:6809/1122192339] boot 2024-03-22T07:16:38.820 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: osdmap e73: 6 total, 6 up, 6 in 2024-03-22T07:16:38.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:16:38.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: pgmap v143: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:38.821 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:38 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 477.970919 secs 2024-03-22T07:16:38.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:38.852 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 477.630366 secs 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: Health check update: Degraded data redundancy: 6130/32856 objects degraded (18.657%), 34 pgs degraded (PG_DEGRADED) 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:16:38.853 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: failed to encode map e73 with expected crc 2024-03-22T07:16:38.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: osd.4 [v2:172.21.15.149:6808/1122192339,v1:172.21.15.149:6809/1122192339] boot 2024-03-22T07:16:38.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: osdmap e73: 6 total, 6 up, 6 in 2024-03-22T07:16:38.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:16:38.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: pgmap v143: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 6130/32856 objects degraded (18.657%) 2024-03-22T07:16:38.854 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:38 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 477.970919 secs 2024-03-22T07:16:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:39 smithi081 ceph-mon[113965]: failed to encode map e74 with expected crc 2024-03-22T07:16:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:39 smithi081 ceph-mon[113965]: osdmap e74: 6 total, 6 up, 6 in 2024-03-22T07:16:40.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:39 smithi081 ceph-mon[113965]: failed to encode map e74 with expected crc 2024-03-22T07:16:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:39 smithi149 ceph-mon[83447]: failed to encode map e74 with expected crc 2024-03-22T07:16:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:39 smithi149 ceph-mon[83447]: osdmap e74: 6 total, 6 up, 6 in 2024-03-22T07:16:40.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:39 smithi149 ceph-mon[83447]: failed to encode map e74 with expected crc 2024-03-22T07:16:41.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:40 smithi081 ceph-mon[113965]: pgmap v145: 65 pgs: 16 peering, 18 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 3323/32856 objects degraded (10.114%) 2024-03-22T07:16:41.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:40 smithi149 ceph-mon[83447]: pgmap v145: 65 pgs: 16 peering, 18 active+undersized+degraded, 31 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 3323/32856 objects degraded (10.114%) 2024-03-22T07:16:42.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:41 smithi081 ceph-mon[113965]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3323/32856 objects degraded (10.114%), 18 pgs degraded) 2024-03-22T07:16:42.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:41 smithi149 ceph-mon[83447]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3323/32856 objects degraded (10.114%), 18 pgs degraded) 2024-03-22T07:16:43.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:42 smithi081 ceph-mon[113965]: pgmap v146: 65 pgs: 16 peering, 49 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:43.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:42 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 482.630469 secs 2024-03-22T07:16:43.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:42 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 482.970989 secs 2024-03-22T07:16:43.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:42 smithi149 ceph-mon[83447]: pgmap v146: 65 pgs: 16 peering, 49 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:43.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:42 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 482.630469 secs 2024-03-22T07:16:43.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:42 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 482.970989 secs 2024-03-22T07:16:45.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:44 smithi081 ceph-mon[113965]: pgmap v147: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:44 smithi149 ceph-mon[83447]: pgmap v147: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:46 smithi081 ceph-mon[113965]: pgmap v148: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:47.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:46 smithi149 ceph-mon[83447]: pgmap v148: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:47.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:47.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:48.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:47 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 487.630606 secs 2024-03-22T07:16:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:47 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 487.630606 secs 2024-03-22T07:16:49.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:48 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 487.971080 secs 2024-03-22T07:16:49.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:48 smithi149 ceph-mon[83447]: pgmap v149: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:49.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:48 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 487.971080 secs 2024-03-22T07:16:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:48 smithi081 ceph-mon[113965]: pgmap v149: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.589+0000 7fb8a4b27700 1 -- 172.21.15.81:0/1257019953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 msgr2=0x7fb8a0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.589+0000 7fb8a4b27700 1 --2- 172.21.15.81:0/1257019953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 0x7fb8a0106e60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb888004940 tx=0x7fb888009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.589+0000 7fb8a4b27700 1 -- 172.21.15.81:0/1257019953 shutdown_connections 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.589+0000 7fb8a4b27700 1 --2- 172.21.15.81:0/1257019953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 0x7fb8a0106e60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.589+0000 7fb8a4b27700 1 --2- 172.21.15.81:0/1257019953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a00fde80 0x7fb8a00fe250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.589+0000 7fb8a4b27700 1 -- 172.21.15.81:0/1257019953 >> 172.21.15.81:0/1257019953 conn(0x7fb8a00f7870 msgr2=0x7fb8a00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.589+0000 7fb8a4b27700 1 -- 172.21.15.81:0/1257019953 shutdown_connections 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.589+0000 7fb8a4b27700 1 -- 172.21.15.81:0/1257019953 wait complete. 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb8a4b27700 1 Processor -- start 2024-03-22T07:16:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb8a4b27700 1 -- start start 2024-03-22T07:16:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb8a4b27700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a00fde80 0x7fb8a0113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb8a4b27700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 0x7fb8a0113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb8a4b27700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a01144a0 con 0x7fb8a00fe820 2024-03-22T07:16:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb8a4b27700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a010daa0 con 0x7fb8a00fde80 2024-03-22T07:16:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb895d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 0x7fb8a0113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb895d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 0x7fb8a0113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42650/0 (socket says 172.21.15.81:42650) 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb895d9b700 1 -- 172.21.15.81:0/2321300722 learned_addr learned my addr 172.21.15.81:0/2321300722 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb89e59c700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a00fde80 0x7fb8a0113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb895d9b700 1 -- 172.21.15.81:0/2321300722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a00fde80 msgr2=0x7fb8a0113910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb895d9b700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a00fde80 0x7fb8a0113910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.590+0000 7fb895d9b700 1 -- 172.21.15.81:0/2321300722 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888005040 con 0x7fb8a00fe820 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.591+0000 7fb895d9b700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 0x7fb8a0113e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb888004890 tx=0x7fb888009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.591+0000 7fb8977fe700 1 -- 172.21.15.81:0/2321300722 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88800e040 con 0x7fb8a00fe820 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.591+0000 7fb8977fe700 1 -- 172.21.15.81:0/2321300722 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb888009af0 con 0x7fb8a00fe820 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.591+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a010dd80 con 0x7fb8a00fe820 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.591+0000 7fb8977fe700 1 -- 172.21.15.81:0/2321300722 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88801c3f0 con 0x7fb8a00fe820 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.591+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a010e2c0 con 0x7fb8a00fe820 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.592+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a0066b10 con 0x7fb8a00fe820 2024-03-22T07:16:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.592+0000 7fb8977fe700 1 -- 172.21.15.81:0/2321300722 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fb88800baf0 con 0x7fb8a00fe820 2024-03-22T07:16:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.593+0000 7fb8977fe700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb88c06d380 0x7fb88c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.593+0000 7fb8977fe700 1 -- 172.21.15.81:0/2321300722 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb88808db50 con 0x7fb8a00fe820 2024-03-22T07:16:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.593+0000 7fb89e59c700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb88c06d380 0x7fb88c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:50.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.593+0000 7fb89e59c700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb88c06d380 0x7fb88c06f830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb890007fd0 tx=0x7fb890009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:50.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.597+0000 7fb8977fe700 1 -- 172.21.15.81:0/2321300722 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb88805b160 con 0x7fb8a00fe820 2024-03-22T07:16:50.808 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.807+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8a01135e0 con 0x7fb88c06d380 2024-03-22T07:16:50.809 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.809+0000 7fb8977fe700 1 -- 172.21.15.81:0/2321300722 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb8a01135e0 con 0x7fb88c06d380 2024-03-22T07:16:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb88c06d380 msgr2=0x7fb88c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb88c06d380 0x7fb88c06f830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb890007fd0 tx=0x7fb890009040 comp rx=0 tx=0).stop 2024-03-22T07:16:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 msgr2=0x7fb8a0113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 0x7fb8a0113e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb888004890 tx=0x7fb888009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 shutdown_connections 2024-03-22T07:16:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb88c06d380 0x7fb88c06f830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8a00fde80 0x7fb8a0113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 --2- 172.21.15.81:0/2321300722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8a00fe820 0x7fb8a0113e50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:50.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 >> 172.21.15.81:0/2321300722 conn(0x7fb8a00f7870 msgr2=0x7fb8a00f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:50.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 shutdown_connections 2024-03-22T07:16:50.812 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.811+0000 7fb8a4b27700 1 -- 172.21.15.81:0/2321300722 wait complete. 2024-03-22T07:16:50.822 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:16:50.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.921+0000 7f38b5088700 1 -- 172.21.15.81:0/1979233345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b0102e10 msgr2=0x7f38b01031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.921+0000 7f38b5088700 1 --2- 172.21.15.81:0/1979233345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b0102e10 0x7f38b01031e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f38a0004940 tx=0x7f38a0009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.921+0000 7f38b5088700 1 -- 172.21.15.81:0/1979233345 shutdown_connections 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 --2- 172.21.15.81:0/1979233345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38b01037b0 0x7f38b00fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 --2- 172.21.15.81:0/1979233345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b0102e10 0x7f38b01031e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 -- 172.21.15.81:0/1979233345 >> 172.21.15.81:0/1979233345 conn(0x7f38b00f8980 msgr2=0x7f38b00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 -- 172.21.15.81:0/1979233345 shutdown_connections 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 -- 172.21.15.81:0/1979233345 wait complete. 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 Processor -- start 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 -- start start 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38b0102e10 0x7f38b0113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b01037b0 0x7f38b01140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01146f0 con 0x7f38b01037b0 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.922+0000 7f38b5088700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b010dae0 con 0x7f38b0102e10 2024-03-22T07:16:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b01037b0 0x7f38b01140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b01037b0 0x7f38b01140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42664/0 (socket says 172.21.15.81:42664) 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a659c700 1 -- 172.21.15.81:0/1748580893 learned_addr learned my addr 172.21.15.81:0/1748580893 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38aed9d700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38b0102e10 0x7f38b0113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a659c700 1 -- 172.21.15.81:0/1748580893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38b0102e10 msgr2=0x7f38b0113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a659c700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38b0102e10 0x7f38b0113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a659c700 1 -- 172.21.15.81:0/1748580893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0005040 con 0x7f38b01037b0 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a659c700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b01037b0 0x7f38b01140a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f389800c800 tx=0x7f3898004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a7fff700 1 -- 172.21.15.81:0/1748580893 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38980106f0 con 0x7f38b01037b0 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a7fff700 1 -- 172.21.15.81:0/1748580893 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3898010d20 con 0x7f38b01037b0 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b010ddc0 con 0x7f38b01037b0 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38a7fff700 1 -- 172.21.15.81:0/1748580893 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38980113a0 con 0x7f38b01037b0 2024-03-22T07:16:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.923+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b010e380 con 0x7f38b01037b0 2024-03-22T07:16:50.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.924+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b0100120 con 0x7f38b01037b0 2024-03-22T07:16:50.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.925+0000 7f38a7fff700 1 -- 172.21.15.81:0/1748580893 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f3898012400 con 0x7f38b01037b0 2024-03-22T07:16:50.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.925+0000 7f38a7fff700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f389c06d390 0x7f389c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:50.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.925+0000 7f38a7fff700 1 -- 172.21.15.81:0/1748580893 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f389808c7d0 con 0x7f38b01037b0 2024-03-22T07:16:50.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.925+0000 7f38aed9d700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f389c06d390 0x7f389c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:50.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.926+0000 7f38aed9d700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f389c06d390 0x7f389c06f840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f38a000b3b0 tx=0x7f38a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:50.930 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:50.930+0000 7f38a7fff700 1 -- 172.21.15.81:0/1748580893 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3898059e60 con 0x7f38b01037b0 2024-03-22T07:16:51.142 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.142+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38b010f220 con 0x7f389c06d390 2024-03-22T07:16:51.143 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.143+0000 7f38a7fff700 1 -- 172.21.15.81:0/1748580893 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f38b010f220 con 0x7f389c06d390 2024-03-22T07:16:51.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f389c06d390 msgr2=0x7f389c06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f389c06d390 0x7f389c06f840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f38a000b3b0 tx=0x7f38a0009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b01037b0 msgr2=0x7f38b01140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b01037b0 0x7f38b01140a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f389800c800 tx=0x7f3898004470 comp rx=0 tx=0).stop 2024-03-22T07:16:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 shutdown_connections 2024-03-22T07:16:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f389c06d390 0x7f389c06f840 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38b0102e10 0x7f38b0113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 --2- 172.21.15.81:0/1748580893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b01037b0 0x7f38b01140a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 >> 172.21.15.81:0/1748580893 conn(0x7f38b00f8980 msgr2=0x7f38b01073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:51.146 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:50 smithi081 ceph-mon[113965]: pgmap v150: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:51.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 shutdown_connections 2024-03-22T07:16:51.147 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.145+0000 7f38b5088700 1 -- 172.21.15.81:0/1748580893 wait complete. 2024-03-22T07:16:51.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.246+0000 7f157d25e700 1 -- 172.21.15.81:0/1698800051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15781037a0 msgr2=0x7f15780fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.248 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.246+0000 7f157d25e700 1 --2- 172.21.15.81:0/1698800051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15781037a0 0x7f15780fce10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1568004940 tx=0x7f1568009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.246+0000 7f157d25e700 1 -- 172.21.15.81:0/1698800051 shutdown_connections 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.246+0000 7f157d25e700 1 --2- 172.21.15.81:0/1698800051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15781037a0 0x7f15780fce10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.246+0000 7f157d25e700 1 --2- 172.21.15.81:0/1698800051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1578102e00 0x7f15781031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.246+0000 7f157d25e700 1 -- 172.21.15.81:0/1698800051 >> 172.21.15.81:0/1698800051 conn(0x7f15780f89b0 msgr2=0x7f15780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.246+0000 7f157d25e700 1 -- 172.21.15.81:0/1698800051 shutdown_connections 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.246+0000 7f157d25e700 1 -- 172.21.15.81:0/1698800051 wait complete. 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f157d25e700 1 Processor -- start 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f157d25e700 1 -- start start 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f157d25e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578102e00 0x7f1578113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f157d25e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15781037a0 0x7f1578113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f157d25e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781145b0 con 0x7f1578102e00 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f157d25e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157810db40 con 0x7f15781037a0 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f1576d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578102e00 0x7f1578113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f1576d9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578102e00 0x7f1578113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42670/0 (socket says 172.21.15.81:42670) 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f1576d9d700 1 -- 172.21.15.81:0/2893008263 learned_addr learned my addr 172.21.15.81:0/2893008263 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f157659c700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15781037a0 0x7f1578113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f1576d9d700 1 -- 172.21.15.81:0/2893008263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15781037a0 msgr2=0x7f1578113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f1576d9d700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15781037a0 0x7f1578113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.247+0000 7f1576d9d700 1 -- 172.21.15.81:0/2893008263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568005040 con 0x7f1578102e00 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.248+0000 7f1576d9d700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578102e00 0x7f1578113990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f156c00b790 tx=0x7f156c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.248+0000 7f155f7fe700 1 -- 172.21.15.81:0/2893008263 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f156c00f6f0 con 0x7f1578102e00 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.248+0000 7f155f7fe700 1 -- 172.21.15.81:0/2893008263 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f156c00fd20 con 0x7f1578102e00 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.248+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157810de20 con 0x7f1578102e00 2024-03-22T07:16:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.248+0000 7f155f7fe700 1 -- 172.21.15.81:0/2893008263 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f156c0103a0 con 0x7f1578102e00 2024-03-22T07:16:51.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.248+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157810e3b0 con 0x7f1578102e00 2024-03-22T07:16:51.251 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.249+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1578100110 con 0x7f1578102e00 2024-03-22T07:16:51.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.249+0000 7f155f7fe700 1 -- 172.21.15.81:0/2893008263 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f156c011400 con 0x7f1578102e00 2024-03-22T07:16:51.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.250+0000 7f155f7fe700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f156006d380 0x7f156006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:51.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.250+0000 7f155f7fe700 1 -- 172.21.15.81:0/2893008263 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f156c08c5a0 con 0x7f1578102e00 2024-03-22T07:16:51.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.250+0000 7f157659c700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f156006d380 0x7f156006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:51.252 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.250+0000 7f157659c700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f156006d380 0x7f156006f830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1568004890 tx=0x7f1568009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:51.255 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.254+0000 7f155f7fe700 1 -- 172.21.15.81:0/2893008263 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f156c0561a0 con 0x7f1578102e00 2024-03-22T07:16:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:50 smithi149 ceph-mon[83447]: pgmap v150: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:51.458 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.458+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1578109fc0 con 0x7f156006d380 2024-03-22T07:16:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.469+0000 7f155f7fe700 1 -- 172.21.15.81:0/2893008263 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1578109fc0 con 0x7f156006d380 2024-03-22T07:16:51.470 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:16:51.470 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (5m) 85s ago 15m 26.0M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:16:51.470 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (3m) 85s ago 15m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:16:51.470 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (3m) 18s ago 15m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:16:51.470 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (5m) 85s ago 15m 84.5M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:16:51.470 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (10m) 85s ago 10m 13.7M - 17.2.7-726-g9c4e0c34 837124da804f 25637addfe1f 2024-03-22T07:16:51.470 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (10m) 85s ago 10m 131M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:16:51.470 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (10m) 18s ago 10m 17.5M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:16:51.470 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (10m) 18s ago 10m 147M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (6m) 85s ago 17m 502M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (6m) 18s ago 14m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (4m) 85s ago 17m 44.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (3m) 18s ago 14m 34.6M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (6m) 85s ago 15m 16.2M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (6m) 18s ago 14m 17.0M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (2m) 85s ago 14m 94.5M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 033c080308b6 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (2m) 85s ago 13m 105M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 8341fe32f3fa 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (91s) 85s ago 13m 12.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 0b8c514dced1 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (55s) 18s ago 12m 104M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c8234f7ed003 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (22s) 18s ago 12m 11.9M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 437455ba97c0 2024-03-22T07:16:51.471 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (11m) 18s ago 11m 184M 4096M 17.2.7-726-g9c4e0c34 837124da804f 1de675bda255 2024-03-22T07:16:51.472 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (5m) 85s ago 15m 59.1M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:16:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f156006d380 msgr2=0x7f156006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f156006d380 0x7f156006f830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1568004890 tx=0x7f1568009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578102e00 msgr2=0x7f1578113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578102e00 0x7f1578113990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f156c00b790 tx=0x7f156c009250 comp rx=0 tx=0).stop 2024-03-22T07:16:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 shutdown_connections 2024-03-22T07:16:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f156006d380 0x7f156006f830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1578102e00 0x7f1578113990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 --2- 172.21.15.81:0/2893008263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15781037a0 0x7f1578113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 >> 172.21.15.81:0/2893008263 conn(0x7f15780f89b0 msgr2=0x7f15780fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 shutdown_connections 2024-03-22T07:16:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.471+0000 7f157d25e700 1 -- 172.21.15.81:0/2893008263 wait complete. 2024-03-22T07:16:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 -- 172.21.15.81:0/2847009296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 msgr2=0x7f6b10102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 --2- 172.21.15.81:0/2847009296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 0x7f6b10102140 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b00004940 tx=0x7f6b00009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 -- 172.21.15.81:0/2847009296 shutdown_connections 2024-03-22T07:16:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 --2- 172.21.15.81:0/2847009296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b10102710 0x7f6b100fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 --2- 172.21.15.81:0/2847009296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 0x7f6b10102140 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 -- 172.21.15.81:0/2847009296 >> 172.21.15.81:0/2847009296 conn(0x7f6b10072bc0 msgr2=0x7f6b10072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 -- 172.21.15.81:0/2847009296 shutdown_connections 2024-03-22T07:16:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 -- 172.21.15.81:0/2847009296 wait complete. 2024-03-22T07:16:51.574 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 Processor -- start 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.572+0000 7f6b15892700 1 -- start start 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b15892700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 0x7f6b10112a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b15892700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b10102710 0x7f6b10112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b15892700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101135d0 con 0x7f6b10101d70 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b15892700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1010c9c0 con 0x7f6b10102710 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b0effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 0x7f6b10112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b0effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 0x7f6b10112a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42700/0 (socket says 172.21.15.81:42700) 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b0effd700 1 -- 172.21.15.81:0/4119696224 learned_addr learned my addr 172.21.15.81:0/4119696224 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b0e7fc700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b10102710 0x7f6b10112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b0effd700 1 -- 172.21.15.81:0/4119696224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b10102710 msgr2=0x7f6b10112f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b0effd700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b10102710 0x7f6b10112f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b0effd700 1 -- 172.21.15.81:0/4119696224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b00005040 con 0x7f6b10101d70 2024-03-22T07:16:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b0effd700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 0x7f6b10112a40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b3b0 tx=0x7f6b00009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6af77fe700 1 -- 172.21.15.81:0/4119696224 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b0000e040 con 0x7f6b10101d70 2024-03-22T07:16:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6af77fe700 1 -- 172.21.15.81:0/4119696224 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b00013420 con 0x7f6b10101d70 2024-03-22T07:16:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6af77fe700 1 -- 172.21.15.81:0/4119696224 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b0001c630 con 0x7f6b10101d70 2024-03-22T07:16:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.573+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1010cc40 con 0x7f6b10101d70 2024-03-22T07:16:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.574+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1010d200 con 0x7f6b10101d70 2024-03-22T07:16:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.574+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b100ff080 con 0x7f6b10101d70 2024-03-22T07:16:51.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.575+0000 7f6af77fe700 1 -- 172.21.15.81:0/4119696224 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f6b00013590 con 0x7f6b10101d70 2024-03-22T07:16:51.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.575+0000 7f6af77fe700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6af806d380 0x7f6af806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:51.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.575+0000 7f6af77fe700 1 -- 172.21.15.81:0/4119696224 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6b0008d9e0 con 0x7f6b10101d70 2024-03-22T07:16:51.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.575+0000 7f6b0e7fc700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6af806d380 0x7f6af806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:51.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.576+0000 7f6b0e7fc700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6af806d380 0x7f6af806f830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b1010db80 tx=0x7f6b0400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:51.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.580+0000 7f6af77fe700 1 -- 172.21.15.81:0/4119696224 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b0005b110 con 0x7f6b10101d70 2024-03-22T07:16:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.854+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b1010e090 con 0x7f6b10101d70 2024-03-22T07:16:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.854+0000 7f6af77fe700 1 -- 172.21.15.81:0/4119696224 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f6b0005aca0 con 0x7f6b10101d70 2024-03-22T07:16:51.855 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:16:51.855 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:16:51.855 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:16:51.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:16:51.855 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 5 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5, 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 9 2024-03-22T07:16:51.856 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:16:51.857 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:16:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.856+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6af806d380 msgr2=0x7f6af806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.856+0000 7f6b15892700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6af806d380 0x7f6af806f830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b1010db80 tx=0x7f6b0400b040 comp rx=0 tx=0).stop 2024-03-22T07:16:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.856+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 msgr2=0x7f6b10112a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.856+0000 7f6b15892700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 0x7f6b10112a40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b3b0 tx=0x7f6b00009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.856+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 shutdown_connections 2024-03-22T07:16:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.856+0000 7f6b15892700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6af806d380 0x7f6af806f830 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.856+0000 7f6b15892700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b10101d70 0x7f6b10112a40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.856+0000 7f6b15892700 1 --2- 172.21.15.81:0/4119696224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b10102710 0x7f6b10112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.856+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 >> 172.21.15.81:0/4119696224 conn(0x7f6b10072bc0 msgr2=0x7f6b100fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.857+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 shutdown_connections 2024-03-22T07:16:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.857+0000 7f6b15892700 1 -- 172.21.15.81:0/4119696224 wait complete. 2024-03-22T07:16:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.957+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1986673082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db4102fd0 msgr2=0x7f0db41033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.957+0000 7f0dbbd6a700 1 --2- 172.21.15.81:0/1986673082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db4102fd0 0x7f0db41033a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ce0 tx=0x7f0dac004d10 comp rx=0 tx=0).stop 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.957+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1986673082 shutdown_connections 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.957+0000 7f0dbbd6a700 1 --2- 172.21.15.81:0/1986673082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0db40fb540 0x7f0db40fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.957+0000 7f0dbbd6a700 1 --2- 172.21.15.81:0/1986673082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db4102fd0 0x7f0db41033a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.957+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1986673082 >> 172.21.15.81:0/1986673082 conn(0x7f0db4071530 msgr2=0x7f0db4071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.957+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1986673082 shutdown_connections 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.957+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1986673082 wait complete. 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0dbbd6a700 1 Processor -- start 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0dbbd6a700 1 -- start start 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0dbbd6a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db40fb540 0x7f0db4113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0dbbd6a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0db4102fd0 0x7f0db4113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0dbbd6a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db4114630 con 0x7f0db40fb540 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0dbbd6a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db410dc10 con 0x7f0db4102fd0 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0db9b06700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db40fb540 0x7f0db4113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0db9b06700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db40fb540 0x7f0db4113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42724/0 (socket says 172.21.15.81:42724) 2024-03-22T07:16:51.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0db9b06700 1 -- 172.21.15.81:0/1904237823 learned_addr learned my addr 172.21.15.81:0/1904237823 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0db9305700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0db4102fd0 0x7f0db4113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0db9b06700 1 -- 172.21.15.81:0/1904237823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0db4102fd0 msgr2=0x7f0db4113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0db9b06700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0db4102fd0 0x7f0db4113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0db9b06700 1 -- 172.21.15.81:0/1904237823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac009940 con 0x7f0db40fb540 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.958+0000 7f0db9b06700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db40fb540 0x7f0db4113aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0dac003400 tx=0x7f0dac004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.959+0000 7f0db27fc700 1 -- 172.21.15.81:0/1904237823 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dac00e040 con 0x7f0db40fb540 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.959+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db410de90 con 0x7f0db40fb540 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.959+0000 7f0db27fc700 1 -- 172.21.15.81:0/1904237823 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dac004610 con 0x7f0db40fb540 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.959+0000 7f0db27fc700 1 -- 172.21.15.81:0/1904237823 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dac01c640 con 0x7f0db40fb540 2024-03-22T07:16:51.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.959+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db410e450 con 0x7f0db40fb540 2024-03-22T07:16:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.960+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db4100270 con 0x7f0db40fb540 2024-03-22T07:16:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.960+0000 7f0db27fc700 1 -- 172.21.15.81:0/1904237823 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f0dac012a70 con 0x7f0db40fb540 2024-03-22T07:16:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.961+0000 7f0db27fc700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f0d9c06d380 0x7f0d9c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.961+0000 7f0db27fc700 1 -- 172.21.15.81:0/1904237823 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0dac08e1e0 con 0x7f0db40fb540 2024-03-22T07:16:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.961+0000 7f0db9305700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f0d9c06d380 0x7f0d9c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:51.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.961+0000 7f0db9305700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f0d9c06d380 0x7f0d9c06f830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0db410f340 tx=0x7f0da800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:51.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:51.965+0000 7f0db27fc700 1 -- 172.21.15.81:0/1904237823 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0dac05b870 con 0x7f0db40fb540 2024-03-22T07:16:52.200 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.200+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0db410a020 con 0x7f0db40fb540 2024-03-22T07:16:52.201 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:51 smithi081 ceph-mon[113965]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:52.201 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:51 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:52.201 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:51 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/4119696224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:52.201 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.200+0000 7f0db27fc700 1 -- 172.21.15.81:0/1904237823 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1858 (secure 0 0 0) 0x7f0dac004e10 con 0x7f0db40fb540 2024-03-22T07:16:52.201 INFO:teuthology.orchestra.run.smithi081.stdout:e22 2024-03-22T07:16:52.201 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:16:52.201 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:16:52.201 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 22 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:12:38.916315+0000 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:16:52.202 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 45 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=14488,1=24251} 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:16:52.203 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{0:14488} state up:active seq 95 join_fscid=1 addr [v2:172.21.15.81:6828/268578316,v1:172.21.15.81:6829/268578316] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{1:24251} state up:active seq 94 join_fscid=1 addr [v2:172.21.15.149:6824/3521694486,v1:172.21.15.149:6825/3521694486] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:14496} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/4012792796,v1:172.21.15.81:6827/4012792796] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:16:52.204 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:16:52.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.202+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f0d9c06d380 msgr2=0x7f0d9c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.202+0000 7f0dbbd6a700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f0d9c06d380 0x7f0d9c06f830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0db410f340 tx=0x7f0da800b040 comp rx=0 tx=0).stop 2024-03-22T07:16:52.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.203+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db40fb540 msgr2=0x7f0db4113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.203+0000 7f0dbbd6a700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db40fb540 0x7f0db4113aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0dac003400 tx=0x7f0dac004d10 comp rx=0 tx=0).stop 2024-03-22T07:16:52.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.203+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 shutdown_connections 2024-03-22T07:16:52.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.203+0000 7f0dbbd6a700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f0d9c06d380 0x7f0d9c06f830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.203+0000 7f0dbbd6a700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0db40fb540 0x7f0db4113aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.203+0000 7f0dbbd6a700 1 --2- 172.21.15.81:0/1904237823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0db4102fd0 0x7f0db4113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.203+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 >> 172.21.15.81:0/1904237823 conn(0x7f0db4071530 msgr2=0x7f0db40fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:52.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.203+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 shutdown_connections 2024-03-22T07:16:52.206 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.203+0000 7f0dbbd6a700 1 -- 172.21.15.81:0/1904237823 wait complete. 2024-03-22T07:16:52.206 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 22 2024-03-22T07:16:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 -- 172.21.15.81:0/105263664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 msgr2=0x7efeac103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 --2- 172.21.15.81:0/105263664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 0x7efeac103340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efea4004940 tx=0x7efea4009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 -- 172.21.15.81:0/105263664 shutdown_connections 2024-03-22T07:16:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 --2- 172.21.15.81:0/105263664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeac0fb3b0 0x7efeac0fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 --2- 172.21.15.81:0/105263664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 0x7efeac103340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 -- 172.21.15.81:0/105263664 >> 172.21.15.81:0/105263664 conn(0x7efeac072310 msgr2=0x7efeac072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 -- 172.21.15.81:0/105263664 shutdown_connections 2024-03-22T07:16:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 -- 172.21.15.81:0/105263664 wait complete. 2024-03-22T07:16:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 Processor -- start 2024-03-22T07:16:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.303+0000 7efeb113f700 1 -- start start 2024-03-22T07:16:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeb113f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeac0fb3b0 0x7efeac113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeb113f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 0x7efeac113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeb113f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac114490 con 0x7efeac102f70 2024-03-22T07:16:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeb113f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac10da90 con 0x7efeac0fb3b0 2024-03-22T07:16:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeaa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 0x7efeac113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeaa59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 0x7efeac113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42736/0 (socket says 172.21.15.81:42736) 2024-03-22T07:16:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeaa59c700 1 -- 172.21.15.81:0/866785265 learned_addr learned my addr 172.21.15.81:0/866785265 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeaad9d700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeac0fb3b0 0x7efeac113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeaa59c700 1 -- 172.21.15.81:0/866785265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeac0fb3b0 msgr2=0x7efeac113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeaa59c700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeac0fb3b0 0x7efeac113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeaa59c700 1 -- 172.21.15.81:0/866785265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea4005040 con 0x7efeac102f70 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeaa59c700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 0x7efeac113e40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efea000cb10 tx=0x7efea0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efe937fe700 1 -- 172.21.15.81:0/866785265 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efea0004200 con 0x7efeac102f70 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac10dd70 con 0x7efeac102f70 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efe937fe700 1 -- 172.21.15.81:0/866785265 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efea000ac70 con 0x7efeac102f70 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efe937fe700 1 -- 172.21.15.81:0/866785265 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efea0003d30 con 0x7efeac102f70 2024-03-22T07:16:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.304+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac10e330 con 0x7efeac102f70 2024-03-22T07:16:52.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.305+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac0fe6d0 con 0x7efeac102f70 2024-03-22T07:16:52.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.306+0000 7efe937fe700 1 -- 172.21.15.81:0/866785265 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7efea000a7a0 con 0x7efeac102f70 2024-03-22T07:16:52.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.306+0000 7efe937fe700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7efe9406d380 0x7efe9406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:52.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.307+0000 7efe937fe700 1 -- 172.21.15.81:0/866785265 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efea008cd40 con 0x7efeac102f70 2024-03-22T07:16:52.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.307+0000 7efeaad9d700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7efe9406d380 0x7efe9406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:52.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.307+0000 7efeaad9d700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7efe9406d380 0x7efe9406f830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efea4004f90 tx=0x7efea4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:52.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.311+0000 7efe937fe700 1 -- 172.21.15.81:0/866785265 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efea005a350 con 0x7efeac102f70 2024-03-22T07:16:52.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:51 smithi149 ceph-mon[83447]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:52.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:51 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:52.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:51 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/4119696224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:52.522 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.521+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efeac10a000 con 0x7efe9406d380 2024-03-22T07:16:52.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.523+0000 7efe937fe700 1 -- 172.21.15.81:0/866785265 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7efeac10a000 con 0x7efe9406d380 2024-03-22T07:16:52.523 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:16:52.523 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:16:52.523 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:16:52.523 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:16:52.523 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:16:52.523 INFO:teuthology.orchestra.run.smithi081.stdout: "mon", 2024-03-22T07:16:52.524 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr", 2024-03-22T07:16:52.524 INFO:teuthology.orchestra.run.smithi081.stdout: "crash" 2024-03-22T07:16:52.524 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:16:52.524 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "11/21 daemons upgraded", 2024-03-22T07:16:52.524 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:16:52.524 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:16:52.524 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:16:52.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7efe9406d380 msgr2=0x7efe9406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7efe9406d380 0x7efe9406f830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efea4004f90 tx=0x7efea4009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:52.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 msgr2=0x7efeac113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 0x7efeac113e40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efea000cb10 tx=0x7efea0007f90 comp rx=0 tx=0).stop 2024-03-22T07:16:52.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 shutdown_connections 2024-03-22T07:16:52.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7efe9406d380 0x7efe9406f830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeac0fb3b0 0x7efeac113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 --2- 172.21.15.81:0/866785265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeac102f70 0x7efeac113e40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 >> 172.21.15.81:0/866785265 conn(0x7efeac072310 msgr2=0x7efeac0f9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:52.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 shutdown_connections 2024-03-22T07:16:52.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.525+0000 7efeb113f700 1 -- 172.21.15.81:0/866785265 wait complete. 2024-03-22T07:16:52.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.625+0000 7f4b27286700 1 -- 172.21.15.81:0/405015363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 msgr2=0x7f4b200fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.625+0000 7f4b27286700 1 --2- 172.21.15.81:0/405015363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 0x7f4b200fb940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4b10004940 tx=0x7f4b10009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.625+0000 7f4b27286700 1 -- 172.21.15.81:0/405015363 shutdown_connections 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.625+0000 7f4b27286700 1 --2- 172.21.15.81:0/405015363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 0x7f4b200fb940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.625+0000 7f4b27286700 1 --2- 172.21.15.81:0/405015363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20102f50 0x7f4b20103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.625+0000 7f4b27286700 1 -- 172.21.15.81:0/405015363 >> 172.21.15.81:0/405015363 conn(0x7f4b200714c0 msgr2=0x7f4b200718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.625+0000 7f4b27286700 1 -- 172.21.15.81:0/405015363 shutdown_connections 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.625+0000 7f4b27286700 1 -- 172.21.15.81:0/405015363 wait complete. 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b27286700 1 Processor -- start 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b27286700 1 -- start start 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b27286700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 0x7f4b20113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b27286700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20102f50 0x7f4b20113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b27286700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20114680 con 0x7f4b200fb4d0 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b27286700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2010dbd0 con 0x7f4b20102f50 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b25022700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 0x7f4b20113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b25022700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 0x7f4b20113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42746/0 (socket says 172.21.15.81:42746) 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b25022700 1 -- 172.21.15.81:0/2357825419 learned_addr learned my addr 172.21.15.81:0/2357825419 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.626+0000 7f4b24821700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20102f50 0x7f4b20113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b25022700 1 -- 172.21.15.81:0/2357825419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20102f50 msgr2=0x7f4b20113fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b25022700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20102f50 0x7f4b20113fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b25022700 1 -- 172.21.15.81:0/2357825419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b10005040 con 0x7f4b200fb4d0 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b24821700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20102f50 0x7f4b20113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b25022700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 0x7f4b20113a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00b790 tx=0x7f4b1c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b15ffb700 1 -- 172.21.15.81:0/2357825419 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1c00f6f0 con 0x7f4b200fb4d0 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b15ffb700 1 -- 172.21.15.81:0/2357825419 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b1c00fd20 con 0x7f4b200fb4d0 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2010de50 con 0x7f4b200fb4d0 2024-03-22T07:16:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b15ffb700 1 -- 172.21.15.81:0/2357825419 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1c0103a0 con 0x7f4b200fb4d0 2024-03-22T07:16:52.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.627+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2010e390 con 0x7f4b200fb4d0 2024-03-22T07:16:52.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.628+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b20100200 con 0x7f4b200fb4d0 2024-03-22T07:16:52.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.628+0000 7f4b15ffb700 1 -- 172.21.15.81:0/2357825419 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f4b1c011400 con 0x7f4b200fb4d0 2024-03-22T07:16:52.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.629+0000 7f4b15ffb700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b0c06d380 0x7f4b0c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:52.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.629+0000 7f4b24821700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b0c06d380 0x7f4b0c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:52.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.629+0000 7f4b15ffb700 1 -- 172.21.15.81:0/2357825419 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4b1c08c640 con 0x7f4b200fb4d0 2024-03-22T07:16:52.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.629+0000 7f4b24821700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b0c06d380 0x7f4b0c06f830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b10004f90 tx=0x7f4b10009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:52.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.634+0000 7f4b15ffb700 1 -- 172.21.15.81:0/2357825419 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4b1c059cd0 con 0x7f4b200fb4d0 2024-03-22T07:16:52.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.906+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4b2010a140 con 0x7f4b200fb4d0 2024-03-22T07:16:52.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.906+0000 7f4b15ffb700 1 -- 172.21.15.81:0/2357825419 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+250 (secure 0 0 0) 0x7f4b1c059860 con 0x7f4b200fb4d0 2024-03-22T07:16:52.906 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_WARN 2 MDSs report slow requests 2024-03-22T07:16:52.906 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-03-22T07:16:52.907 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi081.zfgxzy(mds.0): 1 slow requests are blocked > 180 secs 2024-03-22T07:16:52.907 INFO:teuthology.orchestra.run.smithi081.stdout: mds.cephfs.smithi149.kdtxfw(mds.1): 1 slow requests are blocked > 180 secs 2024-03-22T07:16:52.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b0c06d380 msgr2=0x7f4b0c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b0c06d380 0x7f4b0c06f830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b10004f90 tx=0x7f4b10009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 msgr2=0x7f4b20113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 0x7f4b20113a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00b790 tx=0x7f4b1c009220 comp rx=0 tx=0).stop 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 shutdown_connections 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f4b0c06d380 0x7f4b0c06f830 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b200fb4d0 0x7f4b20113a60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 --2- 172.21.15.81:0/2357825419 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b20102f50 0x7f4b20113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 >> 172.21.15.81:0/2357825419 conn(0x7f4b200714c0 msgr2=0x7f4b200fa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 shutdown_connections 2024-03-22T07:16:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:16:52.908+0000 7f4b27286700 1 -- 172.21.15.81:0/2357825419 wait complete. 2024-03-22T07:16:53.233 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:52 smithi149 ceph-mon[83447]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:53.233 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:52 smithi149 ceph-mon[83447]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:53.233 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:52 smithi149 ceph-mon[83447]: pgmap v151: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:53.233 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:52 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1904237823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:16:53.234 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:52 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 492.630690 secs 2024-03-22T07:16:53.234 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:52 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:16:53.234 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:52 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2357825419' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:16:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:52 smithi081 ceph-mon[113965]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:52 smithi081 ceph-mon[113965]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:52 smithi081 ceph-mon[113965]: pgmap v151: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:52 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1904237823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:16:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:52 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 492.630690 secs 2024-03-22T07:16:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:52 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:16:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:52 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2357825419' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:16:54.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:53 smithi149 ceph-mon[83447]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:54.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:53 smithi149 ceph-mon[83447]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:16:54.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:53 smithi149 ceph-mon[83447]: Upgrade: osd.5 is safe to restart 2024-03-22T07:16:54.281 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:53 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 492.971189 secs 2024-03-22T07:16:54.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:53 smithi081 ceph-mon[113965]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:54.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:53 smithi081 ceph-mon[113965]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:16:54.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:53 smithi081 ceph-mon[113965]: Upgrade: osd.5 is safe to restart 2024-03-22T07:16:54.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:53 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 492.971189 secs 2024-03-22T07:16:55.176 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:54 smithi149 ceph-mon[83447]: pgmap v152: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:55.176 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:54 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:55.176 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:54 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:16:55.176 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:54 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:54 smithi081 ceph-mon[113965]: pgmap v152: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:54 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:16:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:54 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:16:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:54 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:56.041 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:55 smithi149 ceph-mon[83447]: Upgrade: Updating osd.5 2024-03-22T07:16:56.041 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:55 smithi149 ceph-mon[83447]: Deploying daemon osd.5 on smithi149 2024-03-22T07:16:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:55 smithi081 ceph-mon[113965]: Upgrade: Updating osd.5 2024-03-22T07:16:56.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:55 smithi081 ceph-mon[113965]: Deploying daemon osd.5 on smithi149 2024-03-22T07:16:57.293 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:56 smithi149 ceph-mon[83447]: pgmap v153: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:57.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:56 smithi081 ceph-mon[113965]: pgmap v153: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:57.602 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:57 smithi149 systemd[1]: Stopping Ceph osd.5 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:16:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:57 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 497.630803 secs 2024-03-22T07:16:58.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:57 smithi149 ceph-mon[83447]: osd.5 marked itself down and dead 2024-03-22T07:16:58.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:57 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[55376]: 2024-03-22T07:16:57.723+0000 7fe925ec5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:16:58.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:57 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[55376]: 2024-03-22T07:16:57.723+0000 7fe925ec5700 -1 osd.5 74 *** Got signal Terminated *** 2024-03-22T07:16:58.103 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:57 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[55376]: 2024-03-22T07:16:57.723+0000 7fe925ec5700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:16:58.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:57 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 497.630803 secs 2024-03-22T07:16:58.220 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:57 smithi081 ceph-mon[113965]: osd.5 marked itself down and dead 2024-03-22T07:16:58.798 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:58 smithi149 podman[102509]: 2024-03-22 07:16:58.500581627 +0000 UTC m=+0.871975723 container died 1de675bda255adf6eae8985a77600f1c5ee7e763d6eecb5f225840b7ca241118 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:16:58.798 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:58 smithi149 podman[102509]: 2024-03-22 07:16:58.798896016 +0000 UTC m=+1.170290111 container cleanup 1de675bda255adf6eae8985a77600f1c5ee7e763d6eecb5f225840b7ca241118 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9c4e0c3) 2024-03-22T07:16:59.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:58 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 497.971299 secs 2024-03-22T07:16:59.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:58 smithi149 ceph-mon[83447]: pgmap v154: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:59.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:58 smithi149 ceph-mon[83447]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:16:59.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:58 smithi149 ceph-mon[83447]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-22T07:16:59.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:58 smithi149 ceph-mon[83447]: osdmap e75: 6 total, 5 up, 6 in 2024-03-22T07:16:59.103 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:58 smithi149 bash[102509]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5 2024-03-22T07:16:59.103 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:59 smithi149 podman[102526]: 2024-03-22 07:16:59.025180009 +0000 UTC m=+0.523608802 container remove 1de675bda255adf6eae8985a77600f1c5ee7e763d6eecb5f225840b7ca241118 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=quincy-9c4e0c3, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-22T07:16:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:58 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 497.971299 secs 2024-03-22T07:16:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:58 smithi081 ceph-mon[113965]: pgmap v154: 65 pgs: 65 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:16:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:58 smithi081 ceph-mon[113965]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:16:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:58 smithi081 ceph-mon[113965]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-22T07:16:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:58 smithi081 ceph-mon[113965]: osdmap e75: 6 total, 5 up, 6 in 2024-03-22T07:16:59.602 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:59 smithi149 podman[102616]: 2024-03-22 07:16:59.2265849 +0000 UTC m=+0.017963055 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:16:59.602 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:59 smithi149 podman[102616]: 2024-03-22 07:16:59.351117167 +0000 UTC m=+0.142495315 container create 5343db8825e4db1e62f47778690c40a7d6ea230a274bc2817acb5aa0d5dc733b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:16:59.899 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:59 smithi149 podman[102616]: 2024-03-22 07:16:59.603675579 +0000 UTC m=+0.395053737 container init 5343db8825e4db1e62f47778690c40a7d6ea230a274bc2817acb5aa0d5dc733b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:16:59.899 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:59 smithi149 podman[102616]: 2024-03-22 07:16:59.606920998 +0000 UTC m=+0.398299149 container start 5343db8825e4db1e62f47778690c40a7d6ea230a274bc2817acb5aa0d5dc733b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-03-22T07:16:59.899 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:59 smithi149 podman[102616]: 2024-03-22 07:16:59.695575155 +0000 UTC m=+0.486953320 container attach 5343db8825e4db1e62f47778690c40a7d6ea230a274bc2817acb5aa0d5dc733b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-22T07:17:00.189 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:16:59 smithi149 ceph-mon[83447]: osdmap e76: 6 total, 5 up, 6 in 2024-03-22T07:17:00.189 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:16:59 smithi149 podman[102616]: 2024-03-22 07:16:59.900338591 +0000 UTC m=+0.691716750 container died 5343db8825e4db1e62f47778690c40a7d6ea230a274bc2817acb5aa0d5dc733b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:17:00.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:16:59 smithi081 ceph-mon[113965]: osdmap e76: 6 total, 5 up, 6 in 2024-03-22T07:17:00.446 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:00 smithi149 podman[102642]: 2024-03-22 07:17:00.432249103 +0000 UTC m=+0.526383083 container remove 5343db8825e4db1e62f47778690c40a7d6ea230a274bc2817acb5aa0d5dc733b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=squid-db0330b, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-22T07:17:00.852 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:00 smithi149 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-22T07:17:00.852 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:00 smithi149 systemd[1]: Stopped Ceph osd.5 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:17:00.852 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:00 smithi149 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.5.service: Consumed 41.964s CPU time. 2024-03-22T07:17:00.852 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:00 smithi149 systemd[1]: Starting Ceph osd.5 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:17:01.177 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:00 smithi149 ceph-mon[83447]: pgmap v157: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:17:01.177 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:00 smithi149 ceph-mon[83447]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T07:17:01.177 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:01.177 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:01.178 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:00 smithi149 podman[102745]: 2024-03-22 07:17:00.775755717 +0000 UTC m=+0.017893934 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:17:01.178 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:00 smithi149 podman[102745]: 2024-03-22 07:17:00.908274097 +0000 UTC m=+0.150412308 container create e20c966ff5e3887526f72c8a6f9dc29967f7bc06678ed8cac5de56426e83c43b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-22T07:17:01.178 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 podman[102745]: 2024-03-22 07:17:01.094228567 +0000 UTC m=+0.336366781 container init e20c966ff5e3887526f72c8a6f9dc29967f7bc06678ed8cac5de56426e83c43b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-22T07:17:01.178 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 podman[102745]: 2024-03-22 07:17:01.098513699 +0000 UTC m=+0.340651914 container start e20c966ff5e3887526f72c8a6f9dc29967f7bc06678ed8cac5de56426e83c43b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS) 2024-03-22T07:17:01.179 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 podman[102745]: 2024-03-22 07:17:01.177337345 +0000 UTC m=+0.419475568 container attach e20c966ff5e3887526f72c8a6f9dc29967f7bc06678ed8cac5de56426e83c43b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-22T07:17:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:00 smithi081 ceph-mon[113965]: pgmap v157: 65 pgs: 3 peering, 11 stale+active+clean, 51 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail 2024-03-22T07:17:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:00 smithi081 ceph-mon[113965]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-22T07:17:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:01.852 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate[102760]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate[102760]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 bash[102745]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 bash[102745]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 bash[102745]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 bash[102745]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 bash[102745]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 bash[102745]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 bash[102745]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate[102760]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate[102760]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate[102760]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-22T07:17:01.853 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate[102760]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T07:17:01.854 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate[102760]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-22T07:17:01.854 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:01 smithi149 podman[102745]: 2024-03-22 07:17:01.563922482 +0000 UTC m=+0.806060687 container died e20c966ff5e3887526f72c8a6f9dc29967f7bc06678ed8cac5de56426e83c43b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-22T07:17:02.197 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:02 smithi149 ceph-mon[83447]: Health check failed: Degraded data redundancy: 2364/32856 objects degraded (7.195%), 15 pgs degraded (PG_DEGRADED) 2024-03-22T07:17:02.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:02 smithi081 ceph-mon[113965]: Health check failed: Degraded data redundancy: 2364/32856 objects degraded (7.195%), 15 pgs degraded (PG_DEGRADED) 2024-03-22T07:17:02.448 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:02 smithi149 podman[102895]: 2024-03-22 07:17:02.197957534 +0000 UTC m=+0.628866093 container remove e20c966ff5e3887526f72c8a6f9dc29967f7bc06678ed8cac5de56426e83c43b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:17:02.448 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:02 smithi149 podman[102960]: 2024-03-22 07:17:02.315454783 +0000 UTC m=+0.018215440 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:17:02.448 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:02 smithi149 podman[102960]: 2024-03-22 07:17:02.449145936 +0000 UTC m=+0.151906587 container create 0d8d31a867d9bb3b9cd8376978336480a433f96771011c8527823a44be9036d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=squid-db0330b) 2024-03-22T07:17:03.030 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:02 smithi149 podman[102960]: 2024-03-22 07:17:02.710119978 +0000 UTC m=+0.412880631 container init 0d8d31a867d9bb3b9cd8376978336480a433f96771011c8527823a44be9036d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-22T07:17:03.031 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:02 smithi149 podman[102960]: 2024-03-22 07:17:02.713319846 +0000 UTC m=+0.416080495 container start 0d8d31a867d9bb3b9cd8376978336480a433f96771011c8527823a44be9036d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-22T07:17:03.031 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:02 smithi149 ceph-osd[102979]: -- 172.21.15.149:0/523584189 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eea81df1e0 con 0x55eea81b1000 2024-03-22T07:17:03.031 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:02 smithi149 bash[102960]: 0d8d31a867d9bb3b9cd8376978336480a433f96771011c8527823a44be9036d2 2024-03-22T07:17:03.031 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:02 smithi149 systemd[1]: Started Ceph osd.5 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:17:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:03 smithi081 ceph-mon[113965]: pgmap v158: 65 pgs: 15 active+undersized+degraded, 3 peering, 3 stale+active+clean, 44 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 2364/32856 objects degraded (7.195%) 2024-03-22T07:17:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:03 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 502.630924 secs 2024-03-22T07:17:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:03 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 502.971383 secs 2024-03-22T07:17:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:03 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:03 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:03 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:03.352 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:03 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:17:03.258+0000 7fa1a25dc7c0 -1 Falling back to public interface 2024-03-22T07:17:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:03 smithi149 ceph-mon[83447]: pgmap v158: 65 pgs: 15 active+undersized+degraded, 3 peering, 3 stale+active+clean, 44 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 2364/32856 objects degraded (7.195%) 2024-03-22T07:17:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:03 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 502.630924 secs 2024-03-22T07:17:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:03 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 502.971383 secs 2024-03-22T07:17:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:03 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:03 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:03 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:04.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:04 smithi149 ceph-mon[83447]: pgmap v159: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 4795/32856 objects degraded (14.594%) 2024-03-22T07:17:04.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:04 smithi149 ceph-mon[83447]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T07:17:04.568 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:04 smithi081 ceph-mon[113965]: pgmap v159: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 4795/32856 objects degraded (14.594%) 2024-03-22T07:17:04.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:04 smithi081 ceph-mon[113965]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-22T07:17:07.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:06 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:07.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:06 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:07.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:06 smithi081 ceph-mon[113965]: pgmap v160: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 4795/32856 objects degraded (14.594%) 2024-03-22T07:17:07.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:06 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:07.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:06 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:07.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:06 smithi149 ceph-mon[83447]: pgmap v160: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 4795/32856 objects degraded (14.594%) 2024-03-22T07:17:07.506 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:07 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:17:07.126+0000 7fa1a25dc7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-22T07:17:07.781 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:07 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:17:07.542+0000 7fa1a25dc7c0 -1 osd.5 74 log_to_monitors true 2024-03-22T07:17:08.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:07 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:08.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:07 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:08.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:07 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 507.631037 secs 2024-03-22T07:17:08.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:07 smithi149 ceph-mon[83447]: Health check update: Degraded data redundancy: 4795/32856 objects degraded (14.594%), 28 pgs degraded (PG_DEGRADED) 2024-03-22T07:17:08.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:07 smithi149 ceph-mon[83447]: from='osd.5 [v2:172.21.15.149:6816/530633899,v1:172.21.15.149:6817/530633899]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:17:08.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:07 smithi149 ceph-mon[83447]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:17:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:07 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:07 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:07 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 507.631037 secs 2024-03-22T07:17:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:07 smithi081 ceph-mon[113965]: Health check update: Degraded data redundancy: 4795/32856 objects degraded (14.594%), 28 pgs degraded (PG_DEGRADED) 2024-03-22T07:17:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:07 smithi081 ceph-mon[113965]: from='osd.5 [v2:172.21.15.149:6816/530633899,v1:172.21.15.149:6817/530633899]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:17:08.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:07 smithi081 ceph-mon[113965]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:17:08.829 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:17:08 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:17:08.553+0000 7fa193a34700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:17:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:08 smithi149 ceph-mon[83447]: 1 slow requests, 0 included below; oldest blocked for > 507.971499 secs 2024-03-22T07:17:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:08 smithi149 ceph-mon[83447]: pgmap v161: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 4795/32856 objects degraded (14.594%) 2024-03-22T07:17:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:08 smithi149 ceph-mon[83447]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T07:17:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:08 smithi149 ceph-mon[83447]: osdmap e77: 6 total, 5 up, 6 in 2024-03-22T07:17:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:08 smithi149 ceph-mon[83447]: from='osd.5 [v2:172.21.15.149:6816/530633899,v1:172.21.15.149:6817/530633899]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:17:09.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:08 smithi149 ceph-mon[83447]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:17:09.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:08 smithi081 ceph-mon[113965]: 1 slow requests, 0 included below; oldest blocked for > 507.971499 secs 2024-03-22T07:17:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:08 smithi081 ceph-mon[113965]: pgmap v161: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 882 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 4795/32856 objects degraded (14.594%) 2024-03-22T07:17:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:08 smithi081 ceph-mon[113965]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T07:17:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:08 smithi081 ceph-mon[113965]: osdmap e77: 6 total, 5 up, 6 in 2024-03-22T07:17:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:08 smithi081 ceph-mon[113965]: from='osd.5 [v2:172.21.15.149:6816/530633899,v1:172.21.15.149:6817/530633899]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:17:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:08 smithi081 ceph-mon[113965]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-22T07:17:10.020 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:09 smithi149 ceph-mon[83447]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:17:10.020 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:09 smithi149 ceph-mon[83447]: osd.5 [v2:172.21.15.149:6816/530633899,v1:172.21.15.149:6817/530633899] boot 2024-03-22T07:17:10.021 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:09 smithi149 ceph-mon[83447]: osdmap e78: 6 total, 6 up, 6 in 2024-03-22T07:17:10.021 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:09 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:17:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:09 smithi081 ceph-mon[113965]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:17:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:09 smithi081 ceph-mon[113965]: osd.5 [v2:172.21.15.149:6816/530633899,v1:172.21.15.149:6817/530633899] boot 2024-03-22T07:17:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:09 smithi081 ceph-mon[113965]: osdmap e78: 6 total, 6 up, 6 in 2024-03-22T07:17:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:09 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:17:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: pgmap v164: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 4795/32856 objects degraded (14.594%) 2024-03-22T07:17:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:11.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:11.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:11.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:11.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:11.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:11.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:11.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:11.105 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:11.105 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-22T07:17:11.105 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-22T07:17:11.105 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-22T07:17:11.105 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: osdmap e79: 6 total, 6 up, 6 in 2024-03-22T07:17:11.106 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-22T07:17:11.106 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-22T07:17:11.106 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-22T07:17:11.106 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-22T07:17:11.106 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-22T07:17:11.107 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-22T07:17:11.107 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-22T07:17:11.107 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-22T07:17:11.107 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-22T07:17:11.107 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-22T07:17:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: pgmap v164: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 4795/32856 objects degraded (14.594%) 2024-03-22T07:17:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: osdmap e79: 6 total, 6 up, 6 in 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-22T07:17:11.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-22T07:17:11.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-22T07:17:11.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-22T07:17:11.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-22T07:17:11.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-22T07:17:11.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-22T07:17:11.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-22T07:17:11.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-22T07:17:11.850 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081[113961]: 2024-03-22T07:17:11.574+0000 7f102b531700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:17:12.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all osd 2024-03-22T07:17:12.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: Upgrade: Setting require_osd_release to 19 squid 2024-03-22T07:17:12.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-22T07:17:12.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-22T07:17:12.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: osdmap e80: 6 total, 6 up, 6 in 2024-03-22T07:17:12.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-22T07:17:12.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T07:17:12.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:17:12.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: Health check cleared: MDS_SLOW_REQUEST (was: 2 MDSs report slow requests) 2024-03-22T07:17:12.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-22T07:17:12.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: osdmap e81: 6 total, 6 up, 6 in 2024-03-22T07:17:12.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:11 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-22T07:17:12.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all osd 2024-03-22T07:17:12.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: Upgrade: Setting require_osd_release to 19 squid 2024-03-22T07:17:12.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-22T07:17:12.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-22T07:17:12.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: osdmap e80: 6 total, 6 up, 6 in 2024-03-22T07:17:12.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-22T07:17:12.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T07:17:12.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:17:12.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: Health check cleared: MDS_SLOW_REQUEST (was: 2 MDSs report slow requests) 2024-03-22T07:17:12.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-22T07:17:12.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: osdmap e81: 6 total, 6 up, 6 in 2024-03-22T07:17:12.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:11 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-22T07:17:13.014 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:12 smithi081 ceph-mon[113965]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-22T07:17:13.014 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:12 smithi081 ceph-mon[113965]: pgmap v168: 65 pgs: 21 active+undersized+degraded, 44 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 3737/32856 objects degraded (11.374%) 2024-03-22T07:17:13.014 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:12 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.149:6824/2789599242,v1:172.21.15.149:6825/2789599242] up:boot 2024-03-22T07:17:13.015 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:12 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6828/3973077481,v1:172.21.15.81:6829/3973077481] up:boot 2024-03-22T07:17:13.015 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:12 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:17:13.015 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:17:13.015 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:17:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:12 smithi149 ceph-mon[83447]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-22T07:17:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:12 smithi149 ceph-mon[83447]: pgmap v168: 65 pgs: 21 active+undersized+degraded, 44 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 3737/32856 objects degraded (11.374%) 2024-03-22T07:17:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:12 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.149:6824/2789599242,v1:172.21.15.149:6825/2789599242] up:boot 2024-03-22T07:17:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:12 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6828/3973077481,v1:172.21.15.81:6829/3973077481] up:boot 2024-03-22T07:17:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:12 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:17:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:17:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:17:14.045 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:13 smithi081 ceph-mon[113965]: Health check update: Degraded data redundancy: 3737/32856 objects degraded (11.374%), 21 pgs degraded (PG_DEGRADED) 2024-03-22T07:17:14.045 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:13 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:14.045 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:13 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.bnzfrv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:17:14.045 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:13 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:14.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:13 smithi149 ceph-mon[83447]: Health check update: Degraded data redundancy: 3737/32856 objects degraded (11.374%), 21 pgs degraded (PG_DEGRADED) 2024-03-22T07:17:14.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:13 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:14.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:13 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.bnzfrv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:17:14.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:13 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:15.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:14 smithi081 ceph-mon[113965]: Upgrade: Updating mds.cephfs.smithi081.bnzfrv 2024-03-22T07:17:15.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:14 smithi081 ceph-mon[113965]: Deploying daemon mds.cephfs.smithi081.bnzfrv on smithi081 2024-03-22T07:17:15.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:14 smithi081 ceph-mon[113965]: pgmap v169: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:15.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:14 smithi149 ceph-mon[83447]: Upgrade: Updating mds.cephfs.smithi081.bnzfrv 2024-03-22T07:17:15.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:14 smithi149 ceph-mon[83447]: Deploying daemon mds.cephfs.smithi081.bnzfrv on smithi081 2024-03-22T07:17:15.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:14 smithi149 ceph-mon[83447]: pgmap v169: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:16.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:15 smithi081 ceph-mon[113965]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3737/32856 objects degraded (11.374%), 21 pgs degraded) 2024-03-22T07:17:16.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:15 smithi149 ceph-mon[83447]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3737/32856 objects degraded (11.374%), 21 pgs degraded) 2024-03-22T07:17:17.125 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:16 smithi081 ceph-mon[113965]: osdmap e82: 6 total, 6 up, 6 in 2024-03-22T07:17:17.126 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:16 smithi081 ceph-mon[113965]: pgmap v171: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:17.126 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:16 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:17:17.126 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:17.126 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:17.126 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:17:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:16 smithi149 ceph-mon[83447]: osdmap e82: 6 total, 6 up, 6 in 2024-03-22T07:17:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:16 smithi149 ceph-mon[83447]: pgmap v171: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:16 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:17:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:17:19.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:18 smithi081 ceph-mon[113965]: pgmap v172: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:19.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:19.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:19.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:18 smithi149 ceph-mon[83447]: pgmap v172: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:20.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:19 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:boot 2024-03-22T07:17:20.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:19 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:17:20.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:19 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:17:20.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:19 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:boot 2024-03-22T07:17:20.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:19 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:17:20.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:19 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:17:21.203 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:20 smithi081 ceph-mon[113965]: pgmap v173: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:20 smithi149 ceph-mon[83447]: pgmap v173: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:22.168 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:23.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.013+0000 7f96e503a700 1 -- 172.21.15.81:0/3820753463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0101d70 msgr2=0x7f96e0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.013+0000 7f96e503a700 1 --2- 172.21.15.81:0/3820753463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0101d70 0x7f96e0102140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f96d0004940 tx=0x7f96d0009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:23.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.013+0000 7f96e503a700 1 -- 172.21.15.81:0/3820753463 shutdown_connections 2024-03-22T07:17:23.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.013+0000 7f96e503a700 1 --2- 172.21.15.81:0/3820753463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e0102710 0x7f96e00fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.013+0000 7f96e503a700 1 --2- 172.21.15.81:0/3820753463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0101d70 0x7f96e0102140 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.013+0000 7f96e503a700 1 -- 172.21.15.81:0/3820753463 >> 172.21.15.81:0/3820753463 conn(0x7f96e0072bc0 msgr2=0x7f96e0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:23.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.013+0000 7f96e503a700 1 -- 172.21.15.81:0/3820753463 shutdown_connections 2024-03-22T07:17:23.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.013+0000 7f96e503a700 1 -- 172.21.15.81:0/3820753463 wait complete. 2024-03-22T07:17:23.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.014+0000 7f96e503a700 1 Processor -- start 2024-03-22T07:17:23.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.014+0000 7f96e503a700 1 -- start start 2024-03-22T07:17:23.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.014+0000 7f96e503a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e0101d70 0x7f96e0102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:23.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.014+0000 7f96e503a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0102710 0x7f96e0103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:23.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.014+0000 7f96e503a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01049c0 con 0x7f96e0102710 2024-03-22T07:17:23.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.014+0000 7f96e503a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0104b30 con 0x7f96e0101d70 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96de59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0102710 0x7f96e0103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96de59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0102710 0x7f96e0103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47006/0 (socket says 172.21.15.81:47006) 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96de59c700 1 -- 172.21.15.81:0/367408691 learned_addr learned my addr 172.21.15.81:0/367408691 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96ded9d700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e0101d70 0x7f96e0102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96de59c700 1 -- 172.21.15.81:0/367408691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e0101d70 msgr2=0x7f96e0102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96de59c700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e0101d70 0x7f96e0102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96de59c700 1 -- 172.21.15.81:0/367408691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0005040 con 0x7f96e0102710 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96ded9d700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e0101d70 0x7f96e0102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96de59c700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0102710 0x7f96e0103110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c800 tx=0x7f96d4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96c77fe700 1 -- 172.21.15.81:0/367408691 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96d400a760 con 0x7f96e0102710 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96c77fe700 1 -- 172.21.15.81:0/367408691 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96d400ad90 con 0x7f96e0102710 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e0103710 con 0x7f96e0102710 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96c77fe700 1 -- 172.21.15.81:0/367408691 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96d40113a0 con 0x7f96e0102710 2024-03-22T07:17:23.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.015+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e011e7b0 con 0x7f96e0102710 2024-03-22T07:17:23.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.016+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e00fd4c0 con 0x7f96e0102710 2024-03-22T07:17:23.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.017+0000 7f96c77fe700 1 -- 172.21.15.81:0/367408691 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f96d4012400 con 0x7f96e0102710 2024-03-22T07:17:23.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.017+0000 7f96c77fe700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f96c806d380 0x7f96c806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:23.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.017+0000 7f96c77fe700 1 -- 172.21.15.81:0/367408691 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f96d408c9d0 con 0x7f96e0102710 2024-03-22T07:17:23.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.017+0000 7f96ded9d700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f96c806d380 0x7f96c806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:23.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.018+0000 7f96ded9d700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f96c806d380 0x7f96c806f830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f96d0004560 tx=0x7f96d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:23.022 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.022+0000 7f96c77fe700 1 -- 172.21.15.81:0/367408691 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96d4056480 con 0x7f96e0102710 2024-03-22T07:17:23.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.271+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96e004bed0 con 0x7f96c806d380 2024-03-22T07:17:23.271 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:22 smithi081 ceph-mon[113965]: pgmap v174: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:23.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.272+0000 7f96c77fe700 1 -- 172.21.15.81:0/367408691 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f96e004bed0 con 0x7f96c806d380 2024-03-22T07:17:23.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f96c806d380 msgr2=0x7f96c806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f96c806d380 0x7f96c806f830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f96d0004560 tx=0x7f96d0009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:23.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0102710 msgr2=0x7f96e0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0102710 0x7f96e0103110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c800 tx=0x7f96d4004470 comp rx=0 tx=0).stop 2024-03-22T07:17:23.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 shutdown_connections 2024-03-22T07:17:23.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f96c806d380 0x7f96c806f830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f96e0101d70 0x7f96e0102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 --2- 172.21.15.81:0/367408691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96e0102710 0x7f96e0103110 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 >> 172.21.15.81:0/367408691 conn(0x7f96e0072bc0 msgr2=0x7f96e0065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:23.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 shutdown_connections 2024-03-22T07:17:23.276 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.275+0000 7f96e503a700 1 -- 172.21.15.81:0/367408691 wait complete. 2024-03-22T07:17:23.287 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:17:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:22 smithi149 ceph-mon[83447]: pgmap v174: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 -- 172.21.15.81:0/1594092551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a0101d20 msgr2=0x7f46a01020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 --2- 172.21.15.81:0/1594092551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a0101d20 0x7f46a01020f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4690004940 tx=0x7f4690009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 -- 172.21.15.81:0/1594092551 shutdown_connections 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 --2- 172.21.15.81:0/1594092551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a01026c0 0x7f46a00fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 --2- 172.21.15.81:0/1594092551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a0101d20 0x7f46a01020f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 -- 172.21.15.81:0/1594092551 >> 172.21.15.81:0/1594092551 conn(0x7f46a00f78b0 msgr2=0x7f46a00f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 -- 172.21.15.81:0/1594092551 shutdown_connections 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 -- 172.21.15.81:0/1594092551 wait complete. 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 Processor -- start 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.385+0000 7f46a7e83700 1 -- start start 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a7e83700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a0101d20 0x7f46a010eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a7e83700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a01026c0 0x7f46a010f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a7e83700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a010f5c0 con 0x7f46a01026c0 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a7e83700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a010f700 con 0x7f46a0101d20 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a541e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a01026c0 0x7f46a010f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a541e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a01026c0 0x7f46a010f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47022/0 (socket says 172.21.15.81:47022) 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a541e700 1 -- 172.21.15.81:0/2305163726 learned_addr learned my addr 172.21.15.81:0/2305163726 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a5c1f700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a0101d20 0x7f46a010eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a541e700 1 -- 172.21.15.81:0/2305163726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a0101d20 msgr2=0x7f46a010eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a541e700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a0101d20 0x7f46a010eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a541e700 1 -- 172.21.15.81:0/2305163726 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4690005040 con 0x7f46a01026c0 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a541e700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a01026c0 0x7f46a010f080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f469c00c890 tx=0x7f469c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46967fc700 1 -- 172.21.15.81:0/2305163726 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f469c00a700 con 0x7f46a01026c0 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46967fc700 1 -- 172.21.15.81:0/2305163726 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f469c00ad30 con 0x7f46a01026c0 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a0117bb0 con 0x7f46a01026c0 2024-03-22T07:17:23.388 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46967fc700 1 -- 172.21.15.81:0/2305163726 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f469c0113a0 con 0x7f46a01026c0 2024-03-22T07:17:23.389 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.386+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a0117f80 con 0x7f46a01026c0 2024-03-22T07:17:23.389 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.388+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a00fd470 con 0x7f46a01026c0 2024-03-22T07:17:23.389 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.388+0000 7f46967fc700 1 -- 172.21.15.81:0/2305163726 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f469c012400 con 0x7f46a01026c0 2024-03-22T07:17:23.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.388+0000 7f46967fc700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f468c06d380 0x7f468c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:23.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.388+0000 7f46967fc700 1 -- 172.21.15.81:0/2305163726 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f469c08c9c0 con 0x7f46a01026c0 2024-03-22T07:17:23.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.388+0000 7f46a5c1f700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f468c06d380 0x7f468c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:23.390 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.389+0000 7f46a5c1f700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f468c06d380 0x7f468c06f830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4690004f90 tx=0x7f4690009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:23.393 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.393+0000 7f46967fc700 1 -- 172.21.15.81:0/2305163726 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f469c056470 con 0x7f46a01026c0 2024-03-22T07:17:23.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.608+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46a0108e20 con 0x7f468c06d380 2024-03-22T07:17:23.609 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.609+0000 7f46967fc700 1 -- 172.21.15.81:0/2305163726 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f46a0108e20 con 0x7f468c06d380 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.611+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f468c06d380 msgr2=0x7f468c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.611+0000 7f46a7e83700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f468c06d380 0x7f468c06f830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4690004f90 tx=0x7f4690009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.611+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a01026c0 msgr2=0x7f46a010f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.611+0000 7f46a7e83700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a01026c0 0x7f46a010f080 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f469c00c890 tx=0x7f469c0044c0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.611+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 shutdown_connections 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.611+0000 7f46a7e83700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f468c06d380 0x7f468c06f830 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.611+0000 7f46a7e83700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46a0101d20 0x7f46a010eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.611+0000 7f46a7e83700 1 --2- 172.21.15.81:0/2305163726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46a01026c0 0x7f46a010f080 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.611+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 >> 172.21.15.81:0/2305163726 conn(0x7f46a00f78b0 msgr2=0x7f46a0104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.612+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 shutdown_connections 2024-03-22T07:17:23.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.612+0000 7f46a7e83700 1 -- 172.21.15.81:0/2305163726 wait complete. 2024-03-22T07:17:23.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.720+0000 7f84f8736700 1 -- 172.21.15.81:0/1591703510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 msgr2=0x7f84f00fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.720+0000 7f84f8736700 1 --2- 172.21.15.81:0/1591703510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 0x7f84f00fbd80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84e0004940 tx=0x7f84e0009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.720+0000 7f84f8736700 1 -- 172.21.15.81:0/1591703510 shutdown_connections 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.720+0000 7f84f8736700 1 --2- 172.21.15.81:0/1591703510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 0x7f84f00fbd80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.720+0000 7f84f8736700 1 --2- 172.21.15.81:0/1591703510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101d70 0x7f84f0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.720+0000 7f84f8736700 1 -- 172.21.15.81:0/1591703510 >> 172.21.15.81:0/1591703510 conn(0x7f84f0072bc0 msgr2=0x7f84f0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.720+0000 7f84f8736700 1 -- 172.21.15.81:0/1591703510 shutdown_connections 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.720+0000 7f84f8736700 1 -- 172.21.15.81:0/1591703510 wait complete. 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f8736700 1 Processor -- start 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f8736700 1 -- start start 2024-03-22T07:17:23.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f8736700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101d70 0x7f84f010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:23.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f8736700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 0x7f84f010ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:23.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f8736700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f010d3b0 con 0x7f84f0102710 2024-03-22T07:17:23.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f8736700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f010d520 con 0x7f84f0101d70 2024-03-22T07:17:23.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f5cd1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 0x7f84f010ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:23.724 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f5cd1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 0x7f84f010ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47032/0 (socket says 172.21.15.81:47032) 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f5cd1700 1 -- 172.21.15.81:0/2626969590 learned_addr learned my addr 172.21.15.81:0/2626969590 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f64d2700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101d70 0x7f84f010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f5cd1700 1 -- 172.21.15.81:0/2626969590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101d70 msgr2=0x7f84f010c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f5cd1700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101d70 0x7f84f010c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f5cd1700 1 -- 172.21.15.81:0/2626969590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e0005040 con 0x7f84f0102710 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.721+0000 7f84f64d2700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101d70 0x7f84f010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.722+0000 7f84f5cd1700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 0x7f84f010ce70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009100 tx=0x7f84e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.722+0000 7f84e6ffd700 1 -- 172.21.15.81:0/2626969590 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84e000e040 con 0x7f84f0102710 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.722+0000 7f84e6ffd700 1 -- 172.21.15.81:0/2626969590 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84e0013420 con 0x7f84f0102710 2024-03-22T07:17:23.725 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.722+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f010d7a0 con 0x7f84f0102710 2024-03-22T07:17:23.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.722+0000 7f84e6ffd700 1 -- 172.21.15.81:0/2626969590 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84e001c610 con 0x7f84f0102710 2024-03-22T07:17:23.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.722+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f011a320 con 0x7f84f0102710 2024-03-22T07:17:23.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.723+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f0104240 con 0x7f84f0102710 2024-03-22T07:17:23.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.723+0000 7f84e6ffd700 1 -- 172.21.15.81:0/2626969590 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f84e001c890 con 0x7f84f0102710 2024-03-22T07:17:23.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.724+0000 7f84e6ffd700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f84dc06d380 0x7f84dc06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:23.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.724+0000 7f84e6ffd700 1 -- 172.21.15.81:0/2626969590 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f84e008d9f0 con 0x7f84f0102710 2024-03-22T07:17:23.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.724+0000 7f84f64d2700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f84dc06d380 0x7f84dc06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:23.727 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.724+0000 7f84f64d2700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f84dc06d380 0x7f84dc06f830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84ec004d60 tx=0x7f84ec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:23.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.728+0000 7f84e6ffd700 1 -- 172.21.15.81:0/2626969590 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f84e008fe60 con 0x7f84f0102710 2024-03-22T07:17:23.936 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.936+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84f010d7a0 con 0x7f84dc06d380 2024-03-22T07:17:23.946 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.946+0000 7f84e6ffd700 1 -- 172.21.15.81:0/2626969590 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f84f010d7a0 con 0x7f84dc06d380 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (5m) 117s ago 16m 26.0M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (3m) 117s ago 16m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (3m) 18s ago 15m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (5m) 117s ago 16m 84.5M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 starting - - - - 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (10m) 117s ago 10m 131M - 17.2.7-726-g9c4e0c34 837124da804f 87b72412dce2 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (10m) 18s ago 10m 17.6M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (10m) 18s ago 10m 147M - 17.2.7-726-g9c4e0c34 837124da804f 737dfdae839e 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (7m) 117s ago 17m 502M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (6m) 18s ago 15m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (4m) 117s ago 17m 44.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:17:23.947 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (4m) 18s ago 15m 35.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:17:23.948 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (6m) 117s ago 16m 16.2M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:17:23.948 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (6m) 18s ago 15m 16.7M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:17:23.948 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (3m) 117s ago 14m 94.5M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 033c080308b6 2024-03-22T07:17:23.948 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (2m) 117s ago 14m 105M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 8341fe32f3fa 2024-03-22T07:17:23.948 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (2m) 117s ago 13m 12.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 0b8c514dced1 2024-03-22T07:17:23.948 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (87s) 18s ago 13m 105M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c8234f7ed003 2024-03-22T07:17:23.948 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (54s) 18s ago 12m 103M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 437455ba97c0 2024-03-22T07:17:23.948 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (21s) 18s ago 12m 14.9M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0d8d31a867d9 2024-03-22T07:17:23.948 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (6m) 117s ago 15m 59.1M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:17:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f84dc06d380 msgr2=0x7f84dc06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f84dc06d380 0x7f84dc06f830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84ec004d60 tx=0x7f84ec00b040 comp rx=0 tx=0).stop 2024-03-22T07:17:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 msgr2=0x7f84f010ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 0x7f84f010ce70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009100 tx=0x7f84e0009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 shutdown_connections 2024-03-22T07:17:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f84dc06d380 0x7f84dc06f830 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f0101d70 0x7f84f010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 --2- 172.21.15.81:0/2626969590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84f0102710 0x7f84f010ce70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 >> 172.21.15.81:0/2626969590 conn(0x7f84f0072bc0 msgr2=0x7f84f00fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:23.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 shutdown_connections 2024-03-22T07:17:23.950 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:23.948+0000 7f84f8736700 1 -- 172.21.15.81:0/2626969590 wait complete. 2024-03-22T07:17:24.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.050+0000 7fa8b8047700 1 -- 172.21.15.81:0/2406888129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b0102e70 msgr2=0x7fa8b0103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.050+0000 7fa8b8047700 1 --2- 172.21.15.81:0/2406888129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b0102e70 0x7fa8b0103240 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac004940 tx=0x7fa8ac009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:24.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.050+0000 7fa8b8047700 1 -- 172.21.15.81:0/2406888129 shutdown_connections 2024-03-22T07:17:24.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.050+0000 7fa8b8047700 1 --2- 172.21.15.81:0/2406888129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8b00fb4a0 0x7fa8b00fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.050 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.050+0000 7fa8b8047700 1 --2- 172.21.15.81:0/2406888129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b0102e70 0x7fa8b0103240 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.050+0000 7fa8b8047700 1 -- 172.21.15.81:0/2406888129 >> 172.21.15.81:0/2406888129 conn(0x7fa8b00714c0 msgr2=0x7fa8b00718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:24.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.050+0000 7fa8b8047700 1 -- 172.21.15.81:0/2406888129 shutdown_connections 2024-03-22T07:17:24.051 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.050+0000 7fa8b8047700 1 -- 172.21.15.81:0/2406888129 wait complete. 2024-03-22T07:17:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.050+0000 7fa8b8047700 1 Processor -- start 2024-03-22T07:17:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b8047700 1 -- start start 2024-03-22T07:17:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b8047700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b00fb4a0 0x7fa8b0115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b8047700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8b0102e70 0x7fa8b01162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b8047700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b0116920 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b8047700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b010fcf0 con 0x7fa8b0102e70 2024-03-22T07:17:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b5de3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b00fb4a0 0x7fa8b0115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b5de3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b00fb4a0 0x7fa8b0115d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47042/0 (socket says 172.21.15.81:47042) 2024-03-22T07:17:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b5de3700 1 -- 172.21.15.81:0/3579455304 learned_addr learned my addr 172.21.15.81:0/3579455304 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b55e2700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8b0102e70 0x7fa8b01162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b5de3700 1 -- 172.21.15.81:0/3579455304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8b0102e70 msgr2=0x7fa8b01162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b5de3700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8b0102e70 0x7fa8b01162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b5de3700 1 -- 172.21.15.81:0/3579455304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac005040 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.051+0000 7fa8b5de3700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b00fb4a0 0x7fa8b0115d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009180 tx=0x7fa8ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.052+0000 7fa8a67fc700 1 -- 172.21.15.81:0/3579455304 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8ac00e040 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.052+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b010ff70 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.052+0000 7fa8a67fc700 1 -- 172.21.15.81:0/3579455304 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8ac013420 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.052+0000 7fa8a67fc700 1 -- 172.21.15.81:0/3579455304 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8ac01c630 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.052+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b0110530 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.053+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b0065380 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.053+0000 7fa8a67fc700 1 -- 172.21.15.81:0/3579455304 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fa8ac013590 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.054+0000 7fa8a67fc700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa89c06d390 0x7fa89c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:24.054 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.054+0000 7fa8a67fc700 1 -- 172.21.15.81:0/3579455304 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa8ac08db20 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.054+0000 7fa8b55e2700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa89c06d390 0x7fa89c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:24.055 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.054+0000 7fa8b55e2700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa89c06d390 0x7fa89c06f840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0111630 tx=0x7fa8a0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:24.059 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.059+0000 7fa8a67fc700 1 -- 172.21.15.81:0/3579455304 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8ac05b100 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.358+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa8b005f5f0 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.359+0000 7fa8a67fc700 1 -- 172.21.15.81:0/3579455304 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fa8ac05ac90 con 0x7fa8b00fb4a0 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-22T07:17:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:17:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:17:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:17:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-22T07:17:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 11 2024-03-22T07:17:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:17:24.361 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa89c06d390 msgr2=0x7fa89c06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa89c06d390 0x7fa89c06f840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0111630 tx=0x7fa8a0009040 comp rx=0 tx=0).stop 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b00fb4a0 msgr2=0x7fa8b0115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b00fb4a0 0x7fa8b0115d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009180 tx=0x7fa8ac009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 shutdown_connections 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa89c06d390 0x7fa89c06f840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8b00fb4a0 0x7fa8b0115d90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 --2- 172.21.15.81:0/3579455304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8b0102e70 0x7fa8b01162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 >> 172.21.15.81:0/3579455304 conn(0x7fa8b00714c0 msgr2=0x7fa8b0100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 shutdown_connections 2024-03-22T07:17:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.361+0000 7fa8b8047700 1 -- 172.21.15.81:0/3579455304 wait complete. 2024-03-22T07:17:24.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.465+0000 7ff9585d8700 1 -- 172.21.15.81:0/3499214477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9501037b0 msgr2=0x7ff9500fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.465+0000 7ff9585d8700 1 --2- 172.21.15.81:0/3499214477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9501037b0 0x7ff9500fce20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff940004940 tx=0x7ff940009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:24.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.465+0000 7ff9585d8700 1 -- 172.21.15.81:0/3499214477 shutdown_connections 2024-03-22T07:17:24.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.465+0000 7ff9585d8700 1 --2- 172.21.15.81:0/3499214477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9501037b0 0x7ff9500fce20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.465+0000 7ff9585d8700 1 --2- 172.21.15.81:0/3499214477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff950102e10 0x7ff9501031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.465+0000 7ff9585d8700 1 -- 172.21.15.81:0/3499214477 >> 172.21.15.81:0/3499214477 conn(0x7ff9500f8980 msgr2=0x7ff9500fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.465+0000 7ff9585d8700 1 -- 172.21.15.81:0/3499214477 shutdown_connections 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.465+0000 7ff9585d8700 1 -- 172.21.15.81:0/3499214477 wait complete. 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff9585d8700 1 Processor -- start 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff9585d8700 1 -- start start 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff9585d8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff950102e10 0x7ff950113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff9585d8700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9501037b0 0x7ff9501140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff9585d8700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff950114710 con 0x7ff950102e10 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff9585d8700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95010db00 con 0x7ff9501037b0 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff956374700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff950102e10 0x7ff950113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff956374700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff950102e10 0x7ff950113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47048/0 (socket says 172.21.15.81:47048) 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff956374700 1 -- 172.21.15.81:0/2648759016 learned_addr learned my addr 172.21.15.81:0/2648759016 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff955b73700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9501037b0 0x7ff9501140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff956374700 1 -- 172.21.15.81:0/2648759016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9501037b0 msgr2=0x7ff9501140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.468 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff956374700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9501037b0 0x7ff9501140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.466+0000 7ff956374700 1 -- 172.21.15.81:0/2648759016 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940005040 con 0x7ff950102e10 2024-03-22T07:17:24.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.467+0000 7ff956374700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff950102e10 0x7ff950113b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00c8c0 tx=0x7ff94c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:24.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.467+0000 7ff946ffd700 1 -- 172.21.15.81:0/2648759016 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94c00a760 con 0x7ff950102e10 2024-03-22T07:17:24.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.467+0000 7ff946ffd700 1 -- 172.21.15.81:0/2648759016 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff94c00ad90 con 0x7ff950102e10 2024-03-22T07:17:24.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.467+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95010dde0 con 0x7ff950102e10 2024-03-22T07:17:24.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.467+0000 7ff946ffd700 1 -- 172.21.15.81:0/2648759016 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94c0113a0 con 0x7ff950102e10 2024-03-22T07:17:24.469 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.467+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95010e370 con 0x7ff950102e10 2024-03-22T07:17:24.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.468+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9501052e0 con 0x7ff950102e10 2024-03-22T07:17:24.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.468+0000 7ff946ffd700 1 -- 172.21.15.81:0/2648759016 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7ff94c012400 con 0x7ff950102e10 2024-03-22T07:17:24.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.469+0000 7ff946ffd700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff93c06d380 0x7ff93c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:24.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.469+0000 7ff946ffd700 1 -- 172.21.15.81:0/2648759016 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff94c08c8c0 con 0x7ff950102e10 2024-03-22T07:17:24.470 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.469+0000 7ff955b73700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff93c06d380 0x7ff93c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:24.471 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.469+0000 7ff955b73700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff93c06d380 0x7ff93c06f830 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff95010ede0 tx=0x7ff940009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:24.473 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.473+0000 7ff946ffd700 1 -- 172.21.15.81:0/2648759016 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff94c059f40 con 0x7ff950102e10 2024-03-22T07:17:24.774 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.774+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff95010f1d0 con 0x7ff950102e10 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.774+0000 7ff946ffd700 1 -- 172.21.15.81:0/2648759016 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1839 (secure 0 0 0) 0x7ff94c059ad0 con 0x7ff950102e10 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stdout:e26 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 23 2024-03-22T07:17:24.775 INFO:teuthology.orchestra.run.smithi081.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:17:11.575650+0000 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 81 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:17:24.776 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:up {} 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:failed 0,1 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:17:24.777 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:17:24.778 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:34378} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:17:24.778 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{-1:44243} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6824/2789599242,v1:172.21.15.149:6825/2789599242] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:17:24.778 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{-1:44245} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6828/3973077481,v1:172.21.15.81:6829/3973077481] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:17:24.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff93c06d380 msgr2=0x7ff93c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff93c06d380 0x7ff93c06f830 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff95010ede0 tx=0x7ff940009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:24.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff950102e10 msgr2=0x7ff950113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff950102e10 0x7ff950113b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00c8c0 tx=0x7ff94c0044c0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 shutdown_connections 2024-03-22T07:17:24.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff93c06d380 0x7ff93c06f830 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff950102e10 0x7ff950113b80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 --2- 172.21.15.81:0/2648759016 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9501037b0 0x7ff9501140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 >> 172.21.15.81:0/2648759016 conn(0x7ff9500f8980 msgr2=0x7ff9501021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:24.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 shutdown_connections 2024-03-22T07:17:24.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.777+0000 7ff9585d8700 1 -- 172.21.15.81:0/2648759016 wait complete. 2024-03-22T07:17:24.780 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 26 2024-03-22T07:17:24.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 -- 172.21.15.81:0/289560607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c1026c0 msgr2=0x7fbe2c0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 --2- 172.21.15.81:0/289560607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c1026c0 0x7fbe2c0fbd30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe20004940 tx=0x7fbe20009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:24.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 -- 172.21.15.81:0/289560607 shutdown_connections 2024-03-22T07:17:24.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 --2- 172.21.15.81:0/289560607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c1026c0 0x7fbe2c0fbd30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 --2- 172.21.15.81:0/289560607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe2c101d20 0x7fbe2c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 -- 172.21.15.81:0/289560607 >> 172.21.15.81:0/289560607 conn(0x7fbe2c0f78d0 msgr2=0x7fbe2c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:24.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 -- 172.21.15.81:0/289560607 shutdown_connections 2024-03-22T07:17:24.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 -- 172.21.15.81:0/289560607 wait complete. 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 Processor -- start 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.884+0000 7fbe30876700 1 -- start start 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe30876700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c101d20 0x7fbe2c10ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe30876700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe2c1026c0 0x7fbe2c10f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe30876700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c10f790 con 0x7fbe2c101d20 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe30876700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c10f8d0 con 0x7fbe2c1026c0 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe2a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c101d20 0x7fbe2c10ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe2a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c101d20 0x7fbe2c10ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54176/0 (socket says 172.21.15.81:54176) 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe2a59c700 1 -- 172.21.15.81:0/3412880562 learned_addr learned my addr 172.21.15.81:0/3412880562 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe29d9b700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe2c1026c0 0x7fbe2c10f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe2a59c700 1 -- 172.21.15.81:0/3412880562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe2c1026c0 msgr2=0x7fbe2c10f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:24.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe2a59c700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe2c1026c0 0x7fbe2c10f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:24.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe2a59c700 1 -- 172.21.15.81:0/3412880562 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe20005040 con 0x7fbe2c101d20 2024-03-22T07:17:24.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe2a59c700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c101d20 0x7fbe2c10ed10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00eb70 tx=0x7fbe1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:24.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe1affd700 1 -- 172.21.15.81:0/3412880562 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c007cc0 con 0x7fbe2c101d20 2024-03-22T07:17:24.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe1affd700 1 -- 172.21.15.81:0/3412880562 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe1c00f040 con 0x7fbe2c101d20 2024-03-22T07:17:24.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2c117bc0 con 0x7fbe2c101d20 2024-03-22T07:17:24.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe1affd700 1 -- 172.21.15.81:0/3412880562 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c009640 con 0x7fbe2c101d20 2024-03-22T07:17:24.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.885+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c1180d0 con 0x7fbe2c101d20 2024-03-22T07:17:24.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.887+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2c0fd470 con 0x7fbe2c101d20 2024-03-22T07:17:24.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.887+0000 7fbe1affd700 1 -- 172.21.15.81:0/3412880562 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbe1c0097a0 con 0x7fbe2c101d20 2024-03-22T07:17:24.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.887+0000 7fbe1affd700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe1406d380 0x7fbe1406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:24.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.887+0000 7fbe1affd700 1 -- 172.21.15.81:0/3412880562 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbe1c08c570 con 0x7fbe2c101d20 2024-03-22T07:17:24.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.888+0000 7fbe29d9b700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe1406d380 0x7fbe1406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:24.890 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.888+0000 7fbe29d9b700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe1406d380 0x7fbe1406f830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009b30 tx=0x7fbe20009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:24.893 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:24.892+0000 7fbe1affd700 1 -- 172.21.15.81:0/3412880562 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe1c055fa0 con 0x7fbe2c101d20 2024-03-22T07:17:25.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:24 smithi081 ceph-mon[113965]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:25.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:24 smithi081 ceph-mon[113965]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:25.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:24 smithi081 ceph-mon[113965]: pgmap v175: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:25.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:24 smithi081 ceph-mon[113965]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:25.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:24 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:25.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:24 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:25.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:24 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3579455304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:25.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:24 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2648759016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:17:25.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.112+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe2c108e90 con 0x7fbe1406d380 2024-03-22T07:17:25.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.113+0000 7fbe1affd700 1 -- 172.21.15.81:0/3412880562 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbe2c108e90 con 0x7fbe1406d380 2024-03-22T07:17:25.113 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:17:25.113 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "mon", 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr", 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "crash", 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "osd" 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "13/21 daemons upgraded", 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:17:25.114 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:17:25.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.115+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe1406d380 msgr2=0x7fbe1406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:25.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.115+0000 7fbe30876700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe1406d380 0x7fbe1406f830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009b30 tx=0x7fbe20009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:25.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.115+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c101d20 msgr2=0x7fbe2c10ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:25.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.115+0000 7fbe30876700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c101d20 0x7fbe2c10ed10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00eb70 tx=0x7fbe1c00b040 comp rx=0 tx=0).stop 2024-03-22T07:17:25.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.116+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 shutdown_connections 2024-03-22T07:17:25.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.116+0000 7fbe30876700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe1406d380 0x7fbe1406f830 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:25.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.116+0000 7fbe30876700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe2c101d20 0x7fbe2c10ed10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:25.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.116+0000 7fbe30876700 1 --2- 172.21.15.81:0/3412880562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe2c1026c0 0x7fbe2c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:25.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.116+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 >> 172.21.15.81:0/3412880562 conn(0x7fbe2c0f78d0 msgr2=0x7fbe2c0f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:25.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.116+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 shutdown_connections 2024-03-22T07:17:25.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.116+0000 7fbe30876700 1 -- 172.21.15.81:0/3412880562 wait complete. 2024-03-22T07:17:25.223 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.222+0000 7f2a688e3700 1 -- 172.21.15.81:0/2308981311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 msgr2=0x7f2a641031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.222+0000 7f2a688e3700 1 --2- 172.21.15.81:0/2308981311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 0x7f2a641031f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a54004940 tx=0x7f2a54009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.222+0000 7f2a688e3700 1 -- 172.21.15.81:0/2308981311 shutdown_connections 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.222+0000 7f2a688e3700 1 --2- 172.21.15.81:0/2308981311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a641037c0 0x7f2a640fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.222+0000 7f2a688e3700 1 --2- 172.21.15.81:0/2308981311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 0x7f2a641031f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.222+0000 7f2a688e3700 1 -- 172.21.15.81:0/2308981311 >> 172.21.15.81:0/2308981311 conn(0x7f2a640f89d0 msgr2=0x7f2a640fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.222+0000 7f2a688e3700 1 -- 172.21.15.81:0/2308981311 shutdown_connections 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.222+0000 7f2a688e3700 1 -- 172.21.15.81:0/2308981311 wait complete. 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a688e3700 1 Processor -- start 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a688e3700 1 -- start start 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a688e3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 0x7f2a64113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a688e3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a641037c0 0x7f2a64114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a688e3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a641146b0 con 0x7f2a64102e20 2024-03-22T07:17:25.224 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a688e3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6410daf0 con 0x7f2a641037c0 2024-03-22T07:17:25.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a6259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 0x7f2a64113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:25.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a6259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 0x7f2a64113b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54202/0 (socket says 172.21.15.81:54202) 2024-03-22T07:17:25.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a6259c700 1 -- 172.21.15.81:0/4094893305 learned_addr learned my addr 172.21.15.81:0/4094893305 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:25.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a61d9b700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a641037c0 0x7f2a64114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:25.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a6259c700 1 -- 172.21.15.81:0/4094893305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a641037c0 msgr2=0x7f2a64114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:25.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a6259c700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a641037c0 0x7f2a64114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:25.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a6259c700 1 -- 172.21.15.81:0/4094893305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a54005040 con 0x7f2a64102e20 2024-03-22T07:17:25.225 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.223+0000 7f2a6259c700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 0x7f2a64113b20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400b8b0 tx=0x7f2a54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:25.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.224+0000 7f2a52ffd700 1 -- 172.21.15.81:0/4094893305 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a5400e040 con 0x7f2a64102e20 2024-03-22T07:17:25.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.224+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6410dd70 con 0x7f2a64102e20 2024-03-22T07:17:25.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.224+0000 7f2a52ffd700 1 -- 172.21.15.81:0/4094893305 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a54004290 con 0x7f2a64102e20 2024-03-22T07:17:25.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.224+0000 7f2a52ffd700 1 -- 172.21.15.81:0/4094893305 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a5401c3f0 con 0x7f2a64102e20 2024-03-22T07:17:25.226 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.224+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6410e330 con 0x7f2a64102e20 2024-03-22T07:17:25.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.225+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a64100130 con 0x7f2a64102e20 2024-03-22T07:17:25.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.225+0000 7f2a52ffd700 1 -- 172.21.15.81:0/4094893305 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f2a54013420 con 0x7f2a64102e20 2024-03-22T07:17:25.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.226+0000 7f2a52ffd700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2a4c06d380 0x7f2a4c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:25.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.226+0000 7f2a52ffd700 1 -- 172.21.15.81:0/4094893305 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2a5408dab0 con 0x7f2a64102e20 2024-03-22T07:17:25.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.226+0000 7f2a61d9b700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2a4c06d380 0x7f2a4c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:25.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.226+0000 7f2a61d9b700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2a4c06d380 0x7f2a4c06f830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2a6410ec60 tx=0x7f2a5800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:25.231 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.231+0000 7f2a52ffd700 1 -- 172.21.15.81:0/4094893305 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a5405b130 con 0x7f2a64102e20 2024-03-22T07:17:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:24 smithi149 ceph-mon[83447]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:24 smithi149 ceph-mon[83447]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:24 smithi149 ceph-mon[83447]: pgmap v175: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:24 smithi149 ceph-mon[83447]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:24 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:25.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:24 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:25.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:24 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3579455304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:25.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:24 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2648759016' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:17:25.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.522+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2a6410f170 con 0x7f2a64102e20 2024-03-22T07:17:25.523 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.522+0000 7f2a52ffd700 1 -- 172.21.15.81:0/4094893305 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2a5408be60 con 0x7f2a64102e20 2024-03-22T07:17:25.524 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-22T07:17:25.524 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-22T07:17:25.524 INFO:teuthology.orchestra.run.smithi081.stdout: fs cephfs is degraded 2024-03-22T07:17:25.524 INFO:teuthology.orchestra.run.smithi081.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-22T07:17:25.525 INFO:teuthology.orchestra.run.smithi081.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-22T07:17:25.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2a4c06d380 msgr2=0x7f2a4c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:25.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2a4c06d380 0x7f2a4c06f830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2a6410ec60 tx=0x7f2a5800b040 comp rx=0 tx=0).stop 2024-03-22T07:17:25.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 msgr2=0x7f2a64113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:25.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 0x7f2a64113b20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400b8b0 tx=0x7f2a54009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:25.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 shutdown_connections 2024-03-22T07:17:25.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2a4c06d380 0x7f2a4c06f830 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:25.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a64102e20 0x7f2a64113b20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:25.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 --2- 172.21.15.81:0/4094893305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a641037c0 0x7f2a64114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:25.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 >> 172.21.15.81:0/4094893305 conn(0x7f2a640f89d0 msgr2=0x7f2a641073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:25.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 shutdown_connections 2024-03-22T07:17:25.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:25.525+0000 7f2a688e3700 1 -- 172.21.15.81:0/4094893305 wait complete. 2024-03-22T07:17:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:26 smithi081 ceph-mon[113965]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:26 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:26 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/4094893305' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:17:26.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:26 smithi081 ceph-mon[113965]: pgmap v176: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:26 smithi149 ceph-mon[83447]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:26 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:26 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/4094893305' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:17:26.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:26 smithi149 ceph-mon[83447]: pgmap v176: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:29.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:28 smithi081 ceph-mon[113965]: pgmap v177: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:28 smithi149 ceph-mon[83447]: pgmap v177: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:30 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:30.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:30.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:30.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:30.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:30.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:30.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:30 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:31 smithi081 ceph-mon[113965]: pgmap v178: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.zfgxzy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:17:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:31 smithi149 ceph-mon[83447]: pgmap v178: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi081.zfgxzy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:17:31.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:31.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:32 smithi081 ceph-mon[113965]: Upgrade: Updating mds.cephfs.smithi081.zfgxzy 2024-03-22T07:17:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:32 smithi081 ceph-mon[113965]: Deploying daemon mds.cephfs.smithi081.zfgxzy on smithi081 2024-03-22T07:17:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:32 smithi149 ceph-mon[83447]: Upgrade: Updating mds.cephfs.smithi081.zfgxzy 2024-03-22T07:17:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:32 smithi149 ceph-mon[83447]: Deploying daemon mds.cephfs.smithi081.zfgxzy on smithi081 2024-03-22T07:17:33.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:33 smithi081 ceph-mon[113965]: pgmap v179: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:33 smithi149 ceph-mon[83447]: pgmap v179: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:34.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:34 smithi081 ceph-mon[113965]: osdmap e83: 6 total, 6 up, 6 in 2024-03-22T07:17:34.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:34 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:17:34.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:34 smithi081 ceph-mon[113965]: pgmap v181: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:34.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:34 smithi149 ceph-mon[83447]: osdmap e83: 6 total, 6 up, 6 in 2024-03-22T07:17:34.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:34 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:17:34.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:34 smithi149 ceph-mon[83447]: pgmap v181: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:37.052 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:36 smithi081 ceph-mon[113965]: pgmap v182: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:37.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:36 smithi149 ceph-mon[83447]: pgmap v182: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:38.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:38.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:38.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:38.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:38 smithi081 ceph-mon[113965]: pgmap v183: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:38.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:38 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:boot 2024-03-22T07:17:38.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:38 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:17:38.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:38 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:17:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:38 smithi149 ceph-mon[83447]: pgmap v183: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:38 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:boot 2024-03-22T07:17:38.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:38 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:17:38.603 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:38 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:17:41.190 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:40 smithi081 ceph-mon[113965]: pgmap v184: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:41.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:40 smithi149 ceph-mon[83447]: pgmap v184: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:43.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:42 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:42 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:43.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:42 smithi081 ceph-mon[113965]: pgmap v185: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:43.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:42 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:43.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:42 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:43.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:42 smithi149 ceph-mon[83447]: pgmap v185: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:44.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:43 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:44.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:43 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:44.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:43 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:44.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:43 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:45.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:44 smithi081 ceph-mon[113965]: pgmap v186: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:45.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:44 smithi149 ceph-mon[83447]: pgmap v186: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:46.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:45 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:45 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:47.212 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: pgmap v187: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:47.213 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: pgmap v187: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:47.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:47.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:47.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:48 smithi149 ceph-mon[83447]: Upgrade: Updating mds.cephfs.smithi149.kdtxfw 2024-03-22T07:17:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:48 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:48 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.kdtxfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:17:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:48 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:48 smithi149 ceph-mon[83447]: Deploying daemon mds.cephfs.smithi149.kdtxfw on smithi149 2024-03-22T07:17:49.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:48 smithi149 ceph-mon[83447]: pgmap v188: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:48 smithi081 ceph-mon[113965]: Upgrade: Updating mds.cephfs.smithi149.kdtxfw 2024-03-22T07:17:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:48 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:48 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.kdtxfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:17:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:48 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:48 smithi081 ceph-mon[113965]: Deploying daemon mds.cephfs.smithi149.kdtxfw on smithi149 2024-03-22T07:17:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:48 smithi081 ceph-mon[113965]: pgmap v188: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:51.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:50 smithi081 ceph-mon[113965]: pgmap v189: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:50 smithi149 ceph-mon[83447]: pgmap v189: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:52.252 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:51 smithi149 ceph-mon[83447]: osdmap e84: 6 total, 6 up, 6 in 2024-03-22T07:17:52.252 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:51 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:17:52.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:51 smithi081 ceph-mon[113965]: osdmap e84: 6 total, 6 up, 6 in 2024-03-22T07:17:52.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:51 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:17:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:52 smithi081 ceph-mon[113965]: pgmap v191: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:52 smithi149 ceph-mon[83447]: pgmap v191: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:54.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:54 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:54.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:54 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:54.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:54 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:54.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:54 smithi149 ceph-mon[83447]: pgmap v192: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:54.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:54 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.149:6824/3255112562,v1:172.21.15.149:6825/3255112562] up:boot 2024-03-22T07:17:54.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:54 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:17:54.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:54 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:17:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:54 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:54 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:54 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:54 smithi081 ceph-mon[113965]: pgmap v192: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:54 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.149:6824/3255112562,v1:172.21.15.149:6825/3255112562] up:boot 2024-03-22T07:17:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:54 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:17:54.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:54 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:17:55.642 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.640+0000 7fbe803b5700 1 -- 172.21.15.81:0/3020292542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe781037a0 msgr2=0x7fbe780fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.640+0000 7fbe803b5700 1 --2- 172.21.15.81:0/3020292542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe781037a0 0x7fbe780fce10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbe68004940 tx=0x7fbe68009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.640+0000 7fbe803b5700 1 -- 172.21.15.81:0/3020292542 shutdown_connections 2024-03-22T07:17:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.640+0000 7fbe803b5700 1 --2- 172.21.15.81:0/3020292542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe781037a0 0x7fbe780fce10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.640+0000 7fbe803b5700 1 --2- 172.21.15.81:0/3020292542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe78102e00 0x7fbe781031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.640+0000 7fbe803b5700 1 -- 172.21.15.81:0/3020292542 >> 172.21.15.81:0/3020292542 conn(0x7fbe780f89b0 msgr2=0x7fbe780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.640+0000 7fbe803b5700 1 -- 172.21.15.81:0/3020292542 shutdown_connections 2024-03-22T07:17:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.640+0000 7fbe803b5700 1 -- 172.21.15.81:0/3020292542 wait complete. 2024-03-22T07:17:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe803b5700 1 Processor -- start 2024-03-22T07:17:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe803b5700 1 -- start start 2024-03-22T07:17:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe803b5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe78102e00 0x7fbe780702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe803b5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe781037a0 0x7fbe78070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe803b5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe78072210 con 0x7fbe78102e00 2024-03-22T07:17:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe803b5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe78070d40 con 0x7fbe781037a0 2024-03-22T07:17:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe7e151700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe78102e00 0x7fbe780702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe7e151700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe78102e00 0x7fbe780702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52912/0 (socket says 172.21.15.81:52912) 2024-03-22T07:17:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe7e151700 1 -- 172.21.15.81:0/2167900608 learned_addr learned my addr 172.21.15.81:0/2167900608 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe7d950700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe781037a0 0x7fbe78070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe7e151700 1 -- 172.21.15.81:0/2167900608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe781037a0 msgr2=0x7fbe78070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe7e151700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe781037a0 0x7fbe78070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe7e151700 1 -- 172.21.15.81:0/2167900608 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe68005040 con 0x7fbe78102e00 2024-03-22T07:17:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe7d950700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe781037a0 0x7fbe78070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:17:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe7e151700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe78102e00 0x7fbe780702c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbe74002a50 tx=0x7fbe740046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.641+0000 7fbe6effd700 1 -- 172.21.15.81:0/2167900608 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe74007cc0 con 0x7fbe78102e00 2024-03-22T07:17:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.642+0000 7fbe6effd700 1 -- 172.21.15.81:0/2167900608 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe7400f040 con 0x7fbe78102e00 2024-03-22T07:17:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.642+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe78071020 con 0x7fbe78102e00 2024-03-22T07:17:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.642+0000 7fbe6effd700 1 -- 172.21.15.81:0/2167900608 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe7400db70 con 0x7fbe78102e00 2024-03-22T07:17:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.642+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7811a130 con 0x7fbe78102e00 2024-03-22T07:17:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.643+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe781052d0 con 0x7fbe78102e00 2024-03-22T07:17:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.643+0000 7fbe6effd700 1 -- 172.21.15.81:0/2167900608 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbe7401c400 con 0x7fbe78102e00 2024-03-22T07:17:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.643+0000 7fbe6effd700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe6406d380 0x7fbe6406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.644+0000 7fbe6effd700 1 -- 172.21.15.81:0/2167900608 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbe7408c050 con 0x7fbe78102e00 2024-03-22T07:17:55.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.644+0000 7fbe7d950700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe6406d380 0x7fbe6406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:55.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.644+0000 7fbe7d950700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe6406d380 0x7fbe6406f830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800b3b0 tx=0x7fbe68009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:55.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.648+0000 7fbe6effd700 1 -- 172.21.15.81:0/2167900608 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe74055a80 con 0x7fbe78102e00 2024-03-22T07:17:55.860 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.860+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe780717a0 con 0x7fbe6406d380 2024-03-22T07:17:55.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.861+0000 7fbe6effd700 1 -- 172.21.15.81:0/2167900608 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbe780717a0 con 0x7fbe6406d380 2024-03-22T07:17:55.864 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.863+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe6406d380 msgr2=0x7fbe6406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:55.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.863+0000 7fbe803b5700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe6406d380 0x7fbe6406f830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800b3b0 tx=0x7fbe68009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:55.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.863+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe78102e00 msgr2=0x7fbe780702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:55.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.863+0000 7fbe803b5700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe78102e00 0x7fbe780702c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbe74002a50 tx=0x7fbe740046b0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.863+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 shutdown_connections 2024-03-22T07:17:55.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.863+0000 7fbe803b5700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbe6406d380 0x7fbe6406f830 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.865 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.863+0000 7fbe803b5700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe78102e00 0x7fbe780702c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.863+0000 7fbe803b5700 1 --2- 172.21.15.81:0/2167900608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe781037a0 0x7fbe78070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.864+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 >> 172.21.15.81:0/2167900608 conn(0x7fbe780f89b0 msgr2=0x7fbe78102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.864+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 shutdown_connections 2024-03-22T07:17:55.866 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.864+0000 7fbe803b5700 1 -- 172.21.15.81:0/2167900608 wait complete. 2024-03-22T07:17:55.875 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:17:55.968 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.967+0000 7f68c559d700 1 -- 172.21.15.81:0/922831930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0101b40 msgr2=0x7f68c0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:55.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.967+0000 7f68c559d700 1 --2- 172.21.15.81:0/922831930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0101b40 0x7f68c0101f10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004940 tx=0x7f68b0009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:55.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.967+0000 7f68c559d700 1 -- 172.21.15.81:0/922831930 shutdown_connections 2024-03-22T07:17:55.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.967+0000 7f68c559d700 1 --2- 172.21.15.81:0/922831930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c01024e0 0x7f68c00fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.967+0000 7f68c559d700 1 --2- 172.21.15.81:0/922831930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c0101b40 0x7f68c0101f10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.967+0000 7f68c559d700 1 -- 172.21.15.81:0/922831930 >> 172.21.15.81:0/922831930 conn(0x7f68c00f7670 msgr2=0x7f68c00f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:55.969 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.967+0000 7f68c559d700 1 -- 172.21.15.81:0/922831930 shutdown_connections 2024-03-22T07:17:55.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.968+0000 7f68c559d700 1 -- 172.21.15.81:0/922831930 wait complete. 2024-03-22T07:17:55.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.968+0000 7f68c559d700 1 Processor -- start 2024-03-22T07:17:55.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.968+0000 7f68c559d700 1 -- start start 2024-03-22T07:17:55.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.968+0000 7f68c559d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c0101b40 0x7f68c01029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:55.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.968+0000 7f68c559d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c01024e0 0x7f68c0102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:55.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.968+0000 7f68c559d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0103500 con 0x7f68c01024e0 2024-03-22T07:17:55.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.968+0000 7f68c559d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0103640 con 0x7f68c0101b40 2024-03-22T07:17:55.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c01024e0 0x7f68c0102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:55.978 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bf7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c01024e0 0x7f68c0102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52924/0 (socket says 172.21.15.81:52924) 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bf7fe700 1 -- 172.21.15.81:0/570978150 learned_addr learned my addr 172.21.15.81:0/570978150 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bffff700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c0101b40 0x7f68c01029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bf7fe700 1 -- 172.21.15.81:0/570978150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c0101b40 msgr2=0x7f68c01029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bf7fe700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c0101b40 0x7f68c01029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bf7fe700 1 -- 172.21.15.81:0/570978150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b0005040 con 0x7f68c01024e0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bf7fe700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c01024e0 0x7f68c0102ee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68b400b790 tx=0x7f68b4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bcff9700 1 -- 172.21.15.81:0/570978150 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b400f6f0 con 0x7f68c01024e0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c0119ce0 con 0x7f68c01024e0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bcff9700 1 -- 172.21.15.81:0/570978150 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b400fd20 con 0x7f68c01024e0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68bcff9700 1 -- 172.21.15.81:0/570978150 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b40103a0 con 0x7f68c01024e0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.969+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c011a1f0 con 0x7f68c01024e0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.970+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c00fd290 con 0x7f68c01024e0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.971+0000 7f68bcff9700 1 -- 172.21.15.81:0/570978150 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f68b4011400 con 0x7f68c01024e0 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.972+0000 7f68bcff9700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f68a806d380 0x7f68a806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.972+0000 7f68bcff9700 1 -- 172.21.15.81:0/570978150 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f68b408c6e0 con 0x7f68c01024e0 2024-03-22T07:17:55.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.972+0000 7f68bffff700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f68a806d380 0x7f68a806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:55.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.972+0000 7f68bffff700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f68a806d380 0x7f68a806f830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004f90 tx=0x7f68b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:55.980 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:55.977+0000 7f68bcff9700 1 -- 172.21.15.81:0/570978150 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f68b4056190 con 0x7f68c01024e0 2024-03-22T07:17:56.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.191+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68c0108d20 con 0x7f68a806d380 2024-03-22T07:17:56.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.192+0000 7f68bcff9700 1 -- 172.21.15.81:0/570978150 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f68c0108d20 con 0x7f68a806d380 2024-03-22T07:17:56.194 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f68a806d380 msgr2=0x7f68a806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f68a806d380 0x7f68a806f830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004f90 tx=0x7f68b0009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c01024e0 msgr2=0x7f68c0102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c01024e0 0x7f68c0102ee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f68b400b790 tx=0x7f68b4009220 comp rx=0 tx=0).stop 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 shutdown_connections 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f68a806d380 0x7f68a806f830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f68c0101b40 0x7f68c01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 --2- 172.21.15.81:0/570978150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f68c01024e0 0x7f68c0102ee0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 >> 172.21.15.81:0/570978150 conn(0x7f68c00f7670 msgr2=0x7f68c00f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 shutdown_connections 2024-03-22T07:17:56.195 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.194+0000 7f68c559d700 1 -- 172.21.15.81:0/570978150 wait complete. 2024-03-22T07:17:56.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.295+0000 7f97ea4ff700 1 -- 172.21.15.81:0/3294623971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e4102f50 msgr2=0x7f97e4103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.295+0000 7f97ea4ff700 1 --2- 172.21.15.81:0/3294623971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e4102f50 0x7f97e4103320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f97d4004940 tx=0x7f97d4009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:56.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.295+0000 7f97ea4ff700 1 -- 172.21.15.81:0/3294623971 shutdown_connections 2024-03-22T07:17:56.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.295+0000 7f97ea4ff700 1 --2- 172.21.15.81:0/3294623971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97e40fb4d0 0x7f97e40fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.295+0000 7f97ea4ff700 1 --2- 172.21.15.81:0/3294623971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e4102f50 0x7f97e4103320 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.297 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.295+0000 7f97ea4ff700 1 -- 172.21.15.81:0/3294623971 >> 172.21.15.81:0/3294623971 conn(0x7f97e40714c0 msgr2=0x7f97e40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.295+0000 7f97ea4ff700 1 -- 172.21.15.81:0/3294623971 shutdown_connections 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.295+0000 7f97ea4ff700 1 -- 172.21.15.81:0/3294623971 wait complete. 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97ea4ff700 1 Processor -- start 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97ea4ff700 1 -- start start 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97ea4ff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e40fb4d0 0x7f97e40fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97ea4ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97e4102f50 0x7f97e40fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97ea4ff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e40fe600 con 0x7f97e40fb4d0 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97ea4ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e40fd130 con 0x7f97e4102f50 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97e3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e40fb4d0 0x7f97e40fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97e3fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e40fb4d0 0x7f97e40fc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52942/0 (socket says 172.21.15.81:52942) 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97e3fff700 1 -- 172.21.15.81:0/1781761939 learned_addr learned my addr 172.21.15.81:0/1781761939 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97e37fe700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97e4102f50 0x7f97e40fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97e3fff700 1 -- 172.21.15.81:0/1781761939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97e4102f50 msgr2=0x7f97e40fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.298 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97e3fff700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97e4102f50 0x7f97e40fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.296+0000 7f97e3fff700 1 -- 172.21.15.81:0/1781761939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d4005040 con 0x7f97e40fb4d0 2024-03-22T07:17:56.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.297+0000 7f97e3fff700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e40fb4d0 0x7f97e40fc6b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009180 tx=0x7f97d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:56.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.297+0000 7f97e0ff9700 1 -- 172.21.15.81:0/1781761939 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97d400e040 con 0x7f97e40fb4d0 2024-03-22T07:17:56.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.297+0000 7f97e0ff9700 1 -- 172.21.15.81:0/1781761939 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97d4013420 con 0x7f97e40fb4d0 2024-03-22T07:17:56.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.297+0000 7f97e0ff9700 1 -- 172.21.15.81:0/1781761939 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97d401c610 con 0x7f97e40fb4d0 2024-03-22T07:17:56.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.297+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e40fd3b0 con 0x7f97e40fb4d0 2024-03-22T07:17:56.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.297+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e411a0d0 con 0x7f97e40fb4d0 2024-03-22T07:17:56.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.298+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e4100200 con 0x7f97e40fb4d0 2024-03-22T07:17:56.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.298+0000 7f97e0ff9700 1 -- 172.21.15.81:0/1781761939 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f97d4013a00 con 0x7f97e40fb4d0 2024-03-22T07:17:56.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.299+0000 7f97e0ff9700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f97cc06d380 0x7f97cc06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:56.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.299+0000 7f97e0ff9700 1 -- 172.21.15.81:0/1781761939 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f97d408da90 con 0x7f97e40fb4d0 2024-03-22T07:17:56.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.299+0000 7f97e37fe700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f97cc06d380 0x7f97cc06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:56.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.299+0000 7f97e37fe700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f97cc06d380 0x7f97cc06f830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f97e40fdde0 tx=0x7f97d800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:56.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.303+0000 7f97e0ff9700 1 -- 172.21.15.81:0/1781761939 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97d40575f0 con 0x7f97e40fb4d0 2024-03-22T07:17:56.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.506+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f97e40fdb90 con 0x7f97cc06d380 2024-03-22T07:17:56.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.517+0000 7f97e0ff9700 1 -- 172.21.15.81:0/1781761939 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f97e40fdb90 con 0x7f97cc06d380 2024-03-22T07:17:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:17:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (6m) 14s ago 16m 24.4M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:17:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (4m) 14s ago 17m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:17:56.517 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (4m) 0s ago 16m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (6m) 14s ago 16m 87.4M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (38s) 14s ago 11m 12.3M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5405937045b2 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (19s) 14s ago 11m 13.7M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5080d1e80a87 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (11m) 0s ago 11m 17.8M - 17.2.7-726-g9c4e0c34 837124da804f 2086cdd45848 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (3s) 0s ago 11m 13.6M - 19.0.0-1667-gdb0330b1 4f325f8440e9 90fe0ec77272 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (7m) 14s ago 18m 510M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (7m) 0s ago 15m 451M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (5m) 14s ago 18m 49.2M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (4m) 0s ago 15m 40.0M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (7m) 14s ago 16m 17.1M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (7m) 0s ago 16m 16.5M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (3m) 14s ago 15m 98.0M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 033c080308b6 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (3m) 14s ago 14m 108M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 8341fe32f3fa 2024-03-22T07:17:56.518 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (2m) 14s ago 14m 84.7M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 0b8c514dced1 2024-03-22T07:17:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (2m) 0s ago 13m 106M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c8234f7ed003 2024-03-22T07:17:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (87s) 0s ago 13m 102M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 437455ba97c0 2024-03-22T07:17:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (54s) 0s ago 12m 89.5M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0d8d31a867d9 2024-03-22T07:17:56.519 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (6m) 14s ago 16m 63.4M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f97cc06d380 msgr2=0x7f97cc06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f97cc06d380 0x7f97cc06f830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f97e40fdde0 tx=0x7f97d800b040 comp rx=0 tx=0).stop 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e40fb4d0 msgr2=0x7f97e40fc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e40fb4d0 0x7f97e40fc6b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009180 tx=0x7f97d4009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 shutdown_connections 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f97cc06d380 0x7f97cc06f830 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f97e40fb4d0 0x7f97e40fc6b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 --2- 172.21.15.81:0/1781761939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97e4102f50 0x7f97e40fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 >> 172.21.15.81:0/1781761939 conn(0x7f97e40714c0 msgr2=0x7f97e40fa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 shutdown_connections 2024-03-22T07:17:56.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.519+0000 7f97ea4ff700 1 -- 172.21.15.81:0/1781761939 wait complete. 2024-03-22T07:17:56.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.619+0000 7f85b7bd2700 1 -- 172.21.15.81:0/1391491435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0102710 msgr2=0x7f85b00fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.619+0000 7f85b7bd2700 1 --2- 172.21.15.81:0/1391491435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0102710 0x7f85b00fbd80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f85a0004940 tx=0x7f85a0009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:56.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 -- 172.21.15.81:0/1391491435 shutdown_connections 2024-03-22T07:17:56.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 --2- 172.21.15.81:0/1391491435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0102710 0x7f85b00fbd80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 --2- 172.21.15.81:0/1391491435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b0101d70 0x7f85b0102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 -- 172.21.15.81:0/1391491435 >> 172.21.15.81:0/1391491435 conn(0x7f85b0072bc0 msgr2=0x7f85b0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 -- 172.21.15.81:0/1391491435 shutdown_connections 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 -- 172.21.15.81:0/1391491435 wait complete. 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 Processor -- start 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 -- start start 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0101d70 0x7f85b010eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b0102710 0x7f85b010f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b010f6d0 con 0x7f85b0101d70 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b7bd2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b010f810 con 0x7f85b0102710 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.620+0000 7f85b596e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0101d70 0x7f85b010eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85b596e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0101d70 0x7f85b010eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52958/0 (socket says 172.21.15.81:52958) 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85b596e700 1 -- 172.21.15.81:0/3526405591 learned_addr learned my addr 172.21.15.81:0/3526405591 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85b516d700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b0102710 0x7f85b010f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:56.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85b596e700 1 -- 172.21.15.81:0/3526405591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b0102710 msgr2=0x7f85b010f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85b596e700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b0102710 0x7f85b010f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85b596e700 1 -- 172.21.15.81:0/3526405591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0005040 con 0x7f85b0101d70 2024-03-22T07:17:56.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85b596e700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0101d70 0x7f85b010eb40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f85ac002a50 tx=0x7f85ac0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:56.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85a67fc700 1 -- 172.21.15.81:0/3526405591 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85ac007cc0 con 0x7f85b0101d70 2024-03-22T07:17:56.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b0117bb0 con 0x7f85b0101d70 2024-03-22T07:17:56.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85a67fc700 1 -- 172.21.15.81:0/3526405591 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85ac00f040 con 0x7f85b0101d70 2024-03-22T07:17:56.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85a67fc700 1 -- 172.21.15.81:0/3526405591 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85ac00da30 con 0x7f85b0101d70 2024-03-22T07:17:56.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.621+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b01180e0 con 0x7f85b0101d70 2024-03-22T07:17:56.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.622+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b0104240 con 0x7f85b0101d70 2024-03-22T07:17:56.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.623+0000 7f85a67fc700 1 -- 172.21.15.81:0/3526405591 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f85ac00db90 con 0x7f85b0101d70 2024-03-22T07:17:56.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.623+0000 7f85a67fc700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f859c06d380 0x7f859c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:56.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.623+0000 7f85b516d700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f859c06d380 0x7f859c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:56.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.623+0000 7f85a67fc700 1 -- 172.21.15.81:0/3526405591 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f85ac08c5f0 con 0x7f85b0101d70 2024-03-22T07:17:56.627 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.624+0000 7f85b516d700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f859c06d380 0x7f859c06f830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85a0004890 tx=0x7f85a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:56.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.628+0000 7f85a67fc700 1 -- 172.21.15.81:0/3526405591 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85ac0560a0 con 0x7f85b0101d70 2024-03-22T07:17:56.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.902+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f85b004bed0 con 0x7f85b0101d70 2024-03-22T07:17:56.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.903+0000 7f85a67fc700 1 -- 172.21.15.81:0/3526405591 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f85ac059760 con 0x7f85b0101d70 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:17:56.908 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:17:56.909 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-22T07:17:56.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:17:56.909 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:17:56.909 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:17:56.909 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 13 2024-03-22T07:17:56.909 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:17:56.909 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.905+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f859c06d380 msgr2=0x7f859c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.905+0000 7f85b7bd2700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f859c06d380 0x7f859c06f830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85a0004890 tx=0x7f85a0009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.905+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0101d70 msgr2=0x7f85b010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.905+0000 7f85b7bd2700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0101d70 0x7f85b010eb40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f85ac002a50 tx=0x7f85ac0046b0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.905+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 shutdown_connections 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.905+0000 7f85b7bd2700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f859c06d380 0x7f859c06f830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.905+0000 7f85b7bd2700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85b0101d70 0x7f85b010eb40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.905+0000 7f85b7bd2700 1 --2- 172.21.15.81:0/3526405591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b0102710 0x7f85b010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.905+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 >> 172.21.15.81:0/3526405591 conn(0x7f85b0072bc0 msgr2=0x7f85b00fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.906+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 shutdown_connections 2024-03-22T07:17:56.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:56.906+0000 7f85b7bd2700 1 -- 172.21.15.81:0/3526405591 wait complete. 2024-03-22T07:17:57.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.007+0000 7f74e7119700 1 -- 172.21.15.81:0/3692163508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e01026a0 msgr2=0x7f74e00fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.007+0000 7f74e7119700 1 --2- 172.21.15.81:0/3692163508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e01026a0 0x7f74e00fbd10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f74cc004940 tx=0x7f74cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:57.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.007+0000 7f74e7119700 1 -- 172.21.15.81:0/3692163508 shutdown_connections 2024-03-22T07:17:57.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.007+0000 7f74e7119700 1 --2- 172.21.15.81:0/3692163508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e01026a0 0x7f74e00fbd10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.007+0000 7f74e7119700 1 --2- 172.21.15.81:0/3692163508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e0101d00 0x7f74e01020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.007+0000 7f74e7119700 1 -- 172.21.15.81:0/3692163508 >> 172.21.15.81:0/3692163508 conn(0x7f74e00f78d0 msgr2=0x7f74e00f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:57.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.007+0000 7f74e7119700 1 -- 172.21.15.81:0/3692163508 shutdown_connections 2024-03-22T07:17:57.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.007+0000 7f74e7119700 1 -- 172.21.15.81:0/3692163508 wait complete. 2024-03-22T07:17:57.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.008+0000 7f74e7119700 1 Processor -- start 2024-03-22T07:17:57.009 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.008+0000 7f74e7119700 1 -- start start 2024-03-22T07:17:57.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.008+0000 7f74e7119700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e0101d00 0x7f74e0112ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:57.010 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.008+0000 7f74e7119700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e01026a0 0x7f74e0113000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.008+0000 7f74e7119700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e0113650 con 0x7f74e0101d00 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.008+0000 7f74e7119700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e010ca40 con 0x7f74e01026a0 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.008+0000 7f74e4eb5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e0101d00 0x7f74e0112ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.008+0000 7f74e4eb5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e0101d00 0x7f74e0112ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52978/0 (socket says 172.21.15.81:52978) 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74e4eb5700 1 -- 172.21.15.81:0/796166562 learned_addr learned my addr 172.21.15.81:0/796166562 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74dffff700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e01026a0 0x7f74e0113000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74e4eb5700 1 -- 172.21.15.81:0/796166562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e01026a0 msgr2=0x7f74e0113000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74e4eb5700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e01026a0 0x7f74e0113000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74e4eb5700 1 -- 172.21.15.81:0/796166562 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc005040 con 0x7f74e0101d00 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74e4eb5700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e0101d00 0x7f74e0112ac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74d4002a50 tx=0x7f74d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74dd7fa700 1 -- 172.21.15.81:0/796166562 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d4007cc0 con 0x7f74e0101d00 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e010cd20 con 0x7f74e0101d00 2024-03-22T07:17:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74dd7fa700 1 -- 172.21.15.81:0/796166562 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74d400f040 con 0x7f74e0101d00 2024-03-22T07:17:57.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74dd7fa700 1 -- 172.21.15.81:0/796166562 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d400da30 con 0x7f74e0101d00 2024-03-22T07:17:57.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.009+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e010d2b0 con 0x7f74e0101d00 2024-03-22T07:17:57.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.010+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e00fd450 con 0x7f74e0101d00 2024-03-22T07:17:57.012 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.011+0000 7f74dd7fa700 1 -- 172.21.15.81:0/796166562 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f74d400db90 con 0x7f74e0101d00 2024-03-22T07:17:57.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.011+0000 7f74dd7fa700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f74d006d380 0x7f74d006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:57.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.011+0000 7f74dd7fa700 1 -- 172.21.15.81:0/796166562 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f74d408c700 con 0x7f74e0101d00 2024-03-22T07:17:57.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.011+0000 7f74dffff700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f74d006d380 0x7f74d006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:57.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.012+0000 7f74dffff700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f74d006d380 0x7f74d006f830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f74cc004890 tx=0x7f74cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:57.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.016+0000 7f74dd7fa700 1 -- 172.21.15.81:0/796166562 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74d4059c60 con 0x7f74e0101d00 2024-03-22T07:17:57.256 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.256+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f74e0108e30 con 0x7f74e0101d00 2024-03-22T07:17:57.257 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:56 smithi081 ceph-mon[113965]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:57.257 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:56 smithi081 ceph-mon[113965]: pgmap v193: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:57.257 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:57.257 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:56 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:57.257 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:56 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3526405591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.256+0000 7f74dd7fa700 1 -- 172.21.15.81:0/796166562 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1839 (secure 0 0 0) 0x7f74d40597f0 con 0x7f74e0101d00 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stdout:e30 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 23 2024-03-22T07:17:57.258 INFO:teuthology.orchestra.run.smithi081.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:17:11.575650+0000 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 81 2024-03-22T07:17:57.259 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:up {} 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:failed 0,1 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:17:57.260 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:17:57.261 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:17:57.261 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:17:57.261 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:24259} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.149:6826/3446640029,v1:172.21.15.149:6827/3446640029] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:17:57.261 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{-1:34378} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:17:57.261 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:17:57.261 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6824/3255112562,v1:172.21.15.149:6825/3255112562] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f74d006d380 msgr2=0x7f74d006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f74d006d380 0x7f74d006f830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f74cc004890 tx=0x7f74cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e0101d00 msgr2=0x7f74e0112ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e0101d00 0x7f74e0112ac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f74d4002a50 tx=0x7f74d40046b0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 shutdown_connections 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f74d006d380 0x7f74d006f830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e0101d00 0x7f74e0112ac0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 --2- 172.21.15.81:0/796166562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74e01026a0 0x7f74e0113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 >> 172.21.15.81:0/796166562 conn(0x7f74e00f78d0 msgr2=0x7f74e00f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 shutdown_connections 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.259+0000 7f74e7119700 1 -- 172.21.15.81:0/796166562 wait complete. 2024-03-22T07:17:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 30 2024-03-22T07:17:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:56 smithi149 ceph-mon[83447]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:56 smithi149 ceph-mon[83447]: pgmap v193: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:56 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:56 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:56 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3526405591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:57.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.360+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2645899068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 msgr2=0x7fb0c80fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.360+0000 7fb0ce12a700 1 --2- 172.21.15.81:0/2645899068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 0x7fb0c80fbcf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8004940 tx=0x7fb0b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:57.361 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.360+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2645899068 shutdown_connections 2024-03-22T07:17:57.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.360+0000 7fb0ce12a700 1 --2- 172.21.15.81:0/2645899068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 0x7fb0c80fbcf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.360+0000 7fb0ce12a700 1 --2- 172.21.15.81:0/2645899068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c8101ce0 0x7fb0c81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.360+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2645899068 >> 172.21.15.81:0/2645899068 conn(0x7fb0c80f7890 msgr2=0x7fb0c80f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:57.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.361+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2645899068 shutdown_connections 2024-03-22T07:17:57.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.361+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2645899068 wait complete. 2024-03-22T07:17:57.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.361+0000 7fb0ce12a700 1 Processor -- start 2024-03-22T07:17:57.362 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.361+0000 7fb0ce12a700 1 -- start start 2024-03-22T07:17:57.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0ce12a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c8101ce0 0x7fb0c810eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:57.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0ce12a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 0x7fb0c810f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:57.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0ce12a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c810f5d0 con 0x7fb0c8102680 2024-03-22T07:17:57.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0ce12a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c810f710 con 0x7fb0c8101ce0 2024-03-22T07:17:57.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0c6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 0x7fb0c810f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:57.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0c6ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 0x7fb0c810f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52990/0 (socket says 172.21.15.81:52990) 2024-03-22T07:17:57.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0c6ffd700 1 -- 172.21.15.81:0/2658683970 learned_addr learned my addr 172.21.15.81:0/2658683970 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:57.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0c77fe700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c8101ce0 0x7fb0c810eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:57.364 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0c6ffd700 1 -- 172.21.15.81:0/2658683970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c8101ce0 msgr2=0x7fb0c810eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0c6ffd700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c8101ce0 0x7fb0c810eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0c6ffd700 1 -- 172.21.15.81:0/2658683970 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8005040 con 0x7fb0c8102680 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0c6ffd700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 0x7fb0c810f090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009100 tx=0x7fb0b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0affff700 1 -- 172.21.15.81:0/2658683970 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0b800e040 con 0x7fb0c8102680 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c810f990 con 0x7fb0c8102680 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0affff700 1 -- 172.21.15.81:0/2658683970 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0b8004290 con 0x7fb0c8102680 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0affff700 1 -- 172.21.15.81:0/2658683970 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0b801c3f0 con 0x7fb0c8102680 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.362+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c8117f60 con 0x7fb0c8102680 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.363+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c80fd430 con 0x7fb0c8102680 2024-03-22T07:17:57.365 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.364+0000 7fb0affff700 1 -- 172.21.15.81:0/2658683970 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fb0b8013690 con 0x7fb0c8102680 2024-03-22T07:17:57.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.364+0000 7fb0affff700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb0b006d380 0x7fb0b006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:57.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.365+0000 7fb0c77fe700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb0b006d380 0x7fb0b006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:57.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.365+0000 7fb0affff700 1 -- 172.21.15.81:0/2658683970 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb0b808da40 con 0x7fb0c8102680 2024-03-22T07:17:57.366 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.365+0000 7fb0c77fe700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb0b006d380 0x7fb0b006f830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009bf0 tx=0x7fb0bc0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:57.369 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.369+0000 7fb0affff700 1 -- 172.21.15.81:0/2658683970 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0b80574f0 con 0x7fb0c8102680 2024-03-22T07:17:57.581 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.580+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0c81061a0 con 0x7fb0b006d380 2024-03-22T07:17:57.582 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.582+0000 7fb0affff700 1 -- 172.21.15.81:0/2658683970 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb0c81061a0 con 0x7fb0b006d380 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": true, 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [ 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: "mon", 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr", 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: "crash", 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: "osd" 2024-03-22T07:17:57.583 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-03-22T07:17:57.584 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": "15/21 daemons upgraded", 2024-03-22T07:17:57.584 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T07:17:57.584 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:17:57.584 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:17:57.584 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb0b006d380 msgr2=0x7fb0b006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb0b006d380 0x7fb0b006f830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009bf0 tx=0x7fb0bc0091a0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 msgr2=0x7fb0c810f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 0x7fb0c810f090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009100 tx=0x7fb0b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 shutdown_connections 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb0b006d380 0x7fb0b006f830 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c8101ce0 0x7fb0c810eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 --2- 172.21.15.81:0/2658683970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0c8102680 0x7fb0c810f090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 >> 172.21.15.81:0/2658683970 conn(0x7fb0c80f7890 msgr2=0x7fb0c8104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 shutdown_connections 2024-03-22T07:17:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.584+0000 7fb0ce12a700 1 -- 172.21.15.81:0/2658683970 wait complete. 2024-03-22T07:17:57.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.685+0000 7f8281c50700 1 -- 172.21.15.81:0/611656137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 msgr2=0x7f827c1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.685+0000 7f8281c50700 1 --2- 172.21.15.81:0/611656137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 0x7f827c1020f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f826c004940 tx=0x7f826c009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:57.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.685+0000 7f8281c50700 1 -- 172.21.15.81:0/611656137 shutdown_connections 2024-03-22T07:17:57.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.685+0000 7f8281c50700 1 --2- 172.21.15.81:0/611656137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f827c1026c0 0x7f827c0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.685+0000 7f8281c50700 1 --2- 172.21.15.81:0/611656137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 0x7f827c1020f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.685+0000 7f8281c50700 1 -- 172.21.15.81:0/611656137 >> 172.21.15.81:0/611656137 conn(0x7f827c0f7870 msgr2=0x7f827c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:57.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.685+0000 7f8281c50700 1 -- 172.21.15.81:0/611656137 shutdown_connections 2024-03-22T07:17:57.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.685+0000 7f8281c50700 1 -- 172.21.15.81:0/611656137 wait complete. 2024-03-22T07:17:57.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.685+0000 7f8281c50700 1 Processor -- start 2024-03-22T07:17:57.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f8281c50700 1 -- start start 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f8281c50700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 0x7f827c10eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f8281c50700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f827c1026c0 0x7f827c10f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f8281c50700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c10f600 con 0x7f827c101d20 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f8281c50700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c10f740 con 0x7f827c1026c0 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f827b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 0x7f827c10eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f827b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 0x7f827c10eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53010/0 (socket says 172.21.15.81:53010) 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f827b7fe700 1 -- 172.21.15.81:0/1390678628 learned_addr learned my addr 172.21.15.81:0/1390678628 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f827affd700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f827c1026c0 0x7f827c10f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f827b7fe700 1 -- 172.21.15.81:0/1390678628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f827c1026c0 msgr2=0x7f827c10f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f827b7fe700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f827c1026c0 0x7f827c10f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f827b7fe700 1 -- 172.21.15.81:0/1390678628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c005040 con 0x7f827c101d20 2024-03-22T07:17:57.688 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f827b7fe700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 0x7f827c10eb80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f826c00b3b0 tx=0x7f826c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:57.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f8263fff700 1 -- 172.21.15.81:0/1390678628 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826c00e040 con 0x7f827c101d20 2024-03-22T07:17:57.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f8263fff700 1 -- 172.21.15.81:0/1390678628 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f826c00bbc0 con 0x7f827c101d20 2024-03-22T07:17:57.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c10f9c0 con 0x7f827c101d20 2024-03-22T07:17:57.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.686+0000 7f8263fff700 1 -- 172.21.15.81:0/1390678628 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826c01c3f0 con 0x7f827c101d20 2024-03-22T07:17:57.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.687+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c117ef0 con 0x7f827c101d20 2024-03-22T07:17:57.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.687+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c0fd470 con 0x7f827c101d20 2024-03-22T07:17:57.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.688+0000 7f8263fff700 1 -- 172.21.15.81:0/1390678628 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f826c00bd30 con 0x7f827c101d20 2024-03-22T07:17:57.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.688+0000 7f8263fff700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f826406d380 0x7f826406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:57.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.688+0000 7f8263fff700 1 -- 172.21.15.81:0/1390678628 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f826c08dc00 con 0x7f827c101d20 2024-03-22T07:17:57.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.688+0000 7f827affd700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f826406d380 0x7f826406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:57.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.689+0000 7f827affd700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f826406d380 0x7f826406f830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f827c066a60 tx=0x7f8270009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:57.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.693+0000 7f8263fff700 1 -- 172.21.15.81:0/1390678628 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f826c093020 con 0x7f827c101d20 2024-03-22T07:17:57.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.969+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f827c04c710 con 0x7f827c101d20 2024-03-22T07:17:57.970 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.970+0000 7f8263fff700 1 -- 172.21.15.81:0/1390678628 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f826c057630 con 0x7f827c101d20 2024-03-22T07:17:57.971 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-22T07:17:57.971 INFO:teuthology.orchestra.run.smithi081.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-22T07:17:57.971 INFO:teuthology.orchestra.run.smithi081.stdout: fs cephfs is degraded 2024-03-22T07:17:57.971 INFO:teuthology.orchestra.run.smithi081.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-22T07:17:57.971 INFO:teuthology.orchestra.run.smithi081.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-22T07:17:57.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f826406d380 msgr2=0x7f826406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.972 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f826406d380 0x7f826406f830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f827c066a60 tx=0x7f8270009040 comp rx=0 tx=0).stop 2024-03-22T07:17:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 msgr2=0x7f827c10eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 0x7f827c10eb80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f826c00b3b0 tx=0x7f826c009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 shutdown_connections 2024-03-22T07:17:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f826406d380 0x7f826406f830 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f827c101d20 0x7f827c10eb80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 --2- 172.21.15.81:0/1390678628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f827c1026c0 0x7f827c10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 >> 172.21.15.81:0/1390678628 conn(0x7f827c0f7870 msgr2=0x7f827c0f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 shutdown_connections 2024-03-22T07:17:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:17:57.972+0000 7f8281c50700 1 -- 172.21.15.81:0/1390678628 wait complete. 2024-03-22T07:17:58.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:57 smithi149 ceph-mon[83447]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:58.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:57 smithi149 ceph-mon[83447]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:58.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:57 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:58.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:57 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:58.029 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:57 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/796166562' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:17:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:57 smithi081 ceph-mon[113965]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:57 smithi081 ceph-mon[113965]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:57 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:57 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:17:58.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:57 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/796166562' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:17:59.067 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:58 smithi149 ceph-mon[83447]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:59.067 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:58 smithi149 ceph-mon[83447]: pgmap v194: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:59.068 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:17:58 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1390678628' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:17:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:58 smithi081 ceph-mon[113965]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:58 smithi081 ceph-mon[113965]: pgmap v194: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:17:58 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1390678628' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:18:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: pgmap v195: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:01.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:01.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:01.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:01.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:01.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:01.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:01.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:01.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:01.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:01.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: pgmap v195: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:01.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:01.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:01.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:01.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:01.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:01.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:01.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:01.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:01.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:01.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:03.188 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:02 smithi149 ceph-mon[83447]: pgmap v196: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:03.188 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:02 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:03.188 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:02 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.ffewvr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:18:03.188 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:02 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:02 smithi081 ceph-mon[113965]: pgmap v196: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:02 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:02 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi149.ffewvr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:18:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:02 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:04.232 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:03 smithi149 ceph-mon[83447]: Upgrade: Updating mds.cephfs.smithi149.ffewvr 2024-03-22T07:18:04.233 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:03 smithi149 ceph-mon[83447]: Deploying daemon mds.cephfs.smithi149.ffewvr on smithi149 2024-03-22T07:18:04.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:03 smithi081 ceph-mon[113965]: Upgrade: Updating mds.cephfs.smithi149.ffewvr 2024-03-22T07:18:04.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:03 smithi081 ceph-mon[113965]: Deploying daemon mds.cephfs.smithi149.ffewvr on smithi149 2024-03-22T07:18:05.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:04 smithi081 ceph-mon[113965]: pgmap v197: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:04 smithi149 ceph-mon[83447]: pgmap v197: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:06.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:05 smithi081 ceph-mon[113965]: osdmap e85: 6 total, 6 up, 6 in 2024-03-22T07:18:06.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:05 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:18:06.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:05 smithi149 ceph-mon[83447]: osdmap e85: 6 total, 6 up, 6 in 2024-03-22T07:18:06.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:05 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:18:07.226 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:06 smithi149 ceph-mon[83447]: pgmap v199: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:07.226 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:06 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:18:07.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:06 smithi081 ceph-mon[113965]: pgmap v199: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:07.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:06 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:18:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:08 smithi081 ceph-mon[113965]: pgmap v200: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:08 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:08 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:08 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:08 smithi149 ceph-mon[83447]: pgmap v200: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:08 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:08 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:08 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:10.239 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:09 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.149:6826/1411110570,v1:172.21.15.149:6827/1411110570] up:boot 2024-03-22T07:18:10.239 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:09 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:18:10.239 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:09 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:18:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:09 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.149:6826/1411110570,v1:172.21.15.149:6827/1411110570] up:boot 2024-03-22T07:18:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:09 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:18:10.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:09 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:18:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:10 smithi081 ceph-mon[113965]: pgmap v201: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:10 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:10 smithi149 ceph-mon[83447]: pgmap v201: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:10 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:13.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:12 smithi081 ceph-mon[113965]: pgmap v202: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:13.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:13.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:12 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:12 smithi149 ceph-mon[83447]: pgmap v202: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:12 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:15.251 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:14 smithi149 ceph-mon[83447]: pgmap v203: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:15.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:14 smithi081 ceph-mon[113965]: pgmap v203: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:17.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all mds 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi081.bnzfrv"}]': finished 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi081.zfgxzy"}]': finished 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.ffewvr"}]': finished 2024-03-22T07:18:17.070 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:18:17.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.kdtxfw"}]': finished 2024-03-22T07:18:17.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: Upgrade: Setting filesystem cephfs Joinable 2024-03-22T07:18:17.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-22T07:18:17.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: pgmap v204: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:17.071 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:16 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:17.102 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all mds 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi081.bnzfrv"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi081.bnzfrv"}]': finished 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi081.zfgxzy"}]: dispatch 2024-03-22T07:18:17.103 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi081.zfgxzy"}]': finished 2024-03-22T07:18:17.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.ffewvr"}]: dispatch 2024-03-22T07:18:17.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.ffewvr"}]': finished 2024-03-22T07:18:17.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.kdtxfw"}]: dispatch 2024-03-22T07:18:17.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi149.kdtxfw"}]': finished 2024-03-22T07:18:17.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: Upgrade: Setting filesystem cephfs Joinable 2024-03-22T07:18:17.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-22T07:18:17.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: pgmap v204: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:17.104 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:16 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:18.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-22T07:18:18.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Standby daemon mds.cephfs.smithi081.bnzfrv assigned to filesystem cephfs as rank 0 2024-03-22T07:18:18.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Standby daemon mds.cephfs.smithi081.zfgxzy assigned to filesystem cephfs as rank 1 2024-03-22T07:18:18.320 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T07:18:18.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:18:18.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:replay,1=cephfs.smithi081.zfgxzy=up:replay} 2 up:standby 2024-03-22T07:18:18.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all rgw 2024-03-22T07:18:18.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T07:18:18.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.321 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all iscsi 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all nfs 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.322 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Upgrade: Setting container_image for all nvmeof 2024-03-22T07:18:18.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.323 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.324 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.324 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Upgrade: Finalizing container_image settings 2024-03-22T07:18:18.324 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T07:18:18.324 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T07:18:18.324 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.324 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T07:18:18.324 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T07:18:18.324 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-22T07:18:18.325 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T07:18:18.325 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-22T07:18:18.325 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T07:18:18.325 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-22T07:18:18.325 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T07:18:18.325 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T07:18:18.325 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T07:18:18.325 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.326 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.327 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.327 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.327 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: Upgrade: Complete! 2024-03-22T07:18:18.327 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T07:18:18.327 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T07:18:18.327 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:18.327 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:18.328 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:18.328 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.328 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:18.328 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:18.328 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:18.328 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:17 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-22T07:18:18.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Standby daemon mds.cephfs.smithi081.bnzfrv assigned to filesystem cephfs as rank 0 2024-03-22T07:18:18.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Standby daemon mds.cephfs.smithi081.zfgxzy assigned to filesystem cephfs as rank 1 2024-03-22T07:18:18.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T07:18:18.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:18:18.354 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:replay,1=cephfs.smithi081.zfgxzy=up:replay} 2 up:standby 2024-03-22T07:18:18.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all rgw 2024-03-22T07:18:18.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T07:18:18.355 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T07:18:18.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.356 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all iscsi 2024-03-22T07:18:18.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all nfs 2024-03-22T07:18:18.357 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Upgrade: Setting container_image for all nvmeof 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:18.358 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Upgrade: Finalizing container_image settings 2024-03-22T07:18:18.359 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T07:18:18.359 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T07:18:18.359 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.359 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T07:18:18.359 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T07:18:18.359 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-22T07:18:18.359 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T07:18:18.360 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T07:18:18.361 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T07:18:18.361 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T07:18:18.361 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T07:18:18.361 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T07:18:18.361 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T07:18:18.361 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T07:18:18.361 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.361 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.362 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.362 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.362 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.362 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:18:18.362 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: Upgrade: Complete! 2024-03-22T07:18:18.362 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T07:18:18.363 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T07:18:18.363 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:18.363 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:18.363 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:18.363 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:18.363 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:18.364 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:18.364 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:18.364 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:17 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:19.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:18 smithi149 ceph-mon[83447]: pgmap v205: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:19.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:18 smithi081 ceph-mon[113965]: pgmap v205: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:18:21.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:20 smithi081 ceph-mon[113965]: pgmap v206: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 8.3 MiB/s rd, 3 op/s 2024-03-22T07:18:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:20 smithi149 ceph-mon[83447]: pgmap v206: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 8.3 MiB/s rd, 3 op/s 2024-03-22T07:18:22.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:21 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:resolve 2024-03-22T07:18:22.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:21 smithi081 ceph-mon[113965]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:replay,1=cephfs.smithi081.zfgxzy=up:resolve} 2 up:standby 2024-03-22T07:18:22.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:21 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:21 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:resolve 2024-03-22T07:18:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:21 smithi149 ceph-mon[83447]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:replay,1=cephfs.smithi081.zfgxzy=up:resolve} 2 up:standby 2024-03-22T07:18:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:21 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:23.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:22 smithi081 ceph-mon[113965]: pgmap v207: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 6 op/s 2024-03-22T07:18:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:22 smithi149 ceph-mon[83447]: pgmap v207: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 6 op/s 2024-03-22T07:18:24.234 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:23 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:resolve 2024-03-22T07:18:24.234 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:23 smithi081 ceph-mon[113965]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:resolve,1=cephfs.smithi081.zfgxzy=up:resolve} 2 up:standby 2024-03-22T07:18:24.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:23 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:resolve 2024-03-22T07:18:24.253 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:23 smithi149 ceph-mon[83447]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:resolve,1=cephfs.smithi081.zfgxzy=up:resolve} 2 up:standby 2024-03-22T07:18:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:24 smithi081 ceph-mon[113965]: pgmap v208: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 6 op/s 2024-03-22T07:18:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:24 smithi081 ceph-mon[113965]: reconnect by client.14540 v1:172.21.15.149:0/2343999153 after 0.000999981 2024-03-22T07:18:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:24 smithi081 ceph-mon[113965]: reconnect by client.14540 v1:172.21.15.149:0/2343999153 after 0.000999981 2024-03-22T07:18:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:24 smithi081 ceph-mon[113965]: reconnect by client.14534 v1:192.168.0.1:0/323690204 after 0.00299994 2024-03-22T07:18:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:24 smithi081 ceph-mon[113965]: reconnect by client.14534 v1:192.168.0.1:0/323690204 after 0.00299994 2024-03-22T07:18:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:24 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:reconnect 2024-03-22T07:18:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:24 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:reconnect 2024-03-22T07:18:25.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:24 smithi081 ceph-mon[113965]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:reconnect,1=cephfs.smithi081.zfgxzy=up:reconnect} 2 up:standby 2024-03-22T07:18:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:24 smithi149 ceph-mon[83447]: pgmap v208: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 6 op/s 2024-03-22T07:18:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:24 smithi149 ceph-mon[83447]: reconnect by client.14540 v1:172.21.15.149:0/2343999153 after 0.000999981 2024-03-22T07:18:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:24 smithi149 ceph-mon[83447]: reconnect by client.14540 v1:172.21.15.149:0/2343999153 after 0.000999981 2024-03-22T07:18:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:24 smithi149 ceph-mon[83447]: reconnect by client.14534 v1:192.168.0.1:0/323690204 after 0.00299994 2024-03-22T07:18:25.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:24 smithi149 ceph-mon[83447]: reconnect by client.14534 v1:192.168.0.1:0/323690204 after 0.00299994 2024-03-22T07:18:25.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:24 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:reconnect 2024-03-22T07:18:25.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:24 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:reconnect 2024-03-22T07:18:25.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:24 smithi149 ceph-mon[83447]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:reconnect,1=cephfs.smithi081.zfgxzy=up:reconnect} 2 up:standby 2024-03-22T07:18:26.201 INFO:tasks.workunit.client.0.smithi081.stdout:5/321: sync 2024-03-22T07:18:26.201 INFO:tasks.workunit.client.0.smithi081.stdout:9/240: sync 2024-03-22T07:18:26.201 INFO:tasks.workunit.client.0.smithi081.stdout:3/218: sync 2024-03-22T07:18:26.201 INFO:tasks.workunit.client.0.smithi081.stdout:7/437: sync 2024-03-22T07:18:26.201 INFO:tasks.workunit.client.0.smithi081.stdout:4/391: sync 2024-03-22T07:18:26.201 INFO:tasks.workunit.client.0.smithi081.stdout:1/231: rename d6/d2e to d6/d18/d41/d49/d56 0 2024-03-22T07:18:26.202 INFO:tasks.workunit.client.0.smithi081.stdout:0/391: sync 2024-03-22T07:18:26.234 INFO:tasks.workunit.client.0.smithi081.stdout:2/221: rename da/d11/d21/d2c to da/db/d39/d46 0 2024-03-22T07:18:26.236 INFO:tasks.workunit.client.0.smithi081.stdout:0/392: getdents d3/dc 0 2024-03-22T07:18:26.245 INFO:tasks.workunit.client.0.smithi081.stdout:9/241: mknod d2/d2b/c4d 0 2024-03-22T07:18:26.246 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:25 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:rejoin 2024-03-22T07:18:26.246 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:25 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:rejoin 2024-03-22T07:18:26.246 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:25 smithi081 ceph-mon[113965]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:rejoin,1=cephfs.smithi081.zfgxzy=up:rejoin} 2 up:standby 2024-03-22T07:18:26.246 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:25 smithi081 ceph-mon[113965]: daemon mds.cephfs.smithi081.bnzfrv is now active in filesystem cephfs as rank 0 2024-03-22T07:18:26.246 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:25 smithi081 ceph-mon[113965]: daemon mds.cephfs.smithi081.zfgxzy is now active in filesystem cephfs as rank 1 2024-03-22T07:18:26.257 INFO:tasks.workunit.client.0.smithi081.stdout:7/438: mkdir d9/d10/d4b/d9b 0 2024-03-22T07:18:26.269 INFO:tasks.workunit.client.0.smithi081.stdout:6/312: rename d3/da/de/df to d3/da/de/d31/d40/d50/d63 0 2024-03-22T07:18:26.283 INFO:tasks.workunit.client.0.smithi081.stdout:8/285: rename d2/d3/l20 to d2/d3/d28/d2b/d61/l68 0 2024-03-22T07:18:26.331 INFO:tasks.workunit.client.0.smithi081.stdout:6/313: symlink d3/da/de/d31/d40/d50/d63/d5e/l64 0 2024-03-22T07:18:26.343 INFO:tasks.workunit.client.0.smithi081.stdout:7/439: dread - d9/d3f/d52/d83/f8e zero size 2024-03-22T07:18:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:25 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:rejoin 2024-03-22T07:18:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:25 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:rejoin 2024-03-22T07:18:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:25 smithi149 ceph-mon[83447]: fsmap cephfs:2/2 {0=cephfs.smithi081.bnzfrv=up:rejoin,1=cephfs.smithi081.zfgxzy=up:rejoin} 2 up:standby 2024-03-22T07:18:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:25 smithi149 ceph-mon[83447]: daemon mds.cephfs.smithi081.bnzfrv is now active in filesystem cephfs as rank 0 2024-03-22T07:18:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:25 smithi149 ceph-mon[83447]: daemon mds.cephfs.smithi081.zfgxzy is now active in filesystem cephfs as rank 1 2024-03-22T07:18:26.352 INFO:tasks.workunit.client.0.smithi081.stdout:7/440: mknod d9/d10/d16/d1e/d27/c9c 0 2024-03-22T07:18:26.366 INFO:tasks.workunit.client.0.smithi081.stdout:7/441: mkdir d9/d10/d16/d1e/d27/d6d/d8c/d9d 0 2024-03-22T07:18:26.371 INFO:tasks.workunit.client.0.smithi081.stdout:7/442: mkdir d9/d10/d9e 0 2024-03-22T07:18:26.389 INFO:tasks.workunit.client.0.smithi081.stdout:7/443: link d9/d10/d53/f98 d9/d10/d53/d81/d59/f9f 0 2024-03-22T07:18:26.400 INFO:tasks.workunit.client.0.smithi081.stdout:7/444: getdents d9/d10/d16/d1e/d27/d6d 0 2024-03-22T07:18:26.406 INFO:tasks.workunit.client.0.smithi081.stdout:7/445: creat d9/d3f/fa0 x:0 0 0 2024-03-22T07:18:26.452 INFO:tasks.workunit.client.0.smithi081.stdout:7/446: dread - d9/d3f/f79 zero size 2024-03-22T07:18:26.459 INFO:tasks.workunit.client.0.smithi081.stdout:7/447: mkdir d9/d10/d16/d1e/d80/da1 0 2024-03-22T07:18:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:26 smithi081 ceph-mon[113965]: pgmap v209: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 6 op/s 2024-03-22T07:18:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:26 smithi081 ceph-mon[113965]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T07:18:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:26 smithi081 ceph-mon[113965]: Cluster is now healthy 2024-03-22T07:18:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:26 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:active 2024-03-22T07:18:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:26 smithi081 ceph-mon[113965]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:active 2024-03-22T07:18:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:26 smithi081 ceph-mon[113965]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi081.zfgxzy=up:active} 2 up:standby 2024-03-22T07:18:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:26 smithi149 ceph-mon[83447]: pgmap v209: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 6 op/s 2024-03-22T07:18:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:26 smithi149 ceph-mon[83447]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T07:18:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:26 smithi149 ceph-mon[83447]: Cluster is now healthy 2024-03-22T07:18:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:26 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] up:active 2024-03-22T07:18:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:26 smithi149 ceph-mon[83447]: mds.? [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] up:active 2024-03-22T07:18:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:26 smithi149 ceph-mon[83447]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi081.zfgxzy=up:active} 2 up:standby 2024-03-22T07:18:28.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.107+0000 7fed1d639700 1 -- 172.21.15.81:0/2982934643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed18101d70 msgr2=0x7fed18102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.107+0000 7fed1d639700 1 --2- 172.21.15.81:0/2982934643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed18101d70 0x7fed18102140 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fed08004940 tx=0x7fed08009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.107+0000 7fed1d639700 1 -- 172.21.15.81:0/2982934643 shutdown_connections 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.107+0000 7fed1d639700 1 --2- 172.21.15.81:0/2982934643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed180fc5b0 0x7fed180fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.107+0000 7fed1d639700 1 --2- 172.21.15.81:0/2982934643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed18101d70 0x7fed18102140 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.107+0000 7fed1d639700 1 -- 172.21.15.81:0/2982934643 >> 172.21.15.81:0/2982934643 conn(0x7fed180714c0 msgr2=0x7fed180718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.107+0000 7fed1d639700 1 -- 172.21.15.81:0/2982934643 shutdown_connections 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.107+0000 7fed1d639700 1 -- 172.21.15.81:0/2982934643 wait complete. 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed1d639700 1 Processor -- start 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed1d639700 1 -- start start 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed1d639700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed180fc5b0 0x7fed1811a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed1d639700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed18101d70 0x7fed1811a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:28.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed1d639700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1811af00 con 0x7fed180fc5b0 2024-03-22T07:18:28.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed1d639700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1810b8f0 con 0x7fed18101d70 2024-03-22T07:18:28.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed16ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed180fc5b0 0x7fed1811a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:28.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed16ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed180fc5b0 0x7fed1811a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54066/0 (socket says 172.21.15.81:54066) 2024-03-22T07:18:28.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed16ffd700 1 -- 172.21.15.81:0/3648752697 learned_addr learned my addr 172.21.15.81:0/3648752697 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:18:28.110 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.108+0000 7fed167fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed18101d70 0x7fed1811a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fed16ffd700 1 -- 172.21.15.81:0/3648752697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed18101d70 msgr2=0x7fed1811a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fed16ffd700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed18101d70 0x7fed1811a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fed16ffd700 1 -- 172.21.15.81:0/3648752697 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed08005040 con 0x7fed180fc5b0 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fed167fc700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed18101d70 0x7fed1811a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fed16ffd700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed180fc5b0 0x7fed1811a370 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed080043d0 tx=0x7fed08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fecff7fe700 1 -- 172.21.15.81:0/3648752697 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed0800e030 con 0x7fed180fc5b0 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fecff7fe700 1 -- 172.21.15.81:0/3648752697 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed08011e90 con 0x7fed180fc5b0 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fecff7fe700 1 -- 172.21.15.81:0/3648752697 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed08022dc0 con 0x7fed180fc5b0 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1810baf0 con 0x7fed180fc5b0 2024-03-22T07:18:28.111 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.109+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1810c0b0 con 0x7fed180fc5b0 2024-03-22T07:18:28.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.110+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed180fd5c0 con 0x7fed180fc5b0 2024-03-22T07:18:28.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.111+0000 7fecff7fe700 1 -- 172.21.15.81:0/3648752697 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fed080119c0 con 0x7fed180fc5b0 2024-03-22T07:18:28.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.112+0000 7fecff7fe700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fed0006d310 0x7fed0006f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:28.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.112+0000 7fecff7fe700 1 -- 172.21.15.81:0/3648752697 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fed0808def0 con 0x7fed180fc5b0 2024-03-22T07:18:28.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.112+0000 7fed167fc700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fed0006d310 0x7fed0006f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:28.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.113+0000 7fed167fc700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fed0006d310 0x7fed0006f7c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed18115c10 tx=0x7fed0c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:28.121 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.121+0000 7fecff7fe700 1 -- 172.21.15.81:0/3648752697 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed0805b4d0 con 0x7fed180fc5b0 2024-03-22T07:18:28.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.458+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed1810d000 con 0x7fed0006d310 2024-03-22T07:18:28.460 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.460+0000 7fecff7fe700 1 -- 172.21.15.81:0/3648752697 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fed1810d000 con 0x7fed0006d310 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fed0006d310 msgr2=0x7fed0006f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fed0006d310 0x7fed0006f7c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fed18115c10 tx=0x7fed0c00b040 comp rx=0 tx=0).stop 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed180fc5b0 msgr2=0x7fed1811a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed180fc5b0 0x7fed1811a370 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed080043d0 tx=0x7fed08009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 shutdown_connections 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fed0006d310 0x7fed0006f7c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fed180fc5b0 0x7fed1811a370 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 --2- 172.21.15.81:0/3648752697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed18101d70 0x7fed1811a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 >> 172.21.15.81:0/3648752697 conn(0x7fed180714c0 msgr2=0x7fed18104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.464+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 shutdown_connections 2024-03-22T07:18:28.465 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:28.465+0000 7fed1d639700 1 -- 172.21.15.81:0/3648752697 wait complete. 2024-03-22T07:18:28.649 INFO:tasks.workunit.client.0.smithi081.stdout:4/392: write da/d18/d1b/d1e/d37/f6c [17976,28756] 0 2024-03-22T07:18:28.649 INFO:tasks.workunit.client.0.smithi081.stdout:9/242: write d2/d10/f35 [787824,113027] 0 2024-03-22T07:18:28.657 INFO:tasks.workunit.client.0.smithi081.stdout:9/243: mknod d2/d2b/c4e 0 2024-03-22T07:18:28.661 INFO:tasks.workunit.client.0.smithi081.stdout:5/322: write f2 [2880276,97042] 0 2024-03-22T07:18:28.662 INFO:tasks.workunit.client.0.smithi081.stdout:9/244: chown d2/d5/d16/l1a 897 1 2024-03-22T07:18:28.670 INFO:tasks.workunit.client.0.smithi081.stdout:3/219: write db/f12 [797061,75975] 0 2024-03-22T07:18:28.670 INFO:tasks.workunit.client.0.smithi081.stdout:8/286: dread d2/d3/d19/d1e/d13/f47 [4194304,4194304] 0 2024-03-22T07:18:28.685 INFO:tasks.workunit.client.0.smithi081.stdout:2/222: dwrite da/f22 [8388608,4194304] 0 2024-03-22T07:18:28.685 INFO:tasks.workunit.client.0.smithi081.stdout:7/448: dwrite d9/f3c [0,4194304] 0 2024-03-22T07:18:28.686 INFO:tasks.workunit.client.0.smithi081.stdout:4/393: mkdir da/d3d/d7a 0 2024-03-22T07:18:28.686 INFO:tasks.workunit.client.0.smithi081.stdout:1/232: dwrite d6/d18/d41/d49/d56/f44 [0,4194304] 0 2024-03-22T07:18:28.686 INFO:tasks.workunit.client.0.smithi081.stdout:9/245: dread d2/d10/f29 [0,4194304] 0 2024-03-22T07:18:28.686 INFO:tasks.workunit.client.0.smithi081.stdout:5/323: write d4/d10/f24 [1001145,34858] 0 2024-03-22T07:18:28.687 INFO:tasks.workunit.client.0.smithi081.stdout:3/220: write db/d23/f28 [1303763,14145] 0 2024-03-22T07:18:28.697 INFO:tasks.workunit.client.0.smithi081.stdout:7/449: readlink d9/d3f/l5d 0 2024-03-22T07:18:28.698 INFO:tasks.workunit.client.0.smithi081.stdout:3/221: mknod db/d38/c42 0 2024-03-22T07:18:28.700 INFO:tasks.workunit.client.0.smithi081.stdout:1/233: getdents d6/d18/d41/d49/d56/d30 0 2024-03-22T07:18:28.702 INFO:tasks.workunit.client.0.smithi081.stdout:5/324: unlink d4/d10/d20/l38 0 2024-03-22T07:18:28.709 INFO:tasks.workunit.client.0.smithi081.stdout:4/394: truncate da/d18/d1b/d1e/f62 5135418 0 2024-03-22T07:18:28.710 INFO:tasks.workunit.client.0.smithi081.stdout:8/287: dwrite d2/d3/d23/f52 [0,4194304] 0 2024-03-22T07:18:28.711 INFO:tasks.workunit.client.0.smithi081.stdout:7/450: fdatasync d9/d3f/d52/d83/f8e 0 2024-03-22T07:18:28.714 INFO:tasks.workunit.client.0.smithi081.stdout:3/222: creat db/d2e/d3a/f43 x:0 0 0 2024-03-22T07:18:28.716 INFO:tasks.workunit.client.0.smithi081.stdout:2/223: rename da/d11/d32 to da/db/d39/d47 0 2024-03-22T07:18:28.717 INFO:tasks.workunit.client.0.smithi081.stdout:8/288: stat f1 0 2024-03-22T07:18:28.719 INFO:tasks.workunit.client.0.smithi081.stdout:4/395: getdents da/d18/d1b/d26 0 2024-03-22T07:18:28.721 INFO:tasks.workunit.client.0.smithi081.stdout:5/325: fsync d4/df/d22/d49/f4a 0 2024-03-22T07:18:28.724 INFO:tasks.workunit.client.0.smithi081.stdout:4/396: creat da/f7b x:0 0 0 2024-03-22T07:18:28.725 INFO:tasks.workunit.client.0.smithi081.stdout:9/246: dwrite d2/d5/d16/f4c [0,4194304] 0 2024-03-22T07:18:28.727 INFO:tasks.workunit.client.0.smithi081.stdout:3/223: creat db/d2e/f44 x:0 0 0 2024-03-22T07:18:28.731 INFO:tasks.workunit.client.0.smithi081.stdout:9/247: fdatasync d2/d2b/f41 0 2024-03-22T07:18:28.733 INFO:tasks.workunit.client.0.smithi081.stdout:5/326: rmdir d4/df/d22/d4d 39 2024-03-22T07:18:28.734 INFO:tasks.workunit.client.0.smithi081.stdout:3/224: rename db to db/d2e/d45 22 2024-03-22T07:18:28.735 INFO:tasks.workunit.client.0.smithi081.stdout:7/451: symlink d9/d10/d2a/la2 0 2024-03-22T07:18:28.736 INFO:tasks.workunit.client.0.smithi081.stdout:5/327: chown d4/d10/c56 26476283 1 2024-03-22T07:18:28.738 INFO:tasks.workunit.client.0.smithi081.stdout:1/234: dwrite d6/f12 [0,4194304] 0 2024-03-22T07:18:28.740 INFO:tasks.workunit.client.0.smithi081.stdout:5/328: chown d4/d10/c48 52645222 1 2024-03-22T07:18:28.740 INFO:tasks.workunit.client.0.smithi081.stdout:2/224: mknod da/db/d39/d47/d38/c48 0 2024-03-22T07:18:28.741 INFO:tasks.workunit.client.0.smithi081.stdout:8/289: truncate d2/d3/d19/d1e/d13/f60 428177 0 2024-03-22T07:18:28.742 INFO:tasks.workunit.client.0.smithi081.stdout:3/225: creat db/d2e/d3a/f46 x:0 0 0 2024-03-22T07:18:28.743 INFO:tasks.workunit.client.0.smithi081.stdout:1/235: getdents d6 0 2024-03-22T07:18:28.746 INFO:tasks.workunit.client.0.smithi081.stdout:8/290: chown d2/d3/d19/d1e/f43 99433817 1 2024-03-22T07:18:28.747 INFO:tasks.workunit.client.0.smithi081.stdout:3/226: chown db/l21 61 1 2024-03-22T07:18:28.751 INFO:tasks.workunit.client.0.smithi081.stdout:3/227: creat db/d38/f47 x:0 0 0 2024-03-22T07:18:28.754 INFO:tasks.workunit.client.0.smithi081.stdout:2/225: symlink da/db/d39/d47/d3b/d41/l49 0 2024-03-22T07:18:28.754 INFO:tasks.workunit.client.0.smithi081.stdout:1/236: read d6/d18/d1f/d32/f47 [3893368,95815] 0 2024-03-22T07:18:28.757 INFO:tasks.workunit.client.0.smithi081.stdout:2/226: chown da/db/f3f 257 1 2024-03-22T07:18:28.757 INFO:tasks.workunit.client.0.smithi081.stdout:4/397: dread da/d18/d1b/d1e/d37/d1c/f51 [0,4194304] 0 2024-03-22T07:18:28.760 INFO:tasks.workunit.client.0.smithi081.stdout:3/228: mknod db/d2e/c48 0 2024-03-22T07:18:28.760 INFO:tasks.workunit.client.0.smithi081.stdout:9/248: dwrite d2/d10/f29 [0,4194304] 0 2024-03-22T07:18:28.766 INFO:tasks.workunit.client.0.smithi081.stdout:7/452: write d9/d10/d16/d1e/d27/f5a [4361556,14238] 0 2024-03-22T07:18:28.767 INFO:tasks.workunit.client.0.smithi081.stdout:5/329: symlink d4/d10/d20/d25/d2e/l70 0 2024-03-22T07:18:28.768 INFO:tasks.workunit.client.0.smithi081.stdout:8/291: symlink d2/d3/d19/d1e/d13/d22/l69 0 2024-03-22T07:18:28.769 INFO:tasks.workunit.client.0.smithi081.stdout:2/227: mknod da/db/d39/d46/d42/c4a 0 2024-03-22T07:18:28.774 INFO:tasks.workunit.client.0.smithi081.stdout:9/249: link d2/d5/d16/f18 d2/d2b/d49/f4f 0 2024-03-22T07:18:28.775 INFO:tasks.workunit.client.0.smithi081.stdout:1/237: dwrite d6/d18/d1f/f25 [0,4194304] 0 2024-03-22T07:18:28.781 INFO:tasks.workunit.client.0.smithi081.stdout:5/330: unlink d4/d10/d20/d25/d2e/c5c 0 2024-03-22T07:18:28.783 INFO:tasks.workunit.client.0.smithi081.stdout:1/238: fdatasync f1 0 2024-03-22T07:18:28.786 INFO:tasks.workunit.client.0.smithi081.stdout:3/229: dwrite db/f2f [0,4194304] 0 2024-03-22T07:18:28.787 INFO:tasks.workunit.client.0.smithi081.stdout:2/228: symlink da/db/d39/d45/l4b 0 2024-03-22T07:18:28.789 INFO:tasks.workunit.client.0.smithi081.stdout:9/250: getdents d2/d2b/d49 0 2024-03-22T07:18:28.794 INFO:tasks.workunit.client.0.smithi081.stdout:4/398: mkdir da/d18/d42/d4f/d7c 0 2024-03-22T07:18:28.801 INFO:tasks.workunit.client.0.smithi081.stdout:2/229: chown da/c30 68787473 1 2024-03-22T07:18:28.803 INFO:tasks.workunit.client.0.smithi081.stdout:9/251: creat d2/d2b/f50 x:0 0 0 2024-03-22T07:18:28.805 INFO:tasks.workunit.client.0.smithi081.stdout:8/292: truncate d2/d3/d28/d2b/f67 804703 0 2024-03-22T07:18:28.811 INFO:tasks.workunit.client.0.smithi081.stdout:5/331: mknod d4/d10/d20/d1f/d2a/c71 0 2024-03-22T07:18:28.811 INFO:tasks.workunit.client.0.smithi081.stdout:2/230: dread da/d11/f1b [0,4194304] 0 2024-03-22T07:18:28.812 INFO:tasks.workunit.client.0.smithi081.stdout:2/231: creat da/db/d39/d45/f4c x:0 0 0 2024-03-22T07:18:28.813 INFO:tasks.workunit.client.0.smithi081.stdout:7/453: dwrite d9/dc/f4a [0,4194304] 0 2024-03-22T07:18:28.815 INFO:tasks.workunit.client.0.smithi081.stdout:2/232: creat da/d11/f4d x:0 0 0 2024-03-22T07:18:28.817 INFO:tasks.workunit.client.0.smithi081.stdout:1/239: write d6/d18/f27 [3026434,76470] 0 2024-03-22T07:18:28.820 INFO:tasks.workunit.client.0.smithi081.stdout:1/240: unlink d6/d18/d1f/d24/f31 0 2024-03-22T07:18:28.821 INFO:tasks.workunit.client.0.smithi081.stdout:3/230: symlink db/de/l49 0 2024-03-22T07:18:28.821 INFO:tasks.workunit.client.0.smithi081.stdout:9/252: write d2/d10/f29 [54402,23839] 0 2024-03-22T07:18:28.826 INFO:tasks.workunit.client.0.smithi081.stdout:2/233: fdatasync f5 0 2024-03-22T07:18:28.828 INFO:tasks.workunit.client.0.smithi081.stdout:5/332: mknod d4/d9/c72 0 2024-03-22T07:18:28.829 INFO:tasks.workunit.client.0.smithi081.stdout:9/253: fdatasync d2/d5/d16/f4a 0 2024-03-22T07:18:28.830 INFO:tasks.workunit.client.0.smithi081.stdout:7/454: mknod d9/ca3 0 2024-03-22T07:18:28.831 INFO:tasks.workunit.client.0.smithi081.stdout:5/333: chown d4/d10/d20/d25/f6e 31108145 1 2024-03-22T07:18:28.832 INFO:tasks.workunit.client.0.smithi081.stdout:3/231: symlink db/de/l4a 0 2024-03-22T07:18:28.838 INFO:tasks.workunit.client.0.smithi081.stdout:1/241: write d6/d18/f28 [1564832,10112] 0 2024-03-22T07:18:28.839 INFO:tasks.workunit.client.0.smithi081.stdout:9/254: symlink d2/d5/d36/l51 0 2024-03-22T07:18:28.839 INFO:tasks.workunit.client.0.smithi081.stdout:9/255: readlink d2/d5/d16/l1a 0 2024-03-22T07:18:28.844 INFO:tasks.workunit.client.0.smithi081.stdout:7/455: truncate d9/d10/d16/d1e/d6f/f94 332723 0 2024-03-22T07:18:28.851 INFO:tasks.workunit.client.0.smithi081.stdout:4/399: dwrite f3 [0,4194304] 0 2024-03-22T07:18:28.854 INFO:tasks.workunit.client.0.smithi081.stdout:5/334: dwrite d4/d10/f24 [0,4194304] 0 2024-03-22T07:18:28.855 INFO:tasks.workunit.client.0.smithi081.stdout:4/400: chown da/d3d/f5e 6977771 1 2024-03-22T07:18:28.857 INFO:tasks.workunit.client.0.smithi081.stdout:1/242: read d6/d18/d41/d49/d56/d30/f3d [2948794,7883] 0 2024-03-22T07:18:28.857 INFO:tasks.workunit.client.0.smithi081.stdout:7/456: dread f2 [0,4194304] 0 2024-03-22T07:18:28.858 INFO:tasks.workunit.client.0.smithi081.stdout:1/243: rename d6/d18 to d6/d18/d41/d57 22 2024-03-22T07:18:28.858 INFO:tasks.workunit.client.0.smithi081.stdout:1/244: stat d6/d18/d41/d49/d56/d30/f55 0 2024-03-22T07:18:28.864 INFO:tasks.workunit.client.0.smithi081.stdout:5/335: mkdir d4/df/d73 0 2024-03-22T07:18:28.867 INFO:tasks.workunit.client.0.smithi081.stdout:4/401: mknod da/d18/d1b/d26/c7d 0 2024-03-22T07:18:28.870 INFO:tasks.workunit.client.0.smithi081.stdout:5/336: mknod d4/d10/d20/d1f/d2a/c74 0 2024-03-22T07:18:28.873 INFO:tasks.workunit.client.0.smithi081.stdout:4/402: symlink da/d18/d42/l7e 0 2024-03-22T07:18:28.873 INFO:tasks.workunit.client.0.smithi081.stdout:1/245: fdatasync d6/d18/d1f/f37 0 2024-03-22T07:18:28.875 INFO:tasks.workunit.client.0.smithi081.stdout:5/337: mkdir d4/d75 0 2024-03-22T07:18:28.876 INFO:tasks.workunit.client.0.smithi081.stdout:1/246: symlink d6/d18/d41/d49/l58 0 2024-03-22T07:18:28.888 INFO:tasks.workunit.client.0.smithi081.stdout:4/403: dread da/d18/d1b/d1e/f62 [0,4194304] 0 2024-03-22T07:18:28.892 INFO:tasks.workunit.client.0.smithi081.stdout:5/338: read d4/f33 [1149370,130760] 0 2024-03-22T07:18:28.892 INFO:tasks.workunit.client.0.smithi081.stdout:2/234: dwrite da/d11/d21/f2d [0,4194304] 0 2024-03-22T07:18:28.895 INFO:tasks.workunit.client.0.smithi081.stdout:1/247: creat d6/d18/d1f/d32/f59 x:0 0 0 2024-03-22T07:18:28.895 INFO:tasks.workunit.client.0.smithi081.stdout:2/235: dread - da/f28 zero size 2024-03-22T07:18:28.896 INFO:tasks.workunit.client.0.smithi081.stdout:1/248: write d6/d18/d41/d49/d56/f44 [4248633,109971] 0 2024-03-22T07:18:28.897 INFO:tasks.workunit.client.0.smithi081.stdout:2/236: creat da/d11/d21/f4e x:0 0 0 2024-03-22T07:18:28.897 INFO:tasks.workunit.client.0.smithi081.stdout:9/256: dwrite d2/f1e [0,4194304] 0 2024-03-22T07:18:28.899 INFO:tasks.workunit.client.0.smithi081.stdout:3/232: dwrite db/de/f24 [4194304,4194304] 0 2024-03-22T07:18:28.900 INFO:tasks.workunit.client.0.smithi081.stdout:7/457: dwrite d9/d10/d16/d1e/f2c [4194304,4194304] 0 2024-03-22T07:18:28.904 INFO:tasks.workunit.client.0.smithi081.stdout:4/404: dread da/d18/d1b/d34/f64 [0,4194304] 0 2024-03-22T07:18:28.905 INFO:tasks.workunit.client.0.smithi081.stdout:2/237: dread da/f25 [0,4194304] 0 2024-03-22T07:18:28.905 INFO:tasks.workunit.client.0.smithi081.stdout:3/233: mkdir db/de/d4b 0 2024-03-22T07:18:28.910 INFO:tasks.workunit.client.0.smithi081.stdout:1/249: dread d6/f17 [0,4194304] 0 2024-03-22T07:18:28.911 INFO:tasks.workunit.client.0.smithi081.stdout:1/250: chown d6/d18/d1f/d24/d42 1 1 2024-03-22T07:18:28.911 INFO:tasks.workunit.client.0.smithi081.stdout:1/251: write f1 [2696504,65293] 0 2024-03-22T07:18:28.916 INFO:tasks.workunit.client.0.smithi081.stdout:2/238: symlink da/d11/d21/l4f 0 2024-03-22T07:18:28.918 INFO:tasks.workunit.client.0.smithi081.stdout:2/239: mkdir da/db/d39/d47/d3b/d41/d50 0 2024-03-22T07:18:28.918 INFO:tasks.workunit.client.0.smithi081.stdout:2/240: chown da 0 1 2024-03-22T07:18:28.919 INFO:tasks.workunit.client.0.smithi081.stdout:2/241: dread - da/db/d39/d46/f44 zero size 2024-03-22T07:18:28.919 INFO:tasks.workunit.client.0.smithi081.stdout:2/242: truncate da/d11/f4d 717888 0 2024-03-22T07:18:28.919 INFO:tasks.workunit.client.0.smithi081.stdout:2/243: dread - da/db/d39/d45/f4c zero size 2024-03-22T07:18:28.919 INFO:tasks.workunit.client.0.smithi081.stdout:2/244: creat da/db/d39/d47/d3b/d41/f51 x:0 0 0 2024-03-22T07:18:28.920 INFO:tasks.workunit.client.0.smithi081.stdout:3/234: write db/de/f20 [521337,79748] 0 2024-03-22T07:18:28.920 INFO:tasks.workunit.client.0.smithi081.stdout:5/339: dwrite d4/f33 [4194304,4194304] 0 2024-03-22T07:18:28.921 INFO:tasks.workunit.client.0.smithi081.stdout:1/252: write d6/d18/d1f/d24/f3b [845187,54497] 0 2024-03-22T07:18:28.922 INFO:tasks.workunit.client.0.smithi081.stdout:7/458: creat d9/d10/d4b/d61/fa4 x:0 0 0 2024-03-22T07:18:28.925 INFO:tasks.workunit.client.0.smithi081.stdout:2/245: symlink da/db/d39/d45/l52 0 2024-03-22T07:18:28.927 INFO:tasks.workunit.client.0.smithi081.stdout:7/459: stat d9/d3f/d52/d99 0 2024-03-22T07:18:28.928 INFO:tasks.workunit.client.0.smithi081.stdout:3/235: write db/f12 [1522568,13274] 0 2024-03-22T07:18:28.931 INFO:tasks.workunit.client.0.smithi081.stdout:3/236: creat db/de/f4c x:0 0 0 2024-03-22T07:18:28.932 INFO:tasks.workunit.client.0.smithi081.stdout:5/340: chown d4/df/d22/d4d/l55 506 1 2024-03-22T07:18:28.932 INFO:tasks.workunit.client.0.smithi081.stdout:7/460: chown d9/d10/d16/d1e/d27/d70/c72 263330751 1 2024-03-22T07:18:28.932 INFO:tasks.workunit.client.0.smithi081.stdout:4/405: write da/df/f5a [661558,25488] 0 2024-03-22T07:18:28.935 INFO:tasks.workunit.client.0.smithi081.stdout:4/406: dread - da/d18/d1b/d1e/f71 zero size 2024-03-22T07:18:28.937 INFO:tasks.workunit.client.0.smithi081.stdout:2/246: write da/db/d39/d47/f3d [31648,87941] 0 2024-03-22T07:18:28.938 INFO:tasks.workunit.client.0.smithi081.stdout:4/407: chown da/d57 2513627 1 2024-03-22T07:18:28.939 INFO:tasks.workunit.client.0.smithi081.stdout:1/253: dwrite d6/d18/d1f/d32/f4b [0,4194304] 0 2024-03-22T07:18:28.940 INFO:tasks.workunit.client.0.smithi081.stdout:5/341: unlink d4/d10/d20/d1f/d2a/l50 0 2024-03-22T07:18:28.944 INFO:tasks.workunit.client.0.smithi081.stdout:3/237: creat db/d23/f4d x:0 0 0 2024-03-22T07:18:28.946 INFO:tasks.workunit.client.0.smithi081.stdout:7/461: creat d9/d10/d16/d1e/d27/d6d/fa5 x:0 0 0 2024-03-22T07:18:28.946 INFO:tasks.workunit.client.0.smithi081.stdout:7/462: chown d9/d10/d16/d1e/d27/d70 0 1 2024-03-22T07:18:28.948 INFO:tasks.workunit.client.0.smithi081.stdout:4/408: write f0 [4280627,126337] 0 2024-03-22T07:18:28.951 INFO:tasks.workunit.client.0.smithi081.stdout:5/342: write d4/d10/f26 [4523588,88338] 0 2024-03-22T07:18:28.954 INFO:tasks.workunit.client.0.smithi081.stdout:5/343: mknod d4/d10/d20/d1f/c76 0 2024-03-22T07:18:28.955 INFO:tasks.workunit.client.0.smithi081.stdout:5/344: truncate d4/f3e 863799 0 2024-03-22T07:18:28.958 INFO:tasks.workunit.client.0.smithi081.stdout:3/238: dwrite db/d2e/f44 [0,4194304] 0 2024-03-22T07:18:28.960 INFO:tasks.workunit.client.0.smithi081.stdout:3/239: dread - db/d2e/f32 zero size 2024-03-22T07:18:28.962 INFO:tasks.workunit.client.0.smithi081.stdout:7/463: write d9/d10/d16/d1e/d27/f57 [1766054,69089] 0 2024-03-22T07:18:28.965 INFO:tasks.workunit.client.0.smithi081.stdout:7/464: chown d9/d14/f54 105995 1 2024-03-22T07:18:28.966 INFO:tasks.workunit.client.0.smithi081.stdout:5/345: dread d4/d10/d20/d1f/d2f/f37 [0,4194304] 0 2024-03-22T07:18:28.966 INFO:tasks.workunit.client.0.smithi081.stdout:4/409: dread f0 [0,4194304] 0 2024-03-22T07:18:28.969 INFO:tasks.workunit.client.0.smithi081.stdout:4/410: chown da/f1d 14090868 1 2024-03-22T07:18:28.971 INFO:tasks.workunit.client.0.smithi081.stdout:5/346: unlink d4/d10/l2d 0 2024-03-22T07:18:28.972 INFO:tasks.workunit.client.0.smithi081.stdout:4/411: rmdir da/d18/d1b/d1e 39 2024-03-22T07:18:28.973 INFO:tasks.workunit.client.0.smithi081.stdout:5/347: chown d4/df/d22/d49/l64 27434301 1 2024-03-22T07:18:28.978 INFO:tasks.workunit.client.0.smithi081.stdout:5/348: rename d4/d10/d20/d25/d2e/f6f to d4/d75/f77 0 2024-03-22T07:18:28.978 INFO:tasks.workunit.client.0.smithi081.stdout:5/349: fdatasync d4/f33 0 2024-03-22T07:18:28.978 INFO:tasks.workunit.client.0.smithi081.stdout:1/254: dwrite d6/fc [0,4194304] 0 2024-03-22T07:18:28.981 INFO:tasks.workunit.client.0.smithi081.stdout:7/465: write d9/dc/f1a [4547600,16951] 0 2024-03-22T07:18:28.981 INFO:tasks.workunit.client.0.smithi081.stdout:1/255: unlink d6/c45 0 2024-03-22T07:18:28.986 INFO:tasks.workunit.client.0.smithi081.stdout:4/412: creat da/df/d38/f7f x:0 0 0 2024-03-22T07:18:28.986 INFO:tasks.workunit.client.0.smithi081.stdout:4/413: readlink da/d18/l59 0 2024-03-22T07:18:28.987 INFO:tasks.workunit.client.0.smithi081.stdout:4/414: creat da/d18/d6e/f80 x:0 0 0 2024-03-22T07:18:28.987 INFO:tasks.workunit.client.0.smithi081.stdout:1/256: dread f1 [0,4194304] 0 2024-03-22T07:18:28.987 INFO:tasks.workunit.client.0.smithi081.stdout:5/350: dread d4/d10/f1d [0,4194304] 0 2024-03-22T07:18:28.989 INFO:tasks.workunit.client.0.smithi081.stdout:1/257: mknod d6/d18/d1f/d4c/c5a 0 2024-03-22T07:18:28.995 INFO:tasks.workunit.client.0.smithi081.stdout:5/351: creat d4/d10/d20/d4f/f78 x:0 0 0 2024-03-22T07:18:28.997 INFO:tasks.workunit.client.0.smithi081.stdout:5/352: link d4/d10/d20/c41 d4/d10/d20/d25/d2e/c79 0 2024-03-22T07:18:28.998 INFO:tasks.workunit.client.0.smithi081.stdout:5/353: fdatasync d4/df/f6d 0 2024-03-22T07:18:28.999 INFO:tasks.workunit.client.0.smithi081.stdout:5/354: stat d4/l5a 0 2024-03-22T07:18:29.000 INFO:tasks.workunit.client.0.smithi081.stdout:5/355: dread - d4/df/d22/f57 zero size 2024-03-22T07:18:29.006 INFO:tasks.workunit.client.0.smithi081.stdout:5/356: write d4/df/d22/d4d/f6c [424531,27296] 0 2024-03-22T07:18:29.006 INFO:tasks.workunit.client.0.smithi081.stdout:5/357: truncate d4/df/d22/d4d/f6c 1161180 0 2024-03-22T07:18:29.006 INFO:tasks.workunit.client.0.smithi081.stdout:1/258: read d6/d18/d1f/d24/f3f [714615,124295] 0 2024-03-22T07:18:29.008 INFO:tasks.workunit.client.0.smithi081.stdout:1/259: rmdir d6/d18/d1f/d24/d42 0 2024-03-22T07:18:29.009 INFO:tasks.workunit.client.0.smithi081.stdout:7/466: dwrite d9/f75 [0,4194304] 0 2024-03-22T07:18:29.009 INFO:tasks.workunit.client.0.smithi081.stdout:7/467: fdatasync d9/dc/f4a 0 2024-03-22T07:18:29.010 INFO:tasks.workunit.client.0.smithi081.stdout:5/358: mknod d4/df/d22/d4d/c7a 0 2024-03-22T07:18:29.011 INFO:tasks.workunit.client.0.smithi081.stdout:5/359: chown d4/d10 59202 1 2024-03-22T07:18:29.011 INFO:tasks.workunit.client.0.smithi081.stdout:5/360: fdatasync d4/df/d22/f57 0 2024-03-22T07:18:29.013 INFO:tasks.workunit.client.0.smithi081.stdout:1/260: dread d6/f17 [0,4194304] 0 2024-03-22T07:18:29.014 INFO:tasks.workunit.client.0.smithi081.stdout:5/361: mknod d4/df/d22/c7b 0 2024-03-22T07:18:29.014 INFO:tasks.workunit.client.0.smithi081.stdout:1/261: creat d6/f5b x:0 0 0 2024-03-22T07:18:29.016 INFO:tasks.workunit.client.0.smithi081.stdout:5/362: mkdir d4/df/d7c 0 2024-03-22T07:18:29.016 INFO:tasks.workunit.client.0.smithi081.stdout:5/363: fdatasync d4/df/f6d 0 2024-03-22T07:18:29.018 INFO:tasks.workunit.client.0.smithi081.stdout:5/364: creat d4/d10/d20/f7d x:0 0 0 2024-03-22T07:18:29.018 INFO:tasks.workunit.client.0.smithi081.stdout:1/262: mkdir d6/d18/d1f/d24/d48/d5c 0 2024-03-22T07:18:29.019 INFO:tasks.workunit.client.0.smithi081.stdout:1/263: fdatasync d6/d18/d41/d49/d56/d30/f3d 0 2024-03-22T07:18:29.019 INFO:tasks.workunit.client.0.smithi081.stdout:7/468: write d9/dc/f4a [4071274,45514] 0 2024-03-22T07:18:29.020 INFO:tasks.workunit.client.0.smithi081.stdout:4/415: dwrite da/d18/d1b/d34/f64 [0,4194304] 0 2024-03-22T07:18:29.022 INFO:tasks.workunit.client.0.smithi081.stdout:7/469: creat d9/d10/d16/d84/fa6 x:0 0 0 2024-03-22T07:18:29.026 INFO:tasks.workunit.client.0.smithi081.stdout:7/470: mknod d9/d10/d53/ca7 0 2024-03-22T07:18:29.027 INFO:tasks.workunit.client.0.smithi081.stdout:7/471: symlink d9/d10/d9e/la8 0 2024-03-22T07:18:29.030 INFO:tasks.workunit.client.0.smithi081.stdout:7/472: read f1 [627504,47487] 0 2024-03-22T07:18:29.032 INFO:tasks.workunit.client.0.smithi081.stdout:7/473: truncate d9/d3f/f79 980685 0 2024-03-22T07:18:29.032 INFO:tasks.workunit.client.0.smithi081.stdout:7/474: fsync d9/dc/f4a 0 2024-03-22T07:18:29.032 INFO:tasks.workunit.client.0.smithi081.stdout:7/475: getdents d9 0 2024-03-22T07:18:29.034 INFO:tasks.workunit.client.0.smithi081.stdout:7/476: rename d9/d10/d16/d1e/f28 to d9/d10/d2a/fa9 0 2024-03-22T07:18:29.044 INFO:tasks.workunit.client.0.smithi081.stdout:1/264: dwrite d6/d18/d1f/d24/f3b [0,4194304] 0 2024-03-22T07:18:29.044 INFO:tasks.workunit.client.0.smithi081.stdout:5/365: dwrite d4/df/d22/d49/f60 [0,4194304] 0 2024-03-22T07:18:29.045 INFO:tasks.workunit.client.0.smithi081.stdout:5/366: mknod d4/c7e 0 2024-03-22T07:18:29.046 INFO:tasks.workunit.client.0.smithi081.stdout:5/367: dread - d4/d10/d20/f7d zero size 2024-03-22T07:18:29.046 INFO:tasks.workunit.client.0.smithi081.stdout:1/265: rename d6 to d6/d18/d1f/d24/d5d 22 2024-03-22T07:18:29.061 INFO:tasks.workunit.client.0.smithi081.stdout:1/266: dread d6/d18/d41/d49/d56/f3c [0,4194304] 0 2024-03-22T07:18:29.079 INFO:tasks.workunit.client.0.smithi081.stdout:7/477: dwrite d9/f7c [0,4194304] 0 2024-03-22T07:18:29.079 INFO:tasks.workunit.client.0.smithi081.stdout:5/368: dwrite d4/d10/f21 [4194304,4194304] 0 2024-03-22T07:18:29.081 INFO:tasks.workunit.client.0.smithi081.stdout:5/369: readlink d4/d10/d20/d4f/l61 0 2024-03-22T07:18:29.085 INFO:tasks.workunit.client.0.smithi081.stdout:5/370: rename d4/df/d22/d4d/c52 to d4/d10/d20/c7f 0 2024-03-22T07:18:29.089 INFO:tasks.workunit.client.0.smithi081.stdout:7/478: creat d9/d10/d16/d1e/d27/d70/faa x:0 0 0 2024-03-22T07:18:29.090 INFO:tasks.workunit.client.0.smithi081.stdout:7/479: unlink d9/l20 0 2024-03-22T07:18:29.090 INFO:tasks.workunit.client.0.smithi081.stdout:7/480: chown f2 923 1 2024-03-22T07:18:29.093 INFO:tasks.workunit.client.0.smithi081.stdout:1/267: dwrite d6/d18/d41/d49/d56/f3c [0,4194304] 0 2024-03-22T07:18:29.096 INFO:tasks.workunit.client.0.smithi081.stdout:5/371: link d4/d10/c48 d4/d10/d20/d1f/d2f/d45/c80 0 2024-03-22T07:18:29.097 INFO:tasks.workunit.client.0.smithi081.stdout:5/372: rename d4/df/d22/d4d/l55 to d4/d10/d20/d1f/l81 0 2024-03-22T07:18:29.100 INFO:tasks.workunit.client.0.smithi081.stdout:5/373: unlink d4/d10/d20/d1f/d2a/c74 0 2024-03-22T07:18:29.100 INFO:tasks.workunit.client.0.smithi081.stdout:1/268: dread d6/d18/f27 [0,4194304] 0 2024-03-22T07:18:29.101 INFO:tasks.workunit.client.0.smithi081.stdout:5/374: mkdir d4/df/d22/d4d/d82 0 2024-03-22T07:18:29.101 INFO:tasks.workunit.client.0.smithi081.stdout:5/375: fdatasync d4/f3e 0 2024-03-22T07:18:29.103 INFO:tasks.workunit.client.0.smithi081.stdout:1/269: mknod d6/d18/d1f/c5e 0 2024-03-22T07:18:29.118 INFO:tasks.workunit.client.0.smithi081.stdout:5/376: write d4/d9/f2c [1351271,51300] 0 2024-03-22T07:18:29.125 INFO:tasks.workunit.client.0.smithi081.stdout:5/377: write d4/d10/d20/d25/d2e/f3b [728861,118765] 0 2024-03-22T07:18:29.125 INFO:tasks.workunit.client.0.smithi081.stdout:5/378: mknod d4/df/d22/d4d/d82/c83 0 2024-03-22T07:18:29.125 INFO:tasks.workunit.client.0.smithi081.stdout:5/379: read d4/d9/f2c [2408535,118223] 0 2024-03-22T07:18:29.126 INFO:tasks.workunit.client.0.smithi081.stdout:7/481: dwrite d9/d10/d53/f82 [0,4194304] 0 2024-03-22T07:18:29.127 INFO:tasks.workunit.client.0.smithi081.stdout:1/270: dwrite d6/d18/f1d [0,4194304] 0 2024-03-22T07:18:29.127 INFO:tasks.workunit.client.0.smithi081.stdout:1/271: dread - d6/d18/d41/d49/d56/d30/f55 zero size 2024-03-22T07:18:29.129 INFO:tasks.workunit.client.0.smithi081.stdout:1/272: creat d6/d18/d1f/d24/f5f x:0 0 0 2024-03-22T07:18:29.130 INFO:tasks.workunit.client.0.smithi081.stdout:7/482: dread - d9/d10/d53/d81/d59/f9f zero size 2024-03-22T07:18:29.131 INFO:tasks.workunit.client.0.smithi081.stdout:5/380: write d4/d75/f77 [129057,109957] 0 2024-03-22T07:18:29.135 INFO:tasks.workunit.client.0.smithi081.stdout:5/381: symlink d4/d10/d20/d1f/l84 0 2024-03-22T07:18:29.136 INFO:tasks.workunit.client.0.smithi081.stdout:5/382: rename d4/d10/d20/d1f/d2f/f37 to d4/df/d7c/f85 0 2024-03-22T07:18:29.144 INFO:tasks.workunit.client.0.smithi081.stdout:5/383: dread d4/d10/f21 [0,4194304] 0 2024-03-22T07:18:29.146 INFO:tasks.workunit.client.0.smithi081.stdout:5/384: fdatasync d4/d10/d20/d25/f43 0 2024-03-22T07:18:29.155 INFO:tasks.workunit.client.0.smithi081.stdout:5/385: write d4/d10/f26 [3409321,15303] 0 2024-03-22T07:18:29.155 INFO:tasks.workunit.client.0.smithi081.stdout:7/483: dwrite d9/d10/d16/d1e/f32 [0,4194304] 0 2024-03-22T07:18:29.155 INFO:tasks.workunit.client.0.smithi081.stdout:5/386: dread - d4/d10/d20/d25/d2e/f54 zero size 2024-03-22T07:18:29.160 INFO:tasks.workunit.client.0.smithi081.stdout:5/387: read d4/d10/f1d [1458902,94789] 0 2024-03-22T07:18:29.161 INFO:tasks.workunit.client.0.smithi081.stdout:5/388: symlink d4/d10/d20/l86 0 2024-03-22T07:18:29.163 INFO:tasks.workunit.client.0.smithi081.stdout:5/389: unlink f2 0 2024-03-22T07:18:29.164 INFO:tasks.workunit.client.0.smithi081.stdout:5/390: read d4/df/d22/d4d/f6c [1039556,86229] 0 2024-03-22T07:18:29.165 INFO:tasks.workunit.client.0.smithi081.stdout:5/391: chown d4/d10/f24 26 1 2024-03-22T07:18:29.165 INFO:tasks.workunit.client.0.smithi081.stdout:5/392: readlink d4/d9/l69 0 2024-03-22T07:18:29.166 INFO:tasks.workunit.client.0.smithi081.stdout:5/393: creat d4/d10/d20/d1f/d2f/d45/f87 x:0 0 0 2024-03-22T07:18:29.167 INFO:tasks.workunit.client.0.smithi081.stdout:5/394: rmdir d4/df/d22 39 2024-03-22T07:18:29.167 INFO:tasks.workunit.client.0.smithi081.stdout:5/395: chown d4/d10/d20/d1f/d2a/c66 2058529 1 2024-03-22T07:18:29.169 INFO:tasks.workunit.client.0.smithi081.stdout:5/396: creat d4/d10/d20/d1f/d2a/f88 x:0 0 0 2024-03-22T07:18:29.173 INFO:tasks.workunit.client.0.smithi081.stdout:7/484: dwrite d9/d10/d16/d1e/d27/d6d/fa5 [0,4194304] 0 2024-03-22T07:18:29.180 INFO:tasks.workunit.client.0.smithi081.stdout:7/485: symlink d9/d3f/d52/d83/lab 0 2024-03-22T07:18:29.185 INFO:tasks.workunit.client.0.smithi081.stdout:7/486: symlink d9/d3f/d52/d99/lac 0 2024-03-22T07:18:29.191 INFO:tasks.workunit.client.0.smithi081.stdout:5/397: dwrite d4/d10/d20/d1f/d2f/d45/f4c [4194304,4194304] 0 2024-03-22T07:18:29.193 INFO:tasks.workunit.client.0.smithi081.stdout:5/398: link d4/d9/f15 d4/f89 0 2024-03-22T07:18:29.197 INFO:tasks.workunit.client.0.smithi081.stdout:7/487: write d9/d10/d16/d1e/d27/d6d/fa5 [2307884,49724] 0 2024-03-22T07:18:29.197 INFO:tasks.workunit.client.0.smithi081.stdout:7/488: chown d9/d10/d16/d1e/d27/d6d/fa5 1701 1 2024-03-22T07:18:29.220 INFO:tasks.workunit.client.0.smithi081.stdout:5/399: dwrite d4/d10/d20/d1f/d2a/f88 [0,4194304] 0 2024-03-22T07:18:29.220 INFO:tasks.workunit.client.0.smithi081.stdout:5/400: write d4/df/d22/d49/f60 [4353199,126668] 0 2024-03-22T07:18:29.225 INFO:tasks.workunit.client.0.smithi081.stdout:5/401: write d4/d9/f1b [304268,128612] 0 2024-03-22T07:18:29.241 INFO:tasks.workunit.client.0.smithi081.stdout:5/402: dwrite d4/df/d22/f57 [0,4194304] 0 2024-03-22T07:18:29.243 INFO:tasks.workunit.client.0.smithi081.stdout:5/403: symlink d4/d10/d20/d1f/d2f/l8a 0 2024-03-22T07:18:29.247 INFO:tasks.workunit.client.0.smithi081.stdout:5/404: truncate d4/d10/d20/d25/d2e/f31 3991972 0 2024-03-22T07:18:29.249 INFO:tasks.workunit.client.0.smithi081.stdout:5/405: rmdir d4/d10/d20/d4f 39 2024-03-22T07:18:29.249 INFO:tasks.workunit.client.0.smithi081.stdout:5/406: chown d4/d10/f1d 120922444 1 2024-03-22T07:18:29.254 INFO:tasks.workunit.client.0.smithi081.stdout:5/407: rename d4/df/d22/d49/f60 to d4/d10/d20/d25/f8b 0 2024-03-22T07:18:29.258 INFO:tasks.workunit.client.0.smithi081.stdout:5/408: rename d4/d10/d20/d1f/d2f/d45 to d4/df/d73/d8c 0 2024-03-22T07:18:29.262 INFO:tasks.workunit.client.0.smithi081.stdout:5/409: dread f1 [0,4194304] 0 2024-03-22T07:18:29.286 INFO:tasks.workunit.client.0.smithi081.stdout:5/410: dwrite d4/d10/d20/f7d [0,4194304] 0 2024-03-22T07:18:29.287 INFO:tasks.workunit.client.0.smithi081.stdout:5/411: creat d4/d10/f8d x:0 0 0 2024-03-22T07:18:29.292 INFO:tasks.workunit.client.0.smithi081.stdout:5/412: write d4/d10/f21 [3858373,99901] 0 2024-03-22T07:18:29.293 INFO:tasks.workunit.client.0.smithi081.stdout:5/413: fsync d4/df/d22/f58 0 2024-03-22T07:18:29.293 INFO:tasks.workunit.client.0.smithi081.stdout:5/414: fsync d4/df/d22/f58 0 2024-03-22T07:18:29.307 INFO:tasks.workunit.client.0.smithi081.stdout:5/415: dwrite d4/d10/f26 [0,4194304] 0 2024-03-22T07:18:29.308 INFO:tasks.workunit.client.0.smithi081.stdout:5/416: mkdir d4/d10/d20/d1f/d2f/d8e 0 2024-03-22T07:18:29.309 INFO:tasks.workunit.client.0.smithi081.stdout:5/417: dread d4/d9/f15 [0,4194304] 0 2024-03-22T07:18:29.316 INFO:tasks.workunit.client.0.smithi081.stdout:5/418: dread f3 [0,4194304] 0 2024-03-22T07:18:29.317 INFO:tasks.workunit.client.0.smithi081.stdout:5/419: truncate d4/d9/f5b 1042466 0 2024-03-22T07:18:29.317 INFO:tasks.workunit.client.0.smithi081.stdout:5/420: fsync d4/d10/f26 0 2024-03-22T07:18:29.317 INFO:tasks.workunit.client.0.smithi081.stdout:5/421: chown d4/d10/f26 60969 1 2024-03-22T07:18:29.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:28 smithi081 ceph-mon[113965]: pgmap v210: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 255 B/s wr, 10 op/s 2024-03-22T07:18:29.337 INFO:tasks.workunit.client.0.smithi081.stdout:5/422: dwrite d4/d10/f3c [0,4194304] 0 2024-03-22T07:18:29.339 INFO:tasks.workunit.client.0.smithi081.stdout:5/423: link d4/df/d22/c7b d4/df/d73/d8c/c8f 0 2024-03-22T07:18:29.339 INFO:tasks.workunit.client.0.smithi081.stdout:5/424: chown d4/d10/d20/d4f/f78 6715541 1 2024-03-22T07:18:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:28 smithi149 ceph-mon[83447]: pgmap v210: 65 pgs: 65 active+clean; 882 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 255 B/s wr, 10 op/s 2024-03-22T07:18:29.357 INFO:tasks.workunit.client.0.smithi081.stdout:5/425: dwrite d4/d9/f2c [0,4194304] 0 2024-03-22T07:18:29.358 INFO:tasks.workunit.client.0.smithi081.stdout:5/426: mkdir d4/d9/d90 0 2024-03-22T07:18:29.361 INFO:tasks.workunit.client.0.smithi081.stdout:5/427: write d4/d10/d20/d25/f43 [271749,70070] 0 2024-03-22T07:18:29.361 INFO:tasks.workunit.client.0.smithi081.stdout:5/428: chown d4/df/d22/d49 1515544 1 2024-03-22T07:18:29.363 INFO:tasks.workunit.client.0.smithi081.stdout:5/429: write d4/d10/d20/d25/d2e/f3b [609077,50645] 0 2024-03-22T07:18:29.364 INFO:tasks.workunit.client.0.smithi081.stdout:5/430: link d4/d10/d20/d25/f43 d4/d9/f91 0 2024-03-22T07:18:29.364 INFO:tasks.workunit.client.0.smithi081.stdout:5/431: write d4/df/d22/d49/f4a [300607,13294] 0 2024-03-22T07:18:29.364 INFO:tasks.workunit.client.0.smithi081.stdout:5/432: fdatasync f3 0 2024-03-22T07:18:29.364 INFO:tasks.workunit.client.0.smithi081.stdout:5/433: chown d4/d10/d20 208 1 2024-03-22T07:18:29.366 INFO:tasks.workunit.client.0.smithi081.stdout:5/434: link d4/d10/d20/d25/d2e/f3b d4/d10/d20/d1f/d2f/f92 0 2024-03-22T07:18:29.368 INFO:tasks.workunit.client.0.smithi081.stdout:5/435: fdatasync d4/d10/d20/d25/f8b 0 2024-03-22T07:18:29.368 INFO:tasks.workunit.client.0.smithi081.stdout:5/436: read - d4/df/f6d zero size 2024-03-22T07:18:29.369 INFO:tasks.workunit.client.0.smithi081.stdout:5/437: write d4/d9/f91 [1173064,69484] 0 2024-03-22T07:18:29.369 INFO:tasks.workunit.client.0.smithi081.stdout:5/438: dread - d4/df/d22/f58 zero size 2024-03-22T07:18:29.369 INFO:tasks.workunit.client.0.smithi081.stdout:5/439: rmdir d4/d10/d20/d1f 39 2024-03-22T07:18:29.374 INFO:tasks.workunit.client.0.smithi081.stdout:5/440: write d4/df/f1a [3533408,130628] 0 2024-03-22T07:18:29.375 INFO:tasks.workunit.client.0.smithi081.stdout:5/441: getdents d4/df/d22/d4d/d82 0 2024-03-22T07:18:29.384 INFO:tasks.workunit.client.0.smithi081.stdout:5/442: dread d4/df/d22/f57 [0,4194304] 0 2024-03-22T07:18:29.384 INFO:tasks.workunit.client.0.smithi081.stdout:5/443: fsync d4/d9/f2c 0 2024-03-22T07:18:29.384 INFO:tasks.workunit.client.0.smithi081.stdout:5/444: truncate d4/d75/f77 738186 0 2024-03-22T07:18:29.384 INFO:tasks.workunit.client.0.smithi081.stdout:5/445: dread - d4/df/d73/d8c/f62 zero size 2024-03-22T07:18:29.401 INFO:tasks.workunit.client.0.smithi081.stdout:5/446: dwrite d4/df/d73/d8c/f87 [0,4194304] 0 2024-03-22T07:18:29.403 INFO:tasks.workunit.client.0.smithi081.stdout:5/447: mknod d4/df/d22/d4d/c93 0 2024-03-22T07:18:29.404 INFO:tasks.workunit.client.0.smithi081.stdout:5/448: symlink d4/df/d73/l94 0 2024-03-22T07:18:29.411 INFO:tasks.workunit.client.0.smithi081.stdout:5/449: read d4/d10/f21 [4607946,73706] 0 2024-03-22T07:18:29.411 INFO:tasks.workunit.client.0.smithi081.stdout:5/450: readlink d4/df/d73/l94 0 2024-03-22T07:18:29.416 INFO:tasks.workunit.client.0.smithi081.stdout:5/451: dread d4/f33 [0,4194304] 0 2024-03-22T07:18:29.434 INFO:tasks.workunit.client.0.smithi081.stdout:5/452: dwrite d4/f33 [0,4194304] 0 2024-03-22T07:18:29.435 INFO:tasks.workunit.client.0.smithi081.stdout:5/453: read d4/d9/f5b [398209,95614] 0 2024-03-22T07:18:29.456 INFO:tasks.workunit.client.0.smithi081.stdout:5/454: dwrite f3 [0,4194304] 0 2024-03-22T07:18:29.461 INFO:tasks.workunit.client.0.smithi081.stdout:5/455: dread d4/d75/f77 [0,4194304] 0 2024-03-22T07:18:29.461 INFO:tasks.workunit.client.0.smithi081.stdout:5/456: rmdir d4/df 39 2024-03-22T07:18:29.462 INFO:tasks.workunit.client.0.smithi081.stdout:5/457: creat d4/df/d7c/f95 x:0 0 0 2024-03-22T07:18:29.483 INFO:tasks.workunit.client.0.smithi081.stdout:5/458: dwrite d4/fe [4194304,4194304] 0 2024-03-22T07:18:29.484 INFO:tasks.workunit.client.0.smithi081.stdout:5/459: fdatasync d4/d10/f3c 0 2024-03-22T07:18:29.484 INFO:tasks.workunit.client.0.smithi081.stdout:5/460: mknod d4/d10/d20/d25/c96 0 2024-03-22T07:18:29.501 INFO:tasks.workunit.client.0.smithi081.stdout:5/461: dwrite d4/d10/f1d [0,4194304] 0 2024-03-22T07:18:29.501 INFO:tasks.workunit.client.0.smithi081.stdout:5/462: read d4/d10/d20/d25/f43 [35501,119557] 0 2024-03-22T07:18:29.501 INFO:tasks.workunit.client.0.smithi081.stdout:5/463: chown d4/df/d7c 359 1 2024-03-22T07:18:29.501 INFO:tasks.workunit.client.0.smithi081.stdout:5/464: fsync d4/df/f1a 0 2024-03-22T07:18:29.502 INFO:tasks.workunit.client.0.smithi081.stdout:5/465: getdents d4/d9/d90 0 2024-03-22T07:18:29.502 INFO:tasks.workunit.client.0.smithi081.stdout:5/466: symlink d4/d10/d20/d1f/d2f/l97 0 2024-03-22T07:18:29.503 INFO:tasks.workunit.client.0.smithi081.stdout:5/467: getdents d4/d75 0 2024-03-22T07:18:29.504 INFO:tasks.workunit.client.0.smithi081.stdout:5/468: mkdir d4/d75/d98 0 2024-03-22T07:18:29.518 INFO:tasks.workunit.client.0.smithi081.stdout:5/469: dwrite d4/d10/d20/d25/d2e/f3b [0,4194304] 0 2024-03-22T07:18:29.519 INFO:tasks.workunit.client.0.smithi081.stdout:5/470: creat d4/df/d22/d4d/f99 x:0 0 0 2024-03-22T07:18:29.519 INFO:tasks.workunit.client.0.smithi081.stdout:5/471: write d4/df/d7c/f95 [197753,26504] 0 2024-03-22T07:18:29.520 INFO:tasks.workunit.client.0.smithi081.stdout:5/472: mknod d4/df/d73/d8c/c9a 0 2024-03-22T07:18:29.527 INFO:tasks.workunit.client.0.smithi081.stdout:5/473: write d4/d10/d20/d1f/d2a/f88 [1597866,38725] 0 2024-03-22T07:18:29.545 INFO:tasks.workunit.client.0.smithi081.stdout:5/474: dwrite d4/d10/d20/d25/d2e/f3b [4194304,4194304] 0 2024-03-22T07:18:29.548 INFO:tasks.workunit.client.0.smithi081.stdout:5/475: write d4/df/d22/d4d/f63 [670750,30424] 0 2024-03-22T07:18:29.548 INFO:tasks.workunit.client.0.smithi081.stdout:5/476: chown d4/d9/f1b 57 1 2024-03-22T07:18:29.549 INFO:tasks.workunit.client.0.smithi081.stdout:5/477: write d4/df/d22/d4d/f63 [965388,130985] 0 2024-03-22T07:18:30.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:29 smithi081 ceph-mon[113965]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:30 smithi149 ceph-mon[83447]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:31 smithi081 ceph-mon[113965]: pgmap v211: 65 pgs: 65 active+clean; 887 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 440 KiB/s wr, 372 op/s 2024-03-22T07:18:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:31.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:31 smithi149 ceph-mon[83447]: pgmap v211: 65 pgs: 65 active+clean; 887 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 440 KiB/s wr, 372 op/s 2024-03-22T07:18:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:18:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:32.512 INFO:tasks.workunit.client.0.smithi081.stdout:0/393: sync 2024-03-22T07:18:32.661 INFO:tasks.workunit.client.0.smithi081.stdout:3/240: sync 2024-03-22T07:18:32.661 INFO:tasks.workunit.client.0.smithi081.stdout:4/416: sync 2024-03-22T07:18:32.662 INFO:tasks.workunit.client.0.smithi081.stdout:6/314: sync 2024-03-22T07:18:32.662 INFO:tasks.workunit.client.0.smithi081.stdout:8/293: sync 2024-03-22T07:18:32.662 INFO:tasks.workunit.client.0.smithi081.stdout:2/247: sync 2024-03-22T07:18:32.662 INFO:tasks.workunit.client.0.smithi081.stdout:9/257: sync 2024-03-22T07:18:32.687 INFO:tasks.workunit.client.0.smithi081.stdout:9/258: dwrite d2/d2b/f41 [0,4194304] 0 2024-03-22T07:18:32.688 INFO:tasks.workunit.client.0.smithi081.stdout:4/417: dwrite f0 [4194304,4194304] 0 2024-03-22T07:18:32.943 INFO:tasks.workunit.client.0.smithi081.stdout:2/248: creat da/d11/d21/f53 x:0 0 0 2024-03-22T07:18:32.982 INFO:tasks.workunit.client.0.smithi081.stdout:3/241: getdents db/de 0 2024-03-22T07:18:32.990 INFO:tasks.workunit.client.0.smithi081.stdout:4/418: symlink da/d18/d1b/d26/d32/l81 0 2024-03-22T07:18:33.002 INFO:tasks.workunit.client.0.smithi081.stdout:0/394: mknod d3/dc/c80 0 2024-03-22T07:18:33.006 INFO:tasks.workunit.client.0.smithi081.stdout:4/419: symlink da/d18/d42/d4f/d7c/l82 0 2024-03-22T07:18:33.028 INFO:tasks.workunit.client.0.smithi081.stdout:4/420: rename da/d18/d1b/d1e/f71 to da/d18/d42/f83 0 2024-03-22T07:18:33.033 INFO:tasks.workunit.client.0.smithi081.stdout:6/315: link d3/da/de/c18 d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/c65 0 2024-03-22T07:18:33.044 INFO:tasks.workunit.client.0.smithi081.stdout:4/421: dread f9 [0,4194304] 0 2024-03-22T07:18:33.046 INFO:tasks.workunit.client.0.smithi081.stdout:4/422: rmdir da/d3d/d7a 0 2024-03-22T07:18:33.055 INFO:tasks.workunit.client.0.smithi081.stdout:4/423: read da/d18/d1b/d34/f44 [1531850,87847] 0 2024-03-22T07:18:33.196 INFO:tasks.workunit.client.0.smithi081.stdout:2/249: rmdir da/d11/d21 39 2024-03-22T07:18:33.197 INFO:tasks.workunit.client.0.smithi081.stdout:2/250: creat da/db/d39/d47/d3b/d41/f54 x:0 0 0 2024-03-22T07:18:33.197 INFO:tasks.workunit.client.0.smithi081.stdout:2/251: fsync da/db/d39/d47/d3b/d41/f51 0 2024-03-22T07:18:33.201 INFO:tasks.workunit.client.0.smithi081.stdout:9/259: write d2/f20 [843840,16119] 0 2024-03-22T07:18:33.201 INFO:tasks.workunit.client.0.smithi081.stdout:8/294: symlink d2/d3/d19/d1e/d63/l6a 0 2024-03-22T07:18:33.205 INFO:tasks.workunit.client.0.smithi081.stdout:9/260: symlink d2/d10/l52 0 2024-03-22T07:18:33.206 INFO:tasks.workunit.client.0.smithi081.stdout:9/261: creat d2/d10/f53 x:0 0 0 2024-03-22T07:18:33.207 INFO:tasks.workunit.client.0.smithi081.stdout:0/395: symlink d3/dc/d33/d62/l81 0 2024-03-22T07:18:33.208 INFO:tasks.workunit.client.0.smithi081.stdout:9/262: symlink d2/l54 0 2024-03-22T07:18:33.210 INFO:tasks.workunit.client.0.smithi081.stdout:0/396: rmdir d3/dc/d33/d62 39 2024-03-22T07:18:33.213 INFO:tasks.workunit.client.0.smithi081.stdout:4/424: write da/d3d/f5e [648598,77298] 0 2024-03-22T07:18:33.214 INFO:tasks.workunit.client.0.smithi081.stdout:9/263: mknod d2/d2b/d49/c55 0 2024-03-22T07:18:33.220 INFO:tasks.workunit.client.0.smithi081.stdout:4/425: creat da/d18/d42/d4a/f84 x:0 0 0 2024-03-22T07:18:33.223 INFO:tasks.workunit.client.0.smithi081.stdout:4/426: mknod da/d18/d42/c85 0 2024-03-22T07:18:33.226 INFO:tasks.workunit.client.0.smithi081.stdout:0/397: read d3/f13 [561447,38239] 0 2024-03-22T07:18:33.228 INFO:tasks.workunit.client.0.smithi081.stdout:0/398: getdents d3/dc/d33 0 2024-03-22T07:18:33.231 INFO:tasks.workunit.client.0.smithi081.stdout:0/399: truncate d3/d4e/f54 445717 0 2024-03-22T07:18:33.232 INFO:tasks.workunit.client.0.smithi081.stdout:0/400: symlink d3/d6a/l82 0 2024-03-22T07:18:33.241 INFO:tasks.workunit.client.0.smithi081.stdout:9/264: write d2/d2b/f3b [4070903,23064] 0 2024-03-22T07:18:33.243 INFO:tasks.workunit.client.0.smithi081.stdout:9/265: mknod d2/d10/c56 0 2024-03-22T07:18:33.248 INFO:tasks.workunit.client.0.smithi081.stdout:0/401: dread d3/dc/d2b/d39/f3c [0,4194304] 0 2024-03-22T07:18:33.250 INFO:tasks.workunit.client.0.smithi081.stdout:6/316: dwrite d3/da/de/f27 [0,4194304] 0 2024-03-22T07:18:33.251 INFO:tasks.workunit.client.0.smithi081.stdout:3/242: dwrite db/de/f17 [0,4194304] 0 2024-03-22T07:18:33.251 INFO:tasks.workunit.client.0.smithi081.stdout:8/295: dwrite d2/d3/d23/f53 [0,4194304] 0 2024-03-22T07:18:33.251 INFO:tasks.workunit.client.0.smithi081.stdout:2/252: dwrite da/d11/d21/f33 [0,4194304] 0 2024-03-22T07:18:33.252 INFO:tasks.workunit.client.0.smithi081.stdout:2/253: dread da/d11/f1b [0,4194304] 0 2024-03-22T07:18:33.261 INFO:tasks.workunit.client.0.smithi081.stdout:3/243: dread db/f2f [0,4194304] 0 2024-03-22T07:18:33.309 INFO:tasks.workunit.client.0.smithi081.stdout:1/273: sync 2024-03-22T07:18:33.311 INFO:tasks.workunit.client.0.smithi081.stdout:7/489: sync 2024-03-22T07:18:33.311 INFO:tasks.workunit.client.0.smithi081.stdout:7/490: chown d9/d10/d9e 669 1 2024-03-22T07:18:33.311 INFO:tasks.workunit.client.0.smithi081.stdout:5/478: sync 2024-03-22T07:18:33.311 INFO:tasks.workunit.client.0.smithi081.stdout:5/479: read d4/d9/f1b [375661,28423] 0 2024-03-22T07:18:33.311 INFO:tasks.workunit.client.0.smithi081.stdout:4/427: dwrite da/d18/d42/d4f/f5b [0,4194304] 0 2024-03-22T07:18:33.312 INFO:tasks.workunit.client.0.smithi081.stdout:5/480: dread d4/f3e [0,4194304] 0 2024-03-22T07:18:33.312 INFO:tasks.workunit.client.0.smithi081.stdout:3/244: fdatasync db/fd 0 2024-03-22T07:18:33.315 INFO:tasks.workunit.client.0.smithi081.stdout:9/266: mkdir d2/d10/d30/d57 0 2024-03-22T07:18:33.318 INFO:tasks.workunit.client.0.smithi081.stdout:1/274: mkdir d6/d18/d60 0 2024-03-22T07:18:33.318 INFO:tasks.workunit.client.0.smithi081.stdout:1/275: stat d6/d18/d41/d49/d56/d30/f55 0 2024-03-22T07:18:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:33 smithi081 ceph-mon[113965]: pgmap v212: 65 pgs: 65 active+clean; 945 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 14 MiB/s rd, 7.5 MiB/s wr, 607 op/s 2024-03-22T07:18:33.321 INFO:tasks.workunit.client.0.smithi081.stdout:4/428: readlink da/d18/d1b/d1e/d37/l3a 0 2024-03-22T07:18:33.323 INFO:tasks.workunit.client.0.smithi081.stdout:2/254: write da/f13 [268200,118342] 0 2024-03-22T07:18:33.324 INFO:tasks.workunit.client.0.smithi081.stdout:7/491: readlink d9/d10/d16/d1e/d6f/l71 0 2024-03-22T07:18:33.326 INFO:tasks.workunit.client.0.smithi081.stdout:8/296: write d2/d3/f1b [3016402,82836] 0 2024-03-22T07:18:33.328 INFO:tasks.workunit.client.0.smithi081.stdout:2/255: truncate da/d11/f1b 640712 0 2024-03-22T07:18:33.329 INFO:tasks.workunit.client.0.smithi081.stdout:7/492: chown d9/d10/d16/d1e/f4e 22583597 1 2024-03-22T07:18:33.329 INFO:tasks.workunit.client.0.smithi081.stdout:1/276: dread d6/f12 [0,4194304] 0 2024-03-22T07:18:33.330 INFO:tasks.workunit.client.0.smithi081.stdout:4/429: getdents da/d18/d42/d4a 0 2024-03-22T07:18:33.331 INFO:tasks.workunit.client.0.smithi081.stdout:1/277: unlink d6/d18/c1e 0 2024-03-22T07:18:33.335 INFO:tasks.workunit.client.0.smithi081.stdout:6/317: dwrite d3/da/de/f41 [0,4194304] 0 2024-03-22T07:18:33.335 INFO:tasks.workunit.client.0.smithi081.stdout:4/430: mknod da/d18/d42/d4a/c86 0 2024-03-22T07:18:33.336 INFO:tasks.workunit.client.0.smithi081.stdout:5/481: dread d4/df/f46 [0,4194304] 0 2024-03-22T07:18:33.337 INFO:tasks.workunit.client.0.smithi081.stdout:7/493: fsync d9/d10/d2a/d73/f8b 0 2024-03-22T07:18:33.340 INFO:tasks.workunit.client.0.smithi081.stdout:9/267: write d2/d2b/f2d [1796749,28381] 0 2024-03-22T07:18:33.340 INFO:tasks.workunit.client.0.smithi081.stdout:9/268: read d2/d2b/f3b [394633,112360] 0 2024-03-22T07:18:33.342 INFO:tasks.workunit.client.0.smithi081.stdout:4/431: rename da/d18/d6e/f80 to da/f87 0 2024-03-22T07:18:33.344 INFO:tasks.workunit.client.0.smithi081.stdout:1/278: write d6/d18/d1f/d32/f4b [1987918,5989] 0 2024-03-22T07:18:33.344 INFO:tasks.workunit.client.0.smithi081.stdout:2/256: creat da/db/d39/d47/d3b/f55 x:0 0 0 2024-03-22T07:18:33.344 INFO:tasks.workunit.client.0.smithi081.stdout:6/318: dread d3/da/de/d31/d40/d50/d63/f1e [0,4194304] 0 2024-03-22T07:18:33.347 INFO:tasks.workunit.client.0.smithi081.stdout:2/257: dread - da/db/d39/d47/f43 zero size 2024-03-22T07:18:33.349 INFO:tasks.workunit.client.0.smithi081.stdout:3/245: dwrite db/fd [8388608,4194304] 0 2024-03-22T07:18:33.351 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:33 smithi149 ceph-mon[83447]: pgmap v212: 65 pgs: 65 active+clean; 945 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 14 MiB/s rd, 7.5 MiB/s wr, 607 op/s 2024-03-22T07:18:33.355 INFO:tasks.workunit.client.0.smithi081.stdout:1/279: read d6/f35 [107267,8698] 0 2024-03-22T07:18:33.357 INFO:tasks.workunit.client.0.smithi081.stdout:2/258: fdatasync da/db/f35 0 2024-03-22T07:18:33.359 INFO:tasks.workunit.client.0.smithi081.stdout:1/280: truncate f1 983162 0 2024-03-22T07:18:33.362 INFO:tasks.workunit.client.0.smithi081.stdout:4/432: rename da/d18/d1b/l6b to da/d18/d1b/d1e/d5f/l88 0 2024-03-22T07:18:33.363 INFO:tasks.workunit.client.0.smithi081.stdout:2/259: dread - da/db/d39/d47/d3b/d41/f51 zero size 2024-03-22T07:18:33.363 INFO:tasks.workunit.client.0.smithi081.stdout:2/260: truncate da/db/d39/d47/d3b/f55 951872 0 2024-03-22T07:18:33.364 INFO:tasks.workunit.client.0.smithi081.stdout:1/281: creat d6/d18/d41/d49/d56/d30/f61 x:0 0 0 2024-03-22T07:18:33.364 INFO:tasks.workunit.client.0.smithi081.stdout:1/282: fsync d6/fd 0 2024-03-22T07:18:33.366 INFO:tasks.workunit.client.0.smithi081.stdout:2/261: mkdir da/d56 0 2024-03-22T07:18:33.366 INFO:tasks.workunit.client.0.smithi081.stdout:3/246: read db/d2e/f44 [3067877,49981] 0 2024-03-22T07:18:33.369 INFO:tasks.workunit.client.0.smithi081.stdout:3/247: rmdir db/de 39 2024-03-22T07:18:33.371 INFO:tasks.workunit.client.0.smithi081.stdout:2/262: unlink da/f22 0 2024-03-22T07:18:33.373 INFO:tasks.workunit.client.0.smithi081.stdout:2/263: getdents da/db/d39/d47 0 2024-03-22T07:18:33.374 INFO:tasks.workunit.client.0.smithi081.stdout:3/248: creat db/de/f4e x:0 0 0 2024-03-22T07:18:33.376 INFO:tasks.workunit.client.0.smithi081.stdout:3/249: dread - db/d38/f47 zero size 2024-03-22T07:18:33.378 INFO:tasks.workunit.client.0.smithi081.stdout:3/250: creat db/f4f x:0 0 0 2024-03-22T07:18:33.380 INFO:tasks.workunit.client.0.smithi081.stdout:3/251: truncate db/d2e/f44 426659 0 2024-03-22T07:18:33.380 INFO:tasks.workunit.client.0.smithi081.stdout:3/252: write db/d2e/d3a/f46 [696651,52694] 0 2024-03-22T07:18:33.384 INFO:tasks.workunit.client.0.smithi081.stdout:9/269: dwrite d2/f1e [0,4194304] 0 2024-03-22T07:18:33.386 INFO:tasks.workunit.client.0.smithi081.stdout:9/270: unlink d2/d5/c23 0 2024-03-22T07:18:33.388 INFO:tasks.workunit.client.0.smithi081.stdout:9/271: mkdir d2/d10/d4b/d58 0 2024-03-22T07:18:33.388 INFO:tasks.workunit.client.0.smithi081.stdout:9/272: stat d2/d2b/d49 0 2024-03-22T07:18:33.388 INFO:tasks.workunit.client.0.smithi081.stdout:9/273: chown d2/d5/d16/f46 209619 1 2024-03-22T07:18:33.391 INFO:tasks.workunit.client.0.smithi081.stdout:8/297: dwrite d2/d3/d19/d1e/d13/d22/d2f/f40 [0,4194304] 0 2024-03-22T07:18:33.394 INFO:tasks.workunit.client.0.smithi081.stdout:8/298: creat d2/d3/d28/d2b/d61/f6b x:0 0 0 2024-03-22T07:18:33.404 INFO:tasks.workunit.client.0.smithi081.stdout:6/319: dwrite d3/da/de/d31/d40/d50/d63/d26/f33 [4194304,4194304] 0 2024-03-22T07:18:33.404 INFO:tasks.workunit.client.0.smithi081.stdout:4/433: dwrite f3 [0,4194304] 0 2024-03-22T07:18:33.404 INFO:tasks.workunit.client.0.smithi081.stdout:3/253: dwrite db/de/f20 [0,4194304] 0 2024-03-22T07:18:33.404 INFO:tasks.workunit.client.0.smithi081.stdout:5/482: dwrite d4/d10/d20/d25/f6e [0,4194304] 0 2024-03-22T07:18:33.404 INFO:tasks.workunit.client.0.smithi081.stdout:5/483: chown d4/d10/d20/d25/d2e/f31 17835294 1 2024-03-22T07:18:33.413 INFO:tasks.workunit.client.0.smithi081.stdout:5/484: write d4/df/f46 [49370,50917] 0 2024-03-22T07:18:33.418 INFO:tasks.workunit.client.0.smithi081.stdout:5/485: creat d4/d10/d20/d1f/d2a/f9b x:0 0 0 2024-03-22T07:18:33.419 INFO:tasks.workunit.client.0.smithi081.stdout:3/254: mknod db/d38/c50 0 2024-03-22T07:18:33.420 INFO:tasks.workunit.client.0.smithi081.stdout:3/255: chown db/d2e/d3a 73 1 2024-03-22T07:18:33.421 INFO:tasks.workunit.client.0.smithi081.stdout:4/434: symlink da/d18/d42/d4f/l89 0 2024-03-22T07:18:33.422 INFO:tasks.workunit.client.0.smithi081.stdout:9/274: dwrite d2/d2b/d49/f4f [4194304,4194304] 0 2024-03-22T07:18:33.424 INFO:tasks.workunit.client.0.smithi081.stdout:9/275: stat d2/d10/l19 0 2024-03-22T07:18:33.426 INFO:tasks.workunit.client.0.smithi081.stdout:3/256: link db/de/f17 db/d38/f51 0 2024-03-22T07:18:33.427 INFO:tasks.workunit.client.0.smithi081.stdout:1/283: rename d6/c38 to d6/d18/d60/c62 0 2024-03-22T07:18:33.429 INFO:tasks.workunit.client.0.smithi081.stdout:8/299: write d2/d3/d19/d1e/d13/d22/f3a [334934,35980] 0 2024-03-22T07:18:33.430 INFO:tasks.workunit.client.0.smithi081.stdout:9/276: creat d2/d5/d16/f59 x:0 0 0 2024-03-22T07:18:33.430 INFO:tasks.workunit.client.0.smithi081.stdout:1/284: rename d6/d18/d1f/d32/c4d to d6/d18/d1f/d24/c63 0 2024-03-22T07:18:33.435 INFO:tasks.workunit.client.0.smithi081.stdout:6/320: dwrite d3/da/de/f27 [0,4194304] 0 2024-03-22T07:18:33.435 INFO:tasks.workunit.client.0.smithi081.stdout:8/300: creat d2/d3/d28/f6c x:0 0 0 2024-03-22T07:18:33.436 INFO:tasks.workunit.client.0.smithi081.stdout:3/257: creat db/d23/f52 x:0 0 0 2024-03-22T07:18:33.436 INFO:tasks.workunit.client.0.smithi081.stdout:3/258: readlink db/de/l49 0 2024-03-22T07:18:33.438 INFO:tasks.workunit.client.0.smithi081.stdout:2/264: dwrite da/d11/f1f [4194304,4194304] 0 2024-03-22T07:18:33.440 INFO:tasks.workunit.client.0.smithi081.stdout:1/285: write d6/d18/d1f/d32/f47 [2419333,47914] 0 2024-03-22T07:18:33.440 INFO:tasks.workunit.client.0.smithi081.stdout:6/321: creat d3/da/de/d31/f66 x:0 0 0 2024-03-22T07:18:33.440 INFO:tasks.workunit.client.0.smithi081.stdout:8/301: read - d2/d3/d19/d1e/f5f zero size 2024-03-22T07:18:33.443 INFO:tasks.workunit.client.0.smithi081.stdout:9/277: write d2/d2b/d49/f4f [4912694,63242] 0 2024-03-22T07:18:33.443 INFO:tasks.workunit.client.0.smithi081.stdout:8/302: chown d2/d3/d19/d1e/f3e 8 1 2024-03-22T07:18:33.443 INFO:tasks.workunit.client.0.smithi081.stdout:4/435: write da/d18/d1b/d26/f66 [5223759,99487] 0 2024-03-22T07:18:33.444 INFO:tasks.workunit.client.0.smithi081.stdout:3/259: symlink db/d2e/d3a/l53 0 2024-03-22T07:18:33.445 INFO:tasks.workunit.client.0.smithi081.stdout:6/322: rmdir d3/da/de/d31/d40/d50 39 2024-03-22T07:18:33.450 INFO:tasks.workunit.client.0.smithi081.stdout:9/278: mkdir d2/d10/d4b/d5a 0 2024-03-22T07:18:33.450 INFO:tasks.workunit.client.0.smithi081.stdout:4/436: truncate da/d18/d1b/d26/d3e/f6f 508168 0 2024-03-22T07:18:33.451 INFO:tasks.workunit.client.0.smithi081.stdout:3/260: link db/de/f20 db/f54 0 2024-03-22T07:18:33.452 INFO:tasks.workunit.client.0.smithi081.stdout:9/279: chown d2/d2b/l38 607205 1 2024-03-22T07:18:33.452 INFO:tasks.workunit.client.0.smithi081.stdout:1/286: dread d6/fd [0,4194304] 0 2024-03-22T07:18:33.453 INFO:tasks.workunit.client.0.smithi081.stdout:3/261: chown f1 39 1 2024-03-22T07:18:33.455 INFO:tasks.workunit.client.0.smithi081.stdout:3/262: creat db/d2e/f55 x:0 0 0 2024-03-22T07:18:33.458 INFO:tasks.workunit.client.0.smithi081.stdout:9/280: unlink d2/d10/d30/l48 0 2024-03-22T07:18:33.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/323: dread d3/f6 [0,4194304] 0 2024-03-22T07:18:33.459 INFO:tasks.workunit.client.0.smithi081.stdout:2/265: dwrite da/d11/d21/f53 [0,4194304] 0 2024-03-22T07:18:33.464 INFO:tasks.workunit.client.0.smithi081.stdout:3/263: symlink db/d2e/l56 0 2024-03-22T07:18:33.466 INFO:tasks.workunit.client.0.smithi081.stdout:9/281: truncate d2/f11 611565 0 2024-03-22T07:18:33.466 INFO:tasks.workunit.client.0.smithi081.stdout:8/303: write d2/d3/d19/f25 [832235,22691] 0 2024-03-22T07:18:33.466 INFO:tasks.workunit.client.0.smithi081.stdout:8/304: chown d2/d3/d19/d1e/f3e 475 1 2024-03-22T07:18:33.467 INFO:tasks.workunit.client.0.smithi081.stdout:9/282: creat d2/f5b x:0 0 0 2024-03-22T07:18:33.469 INFO:tasks.workunit.client.0.smithi081.stdout:2/266: link da/db/d39/d47/d38/f40 da/db/d39/f57 0 2024-03-22T07:18:33.469 INFO:tasks.workunit.client.0.smithi081.stdout:2/267: read - da/db/d39/d47/f37 zero size 2024-03-22T07:18:33.469 INFO:tasks.workunit.client.0.smithi081.stdout:1/287: dwrite d6/d18/f1d [0,4194304] 0 2024-03-22T07:18:33.469 INFO:tasks.workunit.client.0.smithi081.stdout:2/268: getdents da/db/d39/d47/d38 0 2024-03-22T07:18:33.471 INFO:tasks.workunit.client.0.smithi081.stdout:2/269: creat da/d56/f58 x:0 0 0 2024-03-22T07:18:33.471 INFO:tasks.workunit.client.0.smithi081.stdout:2/270: readlink da/d11/d21/l4f 0 2024-03-22T07:18:33.471 INFO:tasks.workunit.client.0.smithi081.stdout:2/271: fdatasync da/db/d39/d47/f37 0 2024-03-22T07:18:33.474 INFO:tasks.workunit.client.0.smithi081.stdout:6/324: read d3/da/de/f28 [1289163,104524] 0 2024-03-22T07:18:33.475 INFO:tasks.workunit.client.0.smithi081.stdout:2/272: dread da/db/f27 [0,4194304] 0 2024-03-22T07:18:33.478 INFO:tasks.workunit.client.0.smithi081.stdout:8/305: dread d2/d3/f1b [0,4194304] 0 2024-03-22T07:18:33.478 INFO:tasks.workunit.client.0.smithi081.stdout:4/437: rename da/d18/d1b/d1e/d37/f36 to da/d18/d1b/d26/d3e/f8a 0 2024-03-22T07:18:33.479 INFO:tasks.workunit.client.0.smithi081.stdout:6/325: read - d3/da/de/f56 zero size 2024-03-22T07:18:33.484 INFO:tasks.workunit.client.0.smithi081.stdout:1/288: write d6/d18/d1f/d24/f39 [1780380,28187] 0 2024-03-22T07:18:33.492 INFO:tasks.workunit.client.0.smithi081.stdout:6/326: mkdir d3/d67 0 2024-03-22T07:18:33.494 INFO:tasks.workunit.client.0.smithi081.stdout:4/438: rename da/d18/d1b/d26/d3e/c56 to da/d18/c8b 0 2024-03-22T07:18:33.496 INFO:tasks.workunit.client.0.smithi081.stdout:1/289: rename d6/l1c to d6/l64 0 2024-03-22T07:18:33.496 INFO:tasks.workunit.client.0.smithi081.stdout:4/439: write da/d3d/f5e [502365,18775] 0 2024-03-22T07:18:33.499 INFO:tasks.workunit.client.0.smithi081.stdout:9/283: rename d2/d10/f53 to d2/d2b/d49/f5c 0 2024-03-22T07:18:33.500 INFO:tasks.workunit.client.0.smithi081.stdout:6/327: creat d3/da/de/d31/d40/d50/d63/f68 x:0 0 0 2024-03-22T07:18:33.505 INFO:tasks.workunit.client.0.smithi081.stdout:9/284: rename d2/d10/d4b to d2/d5d 0 2024-03-22T07:18:33.507 INFO:tasks.workunit.client.0.smithi081.stdout:9/285: creat d2/d5/d16/f5e x:0 0 0 2024-03-22T07:18:33.507 INFO:tasks.workunit.client.0.smithi081.stdout:9/286: chown d2/f5b 11 1 2024-03-22T07:18:33.508 INFO:tasks.workunit.client.0.smithi081.stdout:6/328: creat d3/da/de/d31/d40/d50/d63/d26/d52/f69 x:0 0 0 2024-03-22T07:18:33.509 INFO:tasks.workunit.client.0.smithi081.stdout:9/287: creat d2/f5f x:0 0 0 2024-03-22T07:18:33.509 INFO:tasks.workunit.client.0.smithi081.stdout:9/288: readlink d2/d10/l19 0 2024-03-22T07:18:33.513 INFO:tasks.workunit.client.0.smithi081.stdout:8/306: dwrite d2/d3/d19/d1e/d13/f60 [0,4194304] 0 2024-03-22T07:18:33.514 INFO:tasks.workunit.client.0.smithi081.stdout:8/307: chown d2/d3/d28/f6c 68520 1 2024-03-22T07:18:33.514 INFO:tasks.workunit.client.0.smithi081.stdout:3/264: dwrite db/fc [0,4194304] 0 2024-03-22T07:18:33.516 INFO:tasks.workunit.client.0.smithi081.stdout:8/308: read - d2/d3/d28/f49 zero size 2024-03-22T07:18:33.518 INFO:tasks.workunit.client.0.smithi081.stdout:6/329: dread d3/da/de/f41 [0,4194304] 0 2024-03-22T07:18:33.518 INFO:tasks.workunit.client.0.smithi081.stdout:6/330: chown d3 3 1 2024-03-22T07:18:33.520 INFO:tasks.workunit.client.0.smithi081.stdout:3/265: creat db/f57 x:0 0 0 2024-03-22T07:18:33.520 INFO:tasks.workunit.client.0.smithi081.stdout:3/266: fdatasync db/de/f26 0 2024-03-22T07:18:33.521 INFO:tasks.workunit.client.0.smithi081.stdout:8/309: getdents d2/d3/d23 0 2024-03-22T07:18:33.523 INFO:tasks.workunit.client.0.smithi081.stdout:6/331: symlink d3/d67/l6a 0 2024-03-22T07:18:33.526 INFO:tasks.workunit.client.0.smithi081.stdout:8/310: chown d2/f5e 755119 1 2024-03-22T07:18:33.529 INFO:tasks.workunit.client.0.smithi081.stdout:3/267: symlink db/de/l58 0 2024-03-22T07:18:33.531 INFO:tasks.workunit.client.0.smithi081.stdout:8/311: mknod d2/d3/d19/d1e/d13/d22/c6d 0 2024-03-22T07:18:33.532 INFO:tasks.workunit.client.0.smithi081.stdout:3/268: rename db/d2e/d3a/f43 to db/d2e/d3a/f59 0 2024-03-22T07:18:33.534 INFO:tasks.workunit.client.0.smithi081.stdout:3/269: fsync db/fd 0 2024-03-22T07:18:33.535 INFO:tasks.workunit.client.0.smithi081.stdout:8/312: truncate d2/d3/d19/d1e/d63/f65 147960 0 2024-03-22T07:18:33.539 INFO:tasks.workunit.client.0.smithi081.stdout:4/440: write da/d18/d1b/d26/d3e/f8a [3298082,52244] 0 2024-03-22T07:18:33.540 INFO:tasks.workunit.client.0.smithi081.stdout:3/270: link db/de/l49 db/d2e/l5a 0 2024-03-22T07:18:33.547 INFO:tasks.workunit.client.0.smithi081.stdout:9/289: dwrite d2/d5/d16/f2f [0,4194304] 0 2024-03-22T07:18:33.549 INFO:tasks.workunit.client.0.smithi081.stdout:8/313: write d2/d3/d19/d1e/d13/d22/f34 [1517287,14303] 0 2024-03-22T07:18:33.549 INFO:tasks.workunit.client.0.smithi081.stdout:9/290: chown d2/f4 414204 1 2024-03-22T07:18:33.554 INFO:tasks.workunit.client.0.smithi081.stdout:9/291: link d2/fe d2/d2b/f60 0 2024-03-22T07:18:33.557 INFO:tasks.workunit.client.0.smithi081.stdout:9/292: link d2/d5/d36/l51 d2/d5/d36/l61 0 2024-03-22T07:18:33.558 INFO:tasks.workunit.client.0.smithi081.stdout:9/293: creat d2/d10/d30/d57/f62 x:0 0 0 2024-03-22T07:18:33.558 INFO:tasks.workunit.client.0.smithi081.stdout:9/294: chown d2/f5f 50510821 1 2024-03-22T07:18:33.558 INFO:tasks.workunit.client.0.smithi081.stdout:9/295: readlink d2/d5/d36/l51 0 2024-03-22T07:18:33.565 INFO:tasks.workunit.client.0.smithi081.stdout:9/296: dread d2/d2b/d49/f4f [4194304,4194304] 0 2024-03-22T07:18:33.566 INFO:tasks.workunit.client.0.smithi081.stdout:6/332: dwrite d3/da/f2a [0,4194304] 0 2024-03-22T07:18:33.566 INFO:tasks.workunit.client.0.smithi081.stdout:6/333: fdatasync d3/da/de/f28 0 2024-03-22T07:18:33.566 INFO:tasks.workunit.client.0.smithi081.stdout:3/271: dwrite db/fd [8388608,4194304] 0 2024-03-22T07:18:33.566 INFO:tasks.workunit.client.0.smithi081.stdout:9/297: creat d2/d5d/d5a/f63 x:0 0 0 2024-03-22T07:18:33.566 INFO:tasks.workunit.client.0.smithi081.stdout:4/441: write da/df/d38/f54 [547691,80893] 0 2024-03-22T07:18:33.573 INFO:tasks.workunit.client.0.smithi081.stdout:3/272: rename db/f57 to db/d2e/f5b 0 2024-03-22T07:18:33.573 INFO:tasks.workunit.client.0.smithi081.stdout:9/298: symlink d2/d2b/l64 0 2024-03-22T07:18:33.574 INFO:tasks.workunit.client.0.smithi081.stdout:3/273: fsync db/f13 0 2024-03-22T07:18:33.578 INFO:tasks.workunit.client.0.smithi081.stdout:4/442: mknod da/d18/d42/d4a/c8c 0 2024-03-22T07:18:33.582 INFO:tasks.workunit.client.0.smithi081.stdout:4/443: mknod da/d18/d42/d4f/d7c/c8d 0 2024-03-22T07:18:33.583 INFO:tasks.workunit.client.0.smithi081.stdout:4/444: dread - da/d18/d1b/d26/f73 zero size 2024-03-22T07:18:33.585 INFO:tasks.workunit.client.0.smithi081.stdout:8/314: dwrite d2/d3/d19/d1e/f43 [0,4194304] 0 2024-03-22T07:18:33.585 INFO:tasks.workunit.client.0.smithi081.stdout:3/274: rmdir db 39 2024-03-22T07:18:33.587 INFO:tasks.workunit.client.0.smithi081.stdout:6/334: rename d3/da/de/f2e to d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f6b 0 2024-03-22T07:18:33.589 INFO:tasks.workunit.client.0.smithi081.stdout:8/315: creat d2/d3/d19/f6e x:0 0 0 2024-03-22T07:18:33.590 INFO:tasks.workunit.client.0.smithi081.stdout:3/275: rmdir db/d38 39 2024-03-22T07:18:33.592 INFO:tasks.workunit.client.0.smithi081.stdout:8/316: readlink d2/d3/d19/d1e/d13/d22/d2f/l31 0 2024-03-22T07:18:33.592 INFO:tasks.workunit.client.0.smithi081.stdout:8/317: dread - d2/d3/d28/d2b/d61/f6b zero size 2024-03-22T07:18:33.592 INFO:tasks.workunit.client.0.smithi081.stdout:8/318: chown d2/d3/d19/d1e/d13/d22/l32 1452 1 2024-03-22T07:18:33.594 INFO:tasks.workunit.client.0.smithi081.stdout:9/299: dwrite d2/d5/f3f [4194304,4194304] 0 2024-03-22T07:18:33.604 INFO:tasks.workunit.client.0.smithi081.stdout:3/276: dread db/de/f17 [0,4194304] 0 2024-03-22T07:18:33.606 INFO:tasks.workunit.client.0.smithi081.stdout:6/335: dread d3/da/de/d31/d40/d50/d63/d26/f44 [0,4194304] 0 2024-03-22T07:18:33.611 INFO:tasks.workunit.client.0.smithi081.stdout:8/319: write f1 [7110021,76826] 0 2024-03-22T07:18:33.612 INFO:tasks.workunit.client.0.smithi081.stdout:8/320: fsync d2/d3/d19/d1e/f56 0 2024-03-22T07:18:33.612 INFO:tasks.workunit.client.0.smithi081.stdout:8/321: write d2/d3/d19/d1e/d13/d22/f3a [300551,31591] 0 2024-03-22T07:18:33.614 INFO:tasks.workunit.client.0.smithi081.stdout:9/300: dwrite d2/d2b/f41 [0,4194304] 0 2024-03-22T07:18:33.615 INFO:tasks.workunit.client.0.smithi081.stdout:9/301: mkdir d2/d5/d65 0 2024-03-22T07:18:33.622 INFO:tasks.workunit.client.0.smithi081.stdout:9/302: write d2/d2b/f41 [891980,68359] 0 2024-03-22T07:18:33.625 INFO:tasks.workunit.client.0.smithi081.stdout:8/322: dread d2/d3/d19/d1e/f5a [0,4194304] 0 2024-03-22T07:18:33.631 INFO:tasks.workunit.client.0.smithi081.stdout:8/323: write d2/d3/d28/d2b/f46 [388513,44464] 0 2024-03-22T07:18:33.632 INFO:tasks.workunit.client.0.smithi081.stdout:8/324: creat d2/d3/d19/d1e/d13/d22/f6f x:0 0 0 2024-03-22T07:18:33.639 INFO:tasks.workunit.client.0.smithi081.stdout:8/325: dread d2/d3/d19/d1e/d13/f60 [0,4194304] 0 2024-03-22T07:18:33.640 INFO:tasks.workunit.client.0.smithi081.stdout:8/326: mkdir d2/d3/d19/d1e/d13/d66/d70 0 2024-03-22T07:18:33.642 INFO:tasks.workunit.client.0.smithi081.stdout:9/303: dwrite d2/d10/d30/d57/f62 [0,4194304] 0 2024-03-22T07:18:33.645 INFO:tasks.workunit.client.0.smithi081.stdout:9/304: mkdir d2/d10/d30/d57/d66 0 2024-03-22T07:18:33.645 INFO:tasks.workunit.client.0.smithi081.stdout:9/305: chown d2 1430184 1 2024-03-22T07:18:33.647 INFO:tasks.workunit.client.0.smithi081.stdout:8/327: mkdir d2/d3/d19/d1e/d13/d22/d2f/d71 0 2024-03-22T07:18:33.649 INFO:tasks.workunit.client.0.smithi081.stdout:8/328: truncate d2/d3/d19/d1e/d63/f65 762786 0 2024-03-22T07:18:33.649 INFO:tasks.workunit.client.0.smithi081.stdout:8/329: readlink d2/d3/d19/d1e/l27 0 2024-03-22T07:18:33.651 INFO:tasks.workunit.client.0.smithi081.stdout:8/330: mknod d2/d3/d19/d1e/d13/d22/d2f/c72 0 2024-03-22T07:18:33.652 INFO:tasks.workunit.client.0.smithi081.stdout:8/331: mkdir d2/d3/d19/d1e/d13/d22/d2f/d73 0 2024-03-22T07:18:33.654 INFO:tasks.workunit.client.0.smithi081.stdout:9/306: read d2/d5/d16/f2e [789650,37779] 0 2024-03-22T07:18:33.655 INFO:tasks.workunit.client.0.smithi081.stdout:9/307: symlink d2/d10/d30/d57/l67 0 2024-03-22T07:18:33.656 INFO:tasks.workunit.client.0.smithi081.stdout:9/308: chown d2/d5/c3a 240520594 1 2024-03-22T07:18:33.656 INFO:tasks.workunit.client.0.smithi081.stdout:9/309: chown d2/f4 85 1 2024-03-22T07:18:33.656 INFO:tasks.workunit.client.0.smithi081.stdout:9/310: chown d2/d2b/d49 5 1 2024-03-22T07:18:33.657 INFO:tasks.workunit.client.0.smithi081.stdout:9/311: chown d2/d2b/c32 561775548 1 2024-03-22T07:18:33.660 INFO:tasks.workunit.client.0.smithi081.stdout:8/332: write d2/d3/d19/d1e/d13/f39 [1083909,9575] 0 2024-03-22T07:18:33.665 INFO:tasks.workunit.client.0.smithi081.stdout:9/312: write d2/f15 [356626,25887] 0 2024-03-22T07:18:33.667 INFO:tasks.workunit.client.0.smithi081.stdout:9/313: rmdir d2/d10/d30 39 2024-03-22T07:18:33.687 INFO:tasks.workunit.client.0.smithi081.stdout:8/333: dwrite d2/d3/d28/d2b/f58 [0,4194304] 0 2024-03-22T07:18:33.690 INFO:tasks.workunit.client.0.smithi081.stdout:8/334: creat d2/d3/d28/d2b/d61/f74 x:0 0 0 2024-03-22T07:18:33.690 INFO:tasks.workunit.client.0.smithi081.stdout:8/335: chown d2/d3/d19/c1a 262045815 1 2024-03-22T07:18:33.691 INFO:tasks.workunit.client.0.smithi081.stdout:8/336: unlink d2/d3/d19/d1e/d13/l59 0 2024-03-22T07:18:33.691 INFO:tasks.workunit.client.0.smithi081.stdout:8/337: symlink d2/d3/d19/d1e/d13/d66/d70/l75 0 2024-03-22T07:18:33.701 INFO:tasks.workunit.client.0.smithi081.stdout:6/336: dwrite d3/da/de/d31/d40/d50/d63/d26/f1d [0,4194304] 0 2024-03-22T07:18:33.706 INFO:tasks.workunit.client.0.smithi081.stdout:9/314: dwrite d2/d2b/f3b [0,4194304] 0 2024-03-22T07:18:33.709 INFO:tasks.workunit.client.0.smithi081.stdout:9/315: rename d2/d5/d16/f5e to d2/d5/f68 0 2024-03-22T07:18:33.709 INFO:tasks.workunit.client.0.smithi081.stdout:9/316: getdents d2/d10/d30/d57/d66 0 2024-03-22T07:18:33.709 INFO:tasks.workunit.client.0.smithi081.stdout:9/317: chown d2/d5/c3a 566 1 2024-03-22T07:18:33.709 INFO:tasks.workunit.client.0.smithi081.stdout:9/318: dread - d2/d2b/f50 zero size 2024-03-22T07:18:33.709 INFO:tasks.workunit.client.0.smithi081.stdout:9/319: fsync d2/d5/d16/f2e 0 2024-03-22T07:18:33.710 INFO:tasks.workunit.client.0.smithi081.stdout:6/337: write d3/da/de/f41 [1256008,107327] 0 2024-03-22T07:18:33.711 INFO:tasks.workunit.client.0.smithi081.stdout:9/320: creat d2/d10/d30/d57/d66/f69 x:0 0 0 2024-03-22T07:18:33.714 INFO:tasks.workunit.client.0.smithi081.stdout:8/338: dwrite d2/d3/f1f [0,4194304] 0 2024-03-22T07:18:33.723 INFO:tasks.workunit.client.0.smithi081.stdout:8/339: link d2/d3/f4 d2/d3/d28/f76 0 2024-03-22T07:18:33.724 INFO:tasks.workunit.client.0.smithi081.stdout:8/340: rmdir d2/d3/d19/d1e/d13/d66 39 2024-03-22T07:18:33.724 INFO:tasks.workunit.client.0.smithi081.stdout:8/341: write d2/d3/d28/d2b/f46 [1192969,96091] 0 2024-03-22T07:18:33.726 INFO:tasks.workunit.client.0.smithi081.stdout:8/342: creat d2/d3/d19/d1e/d13/d22/d2f/f77 x:0 0 0 2024-03-22T07:18:33.734 INFO:tasks.workunit.client.0.smithi081.stdout:9/321: dwrite d2/d10/f35 [0,4194304] 0 2024-03-22T07:18:33.736 INFO:tasks.workunit.client.0.smithi081.stdout:9/322: chown d2/d5/d16/f18 295090190 1 2024-03-22T07:18:33.741 INFO:tasks.workunit.client.0.smithi081.stdout:8/343: mkdir d2/d3/d19/d1e/d13/d22/d38/d78 0 2024-03-22T07:18:33.742 INFO:tasks.workunit.client.0.smithi081.stdout:9/323: rmdir d2 39 2024-03-22T07:18:33.742 INFO:tasks.workunit.client.0.smithi081.stdout:8/344: dread - d2/d3/d19/d1e/f5c zero size 2024-03-22T07:18:33.743 INFO:tasks.workunit.client.0.smithi081.stdout:9/324: rmdir d2/d2b 39 2024-03-22T07:18:33.743 INFO:tasks.workunit.client.0.smithi081.stdout:8/345: mknod d2/d3/d19/d1e/d13/d22/d38/d78/c79 0 2024-03-22T07:18:33.757 INFO:tasks.workunit.client.0.smithi081.stdout:9/325: write d2/d2b/f26 [1146645,32515] 0 2024-03-22T07:18:33.757 INFO:tasks.workunit.client.0.smithi081.stdout:9/326: chown d2/d2b/d49/f4f 4191 1 2024-03-22T07:18:33.757 INFO:tasks.workunit.client.0.smithi081.stdout:9/327: write d2/d2b/d49/f5c [992428,45557] 0 2024-03-22T07:18:33.758 INFO:tasks.workunit.client.0.smithi081.stdout:9/328: mknod d2/d2b/c6a 0 2024-03-22T07:18:33.759 INFO:tasks.workunit.client.0.smithi081.stdout:9/329: rename d2/d5/cb to d2/d5d/c6b 0 2024-03-22T07:18:33.760 INFO:tasks.workunit.client.0.smithi081.stdout:8/346: dwrite d2/d3/d19/d1e/f5f [0,4194304] 0 2024-03-22T07:18:33.760 INFO:tasks.workunit.client.0.smithi081.stdout:9/330: write d2/d2b/d49/f5c [828453,89725] 0 2024-03-22T07:18:33.763 INFO:tasks.workunit.client.0.smithi081.stdout:9/331: unlink d2/d2b/c32 0 2024-03-22T07:18:33.764 INFO:tasks.workunit.client.0.smithi081.stdout:9/332: creat d2/d5/d36/f6c x:0 0 0 2024-03-22T07:18:33.764 INFO:tasks.workunit.client.0.smithi081.stdout:8/347: dread d2/d3/d28/d2b/f46 [0,4194304] 0 2024-03-22T07:18:33.774 INFO:tasks.workunit.client.0.smithi081.stdout:8/348: dread d2/d3/d28/f76 [0,4194304] 0 2024-03-22T07:18:33.776 INFO:tasks.workunit.client.0.smithi081.stdout:8/349: creat d2/d3/d28/d2b/f7a x:0 0 0 2024-03-22T07:18:33.777 INFO:tasks.workunit.client.0.smithi081.stdout:8/350: mknod d2/d3/d28/d2b/d61/c7b 0 2024-03-22T07:18:33.786 INFO:tasks.workunit.client.0.smithi081.stdout:9/333: dwrite d2/d5/d16/d37/f44 [0,4194304] 0 2024-03-22T07:18:33.788 INFO:tasks.workunit.client.0.smithi081.stdout:9/334: truncate d2/d5/f7 1546095 0 2024-03-22T07:18:33.788 INFO:tasks.workunit.client.0.smithi081.stdout:9/335: chown d2/d2b/f2d 0 1 2024-03-22T07:18:33.789 INFO:tasks.workunit.client.0.smithi081.stdout:9/336: chown d2/f15 0 1 2024-03-22T07:18:33.791 INFO:tasks.workunit.client.0.smithi081.stdout:8/351: write d2/d3/d19/d1e/d13/f47 [1545829,92114] 0 2024-03-22T07:18:33.792 INFO:tasks.workunit.client.0.smithi081.stdout:8/352: chown d2/d3/d19/d1e/d13/d22/d2f 1497 1 2024-03-22T07:18:33.792 INFO:tasks.workunit.client.0.smithi081.stdout:9/337: dread d2/f11 [0,4194304] 0 2024-03-22T07:18:33.792 INFO:tasks.workunit.client.0.smithi081.stdout:8/353: mkdir d2/d3/d19/d1e/d13/d22/d2f/d71/d7c 0 2024-03-22T07:18:33.798 INFO:tasks.workunit.client.0.smithi081.stdout:9/338: dread d2/d2b/d49/f4f [4194304,4194304] 0 2024-03-22T07:18:33.800 INFO:tasks.workunit.client.0.smithi081.stdout:9/339: truncate d2/d2b/f43 998492 0 2024-03-22T07:18:33.801 INFO:tasks.workunit.client.0.smithi081.stdout:9/340: write d2/f20 [2511970,73871] 0 2024-03-22T07:18:33.801 INFO:tasks.workunit.client.0.smithi081.stdout:8/354: write d2/d3/d19/d1e/f5a [2497385,63128] 0 2024-03-22T07:18:33.802 INFO:tasks.workunit.client.0.smithi081.stdout:9/341: rename d2/d5/f7 to d2/d5/d16/f6d 0 2024-03-22T07:18:33.803 INFO:tasks.workunit.client.0.smithi081.stdout:9/342: dread - d2/d5/d16/f4a zero size 2024-03-22T07:18:33.807 INFO:tasks.workunit.client.0.smithi081.stdout:9/343: rename d2/l42 to d2/d10/d30/l6e 0 2024-03-22T07:18:33.811 INFO:tasks.workunit.client.0.smithi081.stdout:9/344: link d2/d10/l2c d2/d5d/l6f 0 2024-03-22T07:18:33.811 INFO:tasks.workunit.client.0.smithi081.stdout:9/345: creat d2/d2b/f70 x:0 0 0 2024-03-22T07:18:33.811 INFO:tasks.workunit.client.0.smithi081.stdout:9/346: readlink d2/d10/l19 0 2024-03-22T07:18:33.812 INFO:tasks.workunit.client.0.smithi081.stdout:9/347: chown d2/d5 206560 1 2024-03-22T07:18:33.814 INFO:tasks.workunit.client.0.smithi081.stdout:9/348: rename d2/d5/d16/l1c to d2/d5/d16/l71 0 2024-03-22T07:18:33.814 INFO:tasks.workunit.client.0.smithi081.stdout:9/349: mknod d2/d5d/d5a/c72 0 2024-03-22T07:18:33.817 INFO:tasks.workunit.client.0.smithi081.stdout:8/355: write d2/d3/d19/f48 [1211895,7020] 0 2024-03-22T07:18:33.819 INFO:tasks.workunit.client.0.smithi081.stdout:8/356: symlink d2/d3/d19/d1e/d63/l7d 0 2024-03-22T07:18:33.819 INFO:tasks.workunit.client.0.smithi081.stdout:8/357: write d2/d3/d19/d1e/d13/d22/d2f/f77 [529003,61700] 0 2024-03-22T07:18:33.830 INFO:tasks.workunit.client.0.smithi081.stdout:8/358: dread d2/d3/d23/f52 [0,4194304] 0 2024-03-22T07:18:33.831 INFO:tasks.workunit.client.0.smithi081.stdout:9/350: dwrite d2/d2b/f2d [4194304,4194304] 0 2024-03-22T07:18:33.831 INFO:tasks.workunit.client.0.smithi081.stdout:8/359: read d2/f15 [984323,29332] 0 2024-03-22T07:18:33.845 INFO:tasks.workunit.client.0.smithi081.stdout:9/351: write d2/d5/fd [5359039,83564] 0 2024-03-22T07:18:33.846 INFO:tasks.workunit.client.0.smithi081.stdout:9/352: truncate d2/d10/d30/f45 560445 0 2024-03-22T07:18:33.847 INFO:tasks.workunit.client.0.smithi081.stdout:9/353: mkdir d2/d73 0 2024-03-22T07:18:33.847 INFO:tasks.workunit.client.0.smithi081.stdout:9/354: chown d2/d5/d16/f18 1426 1 2024-03-22T07:18:33.847 INFO:tasks.workunit.client.0.smithi081.stdout:9/355: read - d2/d10/d30/d57/d66/f69 zero size 2024-03-22T07:18:33.848 INFO:tasks.workunit.client.0.smithi081.stdout:8/360: dwrite d2/d3/d19/d1e/f3c [0,4194304] 0 2024-03-22T07:18:33.848 INFO:tasks.workunit.client.0.smithi081.stdout:8/361: dread - d2/d3/d19/d1e/d13/d22/f6f zero size 2024-03-22T07:18:33.858 INFO:tasks.workunit.client.0.smithi081.stdout:9/356: dread d2/f4 [0,4194304] 0 2024-03-22T07:18:33.860 INFO:tasks.workunit.client.0.smithi081.stdout:9/357: mkdir d2/d73/d74 0 2024-03-22T07:18:33.860 INFO:tasks.workunit.client.0.smithi081.stdout:9/358: write d2/d5/f68 [976255,89639] 0 2024-03-22T07:18:33.860 INFO:tasks.workunit.client.0.smithi081.stdout:8/362: read d2/d3/d19/d1e/d13/d22/d2f/f40 [2359627,56180] 0 2024-03-22T07:18:33.861 INFO:tasks.workunit.client.0.smithi081.stdout:8/363: truncate d2/d3/d28/d2b/f67 525369 0 2024-03-22T07:18:33.863 INFO:tasks.workunit.client.0.smithi081.stdout:8/364: unlink d2/d3/d19/d1e/d13/d22/d2f/c5d 0 2024-03-22T07:18:33.870 INFO:tasks.workunit.client.0.smithi081.stdout:8/365: truncate d2/d3/d19/d1e/d13/f47 6096165 0 2024-03-22T07:18:33.878 INFO:tasks.workunit.client.0.smithi081.stdout:9/359: dwrite d2/f15 [0,4194304] 0 2024-03-22T07:18:33.878 INFO:tasks.workunit.client.0.smithi081.stdout:9/360: truncate d2/d10/d30/d57/d66/f69 389761 0 2024-03-22T07:18:33.879 INFO:tasks.workunit.client.0.smithi081.stdout:9/361: chown d2/fe 321 1 2024-03-22T07:18:33.879 INFO:tasks.workunit.client.0.smithi081.stdout:9/362: write d2/d2b/d49/f4f [1701957,1572] 0 2024-03-22T07:18:33.879 INFO:tasks.workunit.client.0.smithi081.stdout:8/366: fsync d2/d3/d19/d1e/d13/f47 0 2024-03-22T07:18:33.883 INFO:tasks.workunit.client.0.smithi081.stdout:8/367: mkdir d2/d3/d28/d2b/d61/d7e 0 2024-03-22T07:18:33.887 INFO:tasks.workunit.client.0.smithi081.stdout:8/368: creat d2/d3/d23/f7f x:0 0 0 2024-03-22T07:18:33.887 INFO:tasks.workunit.client.0.smithi081.stdout:8/369: truncate d2/d3/d19/d1e/f5a 4613080 0 2024-03-22T07:18:33.888 INFO:tasks.workunit.client.0.smithi081.stdout:8/370: chown d2/d3/d19/d1e/d13/d22/l57 5 1 2024-03-22T07:18:33.889 INFO:tasks.workunit.client.0.smithi081.stdout:8/371: creat d2/d3/d19/d1e/d63/f80 x:0 0 0 2024-03-22T07:18:33.894 INFO:tasks.workunit.client.0.smithi081.stdout:8/372: creat d2/d3/d19/d1e/d13/d22/d2f/d71/d7c/f81 x:0 0 0 2024-03-22T07:18:33.895 INFO:tasks.workunit.client.0.smithi081.stdout:8/373: dread - d2/d3/d28/d2b/f7a zero size 2024-03-22T07:18:33.895 INFO:tasks.workunit.client.0.smithi081.stdout:8/374: creat d2/d3/d28/f82 x:0 0 0 2024-03-22T07:18:33.895 INFO:tasks.workunit.client.0.smithi081.stdout:8/375: mknod d2/d3/d19/d1e/d13/d22/d2f/d73/c83 0 2024-03-22T07:18:33.895 INFO:tasks.workunit.client.0.smithi081.stdout:8/376: chown d2/d3/d28/d2b/l62 15461 1 2024-03-22T07:18:33.895 INFO:tasks.workunit.client.0.smithi081.stdout:8/377: symlink d2/d3/d19/d1e/d13/d22/d38/d78/l84 0 2024-03-22T07:18:33.895 INFO:tasks.workunit.client.0.smithi081.stdout:8/378: dread - d2/d3/d28/d2b/f2e zero size 2024-03-22T07:18:33.909 INFO:tasks.workunit.client.0.smithi081.stdout:9/363: dwrite d2/d5/d16/f6d [0,4194304] 0 2024-03-22T07:18:33.911 INFO:tasks.workunit.client.0.smithi081.stdout:9/364: mkdir d2/d10/d30/d57/d66/d75 0 2024-03-22T07:18:33.934 INFO:tasks.workunit.client.0.smithi081.stdout:9/365: dwrite d2/d10/f14 [4194304,4194304] 0 2024-03-22T07:18:33.980 INFO:tasks.workunit.client.0.smithi081.stdout:9/366: dwrite d2/d5/d16/f2e [0,4194304] 0 2024-03-22T07:18:33.989 INFO:tasks.workunit.client.0.smithi081.stdout:9/367: write d2/d10/d30/d57/f62 [628828,37521] 0 2024-03-22T07:18:33.989 INFO:tasks.workunit.client.0.smithi081.stdout:9/368: chown d2/d2b/c4d 977 1 2024-03-22T07:18:33.989 INFO:tasks.workunit.client.0.smithi081.stdout:9/369: fdatasync d2/d2b/f70 0 2024-03-22T07:18:33.990 INFO:tasks.workunit.client.0.smithi081.stdout:9/370: creat d2/d10/d30/d57/f76 x:0 0 0 2024-03-22T07:18:33.991 INFO:tasks.workunit.client.0.smithi081.stdout:9/371: truncate d2/d10/d30/d57/d66/f69 1288594 0 2024-03-22T07:18:34.376 INFO:tasks.workunit.client.0.smithi081.stdout:7/494: sync 2024-03-22T07:18:34.377 INFO:tasks.workunit.client.0.smithi081.stdout:4/445: sync 2024-03-22T07:18:34.377 INFO:tasks.workunit.client.0.smithi081.stdout:1/290: sync 2024-03-22T07:18:34.377 INFO:tasks.workunit.client.0.smithi081.stdout:5/486: sync 2024-03-22T07:18:34.377 INFO:tasks.workunit.client.0.smithi081.stdout:2/273: sync 2024-03-22T07:18:34.377 INFO:tasks.workunit.client.0.smithi081.stdout:0/402: sync 2024-03-22T07:18:34.377 INFO:tasks.workunit.client.0.smithi081.stdout:3/277: sync 2024-03-22T07:18:34.381 INFO:tasks.workunit.client.0.smithi081.stdout:4/446: write da/f79 [888712,29651] 0 2024-03-22T07:18:34.386 INFO:tasks.workunit.client.0.smithi081.stdout:2/274: write da/d11/d21/f53 [799734,120779] 0 2024-03-22T07:18:34.386 INFO:tasks.workunit.client.0.smithi081.stdout:3/278: link db/d38/f51 db/f5c 0 2024-03-22T07:18:34.387 INFO:tasks.workunit.client.0.smithi081.stdout:5/487: write f1 [353676,37218] 0 2024-03-22T07:18:34.389 INFO:tasks.workunit.client.0.smithi081.stdout:0/403: rename d3/l27 to d3/dc/l83 0 2024-03-22T07:18:34.389 INFO:tasks.workunit.client.0.smithi081.stdout:2/275: fsync f6 0 2024-03-22T07:18:34.390 INFO:tasks.workunit.client.0.smithi081.stdout:2/276: dread - da/db/d39/d47/f37 zero size 2024-03-22T07:18:34.391 INFO:tasks.workunit.client.0.smithi081.stdout:0/404: dread - d3/dc/d2b/f49 zero size 2024-03-22T07:18:34.395 INFO:tasks.workunit.client.0.smithi081.stdout:4/447: dread da/d18/d1b/f40 [0,4194304] 0 2024-03-22T07:18:34.395 INFO:tasks.workunit.client.0.smithi081.stdout:2/277: link da/ce da/db/d39/d47/d3b/d41/c59 0 2024-03-22T07:18:34.396 INFO:tasks.workunit.client.0.smithi081.stdout:0/405: creat d3/d32/f84 x:0 0 0 2024-03-22T07:18:34.396 INFO:tasks.workunit.client.0.smithi081.stdout:4/448: dread - da/d18/d42/d4a/f84 zero size 2024-03-22T07:18:34.397 INFO:tasks.workunit.client.0.smithi081.stdout:2/278: rename da/db/d39/d47/d3b to da/db/d39/d47/d3b/d5a 22 2024-03-22T07:18:34.397 INFO:tasks.workunit.client.0.smithi081.stdout:2/279: getdents da/db/d39/d47 0 2024-03-22T07:18:34.402 INFO:tasks.workunit.client.0.smithi081.stdout:2/280: rename da/d11/d21/f2d to da/db/d39/d47/f5b 0 2024-03-22T07:18:34.403 INFO:tasks.workunit.client.0.smithi081.stdout:4/449: mknod da/d18/d1b/d34/c8e 0 2024-03-22T07:18:34.403 INFO:tasks.workunit.client.0.smithi081.stdout:4/450: dread - da/d18/d1b/d26/d3e/f55 zero size 2024-03-22T07:18:34.407 INFO:tasks.workunit.client.0.smithi081.stdout:1/291: dwrite d6/d18/f4f [0,4194304] 0 2024-03-22T07:18:34.409 INFO:tasks.workunit.client.0.smithi081.stdout:3/279: dwrite db/de/f4e [0,4194304] 0 2024-03-22T07:18:34.412 INFO:tasks.workunit.client.0.smithi081.stdout:0/406: dread d3/d32/f5f [0,4194304] 0 2024-03-22T07:18:34.413 INFO:tasks.workunit.client.0.smithi081.stdout:3/280: mkdir db/d2e/d3a/d5d 0 2024-03-22T07:18:34.414 INFO:tasks.workunit.client.0.smithi081.stdout:0/407: getdents d3/dc/d33/d62 0 2024-03-22T07:18:34.415 INFO:tasks.workunit.client.0.smithi081.stdout:5/488: dwrite d4/d10/d20/d25/d2e/f3b [4194304,4194304] 0 2024-03-22T07:18:34.415 INFO:tasks.workunit.client.0.smithi081.stdout:5/489: chown d4/df/d22/f58 5007645 1 2024-03-22T07:18:34.415 INFO:tasks.workunit.client.0.smithi081.stdout:5/490: dread - d4/df/f6d zero size 2024-03-22T07:18:34.419 INFO:tasks.workunit.client.0.smithi081.stdout:5/491: link d4/df/d22/f58 d4/d75/d98/f9c 0 2024-03-22T07:18:34.419 INFO:tasks.workunit.client.0.smithi081.stdout:5/492: fsync d4/f3e 0 2024-03-22T07:18:34.421 INFO:tasks.workunit.client.0.smithi081.stdout:3/281: symlink db/d2e/l5e 0 2024-03-22T07:18:34.422 INFO:tasks.workunit.client.0.smithi081.stdout:0/408: symlink d3/l85 0 2024-03-22T07:18:34.422 INFO:tasks.workunit.client.0.smithi081.stdout:4/451: write da/df/d38/f68 [7091,44298] 0 2024-03-22T07:18:34.423 INFO:tasks.workunit.client.0.smithi081.stdout:2/281: dwrite da/db/d39/d47/d3b/f55 [0,4194304] 0 2024-03-22T07:18:34.424 INFO:tasks.workunit.client.0.smithi081.stdout:5/493: creat d4/df/d73/f9d x:0 0 0 2024-03-22T07:18:34.429 INFO:tasks.workunit.client.0.smithi081.stdout:1/292: rmdir d6/d18 39 2024-03-22T07:18:34.430 INFO:tasks.workunit.client.0.smithi081.stdout:3/282: write db/f2f [49262,99675] 0 2024-03-22T07:18:34.431 INFO:tasks.workunit.client.0.smithi081.stdout:5/494: creat d4/d10/d20/d25/f9e x:0 0 0 2024-03-22T07:18:34.431 INFO:tasks.workunit.client.0.smithi081.stdout:1/293: dread d6/d18/d41/d49/d56/d30/f34 [0,4194304] 0 2024-03-22T07:18:34.432 INFO:tasks.workunit.client.0.smithi081.stdout:0/409: mknod d3/dc/d77/c86 0 2024-03-22T07:18:34.433 INFO:tasks.workunit.client.0.smithi081.stdout:1/294: symlink d6/d18/d1f/d24/l65 0 2024-03-22T07:18:34.433 INFO:tasks.workunit.client.0.smithi081.stdout:1/295: stat d6/d18/f1d 0 2024-03-22T07:18:34.433 INFO:tasks.workunit.client.0.smithi081.stdout:4/452: symlink da/df/l8f 0 2024-03-22T07:18:34.439 INFO:tasks.workunit.client.0.smithi081.stdout:4/453: truncate da/d18/d1b/d26/f43 633308 0 2024-03-22T07:18:34.440 INFO:tasks.workunit.client.0.smithi081.stdout:5/495: dread d4/df/d7c/f85 [0,4194304] 0 2024-03-22T07:18:34.441 INFO:tasks.workunit.client.0.smithi081.stdout:5/496: read d4/df/d22/d4d/f63 [789363,85907] 0 2024-03-22T07:18:34.442 INFO:tasks.workunit.client.0.smithi081.stdout:0/410: mknod d3/dc/d2b/d4d/c87 0 2024-03-22T07:18:34.444 INFO:tasks.workunit.client.0.smithi081.stdout:0/411: creat d3/d4e/f88 x:0 0 0 2024-03-22T07:18:34.444 INFO:tasks.workunit.client.0.smithi081.stdout:5/497: mknod d4/d10/d20/d4f/c9f 0 2024-03-22T07:18:34.446 INFO:tasks.workunit.client.0.smithi081.stdout:0/412: mknod d3/dc/c89 0 2024-03-22T07:18:34.449 INFO:tasks.workunit.client.0.smithi081.stdout:5/498: rename d4/d10/d20/l32 to d4/df/d73/d8c/la0 0 2024-03-22T07:18:34.450 INFO:tasks.workunit.client.0.smithi081.stdout:4/454: dread da/df/d38/f46 [0,4194304] 0 2024-03-22T07:18:34.451 INFO:tasks.workunit.client.0.smithi081.stdout:4/455: truncate da/f7b 736360 0 2024-03-22T07:18:34.451 INFO:tasks.workunit.client.0.smithi081.stdout:0/413: readlink d3/dc/d40/d69/l6e 0 2024-03-22T07:18:34.452 INFO:tasks.workunit.client.0.smithi081.stdout:5/499: rename d4/df/d73/d8c/c80 to d4/d10/ca1 0 2024-03-22T07:18:34.454 INFO:tasks.workunit.client.0.smithi081.stdout:4/456: mkdir da/d18/d42/d4f/d7c/d90 0 2024-03-22T07:18:34.456 INFO:tasks.workunit.client.0.smithi081.stdout:5/500: creat d4/d10/d20/d1f/fa2 x:0 0 0 2024-03-22T07:18:34.456 INFO:tasks.workunit.client.0.smithi081.stdout:5/501: truncate d4/d10/d20/d1f/fa2 131864 0 2024-03-22T07:18:34.457 INFO:tasks.workunit.client.0.smithi081.stdout:0/414: mknod d3/d4e/c8a 0 2024-03-22T07:18:34.461 INFO:tasks.workunit.client.0.smithi081.stdout:5/502: mkdir d4/d10/d20/d25/da3 0 2024-03-22T07:18:34.462 INFO:tasks.workunit.client.0.smithi081.stdout:0/415: mknod d3/dc/d2b/c8b 0 2024-03-22T07:18:34.465 INFO:tasks.workunit.client.0.smithi081.stdout:5/503: symlink d4/d10/la4 0 2024-03-22T07:18:34.465 INFO:tasks.workunit.client.0.smithi081.stdout:0/416: truncate d3/d32/f70 321605 0 2024-03-22T07:18:34.469 INFO:tasks.workunit.client.0.smithi081.stdout:0/417: rename d3/f13 to d3/dc/d2b/d39/f8c 0 2024-03-22T07:18:34.476 INFO:tasks.workunit.client.0.smithi081.stdout:2/282: dwrite da/db/d39/d47/f43 [0,4194304] 0 2024-03-22T07:18:34.476 INFO:tasks.workunit.client.0.smithi081.stdout:1/296: dwrite d6/d18/d1f/d24/f3f [0,4194304] 0 2024-03-22T07:18:34.477 INFO:tasks.workunit.client.0.smithi081.stdout:3/283: dwrite db/d2e/f3e [0,4194304] 0 2024-03-22T07:18:34.478 INFO:tasks.workunit.client.0.smithi081.stdout:5/504: write d4/d9/f2c [3036211,85068] 0 2024-03-22T07:18:34.485 INFO:tasks.workunit.client.0.smithi081.stdout:3/284: mknod db/d2e/d3a/d5d/c5f 0 2024-03-22T07:18:34.488 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch ps' 2024-03-22T07:18:34.490 INFO:tasks.workunit.client.0.smithi081.stdout:1/297: dread d6/d18/d41/d49/d56/f3c [0,4194304] 0 2024-03-22T07:18:34.490 INFO:tasks.workunit.client.0.smithi081.stdout:2/283: rename da/db/d39/d47/d3b/d41/c59 to da/db/d39/d45/c5c 0 2024-03-22T07:18:34.490 INFO:tasks.workunit.client.0.smithi081.stdout:4/457: write da/d18/d1b/d26/d3e/f55 [412355,46087] 0 2024-03-22T07:18:34.492 INFO:tasks.workunit.client.0.smithi081.stdout:5/505: mkdir d4/d75/da5 0 2024-03-22T07:18:34.492 INFO:tasks.workunit.client.0.smithi081.stdout:5/506: chown d4/df/d22/d4d/f63 267 1 2024-03-22T07:18:34.493 INFO:tasks.workunit.client.0.smithi081.stdout:1/298: getdents d6/d18/d41/d49/d56 0 2024-03-22T07:18:34.495 INFO:tasks.workunit.client.0.smithi081.stdout:1/299: symlink d6/d18/d1f/d4c/l66 0 2024-03-22T07:18:34.496 INFO:tasks.workunit.client.0.smithi081.stdout:1/300: truncate d6/d18/d41/f46 819906 0 2024-03-22T07:18:34.497 INFO:tasks.workunit.client.0.smithi081.stdout:3/285: mknod db/d38/c60 0 2024-03-22T07:18:34.498 INFO:tasks.workunit.client.0.smithi081.stdout:5/507: symlink d4/d9/d90/la6 0 2024-03-22T07:18:34.500 INFO:tasks.workunit.client.0.smithi081.stdout:2/284: truncate da/db/d39/f57 1804648 0 2024-03-22T07:18:34.503 INFO:tasks.workunit.client.0.smithi081.stdout:1/301: creat d6/f67 x:0 0 0 2024-03-22T07:18:34.505 INFO:tasks.workunit.client.0.smithi081.stdout:5/508: creat d4/d10/d20/d25/da3/fa7 x:0 0 0 2024-03-22T07:18:34.505 INFO:tasks.workunit.client.0.smithi081.stdout:5/509: write d4/d9/f91 [1956381,24933] 0 2024-03-22T07:18:34.510 INFO:tasks.workunit.client.0.smithi081.stdout:1/302: creat d6/d18/d41/d49/d56/d30/f68 x:0 0 0 2024-03-22T07:18:34.511 INFO:tasks.workunit.client.0.smithi081.stdout:0/418: write d3/d32/f5f [2953848,124254] 0 2024-03-22T07:18:34.511 INFO:tasks.workunit.client.0.smithi081.stdout:5/510: truncate d4/d9/f15 187490 0 2024-03-22T07:18:34.511 INFO:tasks.workunit.client.0.smithi081.stdout:1/303: creat d6/d18/d1f/d24/d48/f69 x:0 0 0 2024-03-22T07:18:34.511 INFO:tasks.workunit.client.0.smithi081.stdout:1/304: chown d6/d18/d60 5 1 2024-03-22T07:18:34.511 INFO:tasks.workunit.client.0.smithi081.stdout:5/511: readlink d4/d10/d20/d1f/l6a 0 2024-03-22T07:18:34.511 INFO:tasks.workunit.client.0.smithi081.stdout:2/285: read da/d11/f1f [7386037,11044] 0 2024-03-22T07:18:34.511 INFO:tasks.workunit.client.0.smithi081.stdout:2/286: chown da 147444 1 2024-03-22T07:18:34.511 INFO:tasks.workunit.client.0.smithi081.stdout:2/287: write f6 [96994,57979] 0 2024-03-22T07:18:34.512 INFO:tasks.workunit.client.0.smithi081.stdout:0/419: rename d3/dc/d2b/f48 to d3/dc/f8d 0 2024-03-22T07:18:34.513 INFO:tasks.workunit.client.0.smithi081.stdout:0/420: chown d3/dc/d2b/c2c 4 1 2024-03-22T07:18:34.513 INFO:tasks.workunit.client.0.smithi081.stdout:9/372: sync 2024-03-22T07:18:34.514 INFO:tasks.workunit.client.0.smithi081.stdout:6/338: sync 2024-03-22T07:18:34.515 INFO:tasks.workunit.client.0.smithi081.stdout:5/512: dread d4/df/d22/d4d/f63 [0,4194304] 0 2024-03-22T07:18:34.515 INFO:tasks.workunit.client.0.smithi081.stdout:3/286: dwrite db/fd [0,4194304] 0 2024-03-22T07:18:34.518 INFO:tasks.workunit.client.0.smithi081.stdout:3/287: creat db/d23/f61 x:0 0 0 2024-03-22T07:18:34.519 INFO:tasks.workunit.client.0.smithi081.stdout:1/305: dread d6/f2a [0,4194304] 0 2024-03-22T07:18:34.519 INFO:tasks.workunit.client.0.smithi081.stdout:1/306: write d6/fd [4358118,25143] 0 2024-03-22T07:18:34.528 INFO:tasks.workunit.client.0.smithi081.stdout:6/339: dread d3/da/de/f28 [0,4194304] 0 2024-03-22T07:18:34.528 INFO:tasks.workunit.client.0.smithi081.stdout:6/340: write d3/da/de/d31/d40/d50/d63/d26/d52/f69 [627553,129130] 0 2024-03-22T07:18:34.528 INFO:tasks.workunit.client.0.smithi081.stdout:5/513: read d4/df/f1a [1515883,60720] 0 2024-03-22T07:18:34.529 INFO:tasks.workunit.client.0.smithi081.stdout:5/514: dread - d4/d10/d20/d4f/f78 zero size 2024-03-22T07:18:34.529 INFO:tasks.workunit.client.0.smithi081.stdout:6/341: chown d3/da/de/f1f 14 1 2024-03-22T07:18:34.532 INFO:tasks.workunit.client.0.smithi081.stdout:3/288: dread db/de/f29 [0,4194304] 0 2024-03-22T07:18:34.533 INFO:tasks.workunit.client.0.smithi081.stdout:9/373: dread d2/d5/d16/f4c [0,4194304] 0 2024-03-22T07:18:34.533 INFO:tasks.workunit.client.0.smithi081.stdout:5/515: mkdir d4/df/da8 0 2024-03-22T07:18:34.534 INFO:tasks.workunit.client.0.smithi081.stdout:3/289: symlink db/de/d4b/l62 0 2024-03-22T07:18:34.537 INFO:tasks.workunit.client.0.smithi081.stdout:6/342: rename d3/da/l34 to d3/da/de/l6c 0 2024-03-22T07:18:34.542 INFO:tasks.workunit.client.0.smithi081.stdout:5/516: symlink d4/la9 0 2024-03-22T07:18:34.542 INFO:tasks.workunit.client.0.smithi081.stdout:6/343: creat d3/da/de/d31/d40/f6d x:0 0 0 2024-03-22T07:18:34.543 INFO:tasks.workunit.client.0.smithi081.stdout:2/288: dwrite da/db/f35 [0,4194304] 0 2024-03-22T07:18:34.543 INFO:tasks.workunit.client.0.smithi081.stdout:5/517: creat d4/d75/faa x:0 0 0 2024-03-22T07:18:34.544 INFO:tasks.workunit.client.0.smithi081.stdout:5/518: creat d4/df/d73/fab x:0 0 0 2024-03-22T07:18:34.546 INFO:tasks.workunit.client.0.smithi081.stdout:1/307: dwrite d6/d18/f1d [0,4194304] 0 2024-03-22T07:18:34.546 INFO:tasks.workunit.client.0.smithi081.stdout:1/308: chown d6/d18/d41/d49/d56 124869 1 2024-03-22T07:18:34.546 INFO:tasks.workunit.client.0.smithi081.stdout:4/458: dwrite f9 [0,4194304] 0 2024-03-22T07:18:34.551 INFO:tasks.workunit.client.0.smithi081.stdout:8/379: sync 2024-03-22T07:18:34.554 INFO:tasks.workunit.client.0.smithi081.stdout:6/344: dread - d3/da/de/f61 zero size 2024-03-22T07:18:34.561 INFO:tasks.workunit.client.0.smithi081.stdout:6/345: chown d3/da/de/d31/d40/d50/d63/d26/d3f/d4c 1093754 1 2024-03-22T07:18:34.564 INFO:tasks.workunit.client.0.smithi081.stdout:5/519: rename d4/d10/l12 to d4/d9/d90/lac 0 2024-03-22T07:18:34.566 INFO:tasks.workunit.client.0.smithi081.stdout:1/309: mkdir d6/d6a 0 2024-03-22T07:18:34.567 INFO:tasks.workunit.client.0.smithi081.stdout:3/290: write db/f1e [942522,116718] 0 2024-03-22T07:18:34.568 INFO:tasks.workunit.client.0.smithi081.stdout:8/380: mknod d2/d3/d28/d2b/d61/c85 0 2024-03-22T07:18:34.573 INFO:tasks.workunit.client.0.smithi081.stdout:3/291: dread - db/d2e/f40 zero size 2024-03-22T07:18:34.577 INFO:tasks.workunit.client.0.smithi081.stdout:7/495: sync 2024-03-22T07:18:34.580 INFO:tasks.workunit.client.0.smithi081.stdout:2/289: write da/db/d39/f57 [539541,25348] 0 2024-03-22T07:18:34.580 INFO:tasks.workunit.client.0.smithi081.stdout:7/496: unlink d9/f7c 0 2024-03-22T07:18:34.580 INFO:tasks.workunit.client.0.smithi081.stdout:7/497: fdatasync d9/d3f/f79 0 2024-03-22T07:18:34.580 INFO:tasks.workunit.client.0.smithi081.stdout:5/520: dwrite d4/df/d22/d4d/f99 [0,4194304] 0 2024-03-22T07:18:34.586 INFO:tasks.workunit.client.0.smithi081.stdout:5/521: creat d4/d75/d98/fad x:0 0 0 2024-03-22T07:18:34.588 INFO:tasks.workunit.client.0.smithi081.stdout:7/498: dread d9/f11 [4194304,4194304] 0 2024-03-22T07:18:34.589 INFO:tasks.workunit.client.0.smithi081.stdout:5/522: rename d4/df/d73/d8c/c8f to d4/df/d22/d4d/cae 0 2024-03-22T07:18:34.590 INFO:tasks.workunit.client.0.smithi081.stdout:7/499: readlink d9/d10/d4b/l4f 0 2024-03-22T07:18:34.592 INFO:tasks.workunit.client.0.smithi081.stdout:5/523: mkdir d4/d10/daf 0 2024-03-22T07:18:34.593 INFO:tasks.workunit.client.0.smithi081.stdout:5/524: rmdir d4/d9 39 2024-03-22T07:18:34.595 INFO:tasks.workunit.client.0.smithi081.stdout:7/500: rename d9/d10/d2a/d73/c87 to d9/d10/cad 0 2024-03-22T07:18:34.597 INFO:tasks.workunit.client.0.smithi081.stdout:5/525: symlink d4/d10/d20/d1f/lb0 0 2024-03-22T07:18:34.597 INFO:tasks.workunit.client.0.smithi081.stdout:5/526: chown d4/df/d22/d4d/f6c 1922 1 2024-03-22T07:18:34.597 INFO:tasks.workunit.client.0.smithi081.stdout:5/527: dread - d4/d75/d98/f9c zero size 2024-03-22T07:18:34.597 INFO:tasks.workunit.client.0.smithi081.stdout:5/528: write d4/df/d7c/f95 [599029,45507] 0 2024-03-22T07:18:34.598 INFO:tasks.workunit.client.0.smithi081.stdout:5/529: creat d4/fb1 x:0 0 0 2024-03-22T07:18:34.599 INFO:tasks.workunit.client.0.smithi081.stdout:5/530: chown d4/d9/d90/lac 501 1 2024-03-22T07:18:34.601 INFO:tasks.workunit.client.0.smithi081.stdout:9/374: dwrite d2/d2b/f60 [0,4194304] 0 2024-03-22T07:18:34.613 INFO:tasks.workunit.client.0.smithi081.stdout:1/310: dwrite d6/d18/d1f/f37 [0,4194304] 0 2024-03-22T07:18:34.615 INFO:tasks.workunit.client.0.smithi081.stdout:1/311: getdents d6 0 2024-03-22T07:18:34.615 INFO:tasks.workunit.client.0.smithi081.stdout:1/312: dread - d6/d18/d41/d49/d56/f4a zero size 2024-03-22T07:18:34.615 INFO:tasks.workunit.client.0.smithi081.stdout:1/313: chown d6/d18/d41/d49/d56/d30/f55 18 1 2024-03-22T07:18:34.615 INFO:tasks.workunit.client.0.smithi081.stdout:1/314: stat d6/d18/d41/d49 0 2024-03-22T07:18:34.616 INFO:tasks.workunit.client.0.smithi081.stdout:1/315: getdents d6/d18 0 2024-03-22T07:18:34.617 INFO:tasks.workunit.client.0.smithi081.stdout:1/316: symlink d6/d18/d1f/l6b 0 2024-03-22T07:18:34.617 INFO:tasks.workunit.client.0.smithi081.stdout:1/317: truncate d6/d18/d41/d49/d56/f3c 902304 0 2024-03-22T07:18:34.619 INFO:tasks.workunit.client.0.smithi081.stdout:1/318: creat d6/d18/d1f/d4c/f6c x:0 0 0 2024-03-22T07:18:34.620 INFO:tasks.workunit.client.0.smithi081.stdout:1/319: readlink d6/d18/d1f/l6b 0 2024-03-22T07:18:34.621 INFO:tasks.workunit.client.0.smithi081.stdout:4/459: dwrite da/d18/d42/d4f/f58 [0,4194304] 0 2024-03-22T07:18:34.621 INFO:tasks.workunit.client.0.smithi081.stdout:2/290: dwrite f5 [4194304,4194304] 0 2024-03-22T07:18:34.621 INFO:tasks.workunit.client.0.smithi081.stdout:1/320: write d6/d18/d1f/d24/f39 [2599629,81842] 0 2024-03-22T07:18:34.622 INFO:tasks.workunit.client.0.smithi081.stdout:9/375: dwrite d2/d2b/f3b [0,4194304] 0 2024-03-22T07:18:34.623 INFO:tasks.workunit.client.0.smithi081.stdout:1/321: truncate d6/d18/d1f/d24/f26 962752 0 2024-03-22T07:18:34.623 INFO:tasks.workunit.client.0.smithi081.stdout:1/322: getdents d6/d18/d41 0 2024-03-22T07:18:34.623 INFO:tasks.workunit.client.0.smithi081.stdout:1/323: chown d6/f17 10 1 2024-03-22T07:18:34.624 INFO:tasks.workunit.client.0.smithi081.stdout:9/376: truncate d2/d10/d30/d57/f62 2921582 0 2024-03-22T07:18:34.625 INFO:tasks.workunit.client.0.smithi081.stdout:4/460: link da/d18/d1b/d34/f64 da/d18/d1b/d26/d32/f91 0 2024-03-22T07:18:34.628 INFO:tasks.workunit.client.0.smithi081.stdout:9/377: dread d2/d5/d16/f4c [0,4194304] 0 2024-03-22T07:18:34.634 INFO:tasks.workunit.client.0.smithi081.stdout:5/531: rename d4/d10/d20/d1f to d4/d10/d20/db2 0 2024-03-22T07:18:34.635 INFO:tasks.workunit.client.0.smithi081.stdout:4/461: dread da/f2c [0,4194304] 0 2024-03-22T07:18:34.640 INFO:tasks.workunit.client.0.smithi081.stdout:5/532: link d4/d10/l14 d4/d75/lb3 0 2024-03-22T07:18:34.640 INFO:tasks.workunit.client.0.smithi081.stdout:5/533: read d4/df/d22/d4d/f63 [668733,66764] 0 2024-03-22T07:18:34.641 INFO:tasks.workunit.client.0.smithi081.stdout:5/534: creat d4/d10/daf/fb4 x:0 0 0 2024-03-22T07:18:34.643 INFO:tasks.workunit.client.0.smithi081.stdout:4/462: dread da/d18/d1b/d26/f43 [0,4194304] 0 2024-03-22T07:18:34.643 INFO:tasks.workunit.client.0.smithi081.stdout:4/463: chown da/d18/d1b/d1e 1919449 1 2024-03-22T07:18:34.644 INFO:tasks.workunit.client.0.smithi081.stdout:1/324: write d6/d18/d41/d49/d56/d30/f55 [588946,36576] 0 2024-03-22T07:18:34.644 INFO:tasks.workunit.client.0.smithi081.stdout:1/325: stat d6/d18/d1f/d4c/c5a 0 2024-03-22T07:18:34.645 INFO:tasks.workunit.client.0.smithi081.stdout:1/326: readlink l5 0 2024-03-22T07:18:34.648 INFO:tasks.workunit.client.0.smithi081.stdout:1/327: rename d6/fd to d6/f6d 0 2024-03-22T07:18:34.649 INFO:tasks.workunit.client.0.smithi081.stdout:5/535: dread d4/df/f46 [0,4194304] 0 2024-03-22T07:18:34.649 INFO:tasks.workunit.client.0.smithi081.stdout:5/536: write d4/d10/d20/d4f/f78 [163519,612] 0 2024-03-22T07:18:34.650 INFO:tasks.workunit.client.0.smithi081.stdout:1/328: mkdir d6/d18/d1f/d24/d48/d6e 0 2024-03-22T07:18:34.650 INFO:tasks.workunit.client.0.smithi081.stdout:9/378: dwrite d2/d10/f14 [0,4194304] 0 2024-03-22T07:18:34.650 INFO:tasks.workunit.client.0.smithi081.stdout:1/329: write d6/d18/f28 [2523427,15232] 0 2024-03-22T07:18:34.650 INFO:tasks.workunit.client.0.smithi081.stdout:9/379: fdatasync d2/d10/f35 0 2024-03-22T07:18:34.651 INFO:tasks.workunit.client.0.smithi081.stdout:9/380: creat d2/d2b/f77 x:0 0 0 2024-03-22T07:18:34.652 INFO:tasks.workunit.client.0.smithi081.stdout:9/381: mkdir d2/d78 0 2024-03-22T07:18:34.653 INFO:tasks.workunit.client.0.smithi081.stdout:1/330: rename d6/d18/d1f/d32/f47 to d6/d18/d1f/d24/f6f 0 2024-03-22T07:18:34.656 INFO:tasks.workunit.client.0.smithi081.stdout:4/464: dread da/d18/d1b/d34/f4c [0,4194304] 0 2024-03-22T07:18:34.656 INFO:tasks.workunit.client.0.smithi081.stdout:9/382: dread d2/d10/d30/f45 [0,4194304] 0 2024-03-22T07:18:34.661 INFO:tasks.workunit.client.0.smithi081.stdout:5/537: dread d4/d10/d20/db2/d2f/f92 [4194304,4194304] 0 2024-03-22T07:18:34.663 INFO:tasks.workunit.client.0.smithi081.stdout:4/465: mkdir da/d18/d1b/d26/d32/d92 0 2024-03-22T07:18:34.665 INFO:tasks.workunit.client.0.smithi081.stdout:5/538: unlink d4/df/d73/d8c/c9a 0 2024-03-22T07:18:34.666 INFO:tasks.workunit.client.0.smithi081.stdout:5/539: mkdir d4/d10/d20/d25/da3/db5 0 2024-03-22T07:18:34.666 INFO:tasks.workunit.client.0.smithi081.stdout:4/466: truncate da/df/d38/f76 290254 0 2024-03-22T07:18:34.667 INFO:tasks.workunit.client.0.smithi081.stdout:5/540: creat d4/d10/d20/d4f/fb6 x:0 0 0 2024-03-22T07:18:34.668 INFO:tasks.workunit.client.0.smithi081.stdout:5/541: rmdir d4/d10/d20/db2/d2a 39 2024-03-22T07:18:34.669 INFO:tasks.workunit.client.0.smithi081.stdout:5/542: getdents d4/df/d22 0 2024-03-22T07:18:34.670 INFO:tasks.workunit.client.0.smithi081.stdout:5/543: creat d4/d10/d20/d25/da3/db5/fb7 x:0 0 0 2024-03-22T07:18:34.671 INFO:tasks.workunit.client.0.smithi081.stdout:5/544: chown d4/df/d22/d49 258251 1 2024-03-22T07:18:34.675 INFO:tasks.workunit.client.0.smithi081.stdout:5/545: rename d4/d10/d20/d25/d2e to d4/d10/d20/d25/db8 0 2024-03-22T07:18:34.676 INFO:tasks.workunit.client.0.smithi081.stdout:5/546: link d4/df/d73/l94 d4/d10/d20/db2/d2f/d8e/lb9 0 2024-03-22T07:18:34.679 INFO:tasks.workunit.client.0.smithi081.stdout:9/383: dwrite d2/d2b/f41 [0,4194304] 0 2024-03-22T07:18:34.682 INFO:tasks.workunit.client.0.smithi081.stdout:5/547: read d4/df/d7c/f85 [474183,122085] 0 2024-03-22T07:18:34.683 INFO:tasks.workunit.client.0.smithi081.stdout:5/548: mknod d4/d10/d20/db2/cba 0 2024-03-22T07:18:34.684 INFO:tasks.workunit.client.0.smithi081.stdout:5/549: creat d4/d9/d90/fbb x:0 0 0 2024-03-22T07:18:34.684 INFO:tasks.workunit.client.0.smithi081.stdout:5/550: symlink d4/d10/d20/db2/d2a/lbc 0 2024-03-22T07:18:34.686 INFO:tasks.workunit.client.0.smithi081.stdout:5/551: write d4/df/d7c/f95 [186282,96703] 0 2024-03-22T07:18:34.690 INFO:tasks.workunit.client.0.smithi081.stdout:5/552: fdatasync d4/df/d7c/f95 0 2024-03-22T07:18:34.691 INFO:tasks.workunit.client.0.smithi081.stdout:5/553: readlink d4/l47 0 2024-03-22T07:18:34.693 INFO:tasks.workunit.client.0.smithi081.stdout:5/554: getdents d4/d10/daf 0 2024-03-22T07:18:34.693 INFO:tasks.workunit.client.0.smithi081.stdout:5/555: fdatasync d4/f89 0 2024-03-22T07:18:34.693 INFO:tasks.workunit.client.0.smithi081.stdout:5/556: write d4/d10/d20/d25/f6e [4953238,100704] 0 2024-03-22T07:18:34.697 INFO:tasks.workunit.client.0.smithi081.stdout:5/557: link d4/df/d22/d49/f4a d4/d75/fbd 0 2024-03-22T07:18:34.700 INFO:tasks.workunit.client.0.smithi081.stdout:5/558: rename d4/d10/d20/l36 to d4/d10/daf/lbe 0 2024-03-22T07:18:34.700 INFO:tasks.workunit.client.0.smithi081.stdout:5/559: dread - d4/d9/d90/fbb zero size 2024-03-22T07:18:34.700 INFO:tasks.workunit.client.0.smithi081.stdout:5/560: chown d4/df/d73/f9d 0 1 2024-03-22T07:18:34.704 INFO:tasks.workunit.client.0.smithi081.stdout:4/467: dwrite da/f1d [4194304,4194304] 0 2024-03-22T07:18:34.708 INFO:tasks.workunit.client.0.smithi081.stdout:9/384: dwrite d2/d10/f14 [4194304,4194304] 0 2024-03-22T07:18:34.709 INFO:tasks.workunit.client.0.smithi081.stdout:4/468: dread da/df/d38/f54 [0,4194304] 0 2024-03-22T07:18:34.710 INFO:tasks.workunit.client.0.smithi081.stdout:9/385: truncate d2/d10/f35 3345528 0 2024-03-22T07:18:34.715 INFO:tasks.workunit.client.0.smithi081.stdout:5/561: write d4/d10/d20/d25/db8/f54 [276617,59089] 0 2024-03-22T07:18:34.715 INFO:tasks.workunit.client.0.smithi081.stdout:9/386: mkdir d2/d10/d79 0 2024-03-22T07:18:34.716 INFO:tasks.workunit.client.0.smithi081.stdout:4/469: link da/d18/d1b/d1e/d37/f6c da/d18/d1b/d26/d3e/f93 0 2024-03-22T07:18:34.720 INFO:tasks.workunit.client.0.smithi081.stdout:9/387: unlink d2/d2b/f50 0 2024-03-22T07:18:34.721 INFO:tasks.workunit.client.0.smithi081.stdout:9/388: dread d2/d10/d30/d57/d66/f69 [0,4194304] 0 2024-03-22T07:18:34.722 INFO:tasks.workunit.client.0.smithi081.stdout:4/470: fsync da/d18/d42/d4f/f6a 0 2024-03-22T07:18:34.724 INFO:tasks.workunit.client.0.smithi081.stdout:9/389: rename d2/d10/d79 to d2/d5/d16/d7a 0 2024-03-22T07:18:34.731 INFO:tasks.workunit.client.0.smithi081.stdout:5/562: dwrite d4/d75/d98/f9c [0,4194304] 0 2024-03-22T07:18:34.731 INFO:tasks.workunit.client.0.smithi081.stdout:5/563: readlink d4/df/d22/l30 0 2024-03-22T07:18:34.736 INFO:tasks.workunit.client.0.smithi081.stdout:9/390: write d2/d5/d16/f2e [686753,41090] 0 2024-03-22T07:18:34.737 INFO:tasks.workunit.client.0.smithi081.stdout:9/391: creat d2/d5/d16/d37/f7b x:0 0 0 2024-03-22T07:18:34.738 INFO:tasks.workunit.client.0.smithi081.stdout:9/392: mknod d2/d5/d16/d7a/c7c 0 2024-03-22T07:18:34.738 INFO:tasks.workunit.client.0.smithi081.stdout:9/393: creat d2/d10/d30/d57/f7d x:0 0 0 2024-03-22T07:18:34.744 INFO:tasks.workunit.client.0.smithi081.stdout:5/564: write d4/d10/f3c [3343834,71966] 0 2024-03-22T07:18:34.745 INFO:tasks.workunit.client.0.smithi081.stdout:5/565: truncate d4/fb1 834198 0 2024-03-22T07:18:34.745 INFO:tasks.workunit.client.0.smithi081.stdout:5/566: readlink d4/df/d73/l94 0 2024-03-22T07:18:34.745 INFO:tasks.workunit.client.0.smithi081.stdout:5/567: readlink d4/d10/d20/d25/db8/l44 0 2024-03-22T07:18:34.746 INFO:tasks.workunit.client.0.smithi081.stdout:5/568: symlink d4/df/d22/d4d/d82/lbf 0 2024-03-22T07:18:34.748 INFO:tasks.workunit.client.0.smithi081.stdout:5/569: mkdir d4/df/d73/d8c/dc0 0 2024-03-22T07:18:34.748 INFO:tasks.workunit.client.0.smithi081.stdout:5/570: chown d4/d10/f21 539184247 1 2024-03-22T07:18:34.768 INFO:tasks.workunit.client.0.smithi081.stdout:9/394: dwrite d2/d10/d30/d57/f76 [0,4194304] 0 2024-03-22T07:18:34.768 INFO:tasks.workunit.client.0.smithi081.stdout:4/471: dwrite da/d18/d1b/d26/d3e/f93 [0,4194304] 0 2024-03-22T07:18:34.769 INFO:tasks.workunit.client.0.smithi081.stdout:9/395: chown d2/f3 2063567 1 2024-03-22T07:18:34.772 INFO:tasks.workunit.client.0.smithi081.stdout:4/472: fdatasync da/df/f65 0 2024-03-22T07:18:34.774 INFO:tasks.workunit.client.0.smithi081.stdout:9/396: creat d2/d73/f7e x:0 0 0 2024-03-22T07:18:34.776 INFO:tasks.workunit.client.0.smithi081.stdout:9/397: truncate d2/d10/d30/f45 1583642 0 2024-03-22T07:18:34.776 INFO:tasks.workunit.client.0.smithi081.stdout:9/398: fdatasync d2/d73/f7e 0 2024-03-22T07:18:34.779 INFO:tasks.workunit.client.0.smithi081.stdout:4/473: rename da/d18/d42/c50 to da/d18/d42/c94 0 2024-03-22T07:18:34.779 INFO:tasks.workunit.client.0.smithi081.stdout:4/474: write da/df/d38/f7f [28543,87840] 0 2024-03-22T07:18:34.780 INFO:tasks.workunit.client.0.smithi081.stdout:9/399: link d2/d5/d16/d7a/c7c d2/d5d/c7f 0 2024-03-22T07:18:34.782 INFO:tasks.workunit.client.0.smithi081.stdout:9/400: creat d2/d10/d30/d57/d66/f80 x:0 0 0 2024-03-22T07:18:34.789 INFO:tasks.workunit.client.0.smithi081.stdout:9/401: write d2/d5/d16/f6d [933892,118137] 0 2024-03-22T07:18:34.789 INFO:tasks.workunit.client.0.smithi081.stdout:4/475: dread da/d18/d1b/d1e/d37/d1c/f51 [0,4194304] 0 2024-03-22T07:18:34.791 INFO:tasks.workunit.client.0.smithi081.stdout:9/402: dread d2/f11 [0,4194304] 0 2024-03-22T07:18:34.792 INFO:tasks.workunit.client.0.smithi081.stdout:9/403: creat d2/d2b/f81 x:0 0 0 2024-03-22T07:18:34.792 INFO:tasks.workunit.client.0.smithi081.stdout:4/476: dread da/d18/d1b/d1e/f41 [0,4194304] 0 2024-03-22T07:18:34.792 INFO:tasks.workunit.client.0.smithi081.stdout:4/477: dread - da/f87 zero size 2024-03-22T07:18:34.794 INFO:tasks.workunit.client.0.smithi081.stdout:4/478: unlink da/d18/d1b/d26/d3e/f93 0 2024-03-22T07:18:34.795 INFO:tasks.workunit.client.0.smithi081.stdout:0/421: sync 2024-03-22T07:18:34.795 INFO:tasks.workunit.client.0.smithi081.stdout:0/422: stat d3/dc/c89 0 2024-03-22T07:18:34.797 INFO:tasks.workunit.client.0.smithi081.stdout:9/404: rename d2/c34 to d2/c82 0 2024-03-22T07:18:34.800 INFO:tasks.workunit.client.0.smithi081.stdout:4/479: truncate da/fb 2674196 0 2024-03-22T07:18:34.802 INFO:tasks.workunit.client.0.smithi081.stdout:9/405: creat d2/d73/d74/f83 x:0 0 0 2024-03-22T07:18:34.803 INFO:tasks.workunit.client.0.smithi081.stdout:9/406: readlink d2/d5/d36/l51 0 2024-03-22T07:18:34.803 INFO:tasks.workunit.client.0.smithi081.stdout:9/407: truncate d2/f5f 239802 0 2024-03-22T07:18:34.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/480: link da/df/l3c da/d18/d1b/d26/l95 0 2024-03-22T07:18:34.810 INFO:tasks.workunit.client.0.smithi081.stdout:4/481: unlink f1 0 2024-03-22T07:18:34.813 INFO:tasks.workunit.client.0.smithi081.stdout:9/408: write d2/d10/d30/f45 [2471006,9627] 0 2024-03-22T07:18:34.813 INFO:tasks.workunit.client.0.smithi081.stdout:9/409: fdatasync d2/d2b/f43 0 2024-03-22T07:18:34.814 INFO:tasks.workunit.client.0.smithi081.stdout:4/482: link da/d18/d1b/d26/l29 da/l96 0 2024-03-22T07:18:34.816 INFO:tasks.workunit.client.0.smithi081.stdout:9/410: creat d2/d5d/d58/f84 x:0 0 0 2024-03-22T07:18:34.820 INFO:tasks.workunit.client.0.smithi081.stdout:9/411: rename d2/d5/d16 to d2/d73/d85 0 2024-03-22T07:18:34.820 INFO:tasks.workunit.client.0.smithi081.stdout:9/412: chown d2/d5d 11 1 2024-03-22T07:18:34.821 INFO:tasks.workunit.client.0.smithi081.stdout:9/413: rmdir d2/d5d 39 2024-03-22T07:18:34.841 INFO:tasks.workunit.client.0.smithi081.stdout:0/423: dwrite d3/dc/f1c [0,4194304] 0 2024-03-22T07:18:34.842 INFO:tasks.workunit.client.0.smithi081.stdout:0/424: unlink d3/dc/d2b/c2c 0 2024-03-22T07:18:34.848 INFO:tasks.workunit.client.0.smithi081.stdout:4/483: dwrite f3 [0,4194304] 0 2024-03-22T07:18:34.849 INFO:tasks.workunit.client.0.smithi081.stdout:9/414: dwrite d2/d2b/f3b [0,4194304] 0 2024-03-22T07:18:34.849 INFO:tasks.workunit.client.0.smithi081.stdout:9/415: fdatasync d2/d2b/f3b 0 2024-03-22T07:18:34.849 INFO:tasks.workunit.client.0.smithi081.stdout:0/425: read d3/dc/f1c [173809,125529] 0 2024-03-22T07:18:34.850 INFO:tasks.workunit.client.0.smithi081.stdout:9/416: mkdir d2/d2b/d49/d86 0 2024-03-22T07:18:34.850 INFO:tasks.workunit.client.0.smithi081.stdout:9/417: read d2/d2b/f43 [590488,122109] 0 2024-03-22T07:18:34.850 INFO:tasks.workunit.client.0.smithi081.stdout:9/418: chown d2/d10/d30/d57/d66/f80 31320 1 2024-03-22T07:18:34.853 INFO:tasks.workunit.client.0.smithi081.stdout:4/484: link da/d18/d1b/d26/c7d da/d18/d1b/d1e/d37/c97 0 2024-03-22T07:18:34.856 INFO:tasks.workunit.client.0.smithi081.stdout:4/485: mkdir da/d18/d1b/d1e/d37/d1c/d74/d98 0 2024-03-22T07:18:34.857 INFO:tasks.workunit.client.0.smithi081.stdout:4/486: truncate da/d18/d1b/d1e/d37/d1c/f51 4005786 0 2024-03-22T07:18:34.858 INFO:tasks.workunit.client.0.smithi081.stdout:9/419: read d2/f1e [1186299,76842] 0 2024-03-22T07:18:34.858 INFO:tasks.workunit.client.0.smithi081.stdout:9/420: readlink d2/d5/d36/l51 0 2024-03-22T07:18:34.860 INFO:tasks.workunit.client.0.smithi081.stdout:9/421: symlink d2/d78/l87 0 2024-03-22T07:18:34.860 INFO:tasks.workunit.client.0.smithi081.stdout:4/487: dread da/df/d38/f76 [0,4194304] 0 2024-03-22T07:18:34.860 INFO:tasks.workunit.client.0.smithi081.stdout:9/422: chown d2/d2b/c39 322458 1 2024-03-22T07:18:34.890 INFO:tasks.workunit.client.0.smithi081.stdout:0/426: dwrite d3/f24 [0,4194304] 0 2024-03-22T07:18:34.890 INFO:tasks.workunit.client.0.smithi081.stdout:9/423: dwrite d2/d73/d85/f6d [0,4194304] 0 2024-03-22T07:18:34.890 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:18:34.893 INFO:tasks.workunit.client.0.smithi081.stdout:0/427: getdents d3/d4e 0 2024-03-22T07:18:34.893 INFO:tasks.workunit.client.0.smithi081.stdout:0/428: chown d3/dc 12 1 2024-03-22T07:18:34.894 INFO:tasks.workunit.client.0.smithi081.stdout:9/424: getdents d2/d10/d30/d57 0 2024-03-22T07:18:34.895 INFO:tasks.workunit.client.0.smithi081.stdout:0/429: creat d3/d6a/f8e x:0 0 0 2024-03-22T07:18:34.896 INFO:tasks.workunit.client.0.smithi081.stdout:9/425: dread d2/f8 [0,4194304] 0 2024-03-22T07:18:34.898 INFO:tasks.workunit.client.0.smithi081.stdout:9/426: creat d2/d73/d85/d7a/f88 x:0 0 0 2024-03-22T07:18:34.898 INFO:tasks.workunit.client.0.smithi081.stdout:9/427: write d2/d73/d85/f59 [739332,91534] 0 2024-03-22T07:18:34.906 INFO:tasks.workunit.client.0.smithi081.stdout:9/428: dread d2/fe [4194304,4194304] 0 2024-03-22T07:18:34.907 INFO:tasks.workunit.client.0.smithi081.stdout:9/429: link d2/fe d2/f89 0 2024-03-22T07:18:34.909 INFO:tasks.workunit.client.0.smithi081.stdout:4/488: dwrite da/f7b [0,4194304] 0 2024-03-22T07:18:34.909 INFO:tasks.workunit.client.0.smithi081.stdout:4/489: dread - da/d18/d42/d4a/f84 zero size 2024-03-22T07:18:34.914 INFO:tasks.workunit.client.0.smithi081.stdout:4/490: write da/d18/d1b/d1e/d37/d1c/f51 [2605006,111512] 0 2024-03-22T07:18:34.919 INFO:tasks.workunit.client.0.smithi081.stdout:9/430: dread d2/d10/f35 [0,4194304] 0 2024-03-22T07:18:34.921 INFO:tasks.workunit.client.0.smithi081.stdout:9/431: dread d2/d2b/d49/f4f [0,4194304] 0 2024-03-22T07:18:34.921 INFO:tasks.workunit.client.0.smithi081.stdout:9/432: chown d2/d10/d30/d57/f7d 234358 1 2024-03-22T07:18:34.923 INFO:tasks.workunit.client.0.smithi081.stdout:9/433: getdents d2/d73 0 2024-03-22T07:18:34.924 INFO:tasks.workunit.client.0.smithi081.stdout:6/346: sync 2024-03-22T07:18:34.924 INFO:tasks.workunit.client.0.smithi081.stdout:6/347: fsync d3/da/de/f1f 0 2024-03-22T07:18:34.924 INFO:tasks.workunit.client.0.smithi081.stdout:4/491: write f3 [1883842,105200] 0 2024-03-22T07:18:34.924 INFO:tasks.workunit.client.0.smithi081.stdout:4/492: fsync da/f2c 0 2024-03-22T07:18:34.924 INFO:tasks.workunit.client.0.smithi081.stdout:4/493: stat da/d18 0 2024-03-22T07:18:34.928 INFO:tasks.workunit.client.0.smithi081.stdout:4/494: chown da/d57/l67 19967 1 2024-03-22T07:18:34.928 INFO:tasks.workunit.client.0.smithi081.stdout:9/434: dread d2/f1e [0,4194304] 0 2024-03-22T07:18:34.928 INFO:tasks.workunit.client.0.smithi081.stdout:9/435: chown d2/f20 0 1 2024-03-22T07:18:34.928 INFO:tasks.workunit.client.0.smithi081.stdout:9/436: chown d2/d10/c56 78 1 2024-03-22T07:18:34.930 INFO:tasks.workunit.client.0.smithi081.stdout:9/437: creat d2/d73/d85/d37/f8a x:0 0 0 2024-03-22T07:18:34.930 INFO:tasks.workunit.client.0.smithi081.stdout:9/438: fsync d2/d2b/f2d 0 2024-03-22T07:18:34.936 INFO:tasks.workunit.client.0.smithi081.stdout:9/439: dread d2/d2b/f60 [4194304,4194304] 0 2024-03-22T07:18:34.937 INFO:tasks.workunit.client.0.smithi081.stdout:9/440: mknod d2/d78/c8b 0 2024-03-22T07:18:34.939 INFO:tasks.workunit.client.0.smithi081.stdout:9/441: link d2/f3 d2/d5/d36/f8c 0 2024-03-22T07:18:34.939 INFO:tasks.workunit.client.0.smithi081.stdout:9/442: chown d2/d73/f7e 0 1 2024-03-22T07:18:34.939 INFO:tasks.workunit.client.0.smithi081.stdout:9/443: symlink d2/d2b/l8d 0 2024-03-22T07:18:34.941 INFO:tasks.workunit.client.0.smithi081.stdout:9/444: unlink d2/d73/d85/d37/f8a 0 2024-03-22T07:18:34.942 INFO:tasks.workunit.client.0.smithi081.stdout:9/445: creat d2/d73/d85/d37/f8e x:0 0 0 2024-03-22T07:18:34.943 INFO:tasks.workunit.client.0.smithi081.stdout:9/446: symlink d2/d2b/d49/d86/l8f 0 2024-03-22T07:18:34.952 INFO:tasks.workunit.client.0.smithi081.stdout:9/447: write d2/f15 [2523426,96199] 0 2024-03-22T07:18:34.952 INFO:tasks.workunit.client.0.smithi081.stdout:4/495: dwrite da/d18/d1b/d1e/d37/f6c [0,4194304] 0 2024-03-22T07:18:34.954 INFO:tasks.workunit.client.0.smithi081.stdout:9/448: rename d2/d73/d74/f83 to d2/d5d/d58/f90 0 2024-03-22T07:18:34.954 INFO:tasks.workunit.client.0.smithi081.stdout:9/449: stat d2/d10/d30/d57/d66/f69 0 2024-03-22T07:18:34.954 INFO:tasks.workunit.client.0.smithi081.stdout:9/450: chown d2/d73/d85/d37/f8e 12139 1 2024-03-22T07:18:34.955 INFO:tasks.workunit.client.0.smithi081.stdout:4/496: creat da/d18/d42/f99 x:0 0 0 2024-03-22T07:18:34.957 INFO:tasks.workunit.client.0.smithi081.stdout:9/451: unlink d2/f5b 0 2024-03-22T07:18:34.957 INFO:tasks.workunit.client.0.smithi081.stdout:9/452: chown d2/d10/d30/d57/f7d 362761 1 2024-03-22T07:18:34.957 INFO:tasks.workunit.client.0.smithi081.stdout:9/453: truncate d2/d2b/f70 675474 0 2024-03-22T07:18:34.959 INFO:tasks.workunit.client.0.smithi081.stdout:9/454: creat d2/d73/d85/d37/f91 x:0 0 0 2024-03-22T07:18:34.960 INFO:tasks.workunit.client.0.smithi081.stdout:9/455: symlink d2/d10/d30/d57/d66/d75/l92 0 2024-03-22T07:18:34.961 INFO:tasks.workunit.client.0.smithi081.stdout:9/456: mkdir d2/d78/d93 0 2024-03-22T07:18:34.962 INFO:tasks.workunit.client.0.smithi081.stdout:9/457: mkdir d2/d10/d30/d57/d66/d75/d94 0 2024-03-22T07:18:34.983 INFO:tasks.workunit.client.0.smithi081.stdout:9/458: dwrite d2/d73/d85/d37/f8e [0,4194304] 0 2024-03-22T07:18:34.983 INFO:tasks.workunit.client.0.smithi081.stdout:4/497: dwrite da/d18/d1b/d1e/f3f [0,4194304] 0 2024-03-22T07:18:34.986 INFO:tasks.workunit.client.0.smithi081.stdout:4/498: creat da/d18/d42/f9a x:0 0 0 2024-03-22T07:18:34.986 INFO:tasks.workunit.client.0.smithi081.stdout:4/499: fsync da/d18/d1b/d26/d3e/f8a 0 2024-03-22T07:18:34.986 INFO:tasks.workunit.client.0.smithi081.stdout:9/459: read d2/d10/d30/f45 [2441854,121685] 0 2024-03-22T07:18:34.986 INFO:tasks.workunit.client.0.smithi081.stdout:9/460: write d2/d2b/d49/f5c [787663,49715] 0 2024-03-22T07:18:34.987 INFO:tasks.workunit.client.0.smithi081.stdout:4/500: creat da/d3d/f9b x:0 0 0 2024-03-22T07:18:34.988 INFO:tasks.workunit.client.0.smithi081.stdout:9/461: symlink d2/d10/d30/d57/d66/d75/l95 0 2024-03-22T07:18:34.991 INFO:tasks.workunit.client.0.smithi081.stdout:9/462: mknod d2/d10/d30/c96 0 2024-03-22T07:18:34.993 INFO:tasks.workunit.client.0.smithi081.stdout:4/501: rename da/d18/d42/d4a/c4d to da/d18/d42/c9c 0 2024-03-22T07:18:34.995 INFO:tasks.workunit.client.0.smithi081.stdout:9/463: mkdir d2/d73/d85/d7a/d97 0 2024-03-22T07:18:34.996 INFO:tasks.workunit.client.0.smithi081.stdout:9/464: dread d2/f5f [0,4194304] 0 2024-03-22T07:18:34.996 INFO:tasks.workunit.client.0.smithi081.stdout:9/465: truncate d2/d2b/f77 334608 0 2024-03-22T07:18:35.018 INFO:tasks.workunit.client.0.smithi081.stdout:4/502: dwrite da/d18/d1b/d34/f44 [0,4194304] 0 2024-03-22T07:18:35.018 INFO:tasks.workunit.client.0.smithi081.stdout:4/503: write da/f87 [168505,20044] 0 2024-03-22T07:18:35.018 INFO:tasks.workunit.client.0.smithi081.stdout:9/466: dwrite d2/d10/f35 [0,4194304] 0 2024-03-22T07:18:35.019 INFO:tasks.workunit.client.0.smithi081.stdout:9/467: mknod d2/d5d/d5a/c98 0 2024-03-22T07:18:35.036 INFO:tasks.workunit.client.0.smithi081.stdout:9/468: dwrite d2/d5/d36/f6c [0,4194304] 0 2024-03-22T07:18:35.036 INFO:tasks.workunit.client.0.smithi081.stdout:9/469: write d2/d73/d85/f59 [1659107,5714] 0 2024-03-22T07:18:35.040 INFO:tasks.workunit.client.0.smithi081.stdout:9/470: symlink d2/d73/d85/d7a/d97/l99 0 2024-03-22T07:18:35.053 INFO:tasks.workunit.client.0.smithi081.stdout:9/471: dwrite d2/d2b/f3b [0,4194304] 0 2024-03-22T07:18:35.053 INFO:tasks.workunit.client.0.smithi081.stdout:9/472: chown d2/f11 130947826 1 2024-03-22T07:18:35.054 INFO:tasks.workunit.client.0.smithi081.stdout:9/473: truncate d2/f25 1139912 0 2024-03-22T07:18:35.056 INFO:tasks.workunit.client.0.smithi081.stdout:9/474: unlink d2/l54 0 2024-03-22T07:18:35.057 INFO:tasks.workunit.client.0.smithi081.stdout:9/475: creat d2/d2b/d49/d86/f9a x:0 0 0 2024-03-22T07:18:35.064 INFO:tasks.workunit.client.0.smithi081.stdout:9/476: dread d2/d5/d36/f8c [0,4194304] 0 2024-03-22T07:18:35.075 INFO:tasks.workunit.client.0.smithi081.stdout:9/477: dwrite d2/d5/f68 [0,4194304] 0 2024-03-22T07:18:35.077 INFO:tasks.workunit.client.0.smithi081.stdout:9/478: mknod d2/d10/d30/d57/c9b 0 2024-03-22T07:18:35.079 INFO:tasks.workunit.client.0.smithi081.stdout:9/479: rename d2/d10/d30/d57/d66/f80 to d2/d10/d30/f9c 0 2024-03-22T07:18:35.100 INFO:tasks.workunit.client.0.smithi081.stdout:9/480: dwrite d2/d2b/f60 [8388608,4194304] 0 2024-03-22T07:18:35.122 INFO:tasks.workunit.client.0.smithi081.stdout:9/481: dwrite d2/d2b/f41 [0,4194304] 0 2024-03-22T07:18:35.122 INFO:tasks.workunit.client.0.smithi081.stdout:9/482: rename d2/d73/d85/d37 to d2/d5/d9d 0 2024-03-22T07:18:35.131 INFO:tasks.workunit.client.0.smithi081.stdout:8/381: sync 2024-03-22T07:18:35.131 INFO:tasks.workunit.client.0.smithi081.stdout:3/292: sync 2024-03-22T07:18:35.136 INFO:tasks.workunit.client.0.smithi081.stdout:3/293: mknod db/d23/c63 0 2024-03-22T07:18:35.138 INFO:tasks.workunit.client.0.smithi081.stdout:3/294: rename db/de/f26 to db/de/f64 0 2024-03-22T07:18:35.144 INFO:tasks.workunit.client.0.smithi081.stdout:3/295: write db/d23/f28 [704129,55382] 0 2024-03-22T07:18:35.144 INFO:tasks.workunit.client.0.smithi081.stdout:3/296: chown db/d2e/l56 2957 1 2024-03-22T07:18:35.144 INFO:tasks.workunit.client.0.smithi081.stdout:3/297: chown db/de/l14 53 1 2024-03-22T07:18:35.145 INFO:tasks.workunit.client.0.smithi081.stdout:3/298: truncate db/d38/f47 883202 0 2024-03-22T07:18:35.145 INFO:tasks.workunit.client.0.smithi081.stdout:3/299: link db/d2e/l5e db/d2e/d3a/d5d/l65 0 2024-03-22T07:18:35.147 INFO:tasks.workunit.client.0.smithi081.stdout:3/300: symlink db/de/l66 0 2024-03-22T07:18:35.147 INFO:tasks.workunit.client.0.smithi081.stdout:9/483: dwrite d2/d73/d85/f2f [0,4194304] 0 2024-03-22T07:18:35.147 INFO:tasks.workunit.client.0.smithi081.stdout:8/382: dwrite d2/d3/d28/d2b/f46 [0,4194304] 0 2024-03-22T07:18:35.149 INFO:tasks.workunit.client.0.smithi081.stdout:9/484: rename d2/d10/f14 to d2/d10/d30/d57/f9e 0 2024-03-22T07:18:35.153 INFO:tasks.workunit.client.0.smithi081.stdout:8/383: fsync d2/f11 0 2024-03-22T07:18:35.155 INFO:tasks.workunit.client.0.smithi081.stdout:9/485: rmdir d2/d10/d30/d57/d66/d75 39 2024-03-22T07:18:35.155 INFO:tasks.workunit.client.0.smithi081.stdout:9/486: write d2/d10/d30/f45 [2172009,127216] 0 2024-03-22T07:18:35.155 INFO:tasks.workunit.client.0.smithi081.stdout:9/487: read - d2/d2b/f81 zero size 2024-03-22T07:18:35.158 INFO:tasks.workunit.client.0.smithi081.stdout:9/488: creat d2/d78/d93/f9f x:0 0 0 2024-03-22T07:18:35.160 INFO:tasks.workunit.client.0.smithi081.stdout:9/489: creat d2/d73/d85/fa0 x:0 0 0 2024-03-22T07:18:35.160 INFO:tasks.workunit.client.0.smithi081.stdout:9/490: dread d2/d2b/f77 [0,4194304] 0 2024-03-22T07:18:35.164 INFO:tasks.workunit.client.0.smithi081.stdout:8/384: write d2/d3/d28/d2b/f67 [1205151,50568] 0 2024-03-22T07:18:35.166 INFO:tasks.workunit.client.0.smithi081.stdout:8/385: mkdir d2/d3/d19/d1e/d13/d66/d70/d86 0 2024-03-22T07:18:35.172 INFO:tasks.workunit.client.0.smithi081.stdout:9/491: write d2/d2b/d49/f4f [4830957,130284] 0 2024-03-22T07:18:35.173 INFO:tasks.workunit.client.0.smithi081.stdout:9/492: symlink d2/d78/la1 0 2024-03-22T07:18:35.174 INFO:tasks.workunit.client.0.smithi081.stdout:3/301: dwrite db/f5c [0,4194304] 0 2024-03-22T07:18:35.179 INFO:tasks.workunit.client.0.smithi081.stdout:8/386: dread d2/d3/d28/f30 [0,4194304] 0 2024-03-22T07:18:35.203 INFO:tasks.workunit.client.0.smithi081.stdout:3/302: dwrite db/f2f [0,4194304] 0 2024-03-22T07:18:35.204 INFO:tasks.workunit.client.0.smithi081.stdout:9/493: dwrite d2/fe [0,4194304] 0 2024-03-22T07:18:35.204 INFO:tasks.workunit.client.0.smithi081.stdout:9/494: write d2/d73/d85/d7a/f88 [293282,93708] 0 2024-03-22T07:18:35.205 INFO:tasks.workunit.client.0.smithi081.stdout:9/495: write d2/fe [13295221,36231] 0 2024-03-22T07:18:35.205 INFO:tasks.workunit.client.0.smithi081.stdout:8/387: dwrite d2/d3/d19/d1e/f5c [0,4194304] 0 2024-03-22T07:18:35.210 INFO:tasks.workunit.client.0.smithi081.stdout:3/303: dread db/de/f29 [0,4194304] 0 2024-03-22T07:18:35.210 INFO:tasks.workunit.client.0.smithi081.stdout:8/388: readlink d2/d3/d19/d1e/d13/d22/d2f/l54 0 2024-03-22T07:18:35.211 INFO:tasks.workunit.client.0.smithi081.stdout:3/304: readlink db/d2e/l3b 0 2024-03-22T07:18:35.212 INFO:tasks.workunit.client.0.smithi081.stdout:8/389: symlink d2/d3/d19/d1e/d13/d22/d2f/d71/l87 0 2024-03-22T07:18:35.213 INFO:tasks.workunit.client.0.smithi081.stdout:9/496: dread d2/d5/f68 [0,4194304] 0 2024-03-22T07:18:35.213 INFO:tasks.workunit.client.0.smithi081.stdout:8/390: mknod d2/d3/d19/d1e/d13/d22/d2f/d71/d7c/c88 0 2024-03-22T07:18:35.214 INFO:tasks.workunit.client.0.smithi081.stdout:9/497: readlink d2/l3c 0 2024-03-22T07:18:35.215 INFO:tasks.workunit.client.0.smithi081.stdout:8/391: read - d2/d3/d28/d2b/f4b zero size 2024-03-22T07:18:35.216 INFO:tasks.workunit.client.0.smithi081.stdout:3/305: dread db/d23/f28 [0,4194304] 0 2024-03-22T07:18:35.217 INFO:tasks.workunit.client.0.smithi081.stdout:9/498: symlink d2/la2 0 2024-03-22T07:18:35.218 INFO:tasks.workunit.client.0.smithi081.stdout:8/392: mknod d2/d3/d19/d1e/d63/c89 0 2024-03-22T07:18:35.218 INFO:tasks.workunit.client.0.smithi081.stdout:8/393: fdatasync d2/d3/d19/d1e/f3c 0 2024-03-22T07:18:35.218 INFO:tasks.workunit.client.0.smithi081.stdout:8/394: chown d2/d3/d28/d2b/d61 2521794 1 2024-03-22T07:18:35.219 INFO:tasks.workunit.client.0.smithi081.stdout:3/306: creat db/d23/f67 x:0 0 0 2024-03-22T07:18:35.222 INFO:tasks.workunit.client.0.smithi081.stdout:9/499: dread d2/f1e [0,4194304] 0 2024-03-22T07:18:35.222 INFO:tasks.workunit.client.0.smithi081.stdout:9/500: chown d2/d5/fd 1747 1 2024-03-22T07:18:35.223 INFO:tasks.workunit.client.0.smithi081.stdout:9/501: write d2/d2b/f77 [272182,121189] 0 2024-03-22T07:18:35.223 INFO:tasks.workunit.client.0.smithi081.stdout:9/502: write d2/d2b/d49/d86/f9a [692227,16066] 0 2024-03-22T07:18:35.223 INFO:tasks.workunit.client.0.smithi081.stdout:9/503: chown d2/d5/d36/l51 37246103 1 2024-03-22T07:18:35.229 INFO:tasks.workunit.client.0.smithi081.stdout:8/395: dread d2/d3/d23/f52 [0,4194304] 0 2024-03-22T07:18:35.230 INFO:tasks.workunit.client.0.smithi081.stdout:8/396: rename d2/d3/d19/d1e/f56 to d2/d3/d19/d1e/f8a 0 2024-03-22T07:18:35.231 INFO:tasks.workunit.client.0.smithi081.stdout:3/307: read db/d2e/f3e [1110927,95799] 0 2024-03-22T07:18:35.231 INFO:tasks.workunit.client.0.smithi081.stdout:8/397: mknod d2/d3/d19/d1e/d13/c8b 0 2024-03-22T07:18:35.235 INFO:tasks.workunit.client.0.smithi081.stdout:9/504: read d2/d5/fd [2863883,23509] 0 2024-03-22T07:18:35.235 INFO:tasks.workunit.client.0.smithi081.stdout:9/505: chown d2/d10/d30/d57/f7d 4156719 1 2024-03-22T07:18:35.236 INFO:tasks.workunit.client.0.smithi081.stdout:9/506: fsync d2/d73/d85/f46 0 2024-03-22T07:18:35.236 INFO:tasks.workunit.client.0.smithi081.stdout:9/507: stat d2/d10/d30 0 2024-03-22T07:18:35.236 INFO:tasks.workunit.client.0.smithi081.stdout:9/508: stat d2/d5d/d5a/c72 0 2024-03-22T07:18:35.313 INFO:tasks.workunit.client.0.smithi081.stdout:8/398: getdents d2/d3/d19/d1e 0 2024-03-22T07:18:35.314 INFO:tasks.workunit.client.0.smithi081.stdout:0/430: sync 2024-03-22T07:18:35.315 INFO:tasks.workunit.client.0.smithi081.stdout:1/331: sync 2024-03-22T07:18:35.315 INFO:tasks.workunit.client.0.smithi081.stdout:6/348: sync 2024-03-22T07:18:35.315 INFO:tasks.workunit.client.0.smithi081.stdout:5/571: sync 2024-03-22T07:18:35.315 INFO:tasks.workunit.client.0.smithi081.stdout:2/291: sync 2024-03-22T07:18:35.315 INFO:tasks.workunit.client.0.smithi081.stdout:7/501: sync 2024-03-22T07:18:35.315 INFO:tasks.workunit.client.0.smithi081.stdout:4/504: sync 2024-03-22T07:18:35.316 INFO:tasks.workunit.client.0.smithi081.stdout:5/572: dread d4/f89 [0,4194304] 0 2024-03-22T07:18:35.317 INFO:tasks.workunit.client.0.smithi081.stdout:1/332: mkdir d6/d18/d1f/d4c/d70 0 2024-03-22T07:18:35.318 INFO:tasks.workunit.client.0.smithi081.stdout:1/333: getdents d6/d18/d41/d49 0 2024-03-22T07:18:35.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:35 smithi081 ceph-mon[113965]: pgmap v213: 65 pgs: 65 active+clean; 1.0 GiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 20 MiB/s rd, 22 MiB/s wr, 681 op/s 2024-03-22T07:18:35.320 INFO:tasks.workunit.client.0.smithi081.stdout:2/292: truncate da/d11/f4d 268581 0 2024-03-22T07:18:35.323 INFO:tasks.workunit.client.0.smithi081.stdout:2/293: symlink da/db/d39/d47/l5d 0 2024-03-22T07:18:35.324 INFO:tasks.workunit.client.0.smithi081.stdout:4/505: mknod da/d18/d42/d4f/c9d 0 2024-03-22T07:18:35.325 INFO:tasks.workunit.client.0.smithi081.stdout:7/502: truncate d9/d10/d2a/fa9 2575775 0 2024-03-22T07:18:35.330 INFO:tasks.workunit.client.0.smithi081.stdout:1/334: creat d6/d18/d1f/f71 x:0 0 0 2024-03-22T07:18:35.330 INFO:tasks.workunit.client.0.smithi081.stdout:6/349: creat d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f6e x:0 0 0 2024-03-22T07:18:35.333 INFO:tasks.workunit.client.0.smithi081.stdout:6/350: creat d3/da/de/d31/d40/d50/f6f x:0 0 0 2024-03-22T07:18:35.333 INFO:tasks.workunit.client.0.smithi081.stdout:6/351: fdatasync d3/da/de/f41 0 2024-03-22T07:18:35.334 INFO:tasks.workunit.client.0.smithi081.stdout:7/503: symlink d9/d10/d16/d84/lae 0 2024-03-22T07:18:35.335 INFO:tasks.workunit.client.0.smithi081.stdout:4/506: dread da/d18/d42/d4f/f58 [0,4194304] 0 2024-03-22T07:18:35.335 INFO:tasks.workunit.client.0.smithi081.stdout:8/399: dwrite d2/d3/d19/d1e/f3c [0,4194304] 0 2024-03-22T07:18:35.336 INFO:tasks.workunit.client.0.smithi081.stdout:6/352: symlink d3/da/l70 0 2024-03-22T07:18:35.339 INFO:tasks.workunit.client.0.smithi081.stdout:4/507: rename da/f2c to da/d18/d6e/f9e 0 2024-03-22T07:18:35.340 INFO:tasks.workunit.client.0.smithi081.stdout:8/400: rename d2/c4d to d2/c8c 0 2024-03-22T07:18:35.341 INFO:tasks.workunit.client.0.smithi081.stdout:6/353: creat d3/da/de/d31/d40/d50/d63/d26/f71 x:0 0 0 2024-03-22T07:18:35.342 INFO:tasks.workunit.client.0.smithi081.stdout:4/508: symlink da/d18/d42/d4f/d7c/l9f 0 2024-03-22T07:18:35.346 INFO:tasks.workunit.client.0.smithi081.stdout:4/509: link da/df/l3c da/d18/d1b/d1e/d37/la0 0 2024-03-22T07:18:35.346 INFO:tasks.workunit.client.0.smithi081.stdout:5/573: dwrite d4/d9/f91 [0,4194304] 0 2024-03-22T07:18:35.346 INFO:tasks.workunit.client.0.smithi081.stdout:6/354: mknod d3/da/de/d31/d40/d50/d63/d26/d3f/c72 0 2024-03-22T07:18:35.348 INFO:tasks.workunit.client.0.smithi081.stdout:4/510: creat da/d18/d1b/d26/d3e/fa1 x:0 0 0 2024-03-22T07:18:35.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:35 smithi149 ceph-mon[83447]: pgmap v213: 65 pgs: 65 active+clean; 1.0 GiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 20 MiB/s rd, 22 MiB/s wr, 681 op/s 2024-03-22T07:18:35.352 INFO:tasks.workunit.client.0.smithi081.stdout:7/504: dread d9/f75 [0,4194304] 0 2024-03-22T07:18:35.353 INFO:tasks.workunit.client.0.smithi081.stdout:5/574: mknod d4/d10/cc1 0 2024-03-22T07:18:35.354 INFO:tasks.workunit.client.0.smithi081.stdout:4/511: mknod da/d57/ca2 0 2024-03-22T07:18:35.355 INFO:tasks.workunit.client.0.smithi081.stdout:5/575: creat d4/d9/fc2 x:0 0 0 2024-03-22T07:18:35.357 INFO:tasks.workunit.client.0.smithi081.stdout:5/576: getdents d4/d10/d20/d4f 0 2024-03-22T07:18:35.358 INFO:tasks.workunit.client.0.smithi081.stdout:4/512: symlink da/d18/d1b/d47/la3 0 2024-03-22T07:18:35.359 INFO:tasks.workunit.client.0.smithi081.stdout:5/577: getdents d4/d10/d20/db2/d2a 0 2024-03-22T07:18:35.360 INFO:tasks.workunit.client.0.smithi081.stdout:4/513: rmdir da/d18/d1b/d26/d32/d92 0 2024-03-22T07:18:35.362 INFO:tasks.workunit.client.0.smithi081.stdout:4/514: link da/d18/d1b/f40 da/d18/d1b/d1e/d37/d1c/d74/d98/fa4 0 2024-03-22T07:18:35.363 INFO:tasks.workunit.client.0.smithi081.stdout:5/578: rmdir d4/d10/d20/d25/da3/db5 39 2024-03-22T07:18:35.365 INFO:tasks.workunit.client.0.smithi081.stdout:4/515: creat da/df/fa5 x:0 0 0 2024-03-22T07:18:35.366 INFO:tasks.workunit.client.0.smithi081.stdout:5/579: symlink d4/d10/d20/d4f/lc3 0 2024-03-22T07:18:35.366 INFO:tasks.workunit.client.0.smithi081.stdout:5/580: write d4/df/f6d [333148,75272] 0 2024-03-22T07:18:35.368 INFO:tasks.workunit.client.0.smithi081.stdout:7/505: dread d9/d10/d16/d1e/d27/d6d/fa5 [0,4194304] 0 2024-03-22T07:18:35.368 INFO:tasks.workunit.client.0.smithi081.stdout:2/294: dwrite da/d11/d21/f33 [4194304,4194304] 0 2024-03-22T07:18:35.368 INFO:tasks.workunit.client.0.smithi081.stdout:2/295: write f6 [2805,115389] 0 2024-03-22T07:18:35.369 INFO:tasks.workunit.client.0.smithi081.stdout:5/581: unlink d4/d75/f77 0 2024-03-22T07:18:35.371 INFO:tasks.workunit.client.0.smithi081.stdout:2/296: symlink da/d56/l5e 0 2024-03-22T07:18:35.371 INFO:tasks.workunit.client.0.smithi081.stdout:1/335: dwrite d6/d18/d41/d49/d56/d30/f3d [0,4194304] 0 2024-03-22T07:18:35.371 INFO:tasks.workunit.client.0.smithi081.stdout:8/401: dwrite d2/d3/d28/f82 [0,4194304] 0 2024-03-22T07:18:35.373 INFO:tasks.workunit.client.0.smithi081.stdout:0/431: dwrite d3/dc/f2a [4194304,4194304] 0 2024-03-22T07:18:35.373 INFO:tasks.workunit.client.0.smithi081.stdout:7/506: getdents d9/d10/d53 0 2024-03-22T07:18:35.373 INFO:tasks.workunit.client.0.smithi081.stdout:7/507: write d9/d10/d16/d1e/d27/f57 [2860795,103391] 0 2024-03-22T07:18:35.376 INFO:tasks.workunit.client.0.smithi081.stdout:8/402: unlink d2/d3/d19/d1e/d63/f80 0 2024-03-22T07:18:35.380 INFO:tasks.workunit.client.0.smithi081.stdout:2/297: symlink da/db/l5f 0 2024-03-22T07:18:35.382 INFO:tasks.workunit.client.0.smithi081.stdout:8/403: rename d2/d3/d19/d1e/d13/d22/l32 to d2/d3/d19/d1e/d13/d66/d70/d86/l8d 0 2024-03-22T07:18:35.385 INFO:tasks.workunit.client.0.smithi081.stdout:1/336: read d6/d18/d1f/d24/f2b [1814710,10946] 0 2024-03-22T07:18:35.385 INFO:tasks.workunit.client.0.smithi081.stdout:1/337: write d6/d18/d1f/d32/f4b [3274871,120955] 0 2024-03-22T07:18:35.385 INFO:tasks.workunit.client.0.smithi081.stdout:1/338: readlink l5 0 2024-03-22T07:18:35.385 INFO:tasks.workunit.client.0.smithi081.stdout:1/339: readlink l5 0 2024-03-22T07:18:35.385 INFO:tasks.workunit.client.0.smithi081.stdout:1/340: fdatasync d6/d18/d41/f46 0 2024-03-22T07:18:35.385 INFO:tasks.workunit.client.0.smithi081.stdout:2/298: mknod da/db/d39/d47/d38/c60 0 2024-03-22T07:18:35.386 INFO:tasks.workunit.client.0.smithi081.stdout:1/341: creat d6/d18/d1f/f72 x:0 0 0 2024-03-22T07:18:35.386 INFO:tasks.workunit.client.0.smithi081.stdout:1/342: chown d6/d18/d1f/d4c/d70 217 1 2024-03-22T07:18:35.386 INFO:tasks.workunit.client.0.smithi081.stdout:1/343: read - d6/f67 zero size 2024-03-22T07:18:35.390 INFO:tasks.workunit.client.0.smithi081.stdout:1/344: truncate d6/d18/d41/d49/d56/f4a 975603 0 2024-03-22T07:18:35.392 INFO:tasks.workunit.client.0.smithi081.stdout:4/516: dwrite da/d18/d1b/d26/d3e/f8a [0,4194304] 0 2024-03-22T07:18:35.392 INFO:tasks.workunit.client.0.smithi081.stdout:5/582: dwrite d4/d10/d20/db2/d2a/f9b [0,4194304] 0 2024-03-22T07:18:35.392 INFO:tasks.workunit.client.0.smithi081.stdout:5/583: chown d4/d10/d20/db2/cba 470 1 2024-03-22T07:18:35.392 INFO:tasks.workunit.client.0.smithi081.stdout:5/584: read - d4/d10/d20/d25/da3/fa7 zero size 2024-03-22T07:18:35.393 INFO:tasks.workunit.client.0.smithi081.stdout:5/585: chown d4/df/f6d 433 1 2024-03-22T07:18:35.393 INFO:tasks.workunit.client.0.smithi081.stdout:5/586: chown d4/d9/f2c 92 1 2024-03-22T07:18:35.394 INFO:tasks.workunit.client.0.smithi081.stdout:8/404: symlink d2/d3/d19/d1e/d13/l8e 0 2024-03-22T07:18:35.395 INFO:tasks.workunit.client.0.smithi081.stdout:8/405: fdatasync d2/d3/d28/d2b/f7a 0 2024-03-22T07:18:35.398 INFO:tasks.workunit.client.0.smithi081.stdout:5/587: creat d4/d10/d20/db2/d2f/fc4 x:0 0 0 2024-03-22T07:18:35.400 INFO:tasks.workunit.client.0.smithi081.stdout:7/508: rename d9/d10/d4b/d61/l8f to d9/d10/d16/d1e/d27/d6d/laf 0 2024-03-22T07:18:35.402 INFO:tasks.workunit.client.0.smithi081.stdout:8/406: link d2/d3/d19/d1e/d13/d22/d38/d78/c79 d2/d3/d19/d1e/d13/d22/d2f/d73/c8f 0 2024-03-22T07:18:35.402 INFO:tasks.workunit.client.0.smithi081.stdout:8/407: chown d2/d3/d19/d1e/d13/f47 325 1 2024-03-22T07:18:35.402 INFO:tasks.workunit.client.0.smithi081.stdout:8/408: stat d2/d3/d19/d1e/d13/d22/d2f/d73/c83 0 2024-03-22T07:18:35.404 INFO:tasks.workunit.client.0.smithi081.stdout:4/517: rename da/d18/d1b/d26/l29 to da/d18/d42/d4f/d7c/d90/la6 0 2024-03-22T07:18:35.404 INFO:tasks.workunit.client.0.smithi081.stdout:4/518: chown da/d18/d42/f99 55759 1 2024-03-22T07:18:35.405 INFO:tasks.workunit.client.0.smithi081.stdout:7/509: creat d9/d10/d16/d1e/d27/fb0 x:0 0 0 2024-03-22T07:18:35.407 INFO:tasks.workunit.client.0.smithi081.stdout:5/588: symlink d4/d10/d20/db2/lc5 0 2024-03-22T07:18:35.410 INFO:tasks.workunit.client.0.smithi081.stdout:8/409: symlink d2/d3/d28/d2b/d61/l90 0 2024-03-22T07:18:35.410 INFO:tasks.workunit.client.0.smithi081.stdout:8/410: stat d2/d3/d19 0 2024-03-22T07:18:35.411 INFO:tasks.workunit.client.0.smithi081.stdout:2/299: dwrite da/db/d39/d47/f37 [0,4194304] 0 2024-03-22T07:18:35.411 INFO:tasks.workunit.client.0.smithi081.stdout:1/345: dwrite d6/d18/d1f/d24/f26 [0,4194304] 0 2024-03-22T07:18:35.414 INFO:tasks.workunit.client.0.smithi081.stdout:4/519: mknod da/ca7 0 2024-03-22T07:18:35.414 INFO:tasks.workunit.client.0.smithi081.stdout:4/520: dread - da/d18/d42/d4a/f84 zero size 2024-03-22T07:18:35.414 INFO:tasks.workunit.client.0.smithi081.stdout:6/355: write d3/da/de/d31/d40/d50/d63/d26/d3f/f7 [6454489,73058] 0 2024-03-22T07:18:35.414 INFO:tasks.workunit.client.0.smithi081.stdout:1/346: link d6/d18/d1f/d4c/f6c d6/d18/d1f/d24/d48/d5c/f73 0 2024-03-22T07:18:35.414 INFO:tasks.workunit.client.0.smithi081.stdout:1/347: fsync d6/d18/d41/d49/d56/d30/f55 0 2024-03-22T07:18:35.415 INFO:tasks.workunit.client.0.smithi081.stdout:1/348: getdents d6/d18/d60 0 2024-03-22T07:18:35.416 INFO:tasks.workunit.client.0.smithi081.stdout:8/411: creat d2/d3/d19/d1e/d13/d22/d38/f91 x:0 0 0 2024-03-22T07:18:35.420 INFO:tasks.workunit.client.0.smithi081.stdout:2/300: symlink da/db/d39/d47/l61 0 2024-03-22T07:18:35.422 INFO:tasks.workunit.client.0.smithi081.stdout:5/589: symlink d4/d10/d20/lc6 0 2024-03-22T07:18:35.422 INFO:tasks.workunit.client.0.smithi081.stdout:5/590: read - d4/d75/d98/fad zero size 2024-03-22T07:18:35.422 INFO:tasks.workunit.client.0.smithi081.stdout:0/432: write d3/dc/d33/d62/f68 [491413,85311] 0 2024-03-22T07:18:35.423 INFO:tasks.workunit.client.0.smithi081.stdout:4/521: chown da/d18/d1b/d26/d3e/l6d 5315 1 2024-03-22T07:18:35.427 INFO:tasks.workunit.client.0.smithi081.stdout:7/510: rename d9/d10/d16/f42 to d9/d3f/d52/fb1 0 2024-03-22T07:18:35.427 INFO:tasks.workunit.client.0.smithi081.stdout:6/356: chown d3/da/de/d31/c3d 1 1 2024-03-22T07:18:35.429 INFO:tasks.workunit.client.0.smithi081.stdout:2/301: stat da/db/d39/d45/c5c 0 2024-03-22T07:18:35.431 INFO:tasks.workunit.client.0.smithi081.stdout:8/412: dread d2/d3/d23/f3b [0,4194304] 0 2024-03-22T07:18:35.436 INFO:tasks.workunit.client.0.smithi081.stdout:5/591: mknod d4/df/d73/d8c/dc0/cc7 0 2024-03-22T07:18:35.437 INFO:tasks.workunit.client.0.smithi081.stdout:0/433: creat d3/d32/f8f x:0 0 0 2024-03-22T07:18:35.444 INFO:tasks.workunit.client.0.smithi081.stdout:1/349: creat d6/d18/f74 x:0 0 0 2024-03-22T07:18:35.446 INFO:tasks.workunit.client.0.smithi081.stdout:6/357: dread - d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f6e zero size 2024-03-22T07:18:35.446 INFO:tasks.workunit.client.0.smithi081.stdout:4/522: dwrite da/d18/d42/d4a/f84 [0,4194304] 0 2024-03-22T07:18:35.447 INFO:tasks.workunit.client.0.smithi081.stdout:2/302: creat da/db/d39/d47/d3b/d41/d50/f62 x:0 0 0 2024-03-22T07:18:35.447 INFO:tasks.workunit.client.0.smithi081.stdout:2/303: read da/f13 [127254,105991] 0 2024-03-22T07:18:35.447 INFO:tasks.workunit.client.0.smithi081.stdout:2/304: dread - da/db/f3f zero size 2024-03-22T07:18:35.449 INFO:tasks.workunit.client.0.smithi081.stdout:8/413: creat d2/d3/d19/d1e/d13/d22/d2f/d71/f92 x:0 0 0 2024-03-22T07:18:35.451 INFO:tasks.workunit.client.0.smithi081.stdout:0/434: fsync d3/dc/f19 0 2024-03-22T07:18:35.452 INFO:tasks.workunit.client.0.smithi081.stdout:7/511: symlink d9/d3f/d52/lb2 0 2024-03-22T07:18:35.453 INFO:tasks.workunit.client.0.smithi081.stdout:6/358: stat d3/da/l4a 0 2024-03-22T07:18:35.460 INFO:tasks.workunit.client.0.smithi081.stdout:1/350: dwrite d6/d18/d1f/d24/f5f [0,4194304] 0 2024-03-22T07:18:35.460 INFO:tasks.workunit.client.0.smithi081.stdout:8/414: creat d2/d3/d19/d1e/d13/d22/d2f/f93 x:0 0 0 2024-03-22T07:18:35.462 INFO:tasks.workunit.client.0.smithi081.stdout:5/592: rename d4/df/d22/d4d/c93 to d4/d10/d20/d25/cc8 0 2024-03-22T07:18:35.462 INFO:tasks.workunit.client.0.smithi081.stdout:1/351: creat d6/d18/d1f/d24/d48/d6e/f75 x:0 0 0 2024-03-22T07:18:35.464 INFO:tasks.workunit.client.0.smithi081.stdout:0/435: fdatasync d3/dc/f43 0 2024-03-22T07:18:35.470 INFO:tasks.workunit.client.0.smithi081.stdout:5/593: rmdir d4/df/da8 0 2024-03-22T07:18:35.470 INFO:tasks.workunit.client.0.smithi081.stdout:4/523: write da/df/d38/f76 [1119548,117651] 0 2024-03-22T07:18:35.470 INFO:tasks.workunit.client.0.smithi081.stdout:4/524: write da/df/d38/f68 [285822,53833] 0 2024-03-22T07:18:35.472 INFO:tasks.workunit.client.0.smithi081.stdout:1/352: mkdir d6/d18/d1f/d76 0 2024-03-22T07:18:35.475 INFO:tasks.workunit.client.0.smithi081.stdout:1/353: chown d6/d18/f21 41299917 1 2024-03-22T07:18:35.477 INFO:tasks.workunit.client.0.smithi081.stdout:1/354: creat d6/d18/d1f/d24/d48/d5c/f77 x:0 0 0 2024-03-22T07:18:35.477 INFO:tasks.workunit.client.0.smithi081.stdout:6/359: symlink d3/da/de/d31/d40/d50/l73 0 2024-03-22T07:18:35.477 INFO:tasks.workunit.client.0.smithi081.stdout:4/525: chown da/d18/d1b/l4e 11166248 1 2024-03-22T07:18:35.478 INFO:tasks.workunit.client.0.smithi081.stdout:1/355: mkdir d6/d18/d78 0 2024-03-22T07:18:35.479 INFO:tasks.workunit.client.0.smithi081.stdout:1/356: chown d6/d18/f28 0 1 2024-03-22T07:18:35.479 INFO:tasks.workunit.client.0.smithi081.stdout:1/357: dread - d6/d18/d1f/d32/f59 zero size 2024-03-22T07:18:35.479 INFO:tasks.workunit.client.0.smithi081.stdout:1/358: write d6/d18/d1f/d24/f3f [4284709,192] 0 2024-03-22T07:18:35.479 INFO:tasks.workunit.client.0.smithi081.stdout:0/436: symlink d3/dc/d2b/l90 0 2024-03-22T07:18:35.480 INFO:tasks.workunit.client.0.smithi081.stdout:4/526: creat da/d18/d6e/fa8 x:0 0 0 2024-03-22T07:18:35.481 INFO:tasks.workunit.client.0.smithi081.stdout:0/437: rmdir d3/dc/d40 39 2024-03-22T07:18:35.491 INFO:tasks.workunit.client.0.smithi081.stdout:7/512: dread d9/d10/d16/d1e/f18 [0,4194304] 0 2024-03-22T07:18:35.494 INFO:tasks.workunit.client.0.smithi081.stdout:8/415: dwrite f1 [0,4194304] 0 2024-03-22T07:18:35.496 INFO:tasks.workunit.client.0.smithi081.stdout:1/359: dread d6/f12 [0,4194304] 0 2024-03-22T07:18:35.496 INFO:tasks.workunit.client.0.smithi081.stdout:1/360: readlink d6/l14 0 2024-03-22T07:18:35.496 INFO:tasks.workunit.client.0.smithi081.stdout:8/416: link d2/d3/d19/d1e/d13/d22/d2f/d73/c8f d2/d3/d19/d1e/d13/d22/d2f/d73/c94 0 2024-03-22T07:18:35.496 INFO:tasks.workunit.client.0.smithi081.stdout:8/417: truncate d2/d3/d19/d1e/d13/d22/f3a 597235 0 2024-03-22T07:18:35.509 INFO:tasks.workunit.client.0.smithi081.stdout:5/594: dwrite d4/d10/d20/d25/f9e [0,4194304] 0 2024-03-22T07:18:35.509 INFO:tasks.workunit.client.0.smithi081.stdout:5/595: readlink d4/d10/d20/db2/d2f/l8a 0 2024-03-22T07:18:35.509 INFO:tasks.workunit.client.0.smithi081.stdout:7/513: write d9/d10/d16/f74 [258974,99995] 0 2024-03-22T07:18:35.509 INFO:tasks.workunit.client.0.smithi081.stdout:7/514: chown d9/c12 80714 1 2024-03-22T07:18:35.513 INFO:tasks.workunit.client.0.smithi081.stdout:7/515: mkdir d9/d3f/db3 0 2024-03-22T07:18:35.515 INFO:tasks.workunit.client.0.smithi081.stdout:1/361: read d6/d18/d1f/d24/f39 [1434675,99497] 0 2024-03-22T07:18:35.515 INFO:tasks.workunit.client.0.smithi081.stdout:1/362: creat d6/d18/d1f/d76/f79 x:0 0 0 2024-03-22T07:18:35.525 INFO:tasks.workunit.client.0.smithi081.stdout:5/596: dread d4/d10/d20/db2/f28 [0,4194304] 0 2024-03-22T07:18:35.530 INFO:tasks.workunit.client.0.smithi081.stdout:7/516: mknod d9/d14/cb4 0 2024-03-22T07:18:35.530 INFO:tasks.workunit.client.0.smithi081.stdout:6/360: dwrite d3/da/de/f28 [0,4194304] 0 2024-03-22T07:18:35.534 INFO:tasks.workunit.client.0.smithi081.stdout:5/597: mkdir d4/df/d73/d8c/dc0/dc9 0 2024-03-22T07:18:35.534 INFO:tasks.workunit.client.0.smithi081.stdout:5/598: readlink d4/df/d73/d8c/la0 0 2024-03-22T07:18:35.534 INFO:tasks.workunit.client.0.smithi081.stdout:2/305: dwrite da/f28 [0,4194304] 0 2024-03-22T07:18:35.534 INFO:tasks.workunit.client.0.smithi081.stdout:7/517: creat d9/d10/d53/fb5 x:0 0 0 2024-03-22T07:18:35.535 INFO:tasks.workunit.client.0.smithi081.stdout:5/599: rmdir d4/d10/d20/d4f 39 2024-03-22T07:18:35.537 INFO:tasks.workunit.client.0.smithi081.stdout:5/600: dread d4/d10/d20/db2/fa2 [0,4194304] 0 2024-03-22T07:18:35.537 INFO:tasks.workunit.client.0.smithi081.stdout:6/361: symlink d3/da/de/d31/d40/d50/d63/d26/l74 0 2024-03-22T07:18:35.539 INFO:tasks.workunit.client.0.smithi081.stdout:7/518: mkdir d9/d10/d16/db6 0 2024-03-22T07:18:35.542 INFO:tasks.workunit.client.0.smithi081.stdout:6/362: mkdir d3/da/de/d31/d40/d75 0 2024-03-22T07:18:35.542 INFO:tasks.workunit.client.0.smithi081.stdout:6/363: stat d3/da/de/f5b 0 2024-03-22T07:18:35.543 INFO:tasks.workunit.client.0.smithi081.stdout:6/364: creat d3/da/de/d31/d40/d50/f76 x:0 0 0 2024-03-22T07:18:35.545 INFO:tasks.workunit.client.0.smithi081.stdout:6/365: getdents d3/da/de/d31 0 2024-03-22T07:18:35.545 INFO:tasks.workunit.client.0.smithi081.stdout:0/438: dwrite d3/dc/d2b/f2f [0,4194304] 0 2024-03-22T07:18:35.546 INFO:tasks.workunit.client.0.smithi081.stdout:5/601: write d4/d10/d20/d25/f6e [68863,96344] 0 2024-03-22T07:18:35.546 INFO:tasks.workunit.client.0.smithi081.stdout:6/366: write d3/da/de/d31/d40/d50/d63/d26/f33 [3879813,80459] 0 2024-03-22T07:18:35.548 INFO:tasks.workunit.client.0.smithi081.stdout:4/527: dwrite da/d18/d42/d4f/f6a [0,4194304] 0 2024-03-22T07:18:35.548 INFO:tasks.workunit.client.0.smithi081.stdout:4/528: chown da/df/f65 467 1 2024-03-22T07:18:35.548 INFO:tasks.workunit.client.0.smithi081.stdout:1/363: dwrite d6/d18/d41/d49/d56/d30/f68 [0,4194304] 0 2024-03-22T07:18:35.549 INFO:tasks.workunit.client.0.smithi081.stdout:0/439: truncate d3/dc/f43 189628 0 2024-03-22T07:18:35.564 INFO:tasks.workunit.client.0.smithi081.stdout:5/602: symlink d4/d75/lca 0 2024-03-22T07:18:35.567 INFO:tasks.workunit.client.0.smithi081.stdout:5/603: creat d4/d75/da5/fcb x:0 0 0 2024-03-22T07:18:35.568 INFO:tasks.workunit.client.0.smithi081.stdout:6/367: link d3/da/l4a d3/da/de/d31/d40/d50/d63/d26/d52/l77 0 2024-03-22T07:18:35.572 INFO:tasks.workunit.client.0.smithi081.stdout:6/368: stat d3/da/de/d31/d40/d50/d63/d26/d3f/c2f 0 2024-03-22T07:18:35.572 INFO:tasks.workunit.client.0.smithi081.stdout:6/369: dread - d3/da/de/f5b zero size 2024-03-22T07:18:35.573 INFO:tasks.workunit.client.0.smithi081.stdout:5/604: getdents d4/d10/d20/db2/d2f 0 2024-03-22T07:18:35.573 INFO:tasks.workunit.client.0.smithi081.stdout:4/529: dread da/d18/d1b/d1e/f3b [0,4194304] 0 2024-03-22T07:18:35.578 INFO:tasks.workunit.client.0.smithi081.stdout:4/530: unlink da/d18/d42/d4f/l89 0 2024-03-22T07:18:35.579 INFO:tasks.workunit.client.0.smithi081.stdout:6/370: link d3/da/de/d31/d40/d50/d63/f68 d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f78 0 2024-03-22T07:18:35.580 INFO:tasks.workunit.client.0.smithi081.stdout:4/531: symlink da/d18/d1b/d26/d32/la9 0 2024-03-22T07:18:35.580 INFO:tasks.workunit.client.0.smithi081.stdout:4/532: write da/df/d38/f7f [234971,3298] 0 2024-03-22T07:18:35.580 INFO:tasks.workunit.client.0.smithi081.stdout:6/371: symlink d3/da/de/d31/l79 0 2024-03-22T07:18:35.581 INFO:tasks.workunit.client.0.smithi081.stdout:6/372: creat d3/da/de/d31/d40/d50/f7a x:0 0 0 2024-03-22T07:18:35.585 INFO:tasks.workunit.client.0.smithi081.stdout:1/364: write d6/f35 [2483438,11946] 0 2024-03-22T07:18:35.586 INFO:tasks.workunit.client.0.smithi081.stdout:2/306: dwrite da/db/f3f [0,4194304] 0 2024-03-22T07:18:35.586 INFO:tasks.workunit.client.0.smithi081.stdout:1/365: mknod d6/d18/d60/c7a 0 2024-03-22T07:18:35.586 INFO:tasks.workunit.client.0.smithi081.stdout:7/519: dwrite d9/dc/f90 [0,4194304] 0 2024-03-22T07:18:35.589 INFO:tasks.workunit.client.0.smithi081.stdout:1/366: creat d6/f7b x:0 0 0 2024-03-22T07:18:35.590 INFO:tasks.workunit.client.0.smithi081.stdout:1/367: mknod d6/d18/d1f/d32/c7c 0 2024-03-22T07:18:35.590 INFO:tasks.workunit.client.0.smithi081.stdout:0/440: mkdir d3/d32/d35/d91 0 2024-03-22T07:18:35.591 INFO:tasks.workunit.client.0.smithi081.stdout:1/368: mkdir d6/d18/d1f/d4c/d70/d7d 0 2024-03-22T07:18:35.591 INFO:tasks.workunit.client.0.smithi081.stdout:1/369: stat d6/d18/d1f/c33 0 2024-03-22T07:18:35.592 INFO:tasks.workunit.client.0.smithi081.stdout:1/370: write d6/d18/d1f/d4c/f6c [72992,119068] 0 2024-03-22T07:18:35.594 INFO:tasks.workunit.client.0.smithi081.stdout:7/520: creat d9/d10/fb7 x:0 0 0 2024-03-22T07:18:35.598 INFO:tasks.workunit.client.0.smithi081.stdout:2/307: write da/d11/d21/f33 [2896950,105660] 0 2024-03-22T07:18:35.599 INFO:tasks.workunit.client.0.smithi081.stdout:0/441: unlink d3/dc/d40/d69/l7c 0 2024-03-22T07:18:35.599 INFO:tasks.workunit.client.0.smithi081.stdout:0/442: fdatasync d3/d32/f70 0 2024-03-22T07:18:35.600 INFO:tasks.workunit.client.0.smithi081.stdout:7/521: mknod d9/d10/d53/d81/d59/d89/cb8 0 2024-03-22T07:18:35.605 INFO:tasks.workunit.client.0.smithi081.stdout:2/308: mkdir da/db/d39/d47/d3b/d63 0 2024-03-22T07:18:35.606 INFO:tasks.workunit.client.0.smithi081.stdout:2/309: chown da/db/d39/d46/f44 458 1 2024-03-22T07:18:35.609 INFO:tasks.workunit.client.0.smithi081.stdout:7/522: link d9/d3f/d52/lb2 d9/d10/d4b/d61/lb9 0 2024-03-22T07:18:35.609 INFO:tasks.workunit.client.0.smithi081.stdout:2/310: getdents da/d56 0 2024-03-22T07:18:35.609 INFO:tasks.workunit.client.0.smithi081.stdout:2/311: dread - da/db/d39/d46/f44 zero size 2024-03-22T07:18:35.612 INFO:tasks.workunit.client.0.smithi081.stdout:2/312: mkdir da/db/d39/d47/d3b/d64 0 2024-03-22T07:18:35.624 INFO:tasks.workunit.client.0.smithi081.stdout:4/533: dwrite da/d18/d1b/f40 [0,4194304] 0 2024-03-22T07:18:35.624 INFO:tasks.workunit.client.0.smithi081.stdout:6/373: dwrite d3/da/de/d31/d40/d50/d63/f37 [0,4194304] 0 2024-03-22T07:18:35.630 INFO:tasks.workunit.client.0.smithi081.stdout:4/534: creat da/d3d/faa x:0 0 0 2024-03-22T07:18:35.631 INFO:tasks.workunit.client.0.smithi081.stdout:4/535: chown f9 1 1 2024-03-22T07:18:35.632 INFO:tasks.workunit.client.0.smithi081.stdout:1/371: rename d6/f12 to d6/d18/d1f/d24/d48/d5c/f7e 0 2024-03-22T07:18:35.633 INFO:tasks.workunit.client.0.smithi081.stdout:1/372: stat d6/f5b 0 2024-03-22T07:18:35.633 INFO:tasks.workunit.client.0.smithi081.stdout:1/373: chown d6/d18/l53 3 1 2024-03-22T07:18:35.634 INFO:tasks.workunit.client.0.smithi081.stdout:4/536: symlink da/d18/d1b/d1e/d37/d1c/d74/lab 0 2024-03-22T07:18:35.634 INFO:tasks.workunit.client.0.smithi081.stdout:6/374: read d3/da/de/f27 [343396,31012] 0 2024-03-22T07:18:35.635 INFO:tasks.workunit.client.0.smithi081.stdout:1/374: creat d6/d18/d41/d49/d56/d30/f7f x:0 0 0 2024-03-22T07:18:35.636 INFO:tasks.workunit.client.0.smithi081.stdout:4/537: mknod da/d18/d42/d4f/d7c/cac 0 2024-03-22T07:18:35.636 INFO:tasks.workunit.client.0.smithi081.stdout:4/538: write da/d18/d1b/d34/f64 [5068192,107786] 0 2024-03-22T07:18:35.636 INFO:tasks.workunit.client.0.smithi081.stdout:1/375: mknod d6/d18/c80 0 2024-03-22T07:18:35.638 INFO:tasks.workunit.client.0.smithi081.stdout:0/443: rename d3/dc/d72 to d3/dc/d40/d69/d92 0 2024-03-22T07:18:35.640 INFO:tasks.workunit.client.0.smithi081.stdout:4/539: dread da/d18/d1b/d34/f4c [0,4194304] 0 2024-03-22T07:18:35.641 INFO:tasks.workunit.client.0.smithi081.stdout:7/523: rename d9/d10/d53/f95 to d9/d3f/d52/d83/fba 0 2024-03-22T07:18:35.643 INFO:tasks.workunit.client.0.smithi081.stdout:6/375: rename d3 to d3/d7b 22 2024-03-22T07:18:35.643 INFO:tasks.workunit.client.0.smithi081.stdout:0/444: rename d3/dc/d33 to d3/dc/d33/d62/d93 22 2024-03-22T07:18:35.643 INFO:tasks.workunit.client.0.smithi081.stdout:6/376: read - d3/da/de/f5b zero size 2024-03-22T07:18:35.645 INFO:tasks.workunit.client.0.smithi081.stdout:7/524: symlink d9/d10/d16/lbb 0 2024-03-22T07:18:35.647 INFO:tasks.workunit.client.0.smithi081.stdout:1/376: rmdir d6/d18/d41/d49/d56/d30 39 2024-03-22T07:18:35.647 INFO:tasks.workunit.client.0.smithi081.stdout:1/377: write d6/d18/d1f/d24/f6f [4524218,27317] 0 2024-03-22T07:18:35.647 INFO:tasks.workunit.client.0.smithi081.stdout:1/378: stat d6/d18/f27 0 2024-03-22T07:18:35.653 INFO:tasks.workunit.client.0.smithi081.stdout:6/377: symlink d3/da/de/l7c 0 2024-03-22T07:18:35.663 INFO:tasks.workunit.client.0.smithi081.stdout:6/378: write d3/da/f2a [2553801,117946] 0 2024-03-22T07:18:35.677 INFO:tasks.workunit.client.0.smithi081.stdout:9/509: sync 2024-03-22T07:18:35.677 INFO:tasks.workunit.client.0.smithi081.stdout:3/308: sync 2024-03-22T07:18:35.678 INFO:tasks.workunit.client.0.smithi081.stdout:3/309: creat db/de/f68 x:0 0 0 2024-03-22T07:18:35.679 INFO:tasks.workunit.client.0.smithi081.stdout:3/310: creat db/d38/f69 x:0 0 0 2024-03-22T07:18:35.679 INFO:tasks.workunit.client.0.smithi081.stdout:3/311: chown db/d23/f4d 208 1 2024-03-22T07:18:35.679 INFO:tasks.workunit.client.0.smithi081.stdout:3/312: dread - db/d2e/f40 zero size 2024-03-22T07:18:35.679 INFO:tasks.workunit.client.0.smithi081.stdout:3/313: mkdir db/de/d4b/d6a 0 2024-03-22T07:18:35.680 INFO:tasks.workunit.client.0.smithi081.stdout:3/314: creat db/d2e/d3a/f6b x:0 0 0 2024-03-22T07:18:35.681 INFO:tasks.workunit.client.0.smithi081.stdout:3/315: creat db/d38/f6c x:0 0 0 2024-03-22T07:18:35.682 INFO:tasks.workunit.client.0.smithi081.stdout:3/316: dread db/d38/f47 [0,4194304] 0 2024-03-22T07:18:35.682 INFO:tasks.workunit.client.0.smithi081.stdout:3/317: chown db/f4f 0 1 2024-03-22T07:18:35.683 INFO:tasks.workunit.client.0.smithi081.stdout:3/318: mknod db/d2e/d3a/c6d 0 2024-03-22T07:18:35.685 INFO:tasks.workunit.client.0.smithi081.stdout:3/319: rename db/de/f1a to db/d38/f6e 0 2024-03-22T07:18:35.688 INFO:tasks.workunit.client.0.smithi081.stdout:0/445: dwrite d3/dc/d2b/f49 [0,4194304] 0 2024-03-22T07:18:35.691 INFO:tasks.workunit.client.0.smithi081.stdout:0/446: creat d3/dc/d40/f94 x:0 0 0 2024-03-22T07:18:35.691 INFO:tasks.workunit.client.0.smithi081.stdout:0/447: read d3/d32/f5f [4025597,103689] 0 2024-03-22T07:18:35.691 INFO:tasks.workunit.client.0.smithi081.stdout:0/448: chown d3/l9 199 1 2024-03-22T07:18:35.692 INFO:tasks.workunit.client.0.smithi081.stdout:0/449: mknod d3/dc/d33/c95 0 2024-03-22T07:18:35.697 INFO:tasks.workunit.client.0.smithi081.stdout:7/525: dwrite d9/dc/f38 [0,4194304] 0 2024-03-22T07:18:35.697 INFO:tasks.workunit.client.0.smithi081.stdout:6/379: dwrite d3/da/de/d31/d40/d50/d63/d26/f44 [0,4194304] 0 2024-03-22T07:18:35.698 INFO:tasks.workunit.client.0.smithi081.stdout:9/510: dwrite d2/d73/d85/f2f [0,4194304] 0 2024-03-22T07:18:35.701 INFO:tasks.workunit.client.0.smithi081.stdout:9/511: symlink d2/d10/d30/d57/d66/d75/d94/la3 0 2024-03-22T07:18:35.701 INFO:tasks.workunit.client.0.smithi081.stdout:9/512: creat d2/d73/d74/fa4 x:0 0 0 2024-03-22T07:18:35.705 INFO:tasks.workunit.client.0.smithi081.stdout:7/526: dread d9/d10/d16/d1e/f4e [0,4194304] 0 2024-03-22T07:18:35.707 INFO:tasks.workunit.client.0.smithi081.stdout:7/527: unlink d9/dc/f33 0 2024-03-22T07:18:35.707 INFO:tasks.workunit.client.0.smithi081.stdout:0/450: write d3/d32/d35/f76 [879385,129393] 0 2024-03-22T07:18:35.708 INFO:tasks.workunit.client.0.smithi081.stdout:0/451: rmdir d3/d32 39 2024-03-22T07:18:35.711 INFO:tasks.workunit.client.0.smithi081.stdout:0/452: creat d3/dc/d33/f96 x:0 0 0 2024-03-22T07:18:35.712 INFO:tasks.workunit.client.0.smithi081.stdout:0/453: creat d3/dc/d33/f97 x:0 0 0 2024-03-22T07:18:35.712 INFO:tasks.workunit.client.0.smithi081.stdout:6/380: write d3/da/de/f28 [150800,28760] 0 2024-03-22T07:18:35.713 INFO:tasks.workunit.client.0.smithi081.stdout:0/454: fsync d3/dc/d2b/f7e 0 2024-03-22T07:18:35.714 INFO:tasks.workunit.client.0.smithi081.stdout:9/513: dread d2/f4 [0,4194304] 0 2024-03-22T07:18:35.721 INFO:tasks.workunit.client.0.smithi081.stdout:7/528: dread d9/d10/d53/f82 [0,4194304] 0 2024-03-22T07:18:35.721 INFO:tasks.workunit.client.0.smithi081.stdout:7/529: mkdir d9/dc/dbc 0 2024-03-22T07:18:35.723 INFO:tasks.workunit.client.0.smithi081.stdout:7/530: link f1 d9/d10/fbd 0 2024-03-22T07:18:35.727 INFO:tasks.workunit.client.0.smithi081.stdout:7/531: write d9/d3f/fa0 [944612,33804] 0 2024-03-22T07:18:35.728 INFO:tasks.workunit.client.0.smithi081.stdout:9/514: write d2/d73/d85/f2f [1356598,35151] 0 2024-03-22T07:18:35.728 INFO:tasks.workunit.client.0.smithi081.stdout:9/515: dread - d2/d5/d9d/f7b zero size 2024-03-22T07:18:35.732 INFO:tasks.workunit.client.0.smithi081.stdout:6/381: dread d3/da/de/d31/d40/d50/d63/d26/f1d [0,4194304] 0 2024-03-22T07:18:35.732 INFO:tasks.workunit.client.0.smithi081.stdout:6/382: truncate d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f78 101006 0 2024-03-22T07:18:35.733 INFO:tasks.workunit.client.0.smithi081.stdout:6/383: truncate d3/da/fd 1034506 0 2024-03-22T07:18:35.734 INFO:tasks.workunit.client.0.smithi081.stdout:6/384: mkdir d3/da/de/d31/d40/d50/d63/d7d 0 2024-03-22T07:18:35.735 INFO:tasks.workunit.client.0.smithi081.stdout:6/385: creat d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f7e x:0 0 0 2024-03-22T07:18:35.736 INFO:tasks.workunit.client.0.smithi081.stdout:6/386: creat d3/da/de/f7f x:0 0 0 2024-03-22T07:18:35.736 INFO:tasks.workunit.client.0.smithi081.stdout:6/387: dread - d3/da/de/f5b zero size 2024-03-22T07:18:35.742 INFO:tasks.workunit.client.0.smithi081.stdout:9/516: dread d2/f40 [0,4194304] 0 2024-03-22T07:18:35.742 INFO:tasks.workunit.client.0.smithi081.stdout:9/517: readlink d2/d10/l19 0 2024-03-22T07:18:35.746 INFO:tasks.workunit.client.0.smithi081.stdout:7/532: dwrite d9/dc/f90 [0,4194304] 0 2024-03-22T07:18:35.754 INFO:tasks.workunit.client.0.smithi081.stdout:7/533: dread d9/d10/d16/d1e/d27/f57 [0,4194304] 0 2024-03-22T07:18:35.773 INFO:tasks.workunit.client.0.smithi081.stdout:7/534: dwrite d9/d10/d16/d1e/d27/d70/faa [0,4194304] 0 2024-03-22T07:18:35.773 INFO:tasks.workunit.client.0.smithi081.stdout:6/388: dwrite d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f5f [0,4194304] 0 2024-03-22T07:18:35.773 INFO:tasks.workunit.client.0.smithi081.stdout:6/389: write d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f6e [412526,19534] 0 2024-03-22T07:18:35.774 INFO:tasks.workunit.client.0.smithi081.stdout:7/535: chown d9/d10/d16/f6b 5573 1 2024-03-22T07:18:35.776 INFO:tasks.workunit.client.0.smithi081.stdout:6/390: truncate d3/da/de/d31/d40/d50/d63/f3b 2414774 0 2024-03-22T07:18:35.796 INFO:tasks.workunit.client.0.smithi081.stdout:6/391: write d3/da/de/d31/d40/d50/d63/f2c [3884,35944] 0 2024-03-22T07:18:35.796 INFO:tasks.workunit.client.0.smithi081.stdout:6/392: truncate d3/da/de/d31/d40/d50/d63/d26/f71 259691 0 2024-03-22T07:18:35.796 INFO:tasks.workunit.client.0.smithi081.stdout:6/393: mknod d3/da/de/d31/d40/d75/c80 0 2024-03-22T07:18:35.797 INFO:tasks.workunit.client.0.smithi081.stdout:7/536: write d9/d10/d16/d1e/d27/f5a [801051,104972] 0 2024-03-22T07:18:35.797 INFO:tasks.workunit.client.0.smithi081.stdout:6/394: mknod d3/c81 0 2024-03-22T07:18:35.817 INFO:tasks.workunit.client.0.smithi081.stdout:6/395: dwrite d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f5f [4194304,4194304] 0 2024-03-22T07:18:35.881 INFO:tasks.workunit.client.0.smithi081.stdout:7/537: unlink d9/cd 0 2024-03-22T07:18:35.884 INFO:tasks.workunit.client.0.smithi081.stdout:6/396: link d3/da/f2a d3/da/de/d31/d40/d50/d63/d5e/f82 0 2024-03-22T07:18:35.884 INFO:tasks.workunit.client.0.smithi081.stdout:6/397: fdatasync d3/da/de/f56 0 2024-03-22T07:18:35.884 INFO:tasks.workunit.client.0.smithi081.stdout:6/398: write d3/da/de/f28 [767290,107281] 0 2024-03-22T07:18:35.884 INFO:tasks.workunit.client.0.smithi081.stdout:6/399: read d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f78 [34764,54239] 0 2024-03-22T07:18:35.887 INFO:tasks.workunit.client.0.smithi081.stdout:7/538: dread d9/d10/d53/f82 [0,4194304] 0 2024-03-22T07:18:35.899 INFO:tasks.workunit.client.0.smithi081.stdout:7/539: dread d9/d10/f2e [0,4194304] 0 2024-03-22T07:18:35.899 INFO:tasks.workunit.client.0.smithi081.stdout:7/540: truncate d9/d10/d53/fb5 514604 0 2024-03-22T07:18:35.900 INFO:tasks.workunit.client.0.smithi081.stdout:7/541: mkdir d9/d10/d53/d81/d59/dbe 0 2024-03-22T07:18:35.902 INFO:tasks.workunit.client.0.smithi081.stdout:6/400: dwrite d3/da/de/f41 [0,4194304] 0 2024-03-22T07:18:35.902 INFO:tasks.workunit.client.0.smithi081.stdout:7/542: creat d9/d10/d4b/d9b/fbf x:0 0 0 2024-03-22T07:18:35.905 INFO:tasks.workunit.client.0.smithi081.stdout:6/401: creat d3/da/f83 x:0 0 0 2024-03-22T07:18:35.908 INFO:tasks.workunit.client.0.smithi081.stdout:6/402: rename d3/da/de/d31/d40/d50/d63/f3b to d3/da/de/d31/d40/d50/d63/d26/f84 0 2024-03-22T07:18:35.910 INFO:tasks.workunit.client.0.smithi081.stdout:6/403: chown d3/da/de/l55 258330766 1 2024-03-22T07:18:35.912 INFO:tasks.workunit.client.0.smithi081.stdout:7/543: read d9/d10/d16/d1e/f2c [7967716,16878] 0 2024-03-22T07:18:35.913 INFO:tasks.workunit.client.0.smithi081.stdout:6/404: link d3/da/de/f61 d3/da/de/d31/d40/d50/f85 0 2024-03-22T07:18:35.913 INFO:tasks.workunit.client.0.smithi081.stdout:6/405: chown d3/da/de/d31/d40/d50/d63/d26/l74 157619 1 2024-03-22T07:18:35.914 INFO:tasks.workunit.client.0.smithi081.stdout:6/406: write d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f7e [378178,15596] 0 2024-03-22T07:18:35.916 INFO:tasks.workunit.client.0.smithi081.stdout:6/407: creat d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f86 x:0 0 0 2024-03-22T07:18:35.917 INFO:tasks.workunit.client.0.smithi081.stdout:6/408: creat d3/da/de/d31/d40/d50/f87 x:0 0 0 2024-03-22T07:18:35.917 INFO:tasks.workunit.client.0.smithi081.stdout:6/409: fdatasync d3/da/de/d31/d40/d50/d63/d5e/f82 0 2024-03-22T07:18:35.919 INFO:tasks.workunit.client.0.smithi081.stdout:6/410: truncate d3/da/fd 926228 0 2024-03-22T07:18:35.921 INFO:tasks.workunit.client.0.smithi081.stdout:7/544: write d9/d10/d16/d1e/d27/d70/f8d [180986,100589] 0 2024-03-22T07:18:35.921 INFO:tasks.workunit.client.0.smithi081.stdout:7/545: stat d9/d10/d16/d1e/d27/l68 0 2024-03-22T07:18:35.921 INFO:tasks.workunit.client.0.smithi081.stdout:7/546: stat d9/d10/d4b/d61/lb9 0 2024-03-22T07:18:35.923 INFO:tasks.workunit.client.0.smithi081.stdout:7/547: mknod d9/d10/d16/d84/cc0 0 2024-03-22T07:18:35.923 INFO:tasks.workunit.client.0.smithi081.stdout:7/548: fdatasync d9/d14/f24 0 2024-03-22T07:18:35.926 INFO:tasks.workunit.client.0.smithi081.stdout:7/549: unlink d9/d10/d16/d1e/f18 0 2024-03-22T07:18:35.934 INFO:tasks.workunit.client.0.smithi081.stdout:7/550: write d9/d10/f2e [2684066,57224] 0 2024-03-22T07:18:35.936 INFO:tasks.workunit.client.0.smithi081.stdout:6/411: dwrite d3/da/de/d31/d40/d50/d63/d26/f1d [0,4194304] 0 2024-03-22T07:18:35.938 INFO:tasks.workunit.client.0.smithi081.stdout:7/551: mkdir d9/d10/d16/d1e/d27/d6d/d8c/dc1 0 2024-03-22T07:18:35.957 INFO:tasks.workunit.client.0.smithi081.stdout:7/552: dwrite d9/d10/fbd [0,4194304] 0 2024-03-22T07:18:35.957 INFO:tasks.workunit.client.0.smithi081.stdout:6/412: dwrite d3/da/de/d31/d40/d50/f76 [0,4194304] 0 2024-03-22T07:18:35.960 INFO:tasks.workunit.client.0.smithi081.stdout:6/413: link d3/da/de/d31/d40/d50/d63/d26/l74 d3/da/de/d31/d40/d50/d63/d7d/l88 0 2024-03-22T07:18:35.963 INFO:tasks.workunit.client.0.smithi081.stdout:8/418: sync 2024-03-22T07:18:35.965 INFO:tasks.workunit.client.0.smithi081.stdout:6/414: symlink d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/l89 0 2024-03-22T07:18:35.965 INFO:tasks.workunit.client.0.smithi081.stdout:6/415: stat d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f86 0 2024-03-22T07:18:35.965 INFO:tasks.workunit.client.0.smithi081.stdout:6/416: dread d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f78 [0,4194304] 0 2024-03-22T07:18:35.965 INFO:tasks.workunit.client.0.smithi081.stdout:8/419: creat d2/d3/f95 x:0 0 0 2024-03-22T07:18:35.968 INFO:tasks.workunit.client.0.smithi081.stdout:8/420: symlink d2/d3/d19/d64/l96 0 2024-03-22T07:18:35.968 INFO:tasks.workunit.client.0.smithi081.stdout:8/421: read d2/d3/d19/d1e/d13/d22/d2f/f40 [2244466,56035] 0 2024-03-22T07:18:35.970 INFO:tasks.workunit.client.0.smithi081.stdout:6/417: dread d3/f6 [0,4194304] 0 2024-03-22T07:18:35.973 INFO:tasks.workunit.client.0.smithi081.stdout:6/418: read d3/da/fd [452292,3142] 0 2024-03-22T07:18:35.973 INFO:tasks.workunit.client.0.smithi081.stdout:6/419: chown d3/da/fd 275744 1 2024-03-22T07:18:35.973 INFO:tasks.workunit.client.0.smithi081.stdout:6/420: write d3/da/de/d31/d40/d50/d63/d26/f71 [372712,82455] 0 2024-03-22T07:18:35.973 INFO:tasks.workunit.client.0.smithi081.stdout:8/422: symlink d2/d3/d23/l97 0 2024-03-22T07:18:35.976 INFO:tasks.workunit.client.0.smithi081.stdout:8/423: mknod d2/d3/c98 0 2024-03-22T07:18:35.976 INFO:tasks.workunit.client.0.smithi081.stdout:8/424: dread - d2/d3/d23/f7f zero size 2024-03-22T07:18:35.976 INFO:tasks.workunit.client.0.smithi081.stdout:6/421: getdents d3/da/de/d31/d40/d50/d63/d26/d3f/d4c 0 2024-03-22T07:18:35.980 INFO:tasks.workunit.client.0.smithi081.stdout:8/425: mkdir d2/d3/d19/d1e/d13/d22/d2f/d99 0 2024-03-22T07:18:35.989 INFO:tasks.workunit.client.0.smithi081.stdout:8/426: write d2/d3/f1f [230344,40779] 0 2024-03-22T07:18:35.990 INFO:tasks.workunit.client.0.smithi081.stdout:8/427: fdatasync d2/d3/d23/f53 0 2024-03-22T07:18:35.990 INFO:tasks.workunit.client.0.smithi081.stdout:8/428: read - d2/f5e zero size 2024-03-22T07:18:35.991 INFO:tasks.workunit.client.0.smithi081.stdout:8/429: rename d2/d3/d19/d1e/d13/d22/d2f/f93 to d2/d3/d19/d1e/f9a 0 2024-03-22T07:18:35.992 INFO:tasks.workunit.client.0.smithi081.stdout:8/430: stat d2/d3/d19/d1e/d13/d22/d2f/l54 0 2024-03-22T07:18:35.992 INFO:tasks.workunit.client.0.smithi081.stdout:8/431: dread - d2/d3/d28/d2b/d61/f6b zero size 2024-03-22T07:18:35.992 INFO:tasks.workunit.client.0.smithi081.stdout:8/432: write d2/d3/d19/d1e/f9a [912647,17580] 0 2024-03-22T07:18:35.997 INFO:tasks.workunit.client.0.smithi081.stdout:6/422: dwrite d3/da/de/f61 [0,4194304] 0 2024-03-22T07:18:36.008 INFO:tasks.workunit.client.0.smithi081.stdout:8/433: dwrite d2/d3/d19/d1e/d13/d22/f34 [0,4194304] 0 2024-03-22T07:18:36.010 INFO:tasks.workunit.client.0.smithi081.stdout:8/434: creat d2/d3/d28/d2b/d61/d7e/f9b x:0 0 0 2024-03-22T07:18:36.010 INFO:tasks.workunit.client.0.smithi081.stdout:8/435: getdents d2/d3/d19/d1e/d13/d22/d38/d78 0 2024-03-22T07:18:36.011 INFO:tasks.workunit.client.0.smithi081.stdout:8/436: mkdir d2/d3/d19/d1e/d13/d22/d2f/d99/d9c 0 2024-03-22T07:18:36.012 INFO:tasks.workunit.client.0.smithi081.stdout:6/423: dwrite d3/da/de/d31/d40/d50/f85 [0,4194304] 0 2024-03-22T07:18:36.012 INFO:tasks.workunit.client.0.smithi081.stdout:8/437: creat d2/d3/d19/d1e/f9d x:0 0 0 2024-03-22T07:18:36.015 INFO:tasks.workunit.client.0.smithi081.stdout:6/424: mknod d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/c8a 0 2024-03-22T07:18:36.015 INFO:tasks.workunit.client.0.smithi081.stdout:6/425: stat d3/da/de/l7c 0 2024-03-22T07:18:36.015 INFO:tasks.workunit.client.0.smithi081.stdout:6/426: write d3/da/de/d31/d40/f6d [809165,115343] 0 2024-03-22T07:18:36.019 INFO:tasks.workunit.client.0.smithi081.stdout:6/427: write d3/da/de/f46 [951266,39775] 0 2024-03-22T07:18:36.023 INFO:tasks.workunit.client.0.smithi081.stdout:6/428: rename d3/d67/l6a to d3/da/de/d31/d40/d50/d63/d5e/l8b 0 2024-03-22T07:18:36.025 INFO:tasks.workunit.client.0.smithi081.stdout:6/429: fsync d3/da/de/d31/d40/d50/f5d 0 2024-03-22T07:18:36.025 INFO:tasks.workunit.client.0.smithi081.stdout:6/430: stat d3/da/de/d31/d40/d50/d63/d26/d3f/f7 0 2024-03-22T07:18:36.034 INFO:tasks.workunit.client.0.smithi081.stdout:8/438: dwrite d2/d3/d23/f3b [0,4194304] 0 2024-03-22T07:18:36.036 INFO:tasks.workunit.client.0.smithi081.stdout:8/439: write d2/f11 [4710201,65393] 0 2024-03-22T07:18:36.039 INFO:tasks.workunit.client.0.smithi081.stdout:8/440: mkdir d2/d3/d9e 0 2024-03-22T07:18:36.045 INFO:tasks.workunit.client.0.smithi081.stdout:8/441: dread d2/d3/d19/d1e/f3c [0,4194304] 0 2024-03-22T07:18:36.063 INFO:tasks.workunit.client.0.smithi081.stdout:8/442: dwrite d2/d3/d19/d1e/d13/d22/d2f/d71/d7c/f81 [0,4194304] 0 2024-03-22T07:18:36.064 INFO:tasks.workunit.client.0.smithi081.stdout:8/443: read d2/d3/d28/d2b/f67 [792692,4819] 0 2024-03-22T07:18:36.067 INFO:tasks.workunit.client.0.smithi081.stdout:8/444: getdents d2/d3 0 2024-03-22T07:18:36.068 INFO:tasks.workunit.client.0.smithi081.stdout:8/445: truncate d2/d3/d28/d2b/f2e 829352 0 2024-03-22T07:18:36.081 INFO:tasks.workunit.client.0.smithi081.stdout:8/446: dwrite f1 [4194304,4194304] 0 2024-03-22T07:18:36.082 INFO:tasks.workunit.client.0.smithi081.stdout:8/447: chown d2/d3/d28/f4a 15539565 1 2024-03-22T07:18:36.083 INFO:tasks.workunit.client.0.smithi081.stdout:8/448: creat d2/d3/d19/d1e/d13/d22/d2f/d99/d9c/f9f x:0 0 0 2024-03-22T07:18:36.091 INFO:tasks.workunit.client.0.smithi081.stdout:8/449: read d2/d3/d23/f52 [401826,4884] 0 2024-03-22T07:18:36.091 INFO:tasks.workunit.client.0.smithi081.stdout:8/450: rmdir d2/d3/d19/d1e/d13/d22/d2f/d99 39 2024-03-22T07:18:36.095 INFO:tasks.workunit.client.0.smithi081.stdout:8/451: rename d2/d3/d19/d1e/d13/d22/d2f/d99 to d2/d3/d19/d1e/d63/da0 0 2024-03-22T07:18:36.095 INFO:tasks.workunit.client.0.smithi081.stdout:8/452: chown d2/d3/d28/d2b/d61/f74 45030 1 2024-03-22T07:18:36.096 INFO:tasks.workunit.client.0.smithi081.stdout:8/453: getdents d2/d3/d19/d1e/d13/d66 0 2024-03-22T07:18:36.097 INFO:tasks.workunit.client.0.smithi081.stdout:8/454: mknod d2/ca1 0 2024-03-22T07:18:36.097 INFO:tasks.workunit.client.0.smithi081.stdout:8/455: creat d2/d3/d23/fa2 x:0 0 0 2024-03-22T07:18:36.099 INFO:tasks.workunit.client.0.smithi081.stdout:8/456: dread d2/d3/d19/d1e/d63/f65 [0,4194304] 0 2024-03-22T07:18:36.100 INFO:tasks.workunit.client.0.smithi081.stdout:8/457: creat d2/d3/d19/d1e/d13/fa3 x:0 0 0 2024-03-22T07:18:36.113 INFO:tasks.workunit.client.0.smithi081.stdout:8/458: write d2/d3/d28/f76 [1741171,104625] 0 2024-03-22T07:18:36.119 INFO:tasks.workunit.client.0.smithi081.stdout:8/459: dread d2/d3/d19/d1e/d13/d22/d2f/f40 [0,4194304] 0 2024-03-22T07:18:36.128 INFO:tasks.workunit.client.0.smithi081.stdout:8/460: dread d2/d3/d28/f76 [0,4194304] 0 2024-03-22T07:18:36.128 INFO:tasks.workunit.client.0.smithi081.stdout:8/461: fsync d2/d3/d23/f53 0 2024-03-22T07:18:36.128 INFO:tasks.workunit.client.0.smithi081.stdout:8/462: symlink d2/d3/d19/d1e/d13/d22/d38/d78/la4 0 2024-03-22T07:18:36.152 INFO:tasks.workunit.client.0.smithi081.stdout:8/463: dwrite d2/f5e [0,4194304] 0 2024-03-22T07:18:36.152 INFO:tasks.workunit.client.0.smithi081.stdout:8/464: chown d2/d3/d28/d2b/l62 5 1 2024-03-22T07:18:36.156 INFO:tasks.workunit.client.0.smithi081.stdout:8/465: dread d2/d3/d19/d1e/f3c [0,4194304] 0 2024-03-22T07:18:36.156 INFO:tasks.workunit.client.0.smithi081.stdout:8/466: chown d2/d3/c98 120630 1 2024-03-22T07:18:36.157 INFO:tasks.workunit.client.0.smithi081.stdout:8/467: getdents d2/d3/d19/d1e/d13 0 2024-03-22T07:18:36.157 INFO:tasks.workunit.client.0.smithi081.stdout:8/468: write d2/d3/d19/d1e/d13/d22/d2f/d71/f92 [685197,94796] 0 2024-03-22T07:18:36.157 INFO:tasks.workunit.client.0.smithi081.stdout:8/469: stat d2/d3/d19/d1e/d13/d22/d2f/f77 0 2024-03-22T07:18:36.158 INFO:tasks.workunit.client.0.smithi081.stdout:8/470: unlink d2/d3/d28/f76 0 2024-03-22T07:18:36.159 INFO:tasks.workunit.client.0.smithi081.stdout:8/471: fdatasync d2/d3/d28/f6c 0 2024-03-22T07:18:36.159 INFO:tasks.workunit.client.0.smithi081.stdout:8/472: mkdir d2/d3/d19/d1e/d13/da5 0 2024-03-22T07:18:36.160 INFO:tasks.workunit.client.0.smithi081.stdout:8/473: write d2/d3/d19/d1e/d13/d22/d2f/d71/d7c/f81 [4423129,104213] 0 2024-03-22T07:18:36.161 INFO:tasks.workunit.client.0.smithi081.stdout:8/474: rename d2/d3/cb to d2/d3/d23/ca6 0 2024-03-22T07:18:36.161 INFO:tasks.workunit.client.0.smithi081.stdout:8/475: fsync d2/d3/d23/f7f 0 2024-03-22T07:18:36.178 INFO:tasks.workunit.client.0.smithi081.stdout:8/476: dwrite d2/d3/d28/f6c [0,4194304] 0 2024-03-22T07:18:36.178 INFO:tasks.workunit.client.0.smithi081.stdout:8/477: truncate d2/d3/d19/d1e/d13/fa3 846117 0 2024-03-22T07:18:36.179 INFO:tasks.workunit.client.0.smithi081.stdout:8/478: creat d2/d3/d28/d2b/d61/d7e/fa7 x:0 0 0 2024-03-22T07:18:36.179 INFO:tasks.workunit.client.0.smithi081.stdout:8/479: write d2/d3/d28/d2b/d61/d7e/fa7 [968392,392] 0 2024-03-22T07:18:36.181 INFO:tasks.workunit.client.0.smithi081.stdout:8/480: creat d2/d3/d28/d2b/fa8 x:0 0 0 2024-03-22T07:18:36.185 INFO:tasks.workunit.client.0.smithi081.stdout:5/605: sync 2024-03-22T07:18:36.185 INFO:tasks.workunit.client.0.smithi081.stdout:2/313: sync 2024-03-22T07:18:36.187 INFO:tasks.workunit.client.0.smithi081.stdout:5/606: symlink d4/d75/d98/lcc 0 2024-03-22T07:18:36.189 INFO:tasks.workunit.client.0.smithi081.stdout:2/314: mknod da/d11/c65 0 2024-03-22T07:18:36.189 INFO:tasks.workunit.client.0.smithi081.stdout:2/315: chown da/db/d39/d47/d3b/d41 2 1 2024-03-22T07:18:36.189 INFO:tasks.workunit.client.0.smithi081.stdout:2/316: creat da/db/d39/d47/d3b/d41/d50/f66 x:0 0 0 2024-03-22T07:18:36.189 INFO:tasks.workunit.client.0.smithi081.stdout:5/607: dread d4/d9/f15 [0,4194304] 0 2024-03-22T07:18:36.190 INFO:tasks.workunit.client.0.smithi081.stdout:2/317: chown da/c19 1073849571 1 2024-03-22T07:18:36.191 INFO:tasks.workunit.client.0.smithi081.stdout:5/608: mknod d4/df/d73/d8c/dc0/dc9/ccd 0 2024-03-22T07:18:36.193 INFO:tasks.workunit.client.0.smithi081.stdout:2/318: mkdir da/db/d39/d46/d42/d67 0 2024-03-22T07:18:36.193 INFO:tasks.workunit.client.0.smithi081.stdout:2/319: fdatasync da/db/d39/d47/f3d 0 2024-03-22T07:18:36.195 INFO:tasks.workunit.client.0.smithi081.stdout:8/481: dwrite d2/d3/d19/d1e/d13/d22/f34 [0,4194304] 0 2024-03-22T07:18:36.196 INFO:tasks.workunit.client.0.smithi081.stdout:8/482: rmdir d2/d3/d19/d64 39 2024-03-22T07:18:36.197 INFO:tasks.workunit.client.0.smithi081.stdout:8/483: creat d2/fa9 x:0 0 0 2024-03-22T07:18:36.198 INFO:tasks.workunit.client.0.smithi081.stdout:8/484: dread d2/d3/d19/d1e/d13/fa3 [0,4194304] 0 2024-03-22T07:18:36.205 INFO:tasks.workunit.client.0.smithi081.stdout:2/320: write da/db/f3f [2812238,100589] 0 2024-03-22T07:18:36.205 INFO:tasks.workunit.client.0.smithi081.stdout:2/321: stat da/d11/c65 0 2024-03-22T07:18:36.206 INFO:tasks.workunit.client.0.smithi081.stdout:2/322: symlink da/d56/l68 0 2024-03-22T07:18:36.209 INFO:tasks.workunit.client.0.smithi081.stdout:2/323: mkdir da/db/d39/d46/d42/d69 0 2024-03-22T07:18:36.209 INFO:tasks.workunit.client.0.smithi081.stdout:2/324: chown da/db/f35 178612 1 2024-03-22T07:18:36.209 INFO:tasks.workunit.client.0.smithi081.stdout:5/609: dwrite d4/df/d73/d8c/f4c [0,4194304] 0 2024-03-22T07:18:36.210 INFO:tasks.workunit.client.0.smithi081.stdout:2/325: mkdir da/db/d39/d47/d3b/d41/d50/d6a 0 2024-03-22T07:18:36.213 INFO:tasks.workunit.client.0.smithi081.stdout:2/326: chown da/l23 9498 1 2024-03-22T07:18:36.213 INFO:tasks.workunit.client.0.smithi081.stdout:2/327: write da/db/d39/d47/f5b [4986738,113876] 0 2024-03-22T07:18:36.213 INFO:tasks.workunit.client.0.smithi081.stdout:8/485: write d2/d3/d19/d1e/d13/d22/f51 [304673,105843] 0 2024-03-22T07:18:36.213 INFO:tasks.workunit.client.0.smithi081.stdout:8/486: stat d2/d3/d19/d1e/d13/d22/d2f/d73/c8f 0 2024-03-22T07:18:36.214 INFO:tasks.workunit.client.0.smithi081.stdout:8/487: rmdir d2/d3/d19/d1e/d13/d66/d70/d86 39 2024-03-22T07:18:36.218 INFO:tasks.workunit.client.0.smithi081.stdout:2/328: write da/db/d39/d46/f44 [328207,37315] 0 2024-03-22T07:18:36.218 INFO:tasks.workunit.client.0.smithi081.stdout:8/488: write d2/d3/d19/d1e/d13/d22/d38/f44 [1520444,43795] 0 2024-03-22T07:18:36.218 INFO:tasks.workunit.client.0.smithi081.stdout:8/489: chown d2/d3/d19/d1e/d13/d22/l57 1993096785 1 2024-03-22T07:18:36.219 INFO:tasks.workunit.client.0.smithi081.stdout:2/329: unlink da/d11/c65 0 2024-03-22T07:18:36.221 INFO:tasks.workunit.client.0.smithi081.stdout:2/330: creat da/d56/f6b x:0 0 0 2024-03-22T07:18:36.223 INFO:tasks.workunit.client.0.smithi081.stdout:8/490: symlink d2/laa 0 2024-03-22T07:18:36.223 INFO:tasks.workunit.client.0.smithi081.stdout:8/491: read d2/d3/d19/f48 [1417847,44631] 0 2024-03-22T07:18:36.223 INFO:tasks.workunit.client.0.smithi081.stdout:8/492: fsync d2/d3/d19/d1e/d13/f60 0 2024-03-22T07:18:36.225 INFO:tasks.workunit.client.0.smithi081.stdout:2/331: unlink da/db/d39/d46/c31 0 2024-03-22T07:18:36.225 INFO:tasks.workunit.client.0.smithi081.stdout:2/332: write da/db/d39/d45/f4c [831308,63400] 0 2024-03-22T07:18:36.226 INFO:tasks.workunit.client.0.smithi081.stdout:2/333: creat da/db/d39/d46/f6c x:0 0 0 2024-03-22T07:18:36.227 INFO:tasks.workunit.client.0.smithi081.stdout:2/334: fdatasync da/db/d39/d47/f43 0 2024-03-22T07:18:36.227 INFO:tasks.workunit.client.0.smithi081.stdout:2/335: chown da/d56/f58 2 1 2024-03-22T07:18:36.228 INFO:tasks.workunit.client.0.smithi081.stdout:2/336: mkdir da/db/d6d 0 2024-03-22T07:18:36.229 INFO:tasks.workunit.client.0.smithi081.stdout:2/337: unlink da/db/d39/d47/c34 0 2024-03-22T07:18:36.229 INFO:tasks.workunit.client.0.smithi081.stdout:2/338: getdents da/db/d39/d47/d3b/d64 0 2024-03-22T07:18:36.230 INFO:tasks.workunit.client.0.smithi081.stdout:2/339: mkdir da/db/d39/d47/d6e 0 2024-03-22T07:18:36.230 INFO:tasks.workunit.client.0.smithi081.stdout:2/340: getdents da/db/d39/d47/d3b/d64 0 2024-03-22T07:18:36.230 INFO:tasks.workunit.client.0.smithi081.stdout:2/341: mknod da/db/d39/d46/d42/d69/c6f 0 2024-03-22T07:18:36.232 INFO:tasks.workunit.client.0.smithi081.stdout:2/342: rename da/db/d39/d47/d38 to da/db/d39/d47/d3b/d41/d50/d6a/d70 0 2024-03-22T07:18:36.232 INFO:tasks.workunit.client.0.smithi081.stdout:2/343: truncate da/f25 2446554 0 2024-03-22T07:18:36.233 INFO:tasks.workunit.client.0.smithi081.stdout:2/344: chown da/db/d39/d46/f2e 343 1 2024-03-22T07:18:36.236 INFO:tasks.workunit.client.0.smithi081.stdout:5/610: dwrite d4/d9/f5b [0,4194304] 0 2024-03-22T07:18:36.242 INFO:tasks.workunit.client.0.smithi081.stdout:8/493: write d2/f24 [2815721,96299] 0 2024-03-22T07:18:36.244 INFO:tasks.workunit.client.0.smithi081.stdout:2/345: getdents da/db/d39/d47 0 2024-03-22T07:18:36.249 INFO:tasks.workunit.client.0.smithi081.stdout:5/611: mknod d4/df/d22/d4d/cce 0 2024-03-22T07:18:36.249 INFO:tasks.workunit.client.0.smithi081.stdout:8/494: mknod d2/d3/d9e/cab 0 2024-03-22T07:18:36.254 INFO:tasks.workunit.client.0.smithi081.stdout:3/320: sync 2024-03-22T07:18:36.254 INFO:tasks.workunit.client.0.smithi081.stdout:1/379: sync 2024-03-22T07:18:36.254 INFO:tasks.workunit.client.0.smithi081.stdout:4/540: sync 2024-03-22T07:18:36.254 INFO:tasks.workunit.client.0.smithi081.stdout:1/380: dread - d6/d18/d1f/f72 zero size 2024-03-22T07:18:36.256 INFO:tasks.workunit.client.0.smithi081.stdout:4/541: unlink da/d18/d1b/d47/la3 0 2024-03-22T07:18:36.258 INFO:tasks.workunit.client.0.smithi081.stdout:4/542: dread da/d18/d1b/d26/d3e/f55 [0,4194304] 0 2024-03-22T07:18:36.260 INFO:tasks.workunit.client.0.smithi081.stdout:4/543: dread da/f79 [0,4194304] 0 2024-03-22T07:18:36.263 INFO:tasks.workunit.client.0.smithi081.stdout:8/495: write d2/d3/d19/d1e/f3e [43901,41152] 0 2024-03-22T07:18:36.271 INFO:tasks.workunit.client.0.smithi081.stdout:4/544: write da/d18/d1b/d26/f43 [699183,60018] 0 2024-03-22T07:18:36.271 INFO:tasks.workunit.client.0.smithi081.stdout:8/496: read d2/d3/d23/f53 [622901,11153] 0 2024-03-22T07:18:36.271 INFO:tasks.workunit.client.0.smithi081.stdout:4/545: chown da/d3d/f5e 48835971 1 2024-03-22T07:18:36.271 INFO:tasks.workunit.client.0.smithi081.stdout:8/497: write d2/d3/f95 [95374,77667] 0 2024-03-22T07:18:36.273 INFO:tasks.workunit.client.0.smithi081.stdout:4/546: unlink da/d18/d1b/f40 0 2024-03-22T07:18:36.273 INFO:tasks.workunit.client.0.smithi081.stdout:4/547: chown da/d18/d1b/d34/c8e 127373343 1 2024-03-22T07:18:36.274 INFO:tasks.workunit.client.0.smithi081.stdout:8/498: truncate d2/d3/d19/d1e/f5f 572338 0 2024-03-22T07:18:36.277 INFO:tasks.workunit.client.0.smithi081.stdout:1/381: link d6/c20 d6/d18/d1f/c81 0 2024-03-22T07:18:36.277 INFO:tasks.workunit.client.0.smithi081.stdout:8/499: read d2/d3/d19/d1e/d13/d22/d2f/d71/f92 [301450,83439] 0 2024-03-22T07:18:36.277 INFO:tasks.workunit.client.0.smithi081.stdout:8/500: fsync d2/f24 0 2024-03-22T07:18:36.302 INFO:tasks.workunit.client.0.smithi081.stdout:3/321: dwrite db/d23/f67 [0,4194304] 0 2024-03-22T07:18:36.303 INFO:tasks.workunit.client.0.smithi081.stdout:5/612: dwrite d4/d10/d20/db2/f28 [0,4194304] 0 2024-03-22T07:18:36.303 INFO:tasks.workunit.client.0.smithi081.stdout:4/548: dwrite da/d18/d1b/d1e/f3b [0,4194304] 0 2024-03-22T07:18:36.303 INFO:tasks.workunit.client.0.smithi081.stdout:4/549: fsync da/df/d38/f68 0 2024-03-22T07:18:36.307 INFO:tasks.workunit.client.0.smithi081.stdout:4/550: creat da/d18/d42/d4f/fad x:0 0 0 2024-03-22T07:18:36.308 INFO:tasks.workunit.client.0.smithi081.stdout:4/551: rename da/df/l3c to da/d18/d1b/lae 0 2024-03-22T07:18:36.310 INFO:tasks.workunit.client.0.smithi081.stdout:4/552: mkdir da/d18/d42/d4f/daf 0 2024-03-22T07:18:36.311 INFO:tasks.workunit.client.0.smithi081.stdout:4/553: creat da/d18/d42/fb0 x:0 0 0 2024-03-22T07:18:36.311 INFO:tasks.workunit.client.0.smithi081.stdout:4/554: chown da/d18/d1b/d1e/d37/d1c/d74/lab 29810 1 2024-03-22T07:18:36.316 INFO:tasks.workunit.client.0.smithi081.stdout:3/322: write db/de/f24 [5615523,26150] 0 2024-03-22T07:18:36.317 INFO:tasks.workunit.client.0.smithi081.stdout:4/555: dread da/d18/f21 [0,4194304] 0 2024-03-22T07:18:36.317 INFO:tasks.workunit.client.0.smithi081.stdout:5/613: write d4/d10/d20/d25/db8/f31 [1793325,82978] 0 2024-03-22T07:18:36.318 INFO:tasks.workunit.client.0.smithi081.stdout:5/614: dread - d4/d10/f8d zero size 2024-03-22T07:18:36.320 INFO:tasks.workunit.client.0.smithi081.stdout:4/556: rename da/d3d/c61 to da/d18/d42/d4f/cb1 0 2024-03-22T07:18:36.322 INFO:tasks.workunit.client.0.smithi081.stdout:5/615: mkdir d4/d10/d20/d25/db8/dcf 0 2024-03-22T07:18:36.325 INFO:tasks.workunit.client.0.smithi081.stdout:5/616: truncate d4/d9/f2c 270045 0 2024-03-22T07:18:36.326 INFO:tasks.workunit.client.0.smithi081.stdout:4/557: mknod da/d18/d1b/d26/cb2 0 2024-03-22T07:18:36.327 INFO:tasks.workunit.client.0.smithi081.stdout:5/617: link d4/df/d73/d8c/dc0/dc9/ccd d4/df/d73/cd0 0 2024-03-22T07:18:36.327 INFO:tasks.workunit.client.0.smithi081.stdout:5/618: write d4/fb1 [444199,25518] 0 2024-03-22T07:18:36.328 INFO:tasks.workunit.client.0.smithi081.stdout:5/619: stat d4/d9/d90 0 2024-03-22T07:18:36.328 INFO:tasks.workunit.client.0.smithi081.stdout:4/558: symlink da/d18/lb3 0 2024-03-22T07:18:36.329 INFO:tasks.workunit.client.0.smithi081.stdout:4/559: getdents da/d18/d1b/d1e/d37/d1c/d74/d98 0 2024-03-22T07:18:36.342 INFO:tasks.workunit.client.0.smithi081.stdout:4/560: write da/d18/d1b/d1e/f3b [1583549,14299] 0 2024-03-22T07:18:36.344 INFO:tasks.workunit.client.0.smithi081.stdout:4/561: symlink da/df/lb4 0 2024-03-22T07:18:36.344 INFO:tasks.workunit.client.0.smithi081.stdout:4/562: chown da/d18/d42/f9a 2521152 1 2024-03-22T07:18:36.346 INFO:tasks.workunit.client.0.smithi081.stdout:5/620: dwrite d4/d10/f24 [4194304,4194304] 0 2024-03-22T07:18:36.380 INFO:tasks.workunit.client.0.smithi081.stdout:5/621: dwrite d4/d10/d20/d25/f8b [0,4194304] 0 2024-03-22T07:18:36.380 INFO:tasks.workunit.client.0.smithi081.stdout:4/563: dwrite da/d18/d1b/d34/f4c [0,4194304] 0 2024-03-22T07:18:36.381 INFO:tasks.workunit.client.0.smithi081.stdout:4/564: creat da/d18/d1b/d26/fb5 x:0 0 0 2024-03-22T07:18:36.381 INFO:tasks.workunit.client.0.smithi081.stdout:5/622: mknod d4/d10/cd1 0 2024-03-22T07:18:36.382 INFO:tasks.workunit.client.0.smithi081.stdout:4/565: mknod da/d18/d6e/cb6 0 2024-03-22T07:18:36.383 INFO:tasks.workunit.client.0.smithi081.stdout:4/566: truncate f3 4567004 0 2024-03-22T07:18:36.383 INFO:tasks.workunit.client.0.smithi081.stdout:4/567: stat f9 0 2024-03-22T07:18:36.383 INFO:tasks.workunit.client.0.smithi081.stdout:5/623: mkdir d4/d10/d20/dd2 0 2024-03-22T07:18:36.383 INFO:tasks.workunit.client.0.smithi081.stdout:4/568: stat da/df/f5a 0 2024-03-22T07:18:36.400 INFO:tasks.workunit.client.0.smithi081.stdout:9/518: sync 2024-03-22T07:18:36.400 INFO:tasks.workunit.client.0.smithi081.stdout:6/431: sync 2024-03-22T07:18:36.400 INFO:tasks.workunit.client.0.smithi081.stdout:7/553: sync 2024-03-22T07:18:36.400 INFO:tasks.workunit.client.0.smithi081.stdout:0/455: sync 2024-03-22T07:18:36.400 INFO:tasks.workunit.client.0.smithi081.stdout:7/554: write d9/f3c [9056935,1794] 0 2024-03-22T07:18:36.402 INFO:tasks.workunit.client.0.smithi081.stdout:9/519: creat d2/d5d/d5a/fa5 x:0 0 0 2024-03-22T07:18:36.402 INFO:tasks.workunit.client.0.smithi081.stdout:0/456: symlink d3/dc/d33/d62/l98 0 2024-03-22T07:18:36.402 INFO:tasks.workunit.client.0.smithi081.stdout:5/624: dwrite d4/d9/f2c [0,4194304] 0 2024-03-22T07:18:36.402 INFO:tasks.workunit.client.0.smithi081.stdout:7/555: creat d9/d10/d16/d1e/d27/d6d/fc2 x:0 0 0 2024-03-22T07:18:36.407 INFO:tasks.workunit.client.0.smithi081.stdout:7/556: rename d9/d3f/d52/fb1 to d9/d10/d9e/fc3 0 2024-03-22T07:18:36.409 INFO:tasks.workunit.client.0.smithi081.stdout:7/557: mknod d9/d10/d53/d81/d59/dbe/cc4 0 2024-03-22T07:18:36.409 INFO:tasks.workunit.client.0.smithi081.stdout:7/558: mknod d9/dc/dbc/cc5 0 2024-03-22T07:18:36.410 INFO:tasks.workunit.client.0.smithi081.stdout:7/559: creat d9/d3f/db3/fc6 x:0 0 0 2024-03-22T07:18:36.411 INFO:tasks.workunit.client.0.smithi081.stdout:9/520: dread d2/d73/d85/f2e [0,4194304] 0 2024-03-22T07:18:36.413 INFO:tasks.workunit.client.0.smithi081.stdout:9/521: creat d2/d78/fa6 x:0 0 0 2024-03-22T07:18:36.413 INFO:tasks.workunit.client.0.smithi081.stdout:9/522: chown d2/d2b/l8d 4579 1 2024-03-22T07:18:36.414 INFO:tasks.workunit.client.0.smithi081.stdout:9/523: unlink d2/f4 0 2024-03-22T07:18:36.415 INFO:tasks.workunit.client.0.smithi081.stdout:4/569: dwrite da/d18/d1b/d1e/f3b [4194304,4194304] 0 2024-03-22T07:18:36.419 INFO:tasks.workunit.client.0.smithi081.stdout:5/625: dwrite d4/d10/d20/d25/f8b [4194304,4194304] 0 2024-03-22T07:18:36.419 INFO:tasks.workunit.client.0.smithi081.stdout:5/626: truncate f3 4827735 0 2024-03-22T07:18:36.421 INFO:tasks.workunit.client.0.smithi081.stdout:5/627: symlink d4/d10/d20/dd2/ld3 0 2024-03-22T07:18:36.423 INFO:tasks.workunit.client.0.smithi081.stdout:0/457: write d3/dc/d33/d62/f63 [902192,82401] 0 2024-03-22T07:18:36.436 INFO:tasks.workunit.client.0.smithi081.stdout:0/458: write d3/dc/f19 [10012,38656] 0 2024-03-22T07:18:36.437 INFO:tasks.workunit.client.0.smithi081.stdout:0/459: mknod d3/d32/d35/d91/c99 0 2024-03-22T07:18:36.466 INFO:tasks.workunit.client.0.smithi081.stdout:9/524: dwrite d2/d73/d85/f46 [0,4194304] 0 2024-03-22T07:18:36.466 INFO:tasks.workunit.client.0.smithi081.stdout:9/525: write d2/d5/fd [3477187,127819] 0 2024-03-22T07:18:36.466 INFO:tasks.workunit.client.0.smithi081.stdout:9/526: write d2/d10/d30/d57/f7d [722245,103524] 0 2024-03-22T07:18:36.466 INFO:tasks.workunit.client.0.smithi081.stdout:4/570: dwrite da/d18/d1b/d26/d3e/f55 [0,4194304] 0 2024-03-22T07:18:36.466 INFO:tasks.workunit.client.0.smithi081.stdout:0/460: dwrite d3/dc/d2b/f2f [0,4194304] 0 2024-03-22T07:18:36.466 INFO:tasks.workunit.client.0.smithi081.stdout:7/560: dwrite d9/d10/d2a/f47 [0,4194304] 0 2024-03-22T07:18:36.468 INFO:tasks.workunit.client.0.smithi081.stdout:0/461: dread d3/dc/d33/d62/f68 [0,4194304] 0 2024-03-22T07:18:36.472 INFO:tasks.workunit.client.0.smithi081.stdout:0/462: creat d3/dc/d40/f9a x:0 0 0 2024-03-22T07:18:36.473 INFO:tasks.workunit.client.0.smithi081.stdout:7/561: getdents d9/d10/d53/d81/d59/d89 0 2024-03-22T07:18:36.484 INFO:tasks.workunit.client.0.smithi081.stdout:9/527: dwrite d2/d73/d85/f2e [4194304,4194304] 0 2024-03-22T07:18:36.487 INFO:tasks.workunit.client.0.smithi081.stdout:9/528: getdents d2/d5d 0 2024-03-22T07:18:36.487 INFO:tasks.workunit.client.0.smithi081.stdout:9/529: readlink d2/d10/l52 0 2024-03-22T07:18:36.487 INFO:tasks.workunit.client.0.smithi081.stdout:9/530: write d2/d2b/f26 [217738,128547] 0 2024-03-22T07:18:36.501 INFO:tasks.workunit.client.0.smithi081.stdout:4/571: dwrite da/d18/f21 [0,4194304] 0 2024-03-22T07:18:36.501 INFO:tasks.workunit.client.0.smithi081.stdout:9/531: dread d2/d2b/f2d [0,4194304] 0 2024-03-22T07:18:36.501 INFO:tasks.workunit.client.0.smithi081.stdout:0/463: dwrite d3/f2e [0,4194304] 0 2024-03-22T07:18:36.514 INFO:tasks.workunit.client.0.smithi081.stdout:4/572: write da/f7b [2496838,19173] 0 2024-03-22T07:18:36.517 INFO:tasks.workunit.client.0.smithi081.stdout:4/573: rename da/d18/d1b/d26/d3e/l6d to da/d18/d1b/d1e/d5f/lb7 0 2024-03-22T07:18:36.518 INFO:tasks.workunit.client.0.smithi081.stdout:4/574: mkdir da/d18/d1b/d1e/d37/d1c/d74/d98/db8 0 2024-03-22T07:18:36.518 INFO:tasks.workunit.client.0.smithi081.stdout:4/575: creat da/df/d38/fb9 x:0 0 0 2024-03-22T07:18:36.519 INFO:tasks.workunit.client.0.smithi081.stdout:4/576: symlink da/d18/d42/lba 0 2024-03-22T07:18:36.520 INFO:tasks.workunit.client.0.smithi081.stdout:4/577: creat da/d18/d1b/d1e/d37/d1c/d74/d98/db8/fbb x:0 0 0 2024-03-22T07:18:36.520 INFO:tasks.workunit.client.0.smithi081.stdout:0/464: dwrite d3/dc/d33/d62/f63 [0,4194304] 0 2024-03-22T07:18:36.523 INFO:tasks.workunit.client.0.smithi081.stdout:4/578: mkdir da/d18/d1b/d26/d32/dbc 0 2024-03-22T07:18:36.523 INFO:tasks.workunit.client.0.smithi081.stdout:0/465: truncate d3/dc/f3f 771180 0 2024-03-22T07:18:36.525 INFO:tasks.workunit.client.0.smithi081.stdout:0/466: getdents d3/d4e 0 2024-03-22T07:18:36.526 INFO:tasks.workunit.client.0.smithi081.stdout:0/467: mkdir d3/d9b 0 2024-03-22T07:18:36.527 INFO:tasks.workunit.client.0.smithi081.stdout:0/468: mknod d3/d32/d35/d74/c9c 0 2024-03-22T07:18:36.528 INFO:tasks.workunit.client.0.smithi081.stdout:0/469: mkdir d3/d32/d35/d91/d9d 0 2024-03-22T07:18:36.529 INFO:tasks.workunit.client.0.smithi081.stdout:0/470: truncate d3/dc/f73 463858 0 2024-03-22T07:18:36.531 INFO:tasks.workunit.client.0.smithi081.stdout:0/471: link d3/dc/d2b/l6d d3/dc/d2b/d39/l9e 0 2024-03-22T07:18:36.531 INFO:tasks.workunit.client.0.smithi081.stdout:4/579: write da/d18/d1b/d26/d3e/f8a [4023092,73518] 0 2024-03-22T07:18:36.532 INFO:tasks.workunit.client.0.smithi081.stdout:0/472: mknod d3/d32/c9f 0 2024-03-22T07:18:36.532 INFO:tasks.workunit.client.0.smithi081.stdout:0/473: truncate d3/dc/d2b/d39/f8c 3303419 0 2024-03-22T07:18:36.534 INFO:tasks.workunit.client.0.smithi081.stdout:4/580: rename da/d18/d6e/cb6 to da/d18/d1b/d1e/d37/d1c/d74/cbd 0 2024-03-22T07:18:36.534 INFO:tasks.workunit.client.0.smithi081.stdout:4/581: fdatasync da/df/f65 0 2024-03-22T07:18:36.537 INFO:tasks.workunit.client.0.smithi081.stdout:4/582: link da/d18/d1b/d26/l95 da/d18/d42/d4f/daf/lbe 0 2024-03-22T07:18:36.540 INFO:tasks.workunit.client.0.smithi081.stdout:4/583: creat da/d18/d1b/d1e/d37/d1c/d74/fbf x:0 0 0 2024-03-22T07:18:36.566 INFO:tasks.workunit.client.0.smithi081.stdout:4/584: dwrite da/d18/d1b/d26/d3e/f55 [0,4194304] 0 2024-03-22T07:18:36.567 INFO:tasks.workunit.client.0.smithi081.stdout:0/474: dwrite d3/f1d [0,4194304] 0 2024-03-22T07:18:36.573 INFO:tasks.workunit.client.0.smithi081.stdout:4/585: dread da/d18/d42/d4f/f5b [0,4194304] 0 2024-03-22T07:18:36.573 INFO:tasks.workunit.client.0.smithi081.stdout:4/586: fsync da/d18/d6e/fa8 0 2024-03-22T07:18:36.573 INFO:tasks.workunit.client.0.smithi081.stdout:4/587: chown da/d18/d1b/d1e/d5f/l88 633178016 1 2024-03-22T07:18:36.583 INFO:tasks.workunit.client.0.smithi081.stdout:0/475: dwrite d3/d6a/f8e [0,4194304] 0 2024-03-22T07:18:36.583 INFO:tasks.workunit.client.0.smithi081.stdout:0/476: readlink d3/d6a/l82 0 2024-03-22T07:18:36.584 INFO:tasks.workunit.client.0.smithi081.stdout:0/477: dread d3/d4e/f54 [0,4194304] 0 2024-03-22T07:18:36.586 INFO:tasks.workunit.client.0.smithi081.stdout:0/478: creat d3/dc/d2b/fa0 x:0 0 0 2024-03-22T07:18:36.586 INFO:tasks.workunit.client.0.smithi081.stdout:0/479: creat d3/d6a/fa1 x:0 0 0 2024-03-22T07:18:36.587 INFO:tasks.workunit.client.0.smithi081.stdout:0/480: creat d3/dc/d2b/fa2 x:0 0 0 2024-03-22T07:18:36.587 INFO:tasks.workunit.client.0.smithi081.stdout:0/481: chown d3/l85 7354 1 2024-03-22T07:18:36.588 INFO:tasks.workunit.client.0.smithi081.stdout:0/482: creat d3/d6a/fa3 x:0 0 0 2024-03-22T07:18:36.590 INFO:tasks.workunit.client.0.smithi081.stdout:0/483: unlink d3/l9 0 2024-03-22T07:18:36.621 INFO:tasks.workunit.client.0.smithi081.stdout:4/588: dwrite da/d18/d1b/d1e/f41 [0,4194304] 0 2024-03-22T07:18:36.621 INFO:tasks.workunit.client.0.smithi081.stdout:0/484: dwrite d3/dc/d33/f97 [0,4194304] 0 2024-03-22T07:18:36.623 INFO:tasks.workunit.client.0.smithi081.stdout:4/589: creat da/d18/d1b/d1e/d5f/fc0 x:0 0 0 2024-03-22T07:18:36.623 INFO:tasks.workunit.client.0.smithi081.stdout:4/590: fsync da/df/d38/f54 0 2024-03-22T07:18:36.624 INFO:tasks.workunit.client.0.smithi081.stdout:0/485: mkdir d3/d9b/da4 0 2024-03-22T07:18:36.625 INFO:tasks.workunit.client.0.smithi081.stdout:4/591: symlink da/d18/d1b/d26/d32/dbc/lc1 0 2024-03-22T07:18:36.626 INFO:tasks.workunit.client.0.smithi081.stdout:4/592: chown da/df 57983 1 2024-03-22T07:18:36.626 INFO:tasks.workunit.client.0.smithi081.stdout:4/593: chown da/d18/d42/f9a 57 1 2024-03-22T07:18:36.626 INFO:tasks.workunit.client.0.smithi081.stdout:0/486: symlink d3/dc/d33/d62/la5 0 2024-03-22T07:18:36.628 INFO:tasks.workunit.client.0.smithi081.stdout:0/487: creat d3/d32/d35/d74/fa6 x:0 0 0 2024-03-22T07:18:36.628 INFO:tasks.workunit.client.0.smithi081.stdout:4/594: symlink da/d18/d1b/d34/lc2 0 2024-03-22T07:18:36.629 INFO:tasks.workunit.client.0.smithi081.stdout:0/488: symlink d3/d9b/la7 0 2024-03-22T07:18:36.641 INFO:tasks.workunit.client.0.smithi081.stdout:0/489: dread d3/f8 [0,4194304] 0 2024-03-22T07:18:36.641 INFO:tasks.workunit.client.0.smithi081.stdout:0/490: write d3/d32/d35/d74/fa6 [470058,98939] 0 2024-03-22T07:18:36.642 INFO:tasks.workunit.client.0.smithi081.stdout:4/595: dwrite da/df/d38/f54 [0,4194304] 0 2024-03-22T07:18:36.645 INFO:tasks.workunit.client.0.smithi081.stdout:4/596: mknod da/d18/d1b/d26/d32/cc3 0 2024-03-22T07:18:36.647 INFO:tasks.workunit.client.0.smithi081.stdout:0/491: write d3/dc/d33/f7d [1779076,43575] 0 2024-03-22T07:18:36.647 INFO:tasks.workunit.client.0.smithi081.stdout:0/492: chown d3/dc/f8d 12 1 2024-03-22T07:18:36.649 INFO:tasks.workunit.client.0.smithi081.stdout:0/493: rename d3/d4e/l61 to d3/dc/d40/la8 0 2024-03-22T07:18:36.650 INFO:tasks.workunit.client.0.smithi081.stdout:0/494: truncate d3/dc/f28 995587 0 2024-03-22T07:18:36.651 INFO:tasks.workunit.client.0.smithi081.stdout:0/495: creat d3/d32/fa9 x:0 0 0 2024-03-22T07:18:36.652 INFO:tasks.workunit.client.0.smithi081.stdout:0/496: creat d3/d9b/faa x:0 0 0 2024-03-22T07:18:36.653 INFO:tasks.workunit.client.0.smithi081.stdout:0/497: chown d3/dc/l26 11561 1 2024-03-22T07:18:36.653 INFO:tasks.workunit.client.0.smithi081.stdout:0/498: readlink d3/l85 0 2024-03-22T07:18:36.654 INFO:tasks.workunit.client.0.smithi081.stdout:0/499: mknod d3/dc/d33/cab 0 2024-03-22T07:18:36.654 INFO:tasks.workunit.client.0.smithi081.stdout:0/500: stat d3/dc/d2b/fa0 0 2024-03-22T07:18:36.654 INFO:tasks.workunit.client.0.smithi081.stdout:0/501: mknod d3/dc/cac 0 2024-03-22T07:18:36.660 INFO:tasks.workunit.client.0.smithi081.stdout:4/597: dwrite da/df/fa5 [0,4194304] 0 2024-03-22T07:18:36.666 INFO:tasks.workunit.client.0.smithi081.stdout:4/598: symlink da/d18/d42/d4f/lc4 0 2024-03-22T07:18:36.666 INFO:tasks.workunit.client.0.smithi081.stdout:4/599: dread - da/d18/d1b/d26/d3e/fa1 zero size 2024-03-22T07:18:36.667 INFO:tasks.workunit.client.0.smithi081.stdout:4/600: creat da/d18/d42/d4f/d7c/d90/fc5 x:0 0 0 2024-03-22T07:18:36.667 INFO:tasks.workunit.client.0.smithi081.stdout:0/502: truncate d3/f1d 2876526 0 2024-03-22T07:18:36.668 INFO:tasks.workunit.client.0.smithi081.stdout:4/601: mkdir da/d18/d42/d4f/d7c/d90/dc6 0 2024-03-22T07:18:36.771 INFO:tasks.workunit.client.0.smithi081.stdout:4/602: truncate da/d18/d1b/d34/f4c 1182193 0 2024-03-22T07:18:36.772 INFO:tasks.workunit.client.0.smithi081.stdout:0/503: symlink d3/d32/lad 0 2024-03-22T07:18:36.773 INFO:tasks.workunit.client.0.smithi081.stdout:4/603: chown da/d18/d1b/d26/d3e/c52 860 1 2024-03-22T07:18:36.774 INFO:tasks.workunit.client.0.smithi081.stdout:0/504: rename d3/dc/d2b/l6d to d3/d32/d35/d91/d9d/lae 0 2024-03-22T07:18:36.776 INFO:tasks.workunit.client.0.smithi081.stdout:0/505: symlink d3/dc/d33/d62/laf 0 2024-03-22T07:18:36.776 INFO:tasks.workunit.client.0.smithi081.stdout:0/506: stat d3/dc/d2b/d39 0 2024-03-22T07:18:36.776 INFO:tasks.workunit.client.0.smithi081.stdout:0/507: fsync d3/dc/d33/d62/f78 0 2024-03-22T07:18:36.777 INFO:tasks.workunit.client.0.smithi081.stdout:4/604: symlink da/d18/d1b/d1e/d37/d1c/d74/lc7 0 2024-03-22T07:18:36.777 INFO:tasks.workunit.client.0.smithi081.stdout:4/605: truncate da/d18/d1b/d1e/d37/d1c/d74/fbf 444553 0 2024-03-22T07:18:36.779 INFO:tasks.workunit.client.0.smithi081.stdout:0/508: unlink d3/dc/d33/d62/l7a 0 2024-03-22T07:18:36.779 INFO:tasks.workunit.client.0.smithi081.stdout:4/606: chown l2 154520 1 2024-03-22T07:18:36.781 INFO:tasks.workunit.client.0.smithi081.stdout:0/509: stat d3/dc/d2b/d39/f3c 0 2024-03-22T07:18:36.786 INFO:tasks.workunit.client.0.smithi081.stdout:4/607: rmdir da/d18/d1b/d26 39 2024-03-22T07:18:36.786 INFO:tasks.workunit.client.0.smithi081.stdout:4/608: fsync da/d18/d1b/d34/f44 0 2024-03-22T07:18:36.786 INFO:tasks.workunit.client.0.smithi081.stdout:4/609: write da/d18/d1b/d26/d3e/fa1 [116745,94246] 0 2024-03-22T07:18:36.786 INFO:tasks.workunit.client.0.smithi081.stdout:4/610: truncate da/d18/d42/d4f/fad 685583 0 2024-03-22T07:18:36.786 INFO:tasks.workunit.client.0.smithi081.stdout:4/611: symlink da/d18/d6e/lc8 2 2024-03-22T07:18:36.788 INFO:tasks.workunit.client.0.smithi081.stdout:4/612: mkdir da/df/d38/dc9 0 2024-03-22T07:18:36.789 INFO:tasks.workunit.client.0.smithi081.stdout:0/510: link d3/dc/d2b/d4d/c71 d3/dc/cb0 0 2024-03-22T07:18:36.789 INFO:tasks.workunit.client.0.smithi081.stdout:0/511: chown d3/dc/d40/f9a 19809345 1 2024-03-22T07:18:36.790 INFO:tasks.workunit.client.0.smithi081.stdout:4/613: rmdir da/d18/d42/d4f/d7c/d90/dc6 0 2024-03-22T07:18:36.790 INFO:tasks.workunit.client.0.smithi081.stdout:0/512: mknod d3/dc/d40/d69/d92/cb1 0 2024-03-22T07:18:36.792 INFO:tasks.workunit.client.0.smithi081.stdout:2/346: sync 2024-03-22T07:18:36.792 INFO:tasks.workunit.client.0.smithi081.stdout:2/347: fdatasync da/db/f35 0 2024-03-22T07:18:36.792 INFO:tasks.workunit.client.0.smithi081.stdout:2/348: chown da/db/d39/d47/f43 2630 1 2024-03-22T07:18:36.793 INFO:tasks.workunit.client.0.smithi081.stdout:4/614: rmdir da/d18/d1b/d1e/d37/d1c/d74/d98/db8 39 2024-03-22T07:18:36.795 INFO:tasks.workunit.client.0.smithi081.stdout:2/349: mkdir da/d11/d21/d71 0 2024-03-22T07:18:36.795 INFO:tasks.workunit.client.0.smithi081.stdout:2/350: mkdir da/db/d6d/d72 0 2024-03-22T07:18:36.796 INFO:tasks.workunit.client.0.smithi081.stdout:2/351: rmdir da/d11/d21 39 2024-03-22T07:18:36.801 INFO:tasks.workunit.client.0.smithi081.stdout:0/513: dread d3/d6a/f8e [0,4194304] 0 2024-03-22T07:18:36.803 INFO:tasks.workunit.client.0.smithi081.stdout:0/514: creat d3/dc/d2b/d4d/fb2 x:0 0 0 2024-03-22T07:18:36.803 INFO:tasks.workunit.client.0.smithi081.stdout:4/615: write da/d18/d1b/d26/d3e/f55 [2837776,72933] 0 2024-03-22T07:18:36.805 INFO:tasks.workunit.client.0.smithi081.stdout:0/515: mkdir d3/dc/d40/d69/db3 0 2024-03-22T07:18:36.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/616: mknod da/d18/d42/cca 0 2024-03-22T07:18:36.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/617: chown da/d18/d42/d4f/d7c/d90 253001 1 2024-03-22T07:18:36.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/618: fsync da/d18/d1b/d1e/f41 0 2024-03-22T07:18:36.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/619: dread - da/d18/d6e/fa8 zero size 2024-03-22T07:18:36.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/620: chown l4 1751 1 2024-03-22T07:18:36.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/621: stat l2 0 2024-03-22T07:18:36.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/622: creat da/d18/d1b/d26/fcb x:0 0 0 2024-03-22T07:18:36.822 INFO:tasks.workunit.client.0.smithi081.stdout:2/352: dwrite da/db/d39/d47/d3b/d41/d50/d6a/d70/f40 [0,4194304] 0 2024-03-22T07:18:36.823 INFO:tasks.workunit.client.0.smithi081.stdout:2/353: read da/db/d39/d45/f4c [50026,86835] 0 2024-03-22T07:18:36.825 INFO:tasks.workunit.client.0.smithi081.stdout:4/623: dwrite da/d18/d1b/d26/d3e/f55 [0,4194304] 0 2024-03-22T07:18:36.836 INFO:tasks.workunit.client.0.smithi081.stdout:1/382: sync 2024-03-22T07:18:36.836 INFO:tasks.workunit.client.0.smithi081.stdout:3/323: sync 2024-03-22T07:18:36.836 INFO:tasks.workunit.client.0.smithi081.stdout:8/501: sync 2024-03-22T07:18:36.836 INFO:tasks.workunit.client.0.smithi081.stdout:3/324: stat db/d2e/l56 0 2024-03-22T07:18:36.836 INFO:tasks.workunit.client.0.smithi081.stdout:8/502: chown d2/d3/d28/f82 24066 1 2024-03-22T07:18:36.852 INFO:tasks.workunit.client.0.smithi081.stdout:1/383: dwrite d6/d18/d1f/d4c/f6c [0,4194304] 0 2024-03-22T07:18:36.853 INFO:tasks.workunit.client.0.smithi081.stdout:1/384: getdents d6/d18/d60 0 2024-03-22T07:18:36.854 INFO:tasks.workunit.client.0.smithi081.stdout:2/354: link da/db/d39/d46/f44 da/db/d6d/f73 0 2024-03-22T07:18:36.855 INFO:tasks.workunit.client.0.smithi081.stdout:1/385: mknod d6/d18/d1f/d24/d48/d5c/c82 0 2024-03-22T07:18:36.856 INFO:tasks.workunit.client.0.smithi081.stdout:3/325: unlink db/de/f17 0 2024-03-22T07:18:36.857 INFO:tasks.workunit.client.0.smithi081.stdout:2/355: creat da/db/d39/d46/d42/f74 x:0 0 0 2024-03-22T07:18:36.858 INFO:tasks.workunit.client.0.smithi081.stdout:4/624: symlink da/d18/d42/lcc 0 2024-03-22T07:18:36.863 INFO:tasks.workunit.client.0.smithi081.stdout:2/356: rename da/db/d39/d45/c5c to da/db/d6d/c75 0 2024-03-22T07:18:36.865 INFO:tasks.workunit.client.0.smithi081.stdout:4/625: stat da/d18/d1b/d1e/d37/c97 0 2024-03-22T07:18:36.866 INFO:tasks.workunit.client.0.smithi081.stdout:3/326: rmdir db/d2e/d3a 39 2024-03-22T07:18:36.871 INFO:tasks.workunit.client.0.smithi081.stdout:4/626: truncate da/d18/d1b/d26/d32/f91 3051190 0 2024-03-22T07:18:36.872 INFO:tasks.workunit.client.0.smithi081.stdout:4/627: write da/d18/d1b/d26/f43 [1283682,67450] 0 2024-03-22T07:18:36.873 INFO:tasks.workunit.client.0.smithi081.stdout:3/327: creat db/d2e/d3a/f6f x:0 0 0 2024-03-22T07:18:36.875 INFO:tasks.workunit.client.0.smithi081.stdout:8/503: dwrite d2/d3/d28/d2b/f4b [0,4194304] 0 2024-03-22T07:18:36.877 INFO:tasks.workunit.client.0.smithi081.stdout:2/357: dread da/db/d39/d46/f36 [0,4194304] 0 2024-03-22T07:18:36.878 INFO:tasks.workunit.client.0.smithi081.stdout:8/504: link d2/d3/d28/d2b/f58 d2/d3/d28/d2b/fac 0 2024-03-22T07:18:36.879 INFO:tasks.workunit.client.0.smithi081.stdout:8/505: mknod d2/d3/d19/d1e/d13/d22/cad 0 2024-03-22T07:18:36.881 INFO:tasks.workunit.client.0.smithi081.stdout:8/506: creat d2/d3/fae x:0 0 0 2024-03-22T07:18:36.882 INFO:tasks.workunit.client.0.smithi081.stdout:8/507: creat d2/d3/d19/d1e/faf x:0 0 0 2024-03-22T07:18:36.884 INFO:tasks.workunit.client.0.smithi081.stdout:2/358: dread da/f28 [0,4194304] 0 2024-03-22T07:18:36.888 INFO:tasks.workunit.client.0.smithi081.stdout:8/508: write d2/d3/d28/d2b/fac [2377968,29205] 0 2024-03-22T07:18:36.890 INFO:tasks.workunit.client.0.smithi081.stdout:8/509: mknod d2/d3/d19/d64/cb0 0 2024-03-22T07:18:36.891 INFO:tasks.workunit.client.0.smithi081.stdout:4/628: dwrite da/d18/d42/d4f/d7c/d90/fc5 [0,4194304] 0 2024-03-22T07:18:36.892 INFO:tasks.workunit.client.0.smithi081.stdout:2/359: read da/d11/d21/f33 [6603842,114672] 0 2024-03-22T07:18:36.892 INFO:tasks.workunit.client.0.smithi081.stdout:8/510: unlink d2/f11 0 2024-03-22T07:18:36.894 INFO:tasks.workunit.client.0.smithi081.stdout:4/629: truncate da/f1d 1948292 0 2024-03-22T07:18:36.895 INFO:tasks.workunit.client.0.smithi081.stdout:2/360: symlink da/db/d39/d47/d3b/d41/d50/l76 0 2024-03-22T07:18:36.895 INFO:tasks.workunit.client.0.smithi081.stdout:4/630: stat da/d18/d42/d4a/c77 0 2024-03-22T07:18:36.897 INFO:tasks.workunit.client.0.smithi081.stdout:4/631: symlink da/d18/d1b/d26/lcd 0 2024-03-22T07:18:36.898 INFO:tasks.workunit.client.0.smithi081.stdout:4/632: mkdir da/d18/d1b/d26/dce 0 2024-03-22T07:18:36.898 INFO:tasks.workunit.client.0.smithi081.stdout:4/633: fsync da/df/f5a 0 2024-03-22T07:18:36.899 INFO:tasks.workunit.client.0.smithi081.stdout:4/634: symlink da/d18/d1b/d26/d32/lcf 0 2024-03-22T07:18:36.900 INFO:tasks.workunit.client.0.smithi081.stdout:4/635: link da/d18/f21 da/d18/d42/d4f/d7c/fd0 0 2024-03-22T07:18:36.901 INFO:tasks.workunit.client.0.smithi081.stdout:4/636: creat da/d18/d1b/d26/d3e/fd1 x:0 0 0 2024-03-22T07:18:36.901 INFO:tasks.workunit.client.0.smithi081.stdout:4/637: readlink da/d18/d42/l7e 0 2024-03-22T07:18:36.902 INFO:tasks.workunit.client.0.smithi081.stdout:4/638: creat da/fd2 x:0 0 0 2024-03-22T07:18:36.902 INFO:tasks.workunit.client.0.smithi081.stdout:4/639: write da/f7b [1801405,27772] 0 2024-03-22T07:18:36.921 INFO:tasks.workunit.client.0.smithi081.stdout:4/640: dwrite da/d18/d1b/d1e/d5f/fc0 [0,4194304] 0 2024-03-22T07:18:36.923 INFO:tasks.workunit.client.0.smithi081.stdout:4/641: creat da/d18/d1b/d1e/d37/d1c/d74/fd3 x:0 0 0 2024-03-22T07:18:36.931 INFO:tasks.workunit.client.0.smithi081.stdout:2/361: dwrite da/db/d39/d47/d3b/d41/d50/d6a/d70/f40 [4194304,4194304] 0 2024-03-22T07:18:36.931 INFO:tasks.workunit.client.0.smithi081.stdout:2/362: chown da/db/f35 783386 1 2024-03-22T07:18:36.933 INFO:tasks.workunit.client.0.smithi081.stdout:4/642: write da/d18/d1b/d34/f44 [3785886,128420] 0 2024-03-22T07:18:36.933 INFO:tasks.workunit.client.0.smithi081.stdout:1/386: write d6/f17 [4027466,66981] 0 2024-03-22T07:18:36.943 INFO:tasks.workunit.client.0.smithi081.stdout:2/363: symlink da/db/d39/d47/d3b/d41/l77 0 2024-03-22T07:18:36.944 INFO:tasks.workunit.client.0.smithi081.stdout:1/387: creat d6/d18/d41/d49/d56/d30/f83 x:0 0 0 2024-03-22T07:18:36.956 INFO:tasks.workunit.client.0.smithi081.stdout:4/643: dwrite da/df/d38/f76 [0,4194304] 0 2024-03-22T07:18:36.956 INFO:tasks.workunit.client.0.smithi081.stdout:4/644: write da/d18/d1b/d1e/d37/d1c/d74/d98/db8/fbb [719664,11951] 0 2024-03-22T07:18:36.956 INFO:tasks.workunit.client.0.smithi081.stdout:4/645: stat l4 0 2024-03-22T07:18:36.959 INFO:tasks.workunit.client.0.smithi081.stdout:4/646: link da/d18/d6e/c78 da/d18/d1b/d26/d3e/cd4 0 2024-03-22T07:18:36.959 INFO:tasks.workunit.client.0.smithi081.stdout:4/647: mknod da/df/cd5 0 2024-03-22T07:18:36.960 INFO:tasks.workunit.client.0.smithi081.stdout:4/648: read da/d18/d1b/d1e/d37/d1c/d74/fbf [1441,62473] 0 2024-03-22T07:18:36.960 INFO:tasks.workunit.client.0.smithi081.stdout:4/649: dread - da/d18/d1b/d26/d3e/fd1 zero size 2024-03-22T07:18:36.960 INFO:tasks.workunit.client.0.smithi081.stdout:1/388: dwrite d6/f7b [0,4194304] 0 2024-03-22T07:18:36.967 INFO:tasks.workunit.client.0.smithi081.stdout:2/364: write da/db/f29 [804789,42043] 0 2024-03-22T07:18:36.981 INFO:tasks.workunit.client.0.smithi081.stdout:4/650: dwrite da/df/d38/f46 [0,4194304] 0 2024-03-22T07:18:36.982 INFO:tasks.workunit.client.0.smithi081.stdout:1/389: rename d6/d18/l36 to d6/d18/d41/d49/d56/d30/l84 0 2024-03-22T07:18:36.982 INFO:tasks.workunit.client.0.smithi081.stdout:1/390: truncate d6/d18/d1f/d24/d48/d6e/f75 921141 0 2024-03-22T07:18:36.996 INFO:tasks.workunit.client.0.smithi081.stdout:1/391: dread d6/d18/d1f/d24/f2b [0,4194304] 0 2024-03-22T07:18:36.999 INFO:tasks.workunit.client.0.smithi081.stdout:4/651: dwrite da/d18/d1b/d26/f43 [0,4194304] 0 2024-03-22T07:18:37.002 INFO:tasks.workunit.client.0.smithi081.stdout:4/652: getdents da/d18/d1b/d1e/d5f 0 2024-03-22T07:18:37.004 INFO:tasks.workunit.client.0.smithi081.stdout:4/653: rmdir da/d18/d1b/d1e/d37/d1c/d74 39 2024-03-22T07:18:37.013 INFO:tasks.workunit.client.0.smithi081.stdout:4/654: dread da/df/f5a [0,4194304] 0 2024-03-22T07:18:37.013 INFO:tasks.workunit.client.0.smithi081.stdout:4/655: write da/d3d/faa [625700,27693] 0 2024-03-22T07:18:37.013 INFO:tasks.workunit.client.0.smithi081.stdout:4/656: truncate da/df/fa5 3959182 0 2024-03-22T07:18:37.015 INFO:tasks.workunit.client.0.smithi081.stdout:4/657: unlink da/d18/d42/lba 0 2024-03-22T07:18:37.016 INFO:tasks.workunit.client.0.smithi081.stdout:4/658: write da/d18/d1b/d1e/d37/d1c/d74/d98/db8/fbb [1028071,47361] 0 2024-03-22T07:18:37.016 INFO:tasks.workunit.client.0.smithi081.stdout:6/432: sync 2024-03-22T07:18:37.016 INFO:tasks.workunit.client.0.smithi081.stdout:7/562: sync 2024-03-22T07:18:37.016 INFO:tasks.workunit.client.0.smithi081.stdout:6/433: chown d3/da/de/d31/d40/d50/f6f 4591634 1 2024-03-22T07:18:37.016 INFO:tasks.workunit.client.0.smithi081.stdout:5/628: sync 2024-03-22T07:18:37.019 INFO:tasks.workunit.client.0.smithi081.stdout:5/629: unlink d4/d10/c48 0 2024-03-22T07:18:37.022 INFO:tasks.workunit.client.0.smithi081.stdout:4/659: dread da/d18/d1b/d26/d32/f91 [0,4194304] 0 2024-03-22T07:18:37.024 INFO:tasks.workunit.client.0.smithi081.stdout:5/630: rename d4/d10/d20/f7d to d4/d75/fd4 0 2024-03-22T07:18:37.026 INFO:tasks.workunit.client.0.smithi081.stdout:4/660: creat da/d18/d1b/d26/dce/fd6 x:0 0 0 2024-03-22T07:18:37.027 INFO:tasks.workunit.client.0.smithi081.stdout:5/631: rmdir d4/df 39 2024-03-22T07:18:37.027 INFO:tasks.workunit.client.0.smithi081.stdout:5/632: chown d4/d10/d20/d4f/f5f 31 1 2024-03-22T07:18:37.028 INFO:tasks.workunit.client.0.smithi081.stdout:4/661: mkdir da/d18/d42/d4f/d7c/d90/dd7 0 2024-03-22T07:18:37.036 INFO:tasks.workunit.client.0.smithi081.stdout:7/563: dwrite d9/d10/d4b/d61/fa4 [0,4194304] 0 2024-03-22T07:18:37.066 INFO:tasks.workunit.client.0.smithi081.stdout:4/662: dwrite da/d3d/f5e [0,4194304] 0 2024-03-22T07:18:37.067 INFO:tasks.workunit.client.0.smithi081.stdout:6/434: dwrite d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f6b [8388608,4194304] 0 2024-03-22T07:18:37.067 INFO:tasks.workunit.client.0.smithi081.stdout:6/435: fdatasync d3/da/de/f56 0 2024-03-22T07:18:37.067 INFO:tasks.workunit.client.0.smithi081.stdout:7/564: dwrite d9/d14/f24 [0,4194304] 0 2024-03-22T07:18:37.075 INFO:tasks.workunit.client.0.smithi081.stdout:6/436: creat d3/da/de/d31/d40/d50/d63/d5e/f8c x:0 0 0 2024-03-22T07:18:37.076 INFO:tasks.workunit.client.0.smithi081.stdout:6/437: mknod d3/da/de/d31/d40/d50/d63/c8d 0 2024-03-22T07:18:37.082 INFO:tasks.workunit.client.0.smithi081.stdout:7/565: dread d9/dc/f1f [0,4194304] 0 2024-03-22T07:18:37.086 INFO:tasks.workunit.client.0.smithi081.stdout:7/566: read d9/d10/d16/f7a [727589,4354] 0 2024-03-22T07:18:37.087 INFO:tasks.workunit.client.0.smithi081.stdout:4/663: dwrite da/d3d/faa [0,4194304] 0 2024-03-22T07:18:37.089 INFO:tasks.workunit.client.0.smithi081.stdout:4/664: truncate f0 270571 0 2024-03-22T07:18:37.090 INFO:tasks.workunit.client.0.smithi081.stdout:4/665: rmdir da/d18/d1b/d1e 39 2024-03-22T07:18:37.091 INFO:tasks.workunit.client.0.smithi081.stdout:4/666: stat da/d18/d1b/d1e/d5f/l88 0 2024-03-22T07:18:37.103 INFO:tasks.workunit.client.0.smithi081.stdout:6/438: dwrite d3/da/de/d31/d40/d50/d63/d26/d52/f69 [0,4194304] 0 2024-03-22T07:18:37.103 INFO:tasks.workunit.client.0.smithi081.stdout:6/439: write d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f6e [1145053,131071] 0 2024-03-22T07:18:37.106 INFO:tasks.workunit.client.0.smithi081.stdout:6/440: creat d3/da/de/f8e x:0 0 0 2024-03-22T07:18:37.113 INFO:tasks.workunit.client.0.smithi081.stdout:7/567: dwrite d9/dc/f38 [0,4194304] 0 2024-03-22T07:18:37.114 INFO:tasks.workunit.client.0.smithi081.stdout:9/532: sync 2024-03-22T07:18:37.115 INFO:tasks.workunit.client.0.smithi081.stdout:4/667: dwrite da/d18/d1b/d26/d3e/f55 [0,4194304] 0 2024-03-22T07:18:37.116 INFO:tasks.workunit.client.0.smithi081.stdout:9/533: symlink d2/d2b/d49/d86/la7 0 2024-03-22T07:18:37.118 INFO:tasks.workunit.client.0.smithi081.stdout:7/568: link d9/d10/l37 d9/d10/d16/d1e/d27/lc7 0 2024-03-22T07:18:37.119 INFO:tasks.workunit.client.0.smithi081.stdout:4/668: read da/d18/d1b/d26/d32/f91 [2869808,126440] 0 2024-03-22T07:18:37.119 INFO:tasks.workunit.client.0.smithi081.stdout:4/669: dread - da/d18/d1b/d26/fcb zero size 2024-03-22T07:18:37.119 INFO:tasks.workunit.client.0.smithi081.stdout:4/670: dread - da/d18/d42/fb0 zero size 2024-03-22T07:18:37.119 INFO:tasks.workunit.client.0.smithi081.stdout:4/671: write da/d18/d1b/d26/fcb [327789,80916] 0 2024-03-22T07:18:37.120 INFO:tasks.workunit.client.0.smithi081.stdout:9/534: creat d2/d73/d85/d7a/d97/fa8 x:0 0 0 2024-03-22T07:18:37.120 INFO:tasks.workunit.client.0.smithi081.stdout:7/569: stat d9/d10/l37 0 2024-03-22T07:18:37.121 INFO:tasks.workunit.client.0.smithi081.stdout:9/535: creat d2/d10/d30/d57/d66/d75/fa9 x:0 0 0 2024-03-22T07:18:37.121 INFO:tasks.workunit.client.0.smithi081.stdout:9/536: truncate d2/d2b/d49/d86/f9a 1288296 0 2024-03-22T07:18:37.121 INFO:tasks.workunit.client.0.smithi081.stdout:7/570: readlink d9/dc/l91 0 2024-03-22T07:18:37.122 INFO:tasks.workunit.client.0.smithi081.stdout:7/571: creat d9/d3f/d52/fc8 x:0 0 0 2024-03-22T07:18:37.123 INFO:tasks.workunit.client.0.smithi081.stdout:9/537: symlink d2/d5/laa 0 2024-03-22T07:18:37.128 INFO:tasks.workunit.client.0.smithi081.stdout:6/441: dwrite d3/da/de/d31/d40/d50/f6f [0,4194304] 0 2024-03-22T07:18:37.131 INFO:tasks.workunit.client.0.smithi081.stdout:6/442: stat d3/da/f53 0 2024-03-22T07:18:37.131 INFO:tasks.workunit.client.0.smithi081.stdout:6/443: stat d3/da/de/d31/d40/d50/f87 0 2024-03-22T07:18:37.132 INFO:tasks.workunit.client.0.smithi081.stdout:9/538: dread d2/d73/d85/f6d [0,4194304] 0 2024-03-22T07:18:37.132 INFO:tasks.workunit.client.0.smithi081.stdout:9/539: write d2/d73/f7e [37206,87459] 0 2024-03-22T07:18:37.133 INFO:tasks.workunit.client.0.smithi081.stdout:6/444: rmdir d3/da/de/d31/d40/d50/d63/d7d 39 2024-03-22T07:18:37.134 INFO:tasks.workunit.client.0.smithi081.stdout:9/540: unlink d2/f25 0 2024-03-22T07:18:37.136 INFO:tasks.workunit.client.0.smithi081.stdout:9/541: symlink d2/d5d/d5a/lab 0 2024-03-22T07:18:37.137 INFO:tasks.workunit.client.0.smithi081.stdout:4/672: dwrite da/d18/d1b/d26/f66 [4194304,4194304] 0 2024-03-22T07:18:37.137 INFO:tasks.workunit.client.0.smithi081.stdout:6/445: rename d3/da/de/f56 to d3/da/f8f 0 2024-03-22T07:18:37.140 INFO:tasks.workunit.client.0.smithi081.stdout:9/542: mknod d2/d10/d30/cac 0 2024-03-22T07:18:37.142 INFO:tasks.workunit.client.0.smithi081.stdout:6/446: mknod d3/da/c90 0 2024-03-22T07:18:37.142 INFO:tasks.workunit.client.0.smithi081.stdout:9/543: mkdir d2/d10/d30/d57/dad 0 2024-03-22T07:18:37.148 INFO:tasks.workunit.client.0.smithi081.stdout:9/544: mkdir d2/d10/d30/d57/d66/d75/d94/dae 0 2024-03-22T07:18:37.150 INFO:tasks.workunit.client.0.smithi081.stdout:4/673: rename da/d18/d42 to da/df/d38/dd8 0 2024-03-22T07:18:37.150 INFO:tasks.workunit.client.0.smithi081.stdout:6/447: mknod d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/c91 0 2024-03-22T07:18:37.151 INFO:tasks.workunit.client.0.smithi081.stdout:9/545: rename d2/d73/d85/f2f to d2/d10/d30/d57/d66/d75/faf 0 2024-03-22T07:18:37.152 INFO:tasks.workunit.client.0.smithi081.stdout:9/546: dread - d2/d78/fa6 zero size 2024-03-22T07:18:37.152 INFO:tasks.workunit.client.0.smithi081.stdout:4/674: creat da/d57/fd9 x:0 0 0 2024-03-22T07:18:37.154 INFO:tasks.workunit.client.0.smithi081.stdout:9/547: creat d2/d10/fb0 x:0 0 0 2024-03-22T07:18:37.154 INFO:tasks.workunit.client.0.smithi081.stdout:9/548: chown d2/d10/d30/f9c 1 1 2024-03-22T07:18:37.154 INFO:tasks.workunit.client.0.smithi081.stdout:9/549: write d2/d2b/f43 [173599,24343] 0 2024-03-22T07:18:37.155 INFO:tasks.workunit.client.0.smithi081.stdout:4/675: creat da/df/d38/dd8/d4a/fda x:0 0 0 2024-03-22T07:18:37.156 INFO:tasks.workunit.client.0.smithi081.stdout:9/550: creat d2/d73/d85/fb1 x:0 0 0 2024-03-22T07:18:37.156 INFO:tasks.workunit.client.0.smithi081.stdout:7/572: dwrite d9/d10/d16/d1e/f58 [0,4194304] 0 2024-03-22T07:18:37.156 INFO:tasks.workunit.client.0.smithi081.stdout:7/573: fsync d9/dc/f90 0 2024-03-22T07:18:37.156 INFO:tasks.workunit.client.0.smithi081.stdout:4/676: getdents da/df/d38/dd8/d4f/d7c/d90 0 2024-03-22T07:18:37.158 INFO:tasks.workunit.client.0.smithi081.stdout:9/551: symlink d2/d10/d30/d57/d66/d75/d94/lb2 0 2024-03-22T07:18:37.159 INFO:tasks.workunit.client.0.smithi081.stdout:4/677: getdents da/d18/d1b/d1e/d37 0 2024-03-22T07:18:37.161 INFO:tasks.workunit.client.0.smithi081.stdout:6/448: write d3/da/de/d31/d40/d50/d63/d26/f33 [389924,65619] 0 2024-03-22T07:18:37.161 INFO:tasks.workunit.client.0.smithi081.stdout:6/449: write d3/da/f83 [901889,105168] 0 2024-03-22T07:18:37.161 INFO:tasks.workunit.client.0.smithi081.stdout:9/552: symlink d2/d10/d30/d57/d66/d75/d94/dae/lb3 0 2024-03-22T07:18:37.162 INFO:tasks.workunit.client.0.smithi081.stdout:9/553: dread d2/d2b/f70 [0,4194304] 0 2024-03-22T07:18:37.163 INFO:tasks.workunit.client.0.smithi081.stdout:4/678: dread da/d18/d1b/d34/f64 [0,4194304] 0 2024-03-22T07:18:37.164 INFO:tasks.workunit.client.0.smithi081.stdout:7/574: rename d9/d10/d4b/l97 to d9/d10/d16/d1e/d27/d6d/d8c/lc9 0 2024-03-22T07:18:37.166 INFO:tasks.workunit.client.0.smithi081.stdout:6/450: link d3/da/f8f d3/da/de/d31/d40/d50/d63/d26/f92 0 2024-03-22T07:18:37.170 INFO:tasks.workunit.client.0.smithi081.stdout:9/554: symlink d2/d2b/lb4 0 2024-03-22T07:18:37.170 INFO:tasks.workunit.client.0.smithi081.stdout:9/555: fsync d2/d10/d30/d57/f76 0 2024-03-22T07:18:37.170 INFO:tasks.workunit.client.0.smithi081.stdout:6/451: truncate d3/da/fd 613978 0 2024-03-22T07:18:37.171 INFO:tasks.workunit.client.0.smithi081.stdout:4/679: mknod da/d18/cdb 0 2024-03-22T07:18:37.177 INFO:tasks.workunit.client.0.smithi081.stdout:7/575: dread d9/d10/d16/d1e/f2c [0,4194304] 0 2024-03-22T07:18:37.179 INFO:tasks.workunit.client.0.smithi081.stdout:6/452: dread d3/da/de/f27 [0,4194304] 0 2024-03-22T07:18:37.180 INFO:tasks.workunit.client.0.smithi081.stdout:7/576: getdents d9 0 2024-03-22T07:18:37.182 INFO:tasks.workunit.client.0.smithi081.stdout:4/680: dread da/d18/d1b/d1e/d37/d1c/d74/d98/fa4 [0,4194304] 0 2024-03-22T07:18:37.190 INFO:tasks.workunit.client.0.smithi081.stdout:7/577: dread d9/dc/f1f [0,4194304] 0 2024-03-22T07:18:37.190 INFO:tasks.workunit.client.0.smithi081.stdout:6/453: dread d3/da/de/f27 [0,4194304] 0 2024-03-22T07:18:37.190 INFO:tasks.workunit.client.0.smithi081.stdout:9/556: dwrite d2/d2b/f3b [0,4194304] 0 2024-03-22T07:18:37.193 INFO:tasks.workunit.client.0.smithi081.stdout:6/454: creat d3/d67/f93 x:0 0 0 2024-03-22T07:18:37.193 INFO:tasks.workunit.client.0.smithi081.stdout:9/557: dread d2/d10/d30/d57/f7d [0,4194304] 0 2024-03-22T07:18:37.201 INFO:tasks.workunit.client.0.smithi081.stdout:6/455: link d3/da/de/d31/d40/d50/d63/d26/d52/l77 d3/da/de/d31/d40/d50/d63/d26/d52/l94 0 2024-03-22T07:18:37.201 INFO:tasks.workunit.client.0.smithi081.stdout:6/456: chown d3/da/de/d31/d40/d50/d63/d5e 1031 1 2024-03-22T07:18:37.201 INFO:tasks.workunit.client.0.smithi081.stdout:6/457: stat d3/da/de/d31/d40/d50/f76 0 2024-03-22T07:18:37.201 INFO:tasks.workunit.client.0.smithi081.stdout:6/458: chown d3/da/de/d31/d40/d50/f7a 52 1 2024-03-22T07:18:37.209 INFO:tasks.workunit.client.0.smithi081.stdout:4/681: dwrite f3 [0,4194304] 0 2024-03-22T07:18:37.209 INFO:tasks.workunit.client.0.smithi081.stdout:7/578: dwrite d9/d10/d16/d1e/d27/f5a [4194304,4194304] 0 2024-03-22T07:18:37.209 INFO:tasks.workunit.client.0.smithi081.stdout:7/579: fdatasync d9/d10/d16/f6b 0 2024-03-22T07:18:37.210 INFO:tasks.workunit.client.0.smithi081.stdout:4/682: truncate da/df/d38/dd8/d4f/d7c/fd0 1422096 0 2024-03-22T07:18:37.212 INFO:tasks.workunit.client.0.smithi081.stdout:7/580: truncate f7 2854783 0 2024-03-22T07:18:37.217 INFO:tasks.workunit.client.0.smithi081.stdout:4/683: mkdir da/d18/ddc 0 2024-03-22T07:18:37.219 INFO:tasks.workunit.client.0.smithi081.stdout:7/581: mknod d9/d10/d16/d1e/d27/d6d/d8c/d9d/cca 0 2024-03-22T07:18:37.219 INFO:tasks.workunit.client.0.smithi081.stdout:7/582: stat d9/d10/d4b/d9b 0 2024-03-22T07:18:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:7/583: link d9/d10/d16/d1e/d27/d6d/fa5 d9/d10/d16/fcb 0 2024-03-22T07:18:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:7/584: fsync d9/d10/d16/d1e/f58 0 2024-03-22T07:18:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:7/585: write f1 [5089941,93127] 0 2024-03-22T07:18:37.222 INFO:tasks.workunit.client.0.smithi081.stdout:6/459: dwrite d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f5f [4194304,4194304] 0 2024-03-22T07:18:37.224 INFO:tasks.workunit.client.0.smithi081.stdout:6/460: mknod d3/da/de/d31/d40/d50/d63/d26/d3f/c95 0 2024-03-22T07:18:37.225 INFO:tasks.workunit.client.0.smithi081.stdout:6/461: symlink d3/da/de/d31/l96 0 2024-03-22T07:18:37.225 INFO:tasks.workunit.client.0.smithi081.stdout:6/462: write d3/da/de/f28 [1162513,60420] 0 2024-03-22T07:18:37.225 INFO:tasks.workunit.client.0.smithi081.stdout:6/463: truncate d3/da/de/d31/d40/d50/d63/f68 503738 0 2024-03-22T07:18:37.235 INFO:tasks.workunit.client.0.smithi081.stdout:7/586: write d9/d10/d16/d1e/f32 [1141596,103475] 0 2024-03-22T07:18:37.235 INFO:tasks.workunit.client.0.smithi081.stdout:7/587: write d9/d3f/db3/fc6 [702417,66891] 0 2024-03-22T07:18:37.235 INFO:tasks.workunit.client.0.smithi081.stdout:9/558: dwrite d2/f11 [0,4194304] 0 2024-03-22T07:18:37.240 INFO:tasks.workunit.client.0.smithi081.stdout:7/588: link d9/dc/f4a d9/d10/d4b/d61/fcc 0 2024-03-22T07:18:37.240 INFO:tasks.workunit.client.0.smithi081.stdout:7/589: chown d9/d10/d2a/d73 49598100 1 2024-03-22T07:18:37.241 INFO:tasks.workunit.client.0.smithi081.stdout:7/590: dread d9/d10/d53/fb5 [0,4194304] 0 2024-03-22T07:18:37.243 INFO:tasks.workunit.client.0.smithi081.stdout:6/464: dwrite d3/da/de/d31/d40/d50/d63/f68 [0,4194304] 0 2024-03-22T07:18:37.244 INFO:tasks.workunit.client.0.smithi081.stdout:4/684: dwrite da/df/d38/fb9 [0,4194304] 0 2024-03-22T07:18:37.244 INFO:tasks.workunit.client.0.smithi081.stdout:7/591: unlink d9/d10/d2a/c86 0 2024-03-22T07:18:37.245 INFO:tasks.workunit.client.0.smithi081.stdout:6/465: dread d3/da/de/d31/d40/d50/d63/d26/f71 [0,4194304] 0 2024-03-22T07:18:37.251 INFO:tasks.workunit.client.0.smithi081.stdout:9/559: dwrite d2/d73/d85/fb1 [0,4194304] 0 2024-03-22T07:18:37.260 INFO:tasks.workunit.client.0.smithi081.stdout:9/560: symlink d2/d10/d30/d57/dad/lb5 0 2024-03-22T07:18:37.261 INFO:tasks.workunit.client.0.smithi081.stdout:9/561: chown d2/d73/d85/f6d 59090 1 2024-03-22T07:18:37.261 INFO:tasks.workunit.client.0.smithi081.stdout:7/592: getdents d9/d10/d16/d1e/d27/d70 0 2024-03-22T07:18:37.261 INFO:tasks.workunit.client.0.smithi081.stdout:7/593: fsync d9/d3f/db3/fc6 0 2024-03-22T07:18:37.265 INFO:tasks.workunit.client.0.smithi081.stdout:9/562: unlink d2/d10/l52 0 2024-03-22T07:18:37.265 INFO:tasks.workunit.client.0.smithi081.stdout:9/563: write d2/d2b/f70 [618803,103752] 0 2024-03-22T07:18:37.265 INFO:tasks.workunit.client.0.smithi081.stdout:7/594: readlink d9/d10/d16/l6a 0 2024-03-22T07:18:37.268 INFO:tasks.workunit.client.0.smithi081.stdout:6/466: rename d3/da/de/d31/d40/d50/d63/f54 to d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f97 0 2024-03-22T07:18:37.270 INFO:tasks.workunit.client.0.smithi081.stdout:9/564: creat d2/d73/d85/fb6 x:0 0 0 2024-03-22T07:18:37.271 INFO:tasks.workunit.client.0.smithi081.stdout:9/565: creat d2/d2b/d49/d86/fb7 x:0 0 0 2024-03-22T07:18:37.272 INFO:tasks.workunit.client.0.smithi081.stdout:7/595: write d9/f75 [3664665,85236] 0 2024-03-22T07:18:37.274 INFO:tasks.workunit.client.0.smithi081.stdout:7/596: getdents d9/d10/d53/d81/d59 0 2024-03-22T07:18:37.274 INFO:tasks.workunit.client.0.smithi081.stdout:7/597: chown d9/d10/d16/d1e/d27/d6d/d8c/lc9 374358 1 2024-03-22T07:18:37.276 INFO:tasks.workunit.client.0.smithi081.stdout:7/598: creat d9/d10/d2a/fcd x:0 0 0 2024-03-22T07:18:37.278 INFO:tasks.workunit.client.0.smithi081.stdout:6/467: write d3/da/de/d31/d40/d50/d63/d26/d3f/f7 [150589,74417] 0 2024-03-22T07:18:37.278 INFO:tasks.workunit.client.0.smithi081.stdout:6/468: truncate d3/da/f53 420827 0 2024-03-22T07:18:37.285 INFO:tasks.workunit.client.0.smithi081.stdout:9/566: dwrite d2/f8 [0,4194304] 0 2024-03-22T07:18:37.287 INFO:tasks.workunit.client.0.smithi081.stdout:9/567: readlink d2/d73/d85/l1d 0 2024-03-22T07:18:37.296 INFO:tasks.workunit.client.0.smithi081.stdout:7/599: dwrite d9/d10/f2e [0,4194304] 0 2024-03-22T07:18:37.297 INFO:tasks.workunit.client.0.smithi081.stdout:7/600: write d9/dc/f4a [4411447,69277] 0 2024-03-22T07:18:37.302 INFO:tasks.workunit.client.0.smithi081.stdout:9/568: write d2/d73/d85/fb1 [696840,24277] 0 2024-03-22T07:18:37.302 INFO:tasks.workunit.client.0.smithi081.stdout:9/569: dread - d2/d2b/f81 zero size 2024-03-22T07:18:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:37 smithi081 ceph-mon[113965]: pgmap v214: 65 pgs: 65 active+clean; 1.0 GiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 17 MiB/s rd, 22 MiB/s wr, 680 op/s 2024-03-22T07:18:37.322 INFO:tasks.workunit.client.0.smithi081.stdout:9/570: dwrite d2/d10/d30/d57/d66/f69 [0,4194304] 0 2024-03-22T07:18:37.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:37 smithi149 ceph-mon[83447]: pgmap v214: 65 pgs: 65 active+clean; 1.0 GiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 17 MiB/s rd, 22 MiB/s wr, 680 op/s 2024-03-22T07:18:37.451 INFO:tasks.workunit.client.0.smithi081.stdout:6/469: dwrite d3/da/de/f5b [0,4194304] 0 2024-03-22T07:18:37.483 INFO:tasks.workunit.client.0.smithi081.stdout:7/601: link f7 d9/d10/d9e/fce 0 2024-03-22T07:18:37.487 INFO:tasks.workunit.client.0.smithi081.stdout:6/470: creat d3/da/de/d31/d40/d50/d63/d26/d3f/f98 x:0 0 0 2024-03-22T07:18:37.491 INFO:tasks.workunit.client.0.smithi081.stdout:7/602: creat d9/d10/d53/d81/d59/fcf x:0 0 0 2024-03-22T07:18:37.495 INFO:tasks.workunit.client.0.smithi081.stdout:7/603: symlink d9/d14/ld0 0 2024-03-22T07:18:37.496 INFO:tasks.workunit.client.0.smithi081.stdout:7/604: getdents d9/d10/d16/d1e/d27 0 2024-03-22T07:18:37.500 INFO:tasks.workunit.client.0.smithi081.stdout:7/605: symlink d9/d10/d16/d1e/d80/ld1 0 2024-03-22T07:18:37.504 INFO:tasks.workunit.client.0.smithi081.stdout:7/606: dread d9/d10/d16/d1e/d6f/f94 [0,4194304] 0 2024-03-22T07:18:37.504 INFO:tasks.workunit.client.0.smithi081.stdout:7/607: read d9/dc/f4a [683582,120323] 0 2024-03-22T07:18:37.505 INFO:tasks.workunit.client.0.smithi081.stdout:7/608: symlink d9/d10/d53/d81/d59/dbe/ld2 0 2024-03-22T07:18:37.506 INFO:tasks.workunit.client.0.smithi081.stdout:7/609: symlink d9/d10/d16/d1e/d27/d70/ld3 0 2024-03-22T07:18:37.560 INFO:tasks.workunit.client.0.smithi081.stdout:6/471: rename d3/da/f2a to d3/da/de/d31/d40/f99 0 2024-03-22T07:18:37.564 INFO:tasks.workunit.client.0.smithi081.stdout:7/610: rename d9/d10/d4b/d61 to d9/d10/d53/d81/d59/d89/dd4 0 2024-03-22T07:18:37.564 INFO:tasks.workunit.client.0.smithi081.stdout:7/611: stat d9/d14/cb4 0 2024-03-22T07:18:37.574 INFO:tasks.workunit.client.0.smithi081.stdout:9/571: truncate d2/d10/d30/d57/d66/f69 1182520 0 2024-03-22T07:18:37.574 INFO:tasks.workunit.client.0.smithi081.stdout:9/572: dread - d2/d5d/d58/f84 zero size 2024-03-22T07:18:37.575 INFO:tasks.workunit.client.0.smithi081.stdout:9/573: write d2/d73/d85/fa0 [876328,49878] 0 2024-03-22T07:18:37.578 INFO:tasks.workunit.client.0.smithi081.stdout:6/472: dwrite d3/da/f8f [0,4194304] 0 2024-03-22T07:18:37.582 INFO:tasks.workunit.client.0.smithi081.stdout:6/473: unlink d3/da/de/f46 0 2024-03-22T07:18:37.583 INFO:tasks.workunit.client.0.smithi081.stdout:6/474: chown d3/l30 348 1 2024-03-22T07:18:37.584 INFO:tasks.workunit.client.0.smithi081.stdout:6/475: creat d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f9a x:0 0 0 2024-03-22T07:18:37.588 INFO:tasks.workunit.client.0.smithi081.stdout:6/476: dread d3/da/de/d31/d40/d50/d63/d26/d3f/f29 [0,4194304] 0 2024-03-22T07:18:37.672 INFO:tasks.workunit.client.0.smithi081.stdout:6/477: creat d3/da/de/f9b x:0 0 0 2024-03-22T07:18:37.673 INFO:tasks.workunit.client.0.smithi081.stdout:6/478: mkdir d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/d9c 0 2024-03-22T07:18:37.673 INFO:tasks.workunit.client.0.smithi081.stdout:6/479: chown d3/da/de/d31/d40 43782 1 2024-03-22T07:18:37.678 INFO:tasks.workunit.client.0.smithi081.stdout:9/574: dwrite d2/d5/fd [4194304,4194304] 0 2024-03-22T07:18:37.696 INFO:tasks.workunit.client.0.smithi081.stdout:6/480: dwrite d3/da/de/d31/d40/d50/d63/d5e/f82 [0,4194304] 0 2024-03-22T07:18:37.696 INFO:tasks.workunit.client.0.smithi081.stdout:6/481: chown d3/da/de/d31/d40/d50/d63/d26/d3f/d5a 21564 1 2024-03-22T07:18:37.697 INFO:tasks.workunit.client.0.smithi081.stdout:6/482: write d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f9a [834495,107906] 0 2024-03-22T07:18:37.740 INFO:tasks.workunit.client.0.smithi081.stdout:6/483: mkdir d3/da/de/d31/d40/d50/d63/d5e/d9d 0 2024-03-22T07:18:37.740 INFO:tasks.workunit.client.0.smithi081.stdout:6/484: chown d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f6b 345269 1 2024-03-22T07:18:37.745 INFO:tasks.workunit.client.0.smithi081.stdout:9/575: rmdir d2/d2b 39 2024-03-22T07:18:37.750 INFO:tasks.workunit.client.0.smithi081.stdout:9/576: dread d2/f20 [0,4194304] 0 2024-03-22T07:18:37.751 INFO:tasks.workunit.client.0.smithi081.stdout:9/577: creat d2/d10/d30/d57/fb8 x:0 0 0 2024-03-22T07:18:37.753 INFO:tasks.workunit.client.0.smithi081.stdout:9/578: symlink d2/d10/d30/d57/d66/d75/lb9 0 2024-03-22T07:18:37.753 INFO:tasks.workunit.client.0.smithi081.stdout:9/579: chown d2/d10/d30/d57/f9e 170389 1 2024-03-22T07:18:37.753 INFO:tasks.workunit.client.0.smithi081.stdout:9/580: chown d2/d78 1628156523 1 2024-03-22T07:18:37.753 INFO:tasks.workunit.client.0.smithi081.stdout:9/581: symlink d2/d78/lba 0 2024-03-22T07:18:37.758 INFO:tasks.workunit.client.0.smithi081.stdout:9/582: dread d2/f12 [0,4194304] 0 2024-03-22T07:18:37.759 INFO:tasks.workunit.client.0.smithi081.stdout:9/583: unlink d2/l3c 0 2024-03-22T07:18:37.760 INFO:tasks.workunit.client.0.smithi081.stdout:9/584: symlink d2/d5d/lbb 0 2024-03-22T07:18:37.760 INFO:tasks.workunit.client.0.smithi081.stdout:6/485: dwrite d3/da/de/d31/d40/d50/f7a [0,4194304] 0 2024-03-22T07:18:37.763 INFO:tasks.workunit.client.0.smithi081.stdout:6/486: creat d3/da/de/d31/d40/d50/d63/d5e/f9e x:0 0 0 2024-03-22T07:18:37.768 INFO:tasks.workunit.client.0.smithi081.stdout:9/585: write d2/d2b/f3b [3131756,1712] 0 2024-03-22T07:18:37.791 INFO:tasks.workunit.client.0.smithi081.stdout:6/487: dwrite d3/da/de/d31/f66 [0,4194304] 0 2024-03-22T07:18:37.793 INFO:tasks.workunit.client.0.smithi081.stdout:6/488: getdents d3/da/de/d31/d40/d50/d63/d26/d3f 0 2024-03-22T07:18:37.794 INFO:tasks.workunit.client.0.smithi081.stdout:6/489: write d3/da/de/d31/d40/d50/f7a [4454466,125919] 0 2024-03-22T07:18:37.794 INFO:tasks.workunit.client.0.smithi081.stdout:6/490: stat d3/da/de/d31/d40/d50/f5d 0 2024-03-22T07:18:37.795 INFO:tasks.workunit.client.0.smithi081.stdout:6/491: rename d3/da/de/d31/d40/d50/d63/d26/f33 to d3/f9f 0 2024-03-22T07:18:37.796 INFO:tasks.workunit.client.0.smithi081.stdout:6/492: mknod d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/ca0 0 2024-03-22T07:18:37.797 INFO:tasks.workunit.client.0.smithi081.stdout:9/586: dwrite d2/d78/fa6 [0,4194304] 0 2024-03-22T07:18:37.801 INFO:tasks.workunit.client.0.smithi081.stdout:9/587: symlink d2/d2b/d49/d86/lbc 0 2024-03-22T07:18:37.801 INFO:tasks.workunit.client.0.smithi081.stdout:9/588: fdatasync d2/f3 0 2024-03-22T07:18:37.814 INFO:tasks.workunit.client.0.smithi081.stdout:6/493: dwrite d3/da/de/d31/d40/d50/f87 [0,4194304] 0 2024-03-22T07:18:37.822 INFO:tasks.workunit.client.0.smithi081.stdout:9/589: dwrite d2/d2b/f43 [0,4194304] 0 2024-03-22T07:18:37.826 INFO:tasks.workunit.client.0.smithi081.stdout:9/590: mknod d2/cbd 0 2024-03-22T07:18:37.965 INFO:tasks.workunit.client.0.smithi081.stdout:8/511: sync 2024-03-22T07:18:37.965 INFO:tasks.workunit.client.0.smithi081.stdout:1/392: sync 2024-03-22T07:18:37.965 INFO:tasks.workunit.client.0.smithi081.stdout:3/328: sync 2024-03-22T07:18:37.965 INFO:tasks.workunit.client.0.smithi081.stdout:1/393: truncate d6/d18/d1f/d32/f59 554726 0 2024-03-22T07:18:37.965 INFO:tasks.workunit.client.0.smithi081.stdout:5/633: sync 2024-03-22T07:18:37.965 INFO:tasks.workunit.client.0.smithi081.stdout:4/685: sync 2024-03-22T07:18:37.965 INFO:tasks.workunit.client.0.smithi081.stdout:0/516: sync 2024-03-22T07:18:37.966 INFO:tasks.workunit.client.0.smithi081.stdout:2/365: sync 2024-03-22T07:18:37.966 INFO:tasks.workunit.client.0.smithi081.stdout:2/366: write da/db/d39/d47/d3b/d41/f51 [103111,87417] 0 2024-03-22T07:18:37.966 INFO:tasks.workunit.client.0.smithi081.stdout:2/367: stat da/db/d39/d47/d3b/d41/l77 0 2024-03-22T07:18:37.966 INFO:tasks.workunit.client.0.smithi081.stdout:3/329: getdents db/d38 0 2024-03-22T07:18:37.966 INFO:tasks.workunit.client.0.smithi081.stdout:4/686: creat da/d18/d1b/fdd x:0 0 0 2024-03-22T07:18:37.967 INFO:tasks.workunit.client.0.smithi081.stdout:4/687: fsync da/d18/d1b/d26/d3e/f55 0 2024-03-22T07:18:37.970 INFO:tasks.workunit.client.0.smithi081.stdout:2/368: symlink da/db/d39/d47/d3b/d41/d50/l78 0 2024-03-22T07:18:37.972 INFO:tasks.workunit.client.0.smithi081.stdout:5/634: dread d4/df/d22/f58 [0,4194304] 0 2024-03-22T07:18:37.977 INFO:tasks.workunit.client.0.smithi081.stdout:3/330: rename db/d2e to db/de/d4b/d6a/d70 0 2024-03-22T07:18:37.977 INFO:tasks.workunit.client.0.smithi081.stdout:0/517: write d3/dc/f2a [3735850,40739] 0 2024-03-22T07:18:37.977 INFO:tasks.workunit.client.0.smithi081.stdout:0/518: fdatasync d3/dc/d33/f96 0 2024-03-22T07:18:37.978 INFO:tasks.workunit.client.0.smithi081.stdout:0/519: chown d3/d32/f84 344210376 1 2024-03-22T07:18:37.978 INFO:tasks.workunit.client.0.smithi081.stdout:0/520: readlink d3/dc/d33/d62/laf 0 2024-03-22T07:18:37.981 INFO:tasks.workunit.client.0.smithi081.stdout:4/688: dread da/d18/f31 [0,4194304] 0 2024-03-22T07:18:37.985 INFO:tasks.workunit.client.0.smithi081.stdout:5/635: link d4/d9/l39 d4/df/d73/d8c/dc0/ld5 0 2024-03-22T07:18:37.987 INFO:tasks.workunit.client.0.smithi081.stdout:8/512: dwrite d2/d3/d19/d1e/d13/d22/f6f [0,4194304] 0 2024-03-22T07:18:37.987 INFO:tasks.workunit.client.0.smithi081.stdout:4/689: creat da/d18/d1b/d26/fde x:0 0 0 2024-03-22T07:18:37.987 INFO:tasks.workunit.client.0.smithi081.stdout:4/690: write da/f79 [1159993,87674] 0 2024-03-22T07:18:37.991 INFO:tasks.workunit.client.0.smithi081.stdout:1/394: write d6/d18/d41/d49/d56/f3e [122888,128418] 0 2024-03-22T07:18:37.991 INFO:tasks.workunit.client.0.smithi081.stdout:1/395: chown l5 397 1 2024-03-22T07:18:37.995 INFO:tasks.workunit.client.0.smithi081.stdout:8/513: link d2/d3/d28/d2b/f4b d2/d3/d19/d1e/d63/da0/d9c/fb1 0 2024-03-22T07:18:37.999 INFO:tasks.workunit.client.0.smithi081.stdout:1/396: mkdir d6/d6a/d85 0 2024-03-22T07:18:37.999 INFO:tasks.workunit.client.0.smithi081.stdout:1/397: write d6/d18/f27 [1660148,82286] 0 2024-03-22T07:18:37.999 INFO:tasks.workunit.client.0.smithi081.stdout:1/398: fsync d6/d18/d1f/d24/d48/d5c/f7e 0 2024-03-22T07:18:37.999 INFO:tasks.workunit.client.0.smithi081.stdout:8/514: dread d2/d3/d19/d1e/f5f [0,4194304] 0 2024-03-22T07:18:38.000 INFO:tasks.workunit.client.0.smithi081.stdout:4/691: mknod da/df/d38/dd8/cdf 0 2024-03-22T07:18:38.003 INFO:tasks.workunit.client.0.smithi081.stdout:3/331: write db/f18 [1319600,28393] 0 2024-03-22T07:18:38.003 INFO:tasks.workunit.client.0.smithi081.stdout:3/332: stat db/de/f20 0 2024-03-22T07:18:38.004 INFO:tasks.workunit.client.0.smithi081.stdout:8/515: link d2/d3/d19/d1e/d13/d22/d38/d78/c79 d2/d3/d19/d1e/d63/da0/cb2 0 2024-03-22T07:18:38.008 INFO:tasks.workunit.client.0.smithi081.stdout:4/692: mknod da/ce0 0 2024-03-22T07:18:38.013 INFO:tasks.workunit.client.0.smithi081.stdout:5/636: dwrite d4/df/d7c/f85 [4194304,4194304] 0 2024-03-22T07:18:38.014 INFO:tasks.workunit.client.0.smithi081.stdout:5/637: write d4/df/d73/f9d [712065,117490] 0 2024-03-22T07:18:38.014 INFO:tasks.workunit.client.0.smithi081.stdout:5/638: chown d4/d10/f8d 0 1 2024-03-22T07:18:38.015 INFO:tasks.workunit.client.0.smithi081.stdout:5/639: getdents d4/df/d22/d49 0 2024-03-22T07:18:38.015 INFO:tasks.workunit.client.0.smithi081.stdout:5/640: fdatasync d4/d9/f91 0 2024-03-22T07:18:38.021 INFO:tasks.workunit.client.0.smithi081.stdout:0/521: dwrite d3/dc/d2b/d39/f65 [0,4194304] 0 2024-03-22T07:18:38.023 INFO:tasks.workunit.client.0.smithi081.stdout:2/369: dwrite da/d11/f1b [0,4194304] 0 2024-03-22T07:18:38.023 INFO:tasks.workunit.client.0.smithi081.stdout:2/370: write da/db/d39/d47/d3b/d41/f51 [1058643,67479] 0 2024-03-22T07:18:38.024 INFO:tasks.workunit.client.0.smithi081.stdout:2/371: truncate da/db/d39/d47/f3d 790515 0 2024-03-22T07:18:38.024 INFO:tasks.workunit.client.0.smithi081.stdout:1/399: dwrite d6/d18/d1f/f25 [0,4194304] 0 2024-03-22T07:18:38.025 INFO:tasks.workunit.client.0.smithi081.stdout:4/693: dread da/f79 [0,4194304] 0 2024-03-22T07:18:38.026 INFO:tasks.workunit.client.0.smithi081.stdout:1/400: symlink d6/d18/d1f/d24/l86 0 2024-03-22T07:18:38.026 INFO:tasks.workunit.client.0.smithi081.stdout:2/372: mknod da/d56/c79 0 2024-03-22T07:18:38.030 INFO:tasks.workunit.client.0.smithi081.stdout:0/522: symlink d3/d32/d35/lb4 0 2024-03-22T07:18:38.031 INFO:tasks.workunit.client.0.smithi081.stdout:5/641: dwrite d4/d9/f15 [0,4194304] 0 2024-03-22T07:18:38.032 INFO:tasks.workunit.client.0.smithi081.stdout:2/373: symlink da/db/d6d/d72/l7a 0 2024-03-22T07:18:38.043 INFO:tasks.workunit.client.0.smithi081.stdout:2/374: mknod da/db/c7b 0 2024-03-22T07:18:38.044 INFO:tasks.workunit.client.0.smithi081.stdout:2/375: chown da/db/c7b 432 1 2024-03-22T07:18:38.044 INFO:tasks.workunit.client.0.smithi081.stdout:2/376: readlink da/l3e 0 2024-03-22T07:18:38.044 INFO:tasks.workunit.client.0.smithi081.stdout:4/694: dwrite da/d18/f21 [0,4194304] 0 2024-03-22T07:18:38.046 INFO:tasks.workunit.client.0.smithi081.stdout:2/377: mknod da/d56/c7c 0 2024-03-22T07:18:38.046 INFO:tasks.workunit.client.0.smithi081.stdout:2/378: dread - da/db/d39/d46/f6c zero size 2024-03-22T07:18:38.046 INFO:tasks.workunit.client.0.smithi081.stdout:2/379: fsync da/d11/d21/f4e 0 2024-03-22T07:18:38.050 INFO:tasks.workunit.client.0.smithi081.stdout:4/695: rename da/d18/lb3 to da/d18/d1b/d1e/d37/d1c/le1 0 2024-03-22T07:18:38.050 INFO:tasks.workunit.client.0.smithi081.stdout:4/696: truncate da/d18/d1b/fdd 426031 0 2024-03-22T07:18:38.064 INFO:tasks.workunit.client.0.smithi081.stdout:0/523: write d3/f2d [350371,110853] 0 2024-03-22T07:18:38.066 INFO:tasks.workunit.client.0.smithi081.stdout:5/642: dwrite d4/f33 [0,4194304] 0 2024-03-22T07:18:38.066 INFO:tasks.workunit.client.0.smithi081.stdout:0/524: link d3/dc/cb0 d3/dc/d40/cb5 0 2024-03-22T07:18:38.074 INFO:tasks.workunit.client.0.smithi081.stdout:2/380: dwrite da/d11/d21/f53 [4194304,4194304] 0 2024-03-22T07:18:38.086 INFO:tasks.workunit.client.0.smithi081.stdout:8/516: dwrite f0 [0,4194304] 0 2024-03-22T07:18:38.086 INFO:tasks.workunit.client.0.smithi081.stdout:3/333: dwrite db/de/d4b/d6a/d70/f40 [0,4194304] 0 2024-03-22T07:18:38.086 INFO:tasks.workunit.client.0.smithi081.stdout:5/643: dwrite d4/d10/d20/d25/db8/f54 [0,4194304] 0 2024-03-22T07:18:38.086 INFO:tasks.workunit.client.0.smithi081.stdout:5/644: truncate d4/d10/daf/fb4 672037 0 2024-03-22T07:18:38.086 INFO:tasks.workunit.client.0.smithi081.stdout:5/645: write d4/df/f1a [4498681,19651] 0 2024-03-22T07:18:38.089 INFO:tasks.workunit.client.0.smithi081.stdout:3/334: creat db/de/f71 x:0 0 0 2024-03-22T07:18:38.092 INFO:tasks.workunit.client.0.smithi081.stdout:8/517: rename d2/d3/d23/f52 to d2/d3/d23/fb3 0 2024-03-22T07:18:38.092 INFO:tasks.workunit.client.0.smithi081.stdout:8/518: dread - d2/d3/d23/f7f zero size 2024-03-22T07:18:38.093 INFO:tasks.workunit.client.0.smithi081.stdout:3/335: symlink db/de/d4b/l72 0 2024-03-22T07:18:38.093 INFO:tasks.workunit.client.0.smithi081.stdout:3/336: fsync db/d23/f4d 0 2024-03-22T07:18:38.093 INFO:tasks.workunit.client.0.smithi081.stdout:2/381: dwrite da/db/d39/d46/f44 [0,4194304] 0 2024-03-22T07:18:38.095 INFO:tasks.workunit.client.0.smithi081.stdout:8/519: mknod d2/d3/d19/d1e/d13/d22/d38/d78/cb4 0 2024-03-22T07:18:38.096 INFO:tasks.workunit.client.0.smithi081.stdout:8/520: fdatasync d2/d3/d28/d2b/d61/d7e/fa7 0 2024-03-22T07:18:38.096 INFO:tasks.workunit.client.0.smithi081.stdout:8/521: readlink d2/d3/la 0 2024-03-22T07:18:38.096 INFO:tasks.workunit.client.0.smithi081.stdout:4/697: write da/d18/d1b/d26/f73 [641098,13216] 0 2024-03-22T07:18:38.102 INFO:tasks.workunit.client.0.smithi081.stdout:2/382: getdents da/d56 0 2024-03-22T07:18:38.105 INFO:tasks.workunit.client.0.smithi081.stdout:2/383: chown da/c2b 10108238 1 2024-03-22T07:18:38.119 INFO:tasks.workunit.client.0.smithi081.stdout:5/646: dwrite f1 [0,4194304] 0 2024-03-22T07:18:38.119 INFO:tasks.workunit.client.0.smithi081.stdout:5/647: creat d4/d10/d20/d25/da3/fd6 x:0 0 0 2024-03-22T07:18:38.120 INFO:tasks.workunit.client.0.smithi081.stdout:5/648: creat d4/d75/da5/fd7 x:0 0 0 2024-03-22T07:18:38.129 INFO:tasks.workunit.client.0.smithi081.stdout:0/525: write d3/d32/f4f [69715,40075] 0 2024-03-22T07:18:38.130 INFO:tasks.workunit.client.0.smithi081.stdout:0/526: creat d3/dc/d40/fb6 x:0 0 0 2024-03-22T07:18:38.131 INFO:tasks.workunit.client.0.smithi081.stdout:8/522: dwrite d2/d3/d28/d2b/f7a [0,4194304] 0 2024-03-22T07:18:38.131 INFO:tasks.workunit.client.0.smithi081.stdout:8/523: rename d2/d3/d19 to d2/d3/d19/d1e/d13/d22/d38/d78/db5 22 2024-03-22T07:18:38.131 INFO:tasks.workunit.client.0.smithi081.stdout:0/527: creat d3/d32/d35/d74/fb7 x:0 0 0 2024-03-22T07:18:38.132 INFO:tasks.workunit.client.0.smithi081.stdout:3/337: dwrite db/f5c [0,4194304] 0 2024-03-22T07:18:38.132 INFO:tasks.workunit.client.0.smithi081.stdout:3/338: readlink db/de/d4b/l72 0 2024-03-22T07:18:38.134 INFO:tasks.workunit.client.0.smithi081.stdout:4/698: dwrite da/d18/d1b/d26/fb5 [0,4194304] 0 2024-03-22T07:18:38.135 INFO:tasks.workunit.client.0.smithi081.stdout:3/339: mknod db/de/d4b/d6a/d70/d3a/c73 0 2024-03-22T07:18:38.137 INFO:tasks.workunit.client.0.smithi081.stdout:3/340: readlink db/de/l2c 0 2024-03-22T07:18:38.139 INFO:tasks.workunit.client.0.smithi081.stdout:3/341: symlink db/de/d4b/l74 0 2024-03-22T07:18:38.143 INFO:tasks.workunit.client.0.smithi081.stdout:2/384: dwrite da/f13 [0,4194304] 0 2024-03-22T07:18:38.145 INFO:tasks.workunit.client.0.smithi081.stdout:4/699: dread da/fb [0,4194304] 0 2024-03-22T07:18:38.145 INFO:tasks.workunit.client.0.smithi081.stdout:2/385: link da/db/d39/d47/d3b/d41/d50/d6a/d70/c60 da/d11/d21/d71/c7d 0 2024-03-22T07:18:38.146 INFO:tasks.workunit.client.0.smithi081.stdout:4/700: mkdir da/df/d38/dc9/de2 0 2024-03-22T07:18:38.146 INFO:tasks.workunit.client.0.smithi081.stdout:4/701: readlink da/df/d38/dd8/l7e 0 2024-03-22T07:18:38.148 INFO:tasks.workunit.client.0.smithi081.stdout:2/386: rename da/db/d39/d46/d42/d69 to da/db/d39/d46/d7e 0 2024-03-22T07:18:38.149 INFO:tasks.workunit.client.0.smithi081.stdout:4/702: symlink da/d57/le3 0 2024-03-22T07:18:38.149 INFO:tasks.workunit.client.0.smithi081.stdout:4/703: write da/f7b [2094590,56032] 0 2024-03-22T07:18:38.149 INFO:tasks.workunit.client.0.smithi081.stdout:4/704: write da/d18/d6e/f9e [1552363,62282] 0 2024-03-22T07:18:38.149 INFO:tasks.workunit.client.0.smithi081.stdout:4/705: write da/d18/d6e/f9e [2606402,5611] 0 2024-03-22T07:18:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:4/706: stat da/d18/d6e 0 2024-03-22T07:18:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:4/707: fdatasync da/f79 0 2024-03-22T07:18:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:2/387: mknod da/db/d39/d47/c7f 0 2024-03-22T07:18:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:2/388: fdatasync da/db/d39/d47/d3b/d41/d50/f66 0 2024-03-22T07:18:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:4/708: rename da/df/d38/dd8/d4f/lc4 to da/d18/ddc/le4 0 2024-03-22T07:18:38.150 INFO:tasks.workunit.client.0.smithi081.stdout:4/709: fdatasync da/d18/f31 0 2024-03-22T07:18:38.151 INFO:tasks.workunit.client.0.smithi081.stdout:2/389: mknod da/d11/d21/d71/c80 0 2024-03-22T07:18:38.153 INFO:tasks.workunit.client.0.smithi081.stdout:5/649: dwrite d4/d10/d20/d25/db8/f31 [0,4194304] 0 2024-03-22T07:18:38.153 INFO:tasks.workunit.client.0.smithi081.stdout:5/650: readlink d4/d10/d20/db2/d2f/l8a 0 2024-03-22T07:18:38.153 INFO:tasks.workunit.client.0.smithi081.stdout:4/710: rename da/d18/d1b/d1e/d37/c97 to da/df/d38/dd8/d4a/ce5 0 2024-03-22T07:18:38.155 INFO:tasks.workunit.client.0.smithi081.stdout:2/390: dread da/db/d39/d45/f4c [0,4194304] 0 2024-03-22T07:18:38.156 INFO:tasks.workunit.client.0.smithi081.stdout:5/651: creat d4/d10/d20/d4f/fd8 x:0 0 0 2024-03-22T07:18:38.156 INFO:tasks.workunit.client.0.smithi081.stdout:5/652: fdatasync d4/df/d73/f9d 0 2024-03-22T07:18:38.156 INFO:tasks.workunit.client.0.smithi081.stdout:5/653: write d4/d9/d90/fbb [53941,80590] 0 2024-03-22T07:18:38.156 INFO:tasks.workunit.client.0.smithi081.stdout:5/654: truncate d4/d9/f1b 779119 0 2024-03-22T07:18:38.158 INFO:tasks.workunit.client.0.smithi081.stdout:8/524: dwrite d2/d3/d23/f3b [0,4194304] 0 2024-03-22T07:18:38.159 INFO:tasks.workunit.client.0.smithi081.stdout:2/391: truncate da/db/f3f 2352745 0 2024-03-22T07:18:38.159 INFO:tasks.workunit.client.0.smithi081.stdout:0/528: dwrite d3/dc/d2b/f2f [0,4194304] 0 2024-03-22T07:18:38.159 INFO:tasks.workunit.client.0.smithi081.stdout:5/655: symlink d4/df/d22/ld9 0 2024-03-22T07:18:38.161 INFO:tasks.workunit.client.0.smithi081.stdout:4/711: rename da/d18/d1b/d1e/d37/d1c/d74/d98 to da/de6 0 2024-03-22T07:18:38.161 INFO:tasks.workunit.client.0.smithi081.stdout:2/392: rename da/db/d39/d47/d3b/d41/d50 to da/db/d39/d47/d3b/d41/d50/d6a/d70/d81 22 2024-03-22T07:18:38.161 INFO:tasks.workunit.client.0.smithi081.stdout:4/712: fsync da/df/d38/dd8/d4f/d7c/fd0 0 2024-03-22T07:18:38.163 INFO:tasks.workunit.client.0.smithi081.stdout:8/525: truncate d2/d3/d19/d1e/d13/d22/f51 285270 0 2024-03-22T07:18:38.163 INFO:tasks.workunit.client.0.smithi081.stdout:8/526: write d2/d3/d28/d2b/f58 [4256905,124543] 0 2024-03-22T07:18:38.163 INFO:tasks.workunit.client.0.smithi081.stdout:8/527: write d2/d3/d19/d1e/d13/d22/d2f/d71/f92 [926022,33960] 0 2024-03-22T07:18:38.172 INFO:tasks.workunit.client.0.smithi081.stdout:5/656: dread d4/f33 [0,4194304] 0 2024-03-22T07:18:38.174 INFO:tasks.workunit.client.0.smithi081.stdout:0/529: getdents d3/d6a 0 2024-03-22T07:18:38.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/393: mknod da/db/c82 0 2024-03-22T07:18:38.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/394: chown da/d56/c7c 959078 1 2024-03-22T07:18:38.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/395: chown da/f25 108 1 2024-03-22T07:18:38.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/396: fsync da/db/d39/d47/d3b/f55 0 2024-03-22T07:18:38.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/397: read - da/db/d39/d46/d42/f74 zero size 2024-03-22T07:18:38.176 INFO:tasks.workunit.client.0.smithi081.stdout:2/398: fdatasync da/db/f29 0 2024-03-22T07:18:38.176 INFO:tasks.workunit.client.0.smithi081.stdout:4/713: mknod da/d18/ce7 0 2024-03-22T07:18:38.176 INFO:tasks.workunit.client.0.smithi081.stdout:4/714: write da/d18/d1b/d26/d3e/fd1 [113593,65454] 0 2024-03-22T07:18:38.176 INFO:tasks.workunit.client.0.smithi081.stdout:4/715: fdatasync da/d18/d1b/d1e/d37/d1c/d74/fd3 0 2024-03-22T07:18:38.176 INFO:tasks.workunit.client.0.smithi081.stdout:4/716: write da/d18/d1b/d26/fb5 [4819079,2840] 0 2024-03-22T07:18:38.177 INFO:tasks.workunit.client.0.smithi081.stdout:8/528: symlink d2/d3/d28/d2b/d61/lb6 0 2024-03-22T07:18:38.179 INFO:tasks.workunit.client.0.smithi081.stdout:5/657: symlink d4/d10/d20/db2/d2f/d8e/lda 0 2024-03-22T07:18:38.180 INFO:tasks.workunit.client.0.smithi081.stdout:0/530: chown d3/f3e 37372124 1 2024-03-22T07:18:38.184 INFO:tasks.workunit.client.0.smithi081.stdout:4/717: creat da/df/d38/dd8/fe8 x:0 0 0 2024-03-22T07:18:38.186 INFO:tasks.workunit.client.0.smithi081.stdout:8/529: mkdir d2/d3/d19/d1e/d63/da0/db7 0 2024-03-22T07:18:38.188 INFO:tasks.workunit.client.0.smithi081.stdout:5/658: truncate d4/d10/d20/d25/db8/f3b 5651096 0 2024-03-22T07:18:38.190 INFO:tasks.workunit.client.0.smithi081.stdout:2/399: mknod da/db/d39/d46/d42/c83 0 2024-03-22T07:18:38.194 INFO:tasks.workunit.client.0.smithi081.stdout:4/718: unlink da/d18/d1b/d1e/d37/la0 0 2024-03-22T07:18:38.194 INFO:tasks.workunit.client.0.smithi081.stdout:8/530: truncate d2/d3/d28/f6c 1647522 0 2024-03-22T07:18:38.194 INFO:tasks.workunit.client.0.smithi081.stdout:4/719: fsync da/d18/d1b/d1e/d37/d1c/f51 0 2024-03-22T07:18:38.194 INFO:tasks.workunit.client.0.smithi081.stdout:8/531: write d2/d3/d19/d1e/f9a [62342,77831] 0 2024-03-22T07:18:38.195 INFO:tasks.workunit.client.0.smithi081.stdout:5/659: mknod d4/d75/cdb 0 2024-03-22T07:18:38.195 INFO:tasks.workunit.client.0.smithi081.stdout:5/660: truncate d4/d75/faa 927879 0 2024-03-22T07:18:38.195 INFO:tasks.workunit.client.0.smithi081.stdout:5/661: truncate d4/d10/d20/d4f/fb6 374067 0 2024-03-22T07:18:38.196 INFO:tasks.workunit.client.0.smithi081.stdout:8/532: truncate d2/f15 841887 0 2024-03-22T07:18:38.196 INFO:tasks.workunit.client.0.smithi081.stdout:8/533: dread - d2/d3/fae zero size 2024-03-22T07:18:38.196 INFO:tasks.workunit.client.0.smithi081.stdout:2/400: write da/db/f35 [3396547,63972] 0 2024-03-22T07:18:38.198 INFO:tasks.workunit.client.0.smithi081.stdout:4/720: link da/d18/d1b/d47/l70 da/df/d38/dd8/d4f/daf/le9 0 2024-03-22T07:18:38.201 INFO:tasks.workunit.client.0.smithi081.stdout:4/721: creat da/d18/ddc/fea x:0 0 0 2024-03-22T07:18:38.201 INFO:tasks.workunit.client.0.smithi081.stdout:8/534: mkdir d2/d3/d19/d1e/d13/d22/db8 0 2024-03-22T07:18:38.202 INFO:tasks.workunit.client.0.smithi081.stdout:8/535: truncate d2/d3/d28/f30 2041308 0 2024-03-22T07:18:38.203 INFO:tasks.workunit.client.0.smithi081.stdout:8/536: write d2/d3/d19/d1e/f3e [497366,121274] 0 2024-03-22T07:18:38.204 INFO:tasks.workunit.client.0.smithi081.stdout:8/537: getdents d2/d3/d19/d1e/d13/d22/d38 0 2024-03-22T07:18:38.206 INFO:tasks.workunit.client.0.smithi081.stdout:0/531: dwrite d3/dc/d2b/f2f [0,4194304] 0 2024-03-22T07:18:38.207 INFO:tasks.workunit.client.0.smithi081.stdout:8/538: rename d2/d3/d19/d1e/d13/d22/c6d to d2/d3/d19/d1e/d13/d22/d2f/d71/cb9 0 2024-03-22T07:18:38.209 INFO:tasks.workunit.client.0.smithi081.stdout:0/532: symlink d3/d32/d35/lb8 0 2024-03-22T07:18:38.209 INFO:tasks.workunit.client.0.smithi081.stdout:0/533: dread - d3/d32/f8f zero size 2024-03-22T07:18:38.210 INFO:tasks.workunit.client.0.smithi081.stdout:0/534: creat d3/d9b/fb9 x:0 0 0 2024-03-22T07:18:38.212 INFO:tasks.workunit.client.0.smithi081.stdout:8/539: mkdir d2/d3/d19/d1e/d13/dba 0 2024-03-22T07:18:38.212 INFO:tasks.workunit.client.0.smithi081.stdout:5/662: write d4/d9/f2c [1647925,97686] 0 2024-03-22T07:18:38.212 INFO:tasks.workunit.client.0.smithi081.stdout:5/663: write f1 [5158462,41991] 0 2024-03-22T07:18:38.214 INFO:tasks.workunit.client.0.smithi081.stdout:0/535: chown d3/dc/d2b/d4d/c6b 551381559 1 2024-03-22T07:18:38.219 INFO:tasks.workunit.client.0.smithi081.stdout:0/536: creat d3/dc/d2b/fba x:0 0 0 2024-03-22T07:18:38.220 INFO:tasks.workunit.client.0.smithi081.stdout:5/664: rmdir d4/d9 39 2024-03-22T07:18:38.223 INFO:tasks.workunit.client.0.smithi081.stdout:0/537: truncate d3/dc/d33/f47 1110777 0 2024-03-22T07:18:38.223 INFO:tasks.workunit.client.0.smithi081.stdout:0/538: write d3/dc/d2b/d4d/fb2 [814569,35080] 0 2024-03-22T07:18:38.224 INFO:tasks.workunit.client.0.smithi081.stdout:5/665: symlink d4/d10/d20/db2/d2f/ldc 0 2024-03-22T07:18:38.224 INFO:tasks.workunit.client.0.smithi081.stdout:2/401: dwrite f6 [0,4194304] 0 2024-03-22T07:18:38.224 INFO:tasks.workunit.client.0.smithi081.stdout:2/402: chown da/db/d39/d47/d3b/d41/d50/f62 264253 1 2024-03-22T07:18:38.231 INFO:tasks.workunit.client.0.smithi081.stdout:5/666: dread d4/d9/d90/fbb [0,4194304] 0 2024-03-22T07:18:38.231 INFO:tasks.workunit.client.0.smithi081.stdout:5/667: write d4/df/d22/d4d/f99 [4412545,56415] 0 2024-03-22T07:18:38.232 INFO:tasks.workunit.client.0.smithi081.stdout:5/668: creat d4/d75/fdd x:0 0 0 2024-03-22T07:18:38.235 INFO:tasks.workunit.client.0.smithi081.stdout:5/669: dread d4/df/d7c/f95 [0,4194304] 0 2024-03-22T07:18:38.239 INFO:tasks.workunit.client.0.smithi081.stdout:2/403: write da/f25 [2582499,75033] 0 2024-03-22T07:18:38.240 INFO:tasks.workunit.client.0.smithi081.stdout:4/722: dwrite da/d18/d1b/d26/fcb [0,4194304] 0 2024-03-22T07:18:38.240 INFO:tasks.workunit.client.0.smithi081.stdout:0/539: dread d3/dc/d33/f97 [0,4194304] 0 2024-03-22T07:18:38.240 INFO:tasks.workunit.client.0.smithi081.stdout:8/540: dwrite d2/d3/d19/f6e [0,4194304] 0 2024-03-22T07:18:38.241 INFO:tasks.workunit.client.0.smithi081.stdout:7/612: sync 2024-03-22T07:18:38.244 INFO:tasks.workunit.client.0.smithi081.stdout:4/723: mknod da/d18/d1b/d1e/d5f/ceb 0 2024-03-22T07:18:38.246 INFO:tasks.workunit.client.0.smithi081.stdout:8/541: mkdir d2/d3/d19/d1e/d13/d22/d38/dbb 0 2024-03-22T07:18:38.248 INFO:tasks.workunit.client.0.smithi081.stdout:4/724: symlink da/d18/ddc/lec 0 2024-03-22T07:18:38.251 INFO:tasks.workunit.client.0.smithi081.stdout:8/542: rename d2/d3/d28/d2b/d61/d7e/f9b to d2/d3/d19/d1e/d13/d22/db8/fbc 0 2024-03-22T07:18:38.251 INFO:tasks.workunit.client.0.smithi081.stdout:8/543: chown d2/d3/d23 183 1 2024-03-22T07:18:38.255 INFO:tasks.workunit.client.0.smithi081.stdout:0/540: read d3/dc/d2b/f31 [4911509,91247] 0 2024-03-22T07:18:38.263 INFO:tasks.workunit.client.0.smithi081.stdout:4/725: dread da/d18/d1b/d26/f66 [0,4194304] 0 2024-03-22T07:18:38.263 INFO:tasks.workunit.client.0.smithi081.stdout:4/726: truncate f9 3012799 0 2024-03-22T07:18:38.263 INFO:tasks.workunit.client.0.smithi081.stdout:4/727: write da/d18/d1b/d26/d3e/f8a [230792,104646] 0 2024-03-22T07:18:38.266 INFO:tasks.workunit.client.0.smithi081.stdout:2/404: dwrite da/db/d39/d47/d3b/d41/d50/d6a/d70/f40 [4194304,4194304] 0 2024-03-22T07:18:38.268 INFO:tasks.workunit.client.0.smithi081.stdout:7/613: dwrite d9/d10/d4b/d9b/fbf [0,4194304] 0 2024-03-22T07:18:38.268 INFO:tasks.workunit.client.0.smithi081.stdout:7/614: truncate d9/d10/f2e 4482588 0 2024-03-22T07:18:38.268 INFO:tasks.workunit.client.0.smithi081.stdout:5/670: dwrite d4/d10/d20/db2/d2f/fc4 [0,4194304] 0 2024-03-22T07:18:38.270 INFO:tasks.workunit.client.0.smithi081.stdout:2/405: rename da/db/d39/d47/d3b/d41/d50/l76 to da/db/d39/d47/d3b/d41/d50/d6a/d70/l84 0 2024-03-22T07:18:38.270 INFO:tasks.workunit.client.0.smithi081.stdout:0/541: dread d3/dc/d33/f5b [0,4194304] 0 2024-03-22T07:18:38.271 INFO:tasks.workunit.client.0.smithi081.stdout:5/671: dread d4/df/d73/f9d [0,4194304] 0 2024-03-22T07:18:38.276 INFO:tasks.workunit.client.0.smithi081.stdout:5/672: creat d4/d10/d20/db2/d2a/fde x:0 0 0 2024-03-22T07:18:38.276 INFO:tasks.workunit.client.0.smithi081.stdout:5/673: truncate d4/d9/fc2 470079 0 2024-03-22T07:18:38.276 INFO:tasks.workunit.client.0.smithi081.stdout:5/674: chown d4/df/d22 10299 1 2024-03-22T07:18:38.277 INFO:tasks.workunit.client.0.smithi081.stdout:2/406: symlink da/db/d39/d45/l85 0 2024-03-22T07:18:38.278 INFO:tasks.workunit.client.0.smithi081.stdout:0/542: symlink d3/dc/d2b/d4d/lbb 0 2024-03-22T07:18:38.279 INFO:tasks.workunit.client.0.smithi081.stdout:5/675: truncate d4/d10/d20/db2/d2a/f9b 2788739 0 2024-03-22T07:18:38.280 INFO:tasks.workunit.client.0.smithi081.stdout:2/407: creat da/db/d39/d47/d3b/d41/f86 x:0 0 0 2024-03-22T07:18:38.280 INFO:tasks.workunit.client.0.smithi081.stdout:2/408: fsync da/db/d39/d47/d3b/d41/d50/d6a/d70/f40 0 2024-03-22T07:18:38.281 INFO:tasks.workunit.client.0.smithi081.stdout:0/543: creat d3/dc/d40/d69/fbc x:0 0 0 2024-03-22T07:18:38.281 INFO:tasks.workunit.client.0.smithi081.stdout:0/544: stat d3/dc/d2b/d39/f8c 0 2024-03-22T07:18:38.282 INFO:tasks.workunit.client.0.smithi081.stdout:2/409: dread da/db/d39/d45/f4c [0,4194304] 0 2024-03-22T07:18:38.282 INFO:tasks.workunit.client.0.smithi081.stdout:2/410: mknod da/db/d39/d46/d7e/c87 0 2024-03-22T07:18:38.283 INFO:tasks.workunit.client.0.smithi081.stdout:2/411: write da/db/d39/d47/d3b/d41/d50/f62 [872376,109309] 0 2024-03-22T07:18:38.284 INFO:tasks.workunit.client.0.smithi081.stdout:2/412: unlink da/db/d39/d47/d3b/d41/l77 0 2024-03-22T07:18:38.286 INFO:tasks.workunit.client.0.smithi081.stdout:8/544: dwrite d2/d3/d19/d1e/f41 [0,4194304] 0 2024-03-22T07:18:38.286 INFO:tasks.workunit.client.0.smithi081.stdout:0/545: rename d3/dc/d2b/l60 to d3/dc/d33/lbd 0 2024-03-22T07:18:38.286 INFO:tasks.workunit.client.0.smithi081.stdout:0/546: dread - d3/d9b/fb9 zero size 2024-03-22T07:18:38.286 INFO:tasks.workunit.client.0.smithi081.stdout:0/547: fdatasync d3/d32/d35/d74/fb7 0 2024-03-22T07:18:38.287 INFO:tasks.workunit.client.0.smithi081.stdout:8/545: creat d2/d3/d9e/fbd x:0 0 0 2024-03-22T07:18:38.289 INFO:tasks.workunit.client.0.smithi081.stdout:4/728: dwrite da/d18/d1b/d1e/f3f [0,4194304] 0 2024-03-22T07:18:38.290 INFO:tasks.workunit.client.0.smithi081.stdout:7/615: dwrite f6 [4194304,4194304] 0 2024-03-22T07:18:38.291 INFO:tasks.workunit.client.0.smithi081.stdout:2/413: rename da/db/l20 to da/db/d39/d47/l88 0 2024-03-22T07:18:38.291 INFO:tasks.workunit.client.0.smithi081.stdout:2/414: write da/db/f3f [2035279,5623] 0 2024-03-22T07:18:38.294 INFO:tasks.workunit.client.0.smithi081.stdout:4/729: mkdir da/df/d38/dd8/d4f/daf/ded 0 2024-03-22T07:18:38.299 INFO:tasks.workunit.client.0.smithi081.stdout:4/730: mknod da/d18/d1b/d1e/d37/cee 0 2024-03-22T07:18:38.299 INFO:tasks.workunit.client.0.smithi081.stdout:4/731: dread - da/d18/d1b/d26/dce/fd6 zero size 2024-03-22T07:18:38.299 INFO:tasks.workunit.client.0.smithi081.stdout:4/732: chown da/d18/d1b/d47 1509729 1 2024-03-22T07:18:38.299 INFO:tasks.workunit.client.0.smithi081.stdout:4/733: stat da/df/d38/dd8/d4f/d7c/l9f 0 2024-03-22T07:18:38.302 INFO:tasks.workunit.client.0.smithi081.stdout:5/676: dwrite d4/d75/da5/fcb [0,4194304] 0 2024-03-22T07:18:38.302 INFO:tasks.workunit.client.0.smithi081.stdout:7/616: dread d9/d10/fbd [4194304,4194304] 0 2024-03-22T07:18:38.303 INFO:tasks.workunit.client.0.smithi081.stdout:4/734: mkdir da/d3d/def 0 2024-03-22T07:18:38.303 INFO:tasks.workunit.client.0.smithi081.stdout:4/735: chown da/df/d38/dd8/d4f/d7c/cac 63207390 1 2024-03-22T07:18:38.303 INFO:tasks.workunit.client.0.smithi081.stdout:4/736: fdatasync da/df/d38/f46 0 2024-03-22T07:18:38.306 INFO:tasks.workunit.client.0.smithi081.stdout:7/617: dread d9/d10/d16/d1e/d27/f57 [0,4194304] 0 2024-03-22T07:18:38.313 INFO:tasks.workunit.client.0.smithi081.stdout:0/548: write d3/f44 [1128640,9134] 0 2024-03-22T07:18:38.314 INFO:tasks.workunit.client.0.smithi081.stdout:8/546: dwrite f0 [4194304,4194304] 0 2024-03-22T07:18:38.315 INFO:tasks.workunit.client.0.smithi081.stdout:2/415: dwrite f5 [8388608,4194304] 0 2024-03-22T07:18:38.316 INFO:tasks.workunit.client.0.smithi081.stdout:5/677: mkdir d4/d10/d20/dd2/ddf 0 2024-03-22T07:18:38.316 INFO:tasks.workunit.client.0.smithi081.stdout:5/678: read d4/d10/d20/db2/fa2 [20100,79420] 0 2024-03-22T07:18:38.318 INFO:tasks.workunit.client.0.smithi081.stdout:4/737: creat da/de6/db8/ff0 x:0 0 0 2024-03-22T07:18:38.318 INFO:tasks.workunit.client.0.smithi081.stdout:4/738: chown da/df/d38/dd8/d4f/d7c/l82 3219421 1 2024-03-22T07:18:38.318 INFO:tasks.workunit.client.0.smithi081.stdout:4/739: chown da/d18/f31 0 1 2024-03-22T07:18:38.319 INFO:tasks.workunit.client.0.smithi081.stdout:7/618: stat d9/d3f/d52/l69 0 2024-03-22T07:18:38.322 INFO:tasks.workunit.client.0.smithi081.stdout:8/547: creat d2/d3/d19/d1e/d13/d22/fbe x:0 0 0 2024-03-22T07:18:38.322 INFO:tasks.workunit.client.0.smithi081.stdout:8/548: read d2/d3/d19/d1e/d13/fa3 [570337,126518] 0 2024-03-22T07:18:38.325 INFO:tasks.workunit.client.0.smithi081.stdout:2/416: mknod da/db/d39/d46/d7e/c89 0 2024-03-22T07:18:38.325 INFO:tasks.workunit.client.0.smithi081.stdout:2/417: chown da/db/d39/d47/d3b/d41/d50/d6a/d70/l84 5110095 1 2024-03-22T07:18:38.327 INFO:tasks.workunit.client.0.smithi081.stdout:8/549: write d2/d3/f4 [4875201,1876] 0 2024-03-22T07:18:38.327 INFO:tasks.workunit.client.0.smithi081.stdout:4/740: dread da/d18/d1b/d26/d3e/f55 [0,4194304] 0 2024-03-22T07:18:38.328 INFO:tasks.workunit.client.0.smithi081.stdout:2/418: symlink da/db/d39/d47/d6e/l8a 0 2024-03-22T07:18:38.328 INFO:tasks.workunit.client.0.smithi081.stdout:2/419: chown da/d11/f1f 3261864 1 2024-03-22T07:18:38.329 INFO:tasks.workunit.client.0.smithi081.stdout:7/619: mknod d9/d10/d53/d81/d59/d89/dd4/cd5 0 2024-03-22T07:18:38.333 INFO:tasks.workunit.client.0.smithi081.stdout:0/549: rename d3/dc/d2b/d4d/f6c to d3/d32/fbe 0 2024-03-22T07:18:38.335 INFO:tasks.workunit.client.0.smithi081.stdout:2/420: getdents da/db/d39/d47 0 2024-03-22T07:18:38.335 INFO:tasks.workunit.client.0.smithi081.stdout:8/550: dread d2/d3/d19/d1e/d63/da0/d9c/fb1 [0,4194304] 0 2024-03-22T07:18:38.337 INFO:tasks.workunit.client.0.smithi081.stdout:5/679: dwrite d4/d9/fc2 [0,4194304] 0 2024-03-22T07:18:38.337 INFO:tasks.workunit.client.0.smithi081.stdout:4/741: mknod da/d18/d1b/d47/cf1 0 2024-03-22T07:18:38.337 INFO:tasks.workunit.client.0.smithi081.stdout:8/551: chown d2/d3/d19/d1e/d13/d22/d2f/l31 107433 1 2024-03-22T07:18:38.339 INFO:tasks.workunit.client.0.smithi081.stdout:4/742: rmdir da/d18/d1b/d1e/d37 39 2024-03-22T07:18:38.341 INFO:tasks.workunit.client.0.smithi081.stdout:2/421: dread da/db/d39/d45/f4c [0,4194304] 0 2024-03-22T07:18:38.350 INFO:tasks.workunit.client.0.smithi081.stdout:6/494: sync 2024-03-22T07:18:38.350 INFO:tasks.workunit.client.0.smithi081.stdout:1/401: sync 2024-03-22T07:18:38.351 INFO:tasks.workunit.client.0.smithi081.stdout:9/591: sync 2024-03-22T07:18:38.355 INFO:tasks.workunit.client.0.smithi081.stdout:2/422: chown da/l2f 1299 1 2024-03-22T07:18:38.361 INFO:tasks.workunit.client.0.smithi081.stdout:8/552: symlink d2/d3/d19/lbf 0 2024-03-22T07:18:38.361 INFO:tasks.workunit.client.0.smithi081.stdout:8/553: write d2/d3/d28/d2b/f67 [356421,128921] 0 2024-03-22T07:18:38.361 INFO:tasks.workunit.client.0.smithi081.stdout:8/554: fdatasync d2/d3/d19/d1e/d13/d22/d2f/d71/f92 0 2024-03-22T07:18:38.361 INFO:tasks.workunit.client.0.smithi081.stdout:8/555: chown d2/d3/d19/d1e/d13/d22/d2f/d71/l87 356 1 2024-03-22T07:18:38.363 INFO:tasks.workunit.client.0.smithi081.stdout:2/423: truncate da/d11/d21/f53 3616362 0 2024-03-22T07:18:38.363 INFO:tasks.workunit.client.0.smithi081.stdout:2/424: chown da/db/d39/d46/f2e 18 1 2024-03-22T07:18:38.363 INFO:tasks.workunit.client.0.smithi081.stdout:2/425: fdatasync da/db/d39/d47/f43 0 2024-03-22T07:18:38.363 INFO:tasks.workunit.client.0.smithi081.stdout:2/426: chown da/db/d39/d47/d3b/d41/d50/d6a 3846006 1 2024-03-22T07:18:38.363 INFO:tasks.workunit.client.0.smithi081.stdout:2/427: dread - da/d56/f6b zero size 2024-03-22T07:18:38.365 INFO:tasks.workunit.client.0.smithi081.stdout:5/680: dread d4/d10/f21 [0,4194304] 0 2024-03-22T07:18:38.365 INFO:tasks.workunit.client.0.smithi081.stdout:8/556: mkdir d2/d3/d19/d1e/d13/d22/d2f/d71/dc0 0 2024-03-22T07:18:38.367 INFO:tasks.workunit.client.0.smithi081.stdout:2/428: truncate da/db/f35 2968171 0 2024-03-22T07:18:38.367 INFO:tasks.workunit.client.0.smithi081.stdout:3/342: sync 2024-03-22T07:18:38.369 INFO:tasks.workunit.client.0.smithi081.stdout:5/681: mkdir d4/d10/d20/d25/db8/de0 0 2024-03-22T07:18:38.376 INFO:tasks.workunit.client.0.smithi081.stdout:3/343: dread db/f1e [0,4194304] 0 2024-03-22T07:18:38.376 INFO:tasks.workunit.client.0.smithi081.stdout:3/344: chown db/de/d4b/l62 94065 1 2024-03-22T07:18:38.377 INFO:tasks.workunit.client.0.smithi081.stdout:2/429: write da/f25 [1955535,18882] 0 2024-03-22T07:18:38.381 INFO:tasks.workunit.client.0.smithi081.stdout:7/620: dwrite d9/f3c [4194304,4194304] 0 2024-03-22T07:18:38.381 INFO:tasks.workunit.client.0.smithi081.stdout:3/345: truncate db/de/d4b/d6a/d70/f32 656466 0 2024-03-22T07:18:38.384 INFO:tasks.workunit.client.0.smithi081.stdout:1/402: dwrite d6/d18/d41/d49/d56/d30/f7f [0,4194304] 0 2024-03-22T07:18:38.386 INFO:tasks.workunit.client.0.smithi081.stdout:3/346: symlink db/de/d4b/d6a/d70/d3a/d5d/l75 0 2024-03-22T07:18:38.388 INFO:tasks.workunit.client.0.smithi081.stdout:7/621: unlink d9/d10/d2a/f4d 0 2024-03-22T07:18:38.391 INFO:tasks.workunit.client.0.smithi081.stdout:3/347: mknod db/de/d4b/c76 0 2024-03-22T07:18:38.392 INFO:tasks.workunit.client.0.smithi081.stdout:2/430: link da/db/c3a da/db/d39/d45/c8b 0 2024-03-22T07:18:38.394 INFO:tasks.workunit.client.0.smithi081.stdout:3/348: truncate f1 2979910 0 2024-03-22T07:18:38.395 INFO:tasks.workunit.client.0.smithi081.stdout:2/431: getdents da/db/d39/d47/d3b/d41/d50/d6a/d70 0 2024-03-22T07:18:38.395 INFO:tasks.workunit.client.0.smithi081.stdout:2/432: stat da/db/d39/d47/d3b/f55 0 2024-03-22T07:18:38.400 INFO:tasks.workunit.client.0.smithi081.stdout:2/433: rename da/db/d39/d47/f37 to da/db/d39/d45/f8c 0 2024-03-22T07:18:38.400 INFO:tasks.workunit.client.0.smithi081.stdout:2/434: stat da/db/d39/d47/d3b/d41/d50/f66 0 2024-03-22T07:18:38.400 INFO:tasks.workunit.client.0.smithi081.stdout:2/435: stat da/db/d39/d47/d3b/d41/l49 0 2024-03-22T07:18:38.401 INFO:tasks.workunit.client.0.smithi081.stdout:2/436: rename da/db/d39/d47/c7f to da/db/d39/d47/d3b/d63/c8d 0 2024-03-22T07:18:38.401 INFO:tasks.workunit.client.0.smithi081.stdout:2/437: chown da/db/d39/d46/f36 5029386 1 2024-03-22T07:18:38.401 INFO:tasks.workunit.client.0.smithi081.stdout:2/438: write da/db/d39/d46/f36 [1196831,28588] 0 2024-03-22T07:18:38.406 INFO:tasks.workunit.client.0.smithi081.stdout:1/403: dread d6/d18/d1f/d24/f6f [0,4194304] 0 2024-03-22T07:18:38.414 INFO:tasks.workunit.client.0.smithi081.stdout:2/439: read f5 [3581465,51021] 0 2024-03-22T07:18:38.414 INFO:tasks.workunit.client.0.smithi081.stdout:9/592: dwrite d2/f5f [0,4194304] 0 2024-03-22T07:18:38.416 INFO:tasks.workunit.client.0.smithi081.stdout:2/440: getdents da/db/d39/d47/d3b/d41/d50 0 2024-03-22T07:18:38.417 INFO:tasks.workunit.client.0.smithi081.stdout:2/441: mknod da/db/d39/d47/d3b/d63/c8e 0 2024-03-22T07:18:38.418 INFO:tasks.workunit.client.0.smithi081.stdout:2/442: read da/d11/f4d [147279,73894] 0 2024-03-22T07:18:38.418 INFO:tasks.workunit.client.0.smithi081.stdout:2/443: dread - da/db/d39/d46/f6c zero size 2024-03-22T07:18:38.419 INFO:tasks.workunit.client.0.smithi081.stdout:2/444: creat da/db/d39/d47/d3b/d64/f8f x:0 0 0 2024-03-22T07:18:38.420 INFO:tasks.workunit.client.0.smithi081.stdout:2/445: link da/db/d39/d47/d3b/d64/f8f da/d11/d21/f90 0 2024-03-22T07:18:38.421 INFO:tasks.workunit.client.0.smithi081.stdout:2/446: mknod da/c91 0 2024-03-22T07:18:38.423 INFO:tasks.workunit.client.0.smithi081.stdout:2/447: link da/c91 da/db/d39/d45/c92 0 2024-03-22T07:18:38.423 INFO:tasks.workunit.client.0.smithi081.stdout:2/448: fdatasync da/db/d39/d47/d3b/d41/f86 0 2024-03-22T07:18:38.423 INFO:tasks.workunit.client.0.smithi081.stdout:2/449: readlink da/db/d39/d47/l5d 0 2024-03-22T07:18:38.424 INFO:tasks.workunit.client.0.smithi081.stdout:2/450: mkdir da/db/d39/d47/d3b/d93 0 2024-03-22T07:18:38.426 INFO:tasks.workunit.client.0.smithi081.stdout:2/451: rename da/d56/l68 to da/db/d39/d47/d6e/l94 0 2024-03-22T07:18:38.426 INFO:tasks.workunit.client.0.smithi081.stdout:2/452: readlink da/db/d39/d47/l61 0 2024-03-22T07:18:38.426 INFO:tasks.workunit.client.0.smithi081.stdout:2/453: chown da/db/d39/d46/d42/f74 14656374 1 2024-03-22T07:18:38.426 INFO:tasks.workunit.client.0.smithi081.stdout:9/593: write d2/f11 [3198538,109160] 0 2024-03-22T07:18:38.426 INFO:tasks.workunit.client.0.smithi081.stdout:8/557: dwrite d2/d3/d19/f48 [0,4194304] 0 2024-03-22T07:18:38.426 INFO:tasks.workunit.client.0.smithi081.stdout:8/558: fsync d2/d3/d19/d1e/f9d 0 2024-03-22T07:18:38.428 INFO:tasks.workunit.client.0.smithi081.stdout:4/743: dwrite da/d18/d1b/d26/d3e/f8a [0,4194304] 0 2024-03-22T07:18:38.429 INFO:tasks.workunit.client.0.smithi081.stdout:6/495: dwrite d3/da/f45 [0,4194304] 0 2024-03-22T07:18:38.429 INFO:tasks.workunit.client.0.smithi081.stdout:1/404: dwrite d6/d18/f4f [4194304,4194304] 0 2024-03-22T07:18:38.429 INFO:tasks.workunit.client.0.smithi081.stdout:1/405: write d6/f35 [1717299,18707] 0 2024-03-22T07:18:38.429 INFO:tasks.workunit.client.0.smithi081.stdout:5/682: dwrite d4/d10/d20/d25/db8/f3b [0,4194304] 0 2024-03-22T07:18:38.429 INFO:tasks.workunit.client.0.smithi081.stdout:5/683: fsync d4/fe 0 2024-03-22T07:18:38.429 INFO:tasks.workunit.client.0.smithi081.stdout:2/454: rename da/db/d39/f57 to da/db/d39/d47/d3b/d41/d50/d6a/d70/f95 0 2024-03-22T07:18:38.429 INFO:tasks.workunit.client.0.smithi081.stdout:3/349: dwrite db/de/f24 [8388608,4194304] 0 2024-03-22T07:18:38.429 INFO:tasks.workunit.client.0.smithi081.stdout:3/350: chown db/de/d4b/d6a/d70/d3a/c6d 50481289 1 2024-03-22T07:18:38.430 INFO:tasks.workunit.client.0.smithi081.stdout:1/406: getdents d6/d18/d1f/d24/d48/d6e 0 2024-03-22T07:18:38.430 INFO:tasks.workunit.client.0.smithi081.stdout:1/407: write f1 [280015,83595] 0 2024-03-22T07:18:38.430 INFO:tasks.workunit.client.0.smithi081.stdout:9/594: creat d2/d10/d30/d57/dad/fbe x:0 0 0 2024-03-22T07:18:38.431 INFO:tasks.workunit.client.0.smithi081.stdout:8/559: chown d2/d3/d19/d1e/d13/d22/l42 7 1 2024-03-22T07:18:38.431 INFO:tasks.workunit.client.0.smithi081.stdout:1/408: creat d6/d18/d1f/f87 x:0 0 0 2024-03-22T07:18:38.431 INFO:tasks.workunit.client.0.smithi081.stdout:1/409: write d6/d18/d1f/d24/f6f [1784528,65547] 0 2024-03-22T07:18:38.431 INFO:tasks.workunit.client.0.smithi081.stdout:1/410: stat d6/f35 0 2024-03-22T07:18:38.436 INFO:tasks.workunit.client.0.smithi081.stdout:6/496: creat d3/da/de/d31/d40/d50/d63/d26/d3f/fa1 x:0 0 0 2024-03-22T07:18:38.438 INFO:tasks.workunit.client.0.smithi081.stdout:5/684: truncate d4/f33 644243 0 2024-03-22T07:18:38.438 INFO:tasks.workunit.client.0.smithi081.stdout:5/685: write d4/d10/d20/d25/f8b [9031241,56857] 0 2024-03-22T07:18:38.438 INFO:tasks.workunit.client.0.smithi081.stdout:5/686: fdatasync d4/d10/d20/db2/d2f/f92 0 2024-03-22T07:18:38.438 INFO:tasks.workunit.client.0.smithi081.stdout:5/687: fsync d4/df/d22/d49/f4a 0 2024-03-22T07:18:38.439 INFO:tasks.workunit.client.0.smithi081.stdout:2/455: mknod da/db/d39/d46/d7e/c96 0 2024-03-22T07:18:38.440 INFO:tasks.workunit.client.0.smithi081.stdout:2/456: chown da/db/d6d/d72/l7a 2125840902 1 2024-03-22T07:18:38.441 INFO:tasks.workunit.client.0.smithi081.stdout:9/595: write d2/d5/d36/f6c [131957,64052] 0 2024-03-22T07:18:38.444 INFO:tasks.workunit.client.0.smithi081.stdout:2/457: dread da/db/d39/d46/f36 [0,4194304] 0 2024-03-22T07:18:38.447 INFO:tasks.workunit.client.0.smithi081.stdout:5/688: dread d4/d9/f15 [0,4194304] 0 2024-03-22T07:18:38.448 INFO:tasks.workunit.client.0.smithi081.stdout:4/744: getdents da/d18/d1b/d1e/d37/d1c/d74 0 2024-03-22T07:18:38.449 INFO:tasks.workunit.client.0.smithi081.stdout:6/497: symlink d3/d67/la2 0 2024-03-22T07:18:38.450 INFO:tasks.workunit.client.0.smithi081.stdout:6/498: dread - d3/da/de/d31/d40/d50/d63/d5e/f9e zero size 2024-03-22T07:18:38.454 INFO:tasks.workunit.client.0.smithi081.stdout:9/596: symlink d2/d10/d30/d57/d66/d75/d94/dae/lbf 0 2024-03-22T07:18:38.456 INFO:tasks.workunit.client.0.smithi081.stdout:2/458: rmdir da/db/d39/d47/d3b/d93 0 2024-03-22T07:18:38.456 INFO:tasks.workunit.client.0.smithi081.stdout:2/459: fdatasync da/db/d39/d47/f3d 0 2024-03-22T07:18:38.456 INFO:tasks.workunit.client.0.smithi081.stdout:2/460: chown da/db/d39/d47/d3b/d41/d50/d6a/d70/c60 12 1 2024-03-22T07:18:38.457 INFO:tasks.workunit.client.0.smithi081.stdout:5/689: mkdir d4/df/d73/d8c/dc0/de1 0 2024-03-22T07:18:38.458 INFO:tasks.workunit.client.0.smithi081.stdout:4/745: creat da/d18/d1b/d26/d32/dbc/ff2 x:0 0 0 2024-03-22T07:18:38.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/499: creat d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/fa3 x:0 0 0 2024-03-22T07:18:38.460 INFO:tasks.workunit.client.0.smithi081.stdout:3/351: rename f1 to db/f77 0 2024-03-22T07:18:38.461 INFO:tasks.workunit.client.0.smithi081.stdout:7/622: dwrite d9/d10/d9e/fc3 [0,4194304] 0 2024-03-22T07:18:38.462 INFO:tasks.workunit.client.0.smithi081.stdout:9/597: unlink d2/d2b/f43 0 2024-03-22T07:18:38.463 INFO:tasks.workunit.client.0.smithi081.stdout:1/411: rename d6/d18/f1d to d6/d18/d1f/d4c/d70/d7d/f88 0 2024-03-22T07:18:38.463 INFO:tasks.workunit.client.0.smithi081.stdout:2/461: creat da/db/d39/d46/d42/d67/f97 x:0 0 0 2024-03-22T07:18:38.464 INFO:tasks.workunit.client.0.smithi081.stdout:4/746: creat da/d18/d1b/d1e/d37/ff3 x:0 0 0 2024-03-22T07:18:38.467 INFO:tasks.workunit.client.0.smithi081.stdout:3/352: creat db/f78 x:0 0 0 2024-03-22T07:18:38.471 INFO:tasks.workunit.client.0.smithi081.stdout:7/623: creat d9/d10/d16/d1e/d27/d6d/d8c/d9d/fd6 x:0 0 0 2024-03-22T07:18:38.472 INFO:tasks.workunit.client.0.smithi081.stdout:9/598: chown d2/d2b/c47 189689956 1 2024-03-22T07:18:38.473 INFO:tasks.workunit.client.0.smithi081.stdout:5/690: rename d4/d75/d98/lcc to d4/df/d73/d8c/dc0/le2 0 2024-03-22T07:18:38.474 INFO:tasks.workunit.client.0.smithi081.stdout:2/462: creat da/d11/d21/f98 x:0 0 0 2024-03-22T07:18:38.475 INFO:tasks.workunit.client.0.smithi081.stdout:1/412: rename d6/d18/d1f/d24/f3f to d6/d18/d78/f89 0 2024-03-22T07:18:38.477 INFO:tasks.workunit.client.0.smithi081.stdout:1/413: rmdir d6/d18/d1f/d76 39 2024-03-22T07:18:38.478 INFO:tasks.workunit.client.0.smithi081.stdout:4/747: rmdir da/df/d38/dc9/de2 0 2024-03-22T07:18:38.479 INFO:tasks.workunit.client.0.smithi081.stdout:3/353: dread db/de/f24 [4194304,4194304] 0 2024-03-22T07:18:38.479 INFO:tasks.workunit.client.0.smithi081.stdout:5/691: rmdir d4/d10/d20/d25/da3/db5 39 2024-03-22T07:18:38.481 INFO:tasks.workunit.client.0.smithi081.stdout:2/463: mkdir da/db/d39/d47/d6e/d99 0 2024-03-22T07:18:38.481 INFO:tasks.workunit.client.0.smithi081.stdout:2/464: chown da/db 26017108 1 2024-03-22T07:18:38.481 INFO:tasks.workunit.client.0.smithi081.stdout:2/465: stat da/db/d39/d46/d42 0 2024-03-22T07:18:38.481 INFO:tasks.workunit.client.0.smithi081.stdout:2/466: write da/d56/f58 [840802,14172] 0 2024-03-22T07:18:38.483 INFO:tasks.workunit.client.0.smithi081.stdout:7/624: dread d9/dc/f90 [0,4194304] 0 2024-03-22T07:18:38.486 INFO:tasks.workunit.client.0.smithi081.stdout:3/354: mknod db/de/d4b/d6a/d70/d3a/d5d/c79 0 2024-03-22T07:18:38.486 INFO:tasks.workunit.client.0.smithi081.stdout:3/355: write db/de/d4b/d6a/d70/f44 [969644,32816] 0 2024-03-22T07:18:38.488 INFO:tasks.workunit.client.0.smithi081.stdout:2/467: mkdir da/db/d39/d47/d3b/d41/d50/d9a 0 2024-03-22T07:18:38.490 INFO:tasks.workunit.client.0.smithi081.stdout:9/599: rename d2/d73/d85/l1a to d2/d2b/d49/lc0 0 2024-03-22T07:18:38.491 INFO:tasks.workunit.client.0.smithi081.stdout:1/414: symlink d6/l8a 0 2024-03-22T07:18:38.493 INFO:tasks.workunit.client.0.smithi081.stdout:1/415: mknod d6/d18/d1f/d24/d48/c8b 0 2024-03-22T07:18:38.494 INFO:tasks.workunit.client.0.smithi081.stdout:1/416: mknod d6/d18/d1f/d4c/c8c 0 2024-03-22T07:18:38.495 INFO:tasks.workunit.client.0.smithi081.stdout:1/417: write d6/d18/d78/f89 [5272560,107917] 0 2024-03-22T07:18:38.495 INFO:tasks.workunit.client.0.smithi081.stdout:1/418: chown d6/f2a 36688 1 2024-03-22T07:18:38.496 INFO:tasks.workunit.client.0.smithi081.stdout:3/356: creat db/de/d4b/d6a/d70/d3a/f7a x:0 0 0 2024-03-22T07:18:38.498 INFO:tasks.workunit.client.0.smithi081.stdout:9/600: mkdir d2/d10/d30/d57/d66/d75/d94/dae/dc1 0 2024-03-22T07:18:38.501 INFO:tasks.workunit.client.0.smithi081.stdout:2/468: link da/db/d39/d47/d3b/d41/d50/l78 da/d11/l9b 0 2024-03-22T07:18:38.502 INFO:tasks.workunit.client.0.smithi081.stdout:1/419: truncate d6/d18/d1f/d24/f6f 2868335 0 2024-03-22T07:18:38.505 INFO:tasks.workunit.client.0.smithi081.stdout:9/601: mkdir d2/d5d/d58/dc2 0 2024-03-22T07:18:38.505 INFO:tasks.workunit.client.0.smithi081.stdout:9/602: readlink d2/d78/lba 0 2024-03-22T07:18:38.505 INFO:tasks.workunit.client.0.smithi081.stdout:9/603: stat d2/d10/d30/d57/dad/lb5 0 2024-03-22T07:18:38.506 INFO:tasks.workunit.client.0.smithi081.stdout:9/604: write d2/d5d/d58/f84 [949422,122376] 0 2024-03-22T07:18:38.506 INFO:tasks.workunit.client.0.smithi081.stdout:9/605: write d2/d5/d36/f8c [4957433,64649] 0 2024-03-22T07:18:38.506 INFO:tasks.workunit.client.0.smithi081.stdout:3/357: read db/de/f29 [1430140,44198] 0 2024-03-22T07:18:38.506 INFO:tasks.workunit.client.0.smithi081.stdout:4/748: dwrite da/fd2 [0,4194304] 0 2024-03-22T07:18:38.507 INFO:tasks.workunit.client.0.smithi081.stdout:2/469: symlink da/db/d39/d47/d3b/d64/l9c 0 2024-03-22T07:18:38.507 INFO:tasks.workunit.client.0.smithi081.stdout:5/692: dwrite d4/df/d22/f58 [0,4194304] 0 2024-03-22T07:18:38.507 INFO:tasks.workunit.client.0.smithi081.stdout:5/693: fsync d4/d75/d98/fad 0 2024-03-22T07:18:38.518 INFO:tasks.workunit.client.0.smithi081.stdout:2/470: unlink da/db/d39/d46/d42/d67/f97 0 2024-03-22T07:18:38.524 INFO:tasks.workunit.client.0.smithi081.stdout:2/471: creat da/db/d39/d47/d3b/d41/d50/f9d x:0 0 0 2024-03-22T07:18:38.528 INFO:tasks.workunit.client.0.smithi081.stdout:2/472: mknod da/db/d39/d47/c9e 0 2024-03-22T07:18:38.529 INFO:tasks.workunit.client.0.smithi081.stdout:4/749: rename da/d18/d1b/d26/d32/dbc to da/df/df4 0 2024-03-22T07:18:38.549 INFO:tasks.workunit.client.0.smithi081.stdout:9/606: dwrite d2/f11 [0,4194304] 0 2024-03-22T07:18:38.550 INFO:tasks.workunit.client.0.smithi081.stdout:1/420: dwrite d6/d18/d41/d49/d56/d30/f83 [0,4194304] 0 2024-03-22T07:18:38.552 INFO:tasks.workunit.client.0.smithi081.stdout:9/607: mknod d2/d5d/d5a/cc3 0 2024-03-22T07:18:38.552 INFO:tasks.workunit.client.0.smithi081.stdout:9/608: creat d2/d10/d30/d57/d66/d75/fc4 x:0 0 0 2024-03-22T07:18:38.557 INFO:tasks.workunit.client.0.smithi081.stdout:1/421: dread d6/f17 [0,4194304] 0 2024-03-22T07:18:38.557 INFO:tasks.workunit.client.0.smithi081.stdout:1/422: fsync d6/d18/d1f/d24/d48/d5c/f73 0 2024-03-22T07:18:38.564 INFO:tasks.workunit.client.0.smithi081.stdout:6/500: dwrite d3/f3c [0,4194304] 0 2024-03-22T07:18:38.565 INFO:tasks.workunit.client.0.smithi081.stdout:6/501: symlink d3/da/de/d31/d40/d50/d63/d5e/d9d/la4 0 2024-03-22T07:18:38.565 INFO:tasks.workunit.client.0.smithi081.stdout:6/502: chown d3/da/de/d31/d40/d50/d63/d26/d3f/fa1 0 1 2024-03-22T07:18:38.570 INFO:tasks.workunit.client.0.smithi081.stdout:1/423: dread d6/d18/d1f/d24/f2b [0,4194304] 0 2024-03-22T07:18:38.570 INFO:tasks.workunit.client.0.smithi081.stdout:1/424: dread - d6/d18/d1f/f87 zero size 2024-03-22T07:18:38.570 INFO:tasks.workunit.client.0.smithi081.stdout:6/503: rename d3/da/de/d31/d40/d50/d63/d26/d52/l77 to d3/da/de/d31/d40/la5 0 2024-03-22T07:18:38.571 INFO:tasks.workunit.client.0.smithi081.stdout:1/425: creat d6/d18/d1f/d76/f8d x:0 0 0 2024-03-22T07:18:38.571 INFO:tasks.workunit.client.0.smithi081.stdout:1/426: fdatasync d6/d18/d1f/d32/f4b 0 2024-03-22T07:18:38.571 INFO:tasks.workunit.client.0.smithi081.stdout:1/427: rmdir d6/d18/d1f/d32 39 2024-03-22T07:18:38.573 INFO:tasks.workunit.client.0.smithi081.stdout:5/694: dwrite d4/d75/d98/fad [0,4194304] 0 2024-03-22T07:18:38.573 INFO:tasks.workunit.client.0.smithi081.stdout:5/695: write d4/df/d73/f9d [1683239,127544] 0 2024-03-22T07:18:38.573 INFO:tasks.workunit.client.0.smithi081.stdout:5/696: fdatasync d4/df/f46 0 2024-03-22T07:18:38.573 INFO:tasks.workunit.client.0.smithi081.stdout:2/473: dwrite da/db/d39/d47/d3b/d41/f54 [0,4194304] 0 2024-03-22T07:18:38.573 INFO:tasks.workunit.client.0.smithi081.stdout:2/474: chown da/db/d39/d47/d3b/d41/l49 670074594 1 2024-03-22T07:18:38.574 INFO:tasks.workunit.client.0.smithi081.stdout:4/750: dwrite da/d18/d1b/d1e/d37/d1c/d74/fbf [0,4194304] 0 2024-03-22T07:18:38.574 INFO:tasks.workunit.client.0.smithi081.stdout:3/358: dwrite db/f5c [0,4194304] 0 2024-03-22T07:18:38.574 INFO:tasks.workunit.client.0.smithi081.stdout:7/625: dwrite d9/d10/d53/d81/d59/f9f [0,4194304] 0 2024-03-22T07:18:38.576 INFO:tasks.workunit.client.0.smithi081.stdout:5/697: creat d4/df/d22/d4d/fe3 x:0 0 0 2024-03-22T07:18:38.582 INFO:tasks.workunit.client.0.smithi081.stdout:6/504: dread d3/da/de/d31/f66 [0,4194304] 0 2024-03-22T07:18:38.584 INFO:tasks.workunit.client.0.smithi081.stdout:3/359: creat db/de/f7b x:0 0 0 2024-03-22T07:18:38.584 INFO:tasks.workunit.client.0.smithi081.stdout:3/360: write db/f4f [194326,120726] 0 2024-03-22T07:18:38.585 INFO:tasks.workunit.client.0.smithi081.stdout:9/609: truncate d2/f8 3918919 0 2024-03-22T07:18:38.585 INFO:tasks.workunit.client.0.smithi081.stdout:9/610: chown d2/d2b/l38 252 1 2024-03-22T07:18:38.591 INFO:tasks.workunit.client.0.smithi081.stdout:1/428: dread d6/d18/d1f/d24/d48/d5c/f73 [0,4194304] 0 2024-03-22T07:18:38.592 INFO:tasks.workunit.client.0.smithi081.stdout:7/626: mknod d9/d10/d16/d1e/d27/d70/cd7 0 2024-03-22T07:18:38.592 INFO:tasks.workunit.client.0.smithi081.stdout:7/627: write d9/d10/d16/f7a [201794,66664] 0 2024-03-22T07:18:38.592 INFO:tasks.workunit.client.0.smithi081.stdout:5/698: mkdir d4/df/d7c/de4 0 2024-03-22T07:18:38.592 INFO:tasks.workunit.client.0.smithi081.stdout:4/751: mknod da/df/d38/dd8/cf5 0 2024-03-22T07:18:38.592 INFO:tasks.workunit.client.0.smithi081.stdout:2/475: write f6 [3140855,119030] 0 2024-03-22T07:18:38.592 INFO:tasks.workunit.client.0.smithi081.stdout:2/476: write da/db/d39/d46/d42/f74 [348210,58183] 0 2024-03-22T07:18:38.592 INFO:tasks.workunit.client.0.smithi081.stdout:2/477: chown f6 0 1 2024-03-22T07:18:38.592 INFO:tasks.workunit.client.0.smithi081.stdout:1/429: symlink d6/d6a/d85/l8e 0 2024-03-22T07:18:38.593 INFO:tasks.workunit.client.0.smithi081.stdout:6/505: dread d3/da/de/f28 [0,4194304] 0 2024-03-22T07:18:38.593 INFO:tasks.workunit.client.0.smithi081.stdout:4/752: chown da/d18/d1b/d1e/d5f/l88 1 1 2024-03-22T07:18:38.594 INFO:tasks.workunit.client.0.smithi081.stdout:4/753: write da/d18/d1b/d26/d32/f91 [2730980,77003] 0 2024-03-22T07:18:38.595 INFO:tasks.workunit.client.0.smithi081.stdout:9/611: dread d2/d5/f3f [0,4194304] 0 2024-03-22T07:18:38.603 INFO:tasks.workunit.client.0.smithi081.stdout:1/430: creat d6/d6a/f8f x:0 0 0 2024-03-22T07:18:38.603 INFO:tasks.workunit.client.0.smithi081.stdout:6/506: read d3/da/de/d31/d40/d50/f85 [18331,105489] 0 2024-03-22T07:18:38.603 INFO:tasks.workunit.client.0.smithi081.stdout:7/628: rename d9/d10/d53/d81/d59/d89/cb8 to d9/cd8 0 2024-03-22T07:18:38.603 INFO:tasks.workunit.client.0.smithi081.stdout:7/629: chown d9/d10/d53/f98 24652 1 2024-03-22T07:18:38.603 INFO:tasks.workunit.client.0.smithi081.stdout:4/754: readlink da/d18/l59 0 2024-03-22T07:18:38.603 INFO:tasks.workunit.client.0.smithi081.stdout:2/478: symlink da/db/d39/d46/d42/l9f 0 2024-03-22T07:18:38.603 INFO:tasks.workunit.client.0.smithi081.stdout:2/479: write da/db/d6d/f73 [4691184,41566] 0 2024-03-22T07:18:38.603 INFO:tasks.workunit.client.0.smithi081.stdout:2/480: stat da/c19 0 2024-03-22T07:18:38.604 INFO:tasks.workunit.client.0.smithi081.stdout:9/612: dread d2/d5/d36/f6c [0,4194304] 0 2024-03-22T07:18:38.606 INFO:tasks.workunit.client.0.smithi081.stdout:1/431: creat d6/d6a/d85/f90 x:0 0 0 2024-03-22T07:18:38.606 INFO:tasks.workunit.client.0.smithi081.stdout:3/361: rmdir db/de 39 2024-03-22T07:18:38.606 INFO:tasks.workunit.client.0.smithi081.stdout:3/362: chown db/de/l14 607354 1 2024-03-22T07:18:38.606 INFO:tasks.workunit.client.0.smithi081.stdout:3/363: truncate db/de/d4b/d6a/d70/f5b 228860 0 2024-03-22T07:18:38.607 INFO:tasks.workunit.client.0.smithi081.stdout:6/507: mkdir d3/da/de/d31/d40/d50/d63/d5e/d9d/da6 0 2024-03-22T07:18:38.611 INFO:tasks.workunit.client.0.smithi081.stdout:5/699: rename d4/df/d7c/f95 to d4/df/d73/d8c/dc0/fe5 0 2024-03-22T07:18:38.611 INFO:tasks.workunit.client.0.smithi081.stdout:5/700: chown d4/d10/d20/db2/d2f/d8e/lb9 252 1 2024-03-22T07:18:38.613 INFO:tasks.workunit.client.0.smithi081.stdout:7/630: mkdir d9/d10/d16/db6/dd9 0 2024-03-22T07:18:38.614 INFO:tasks.workunit.client.0.smithi081.stdout:1/432: dread d6/d18/d1f/d32/f4b [0,4194304] 0 2024-03-22T07:18:38.616 INFO:tasks.workunit.client.0.smithi081.stdout:9/613: dread d2/d10/d30/d57/f9e [4194304,4194304] 0 2024-03-22T07:18:38.617 INFO:tasks.workunit.client.0.smithi081.stdout:9/614: fdatasync d2/d73/d85/f4a 0 2024-03-22T07:18:38.617 INFO:tasks.workunit.client.0.smithi081.stdout:9/615: creat d2/d78/fc5 x:0 0 0 2024-03-22T07:18:38.617 INFO:tasks.workunit.client.0.smithi081.stdout:9/616: creat d2/d78/d93/fc6 x:0 0 0 2024-03-22T07:18:38.619 INFO:tasks.workunit.client.0.smithi081.stdout:4/755: mknod da/d18/d1b/d34/cf6 0 2024-03-22T07:18:38.620 INFO:tasks.workunit.client.0.smithi081.stdout:1/433: link d6/d18/d60/c62 d6/d18/d1f/d24/d48/d6e/c91 0 2024-03-22T07:18:38.623 INFO:tasks.workunit.client.0.smithi081.stdout:7/631: unlink d9/d10/d2a/f7d 0 2024-03-22T07:18:38.623 INFO:tasks.workunit.client.0.smithi081.stdout:7/632: dread - d9/d10/d2a/fcd zero size 2024-03-22T07:18:38.624 INFO:tasks.workunit.client.0.smithi081.stdout:4/756: creat da/ff7 x:0 0 0 2024-03-22T07:18:38.626 INFO:tasks.workunit.client.0.smithi081.stdout:9/617: symlink d2/d2b/lc7 0 2024-03-22T07:18:38.627 INFO:tasks.workunit.client.0.smithi081.stdout:9/618: mknod d2/d2b/d49/cc8 0 2024-03-22T07:18:38.628 INFO:tasks.workunit.client.0.smithi081.stdout:9/619: creat d2/d78/fc9 x:0 0 0 2024-03-22T07:18:38.629 INFO:tasks.workunit.client.0.smithi081.stdout:1/434: write d6/d18/d1f/d24/f26 [117358,61952] 0 2024-03-22T07:18:38.633 INFO:tasks.workunit.client.0.smithi081.stdout:1/435: mkdir d6/d18/d1f/d92 0 2024-03-22T07:18:38.633 INFO:tasks.workunit.client.0.smithi081.stdout:1/436: fdatasync d6/d18/d1f/d24/d48/d5c/f77 0 2024-03-22T07:18:38.633 INFO:tasks.workunit.client.0.smithi081.stdout:1/437: dread d6/d18/d1f/d32/f59 [0,4194304] 0 2024-03-22T07:18:38.635 INFO:tasks.workunit.client.0.smithi081.stdout:1/438: mkdir d6/d18/d41/d49/d56/d30/d93 0 2024-03-22T07:18:38.642 INFO:tasks.workunit.client.0.smithi081.stdout:6/508: write d3/da/f57 [830582,111799] 0 2024-03-22T07:18:38.642 INFO:tasks.workunit.client.0.smithi081.stdout:6/509: fdatasync d3/da/de/f41 0 2024-03-22T07:18:38.644 INFO:tasks.workunit.client.0.smithi081.stdout:2/481: dwrite f6 [0,4194304] 0 2024-03-22T07:18:38.644 INFO:tasks.workunit.client.0.smithi081.stdout:2/482: write da/db/d39/d47/d3b/d41/f51 [1301785,71985] 0 2024-03-22T07:18:38.646 INFO:tasks.workunit.client.0.smithi081.stdout:6/510: dread d3/da/de/d31/d40/d50/d63/f2c [0,4194304] 0 2024-03-22T07:18:38.646 INFO:tasks.workunit.client.0.smithi081.stdout:6/511: read - d3/da/de/d31/d40/d50/d63/d26/d3f/d4c/f86 zero size 2024-03-22T07:18:38.646 INFO:tasks.workunit.client.0.smithi081.stdout:3/364: dwrite db/de/f24 [12582912,4194304] 0 2024-03-22T07:18:38.646 INFO:tasks.workunit.client.0.smithi081.stdout:4/757: dwrite da/df/d38/dd8/d4f/f5b [0,4194304] 0 2024-03-22T07:18:38.649 INFO:tasks.workunit.client.0.smithi081.stdout:2/483: rmdir da/db/d39/d47/d6e/d99 0 2024-03-22T07:18:38.651 INFO:tasks.workunit.client.0.smithi081.stdout:6/512: creat d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/fa7 x:0 0 0 2024-03-22T07:18:38.652 INFO:tasks.workunit.client.0.smithi081.stdout:3/365: creat db/de/d4b/d6a/f7c x:0 0 0 2024-03-22T07:18:38.652 INFO:tasks.workunit.client.0.smithi081.stdout:3/366: fsync db/de/f4e 0 2024-03-22T07:18:38.654 INFO:tasks.workunit.client.0.smithi081.stdout:4/758: creat da/d3d/ff8 x:0 0 0 2024-03-22T07:18:38.655 INFO:tasks.workunit.client.0.smithi081.stdout:2/484: rename da/d11/d21/d71/c7d to da/db/d39/d47/d3b/d63/ca0 0 2024-03-22T07:18:38.657 INFO:tasks.workunit.client.0.smithi081.stdout:1/439: dwrite d6/f67 [0,4194304] 0 2024-03-22T07:18:38.658 INFO:tasks.workunit.client.0.smithi081.stdout:9/620: dwrite d2/d10/f29 [0,4194304] 0 2024-03-22T07:18:38.658 INFO:tasks.workunit.client.0.smithi081.stdout:4/759: creat da/df/d38/dd8/d4f/daf/ff9 x:0 0 0 2024-03-22T07:18:38.660 INFO:tasks.workunit.client.0.smithi081.stdout:2/485: creat da/db/d39/d47/fa1 x:0 0 0 2024-03-22T07:18:38.660 INFO:tasks.workunit.client.0.smithi081.stdout:6/513: mkdir d3/da/de/d31/d40/da8 0 2024-03-22T07:18:38.661 INFO:tasks.workunit.client.0.smithi081.stdout:6/514: stat d3/da/de/d31/d40/d50/d63/d5e/d9d 0 2024-03-22T07:18:38.661 INFO:tasks.workunit.client.0.smithi081.stdout:2/486: write da/db/d39/d47/f5b [4720119,23804] 0 2024-03-22T07:18:38.662 INFO:tasks.workunit.client.0.smithi081.stdout:9/621: mkdir d2/d10/d30/d57/d66/d75/dca 0 2024-03-22T07:18:38.662 INFO:tasks.workunit.client.0.smithi081.stdout:1/440: symlink d6/d18/d1f/d24/l94 0 2024-03-22T07:18:38.663 INFO:tasks.workunit.client.0.smithi081.stdout:1/441: chown d6 1064160 1 2024-03-22T07:18:38.666 INFO:tasks.workunit.client.0.smithi081.stdout:2/487: creat da/d11/d21/fa2 x:0 0 0 2024-03-22T07:18:38.667 INFO:tasks.workunit.client.0.smithi081.stdout:9/622: link d2/d2b/d49/lc0 d2/d5d/lcb 0 2024-03-22T07:18:38.668 INFO:tasks.workunit.client.0.smithi081.stdout:4/760: truncate da/d18/d1b/d1e/f62 105428 0 2024-03-22T07:18:38.668 INFO:tasks.workunit.client.0.smithi081.stdout:4/761: stat da/df/df4 0 2024-03-22T07:18:38.668 INFO:tasks.workunit.client.0.smithi081.stdout:4/762: fsync da/df/d38/dd8/d4a/fda 0 2024-03-22T07:18:38.669 INFO:tasks.workunit.client.0.smithi081.stdout:9/623: read d2/d10/d30/d57/f7d [412199,90890] 0 2024-03-22T07:18:38.672 INFO:tasks.workunit.client.0.smithi081.stdout:9/624: symlink d2/d10/d30/d57/d66/d75/d94/dae/lcc 0 2024-03-22T07:18:38.672 INFO:tasks.workunit.client.0.smithi081.stdout:9/625: creat d2/d5d/d5a/fcd x:0 0 0 2024-03-22T07:18:38.673 INFO:tasks.workunit.client.0.smithi081.stdout:1/442: dread d6/d18/d41/d49/d56/d30/f3d [0,4194304] 0 2024-03-22T07:18:38.676 INFO:tasks.workunit.client.0.smithi081.stdout:2/488: read da/db/d39/d45/f8c [4119224,6084] 0 2024-03-22T07:18:38.676 INFO:tasks.workunit.client.0.smithi081.stdout:2/489: readlink da/db/d39/d47/d6e/l8a 0 2024-03-22T07:18:38.676 INFO:tasks.workunit.client.0.smithi081.stdout:7/633: dwrite f7 [0,4194304] 0 2024-03-22T07:18:38.676 INFO:tasks.workunit.client.0.smithi081.stdout:7/634: stat d9/d10/d16/d1e/f32 0 2024-03-22T07:18:38.677 INFO:tasks.workunit.client.0.smithi081.stdout:2/490: getdents da 0 2024-03-22T07:18:38.677 INFO:tasks.workunit.client.0.smithi081.stdout:2/491: chown da/db/d39/d47/f3d 0 1 2024-03-22T07:18:38.677 INFO:tasks.workunit.client.0.smithi081.stdout:2/492: fsync f5 0 2024-03-22T07:18:38.677 INFO:tasks.workunit.client.0.smithi081.stdout:2/493: chown da/db/d39/d47/d3b/d41/l49 217 1 2024-03-22T07:18:38.679 INFO:tasks.workunit.client.0.smithi081.stdout:7/635: creat d9/d10/d16/fda x:0 0 0 2024-03-22T07:18:38.682 INFO:tasks.workunit.client.0.smithi081.stdout:3/367: dwrite db/de/f20 [0,4194304] 0 2024-03-22T07:18:38.684 INFO:tasks.workunit.client.0.smithi081.stdout:1/443: read d6/d18/d78/f89 [1415862,65731] 0 2024-03-22T07:18:38.688 INFO:tasks.workunit.client.0.smithi081.stdout:2/494: read da/db/d39/d47/d3b/f55 [534446,8907] 0 2024-03-22T07:18:38.691 INFO:tasks.workunit.client.0.smithi081.stdout:7/636: read d9/d3f/f65 [854720,115292] 0 2024-03-22T07:18:38.691 INFO:tasks.workunit.client.0.smithi081.stdout:7/637: fsync d9/d10/d53/d81/d59/fcf 0 2024-03-22T07:18:38.691 INFO:tasks.workunit.client.0.smithi081.stdout:3/368: dread db/de/d4b/d6a/d70/f5b [0,4194304] 0 2024-03-22T07:18:38.691 INFO:tasks.workunit.client.0.smithi081.stdout:1/444: creat d6/d18/d41/d49/d56/f95 x:0 0 0 2024-03-22T07:18:38.692 INFO:tasks.workunit.client.0.smithi081.stdout:2/495: truncate da/db/f27 279721 0 2024-03-22T07:18:38.693 INFO:tasks.workunit.client.0.smithi081.stdout:1/445: mknod d6/d18/d1f/c96 0 2024-03-22T07:18:38.694 INFO:tasks.workunit.client.0.smithi081.stdout:3/369: creat db/de/f7d x:0 0 0 2024-03-22T07:18:38.697 INFO:tasks.workunit.client.0.smithi081.stdout:7/638: symlink d9/d3f/d52/ldb 0 2024-03-22T07:18:38.699 INFO:tasks.workunit.client.0.smithi081.stdout:3/370: unlink db/de/d4b/d6a/d70/d3a/d5d/l65 0 2024-03-22T07:18:38.701 INFO:tasks.workunit.client.0.smithi081.stdout:2/496: link da/db/l5f da/db/d39/d47/d3b/d41/la3 0 2024-03-22T07:18:38.701 INFO:tasks.workunit.client.0.smithi081.stdout:2/497: creat da/db/d39/d46/d42/d67/fa4 x:0 0 0 2024-03-22T07:18:38.702 INFO:tasks.workunit.client.0.smithi081.stdout:7/639: creat d9/d10/d4b/d9b/fdc x:0 0 0 2024-03-22T07:18:38.705 INFO:tasks.workunit.client.0.smithi081.stdout:7/640: creat d9/d10/d16/d1e/d27/d6d/d8c/dc1/fdd x:0 0 0 2024-03-22T07:18:38.706 INFO:tasks.workunit.client.0.smithi081.stdout:3/371: link db/l27 db/de/d4b/l7e 0 2024-03-22T07:18:38.706 INFO:tasks.workunit.client.0.smithi081.stdout:3/372: chown db/d23/f4d 258 1 2024-03-22T07:18:38.708 INFO:tasks.workunit.client.0.smithi081.stdout:3/373: symlink db/d23/l7f 0 2024-03-22T07:18:38.709 INFO:tasks.workunit.client.0.smithi081.stdout:3/374: creat db/d38/f80 x:0 0 0 2024-03-22T07:18:38.711 INFO:tasks.workunit.client.0.smithi081.stdout:3/375: symlink db/de/l81 0 2024-03-22T07:18:38.711 INFO:tasks.workunit.client.0.smithi081.stdout:3/376: write db/de/d4b/d6a/d70/d3a/f6b [919768,44562] 0 2024-03-22T07:18:38.712 INFO:tasks.workunit.client.0.smithi081.stdout:3/377: chown db/f18 7578 1 2024-03-22T07:18:38.712 INFO:tasks.workunit.client.0.smithi081.stdout:3/378: dread - db/d23/f4d zero size 2024-03-22T07:18:38.717 INFO:tasks.workunit.client.0.smithi081.stdout:2/498: dread da/d11/d21/f33 [0,4194304] 0 2024-03-22T07:18:38.717 INFO:tasks.workunit.client.0.smithi081.stdout:2/499: write da/d11/d21/fa2 [873131,124076] 0 2024-03-22T07:18:38.717 INFO:tasks.workunit.client.0.smithi081.stdout:2/500: stat da/db/d6d/d72/l7a 0 2024-03-22T07:18:38.719 INFO:tasks.workunit.client.0.smithi081.stdout:1/446: dwrite d6/d6a/d85/f90 [0,4194304] 0 2024-03-22T07:18:38.719 INFO:tasks.workunit.client.0.smithi081.stdout:6/515: dwrite d3/da/f59 [0,4194304] 0 2024-03-22T07:18:38.725 INFO:tasks.workunit.client.0.smithi081.stdout:6/516: rename d3/da/de/d31/d40/d50/d63/d26/f1d to d3/da/de/d31/fa9 0 2024-03-22T07:18:38.729 INFO:tasks.workunit.client.0.smithi081.stdout:6/517: read d3/da/de/d31/d40/d50/d63/f1e [88100,82849] 0 2024-03-22T07:18:38.732 INFO:tasks.workunit.client.0.smithi081.stdout:7/641: dwrite d9/d3f/db3/fc6 [0,4194304] 0 2024-03-22T07:18:38.732 INFO:tasks.workunit.client.0.smithi081.stdout:7/642: chown d9/d10/d53/d81/d59/d89/dd4/cd5 1 1 2024-03-22T07:18:38.736 INFO:tasks.workunit.client.0.smithi081.stdout:7/643: rename d9/dc/l91 to d9/d10/d2a/d73/lde 0 2024-03-22T07:18:38.738 INFO:tasks.workunit.client.0.smithi081.stdout:7/644: stat d9/d10/d2a/c2d 0 2024-03-22T07:18:38.741 INFO:tasks.workunit.client.0.smithi081.stdout:7/645: rmdir d9/d3f/db3 39 2024-03-22T07:18:38.741 INFO:tasks.workunit.client.0.smithi081.stdout:7/646: stat d9/d10/d16/d1e/d27/d70/f8d 0 2024-03-22T07:18:38.741 INFO:tasks.workunit.client.0.smithi081.stdout:7/647: stat d9/d10/d16/d1e/d27/l3e 0 2024-03-22T07:18:38.741 INFO:tasks.workunit.client.0.smithi081.stdout:7/648: fsync d9/d3f/fa0 0 2024-03-22T07:18:38.741 INFO:tasks.workunit.client.0.smithi081.stdout:6/518: write d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/f6b [4820121,69355] 0 2024-03-22T07:18:38.741 INFO:tasks.workunit.client.0.smithi081.stdout:7/649: read - d9/d3f/d52/d83/fba zero size 2024-03-22T07:18:38.743 INFO:tasks.workunit.client.0.smithi081.stdout:6/519: creat d3/da/de/d31/d40/d50/d63/faa x:0 0 0 2024-03-22T07:18:38.744 INFO:tasks.workunit.client.0.smithi081.stdout:6/520: truncate d3/da/f83 709855 0 2024-03-22T07:18:38.746 INFO:tasks.workunit.client.0.smithi081.stdout:6/521: mknod d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/cab 0 2024-03-22T07:18:38.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/501: dwrite da/db/d39/d46/f36 [0,4194304] 0 2024-03-22T07:18:38.747 INFO:tasks.workunit.client.0.smithi081.stdout:2/502: truncate da/d11/f1b 3095712 0 2024-03-22T07:18:38.747 INFO:tasks.workunit.client.0.smithi081.stdout:1/447: dwrite d6/d18/f27 [0,4194304] 0 2024-03-22T07:18:38.749 INFO:tasks.workunit.client.0.smithi081.stdout:1/448: truncate d6/d18/d1f/d32/f4b 3059596 0 2024-03-22T07:18:38.750 INFO:tasks.workunit.client.0.smithi081.stdout:2/503: mknod da/d11/ca5 0 2024-03-22T07:18:38.755 INFO:tasks.workunit.client.0.smithi081.stdout:6/522: write d3/da/de/d31/d40/d50/f87 [1561527,61168] 0 2024-03-22T07:18:38.760 INFO:tasks.workunit.client.0.smithi081.stdout:6/523: unlink d3/da/d48/c62 0 2024-03-22T07:18:38.760 INFO:tasks.workunit.client.0.smithi081.stdout:6/524: fdatasync d3/da/de/f9b 0 2024-03-22T07:18:38.764 INFO:tasks.workunit.client.0.smithi081.stdout:7/650: dwrite d9/d10/d53/f82 [0,4194304] 0 2024-03-22T07:18:38.767 INFO:tasks.workunit.client.0.smithi081.stdout:7/651: creat d9/d10/d16/d1e/d27/d6d/d8c/fdf x:0 0 0 2024-03-22T07:18:38.776 INFO:tasks.workunit.client.0.smithi081.stdout:7/652: rename d9/d10/d16/d1e/d27 to d9/d10/d53/d81/d59/d89/dd4/de0 0 2024-03-22T07:18:38.777 INFO:tasks.workunit.client.0.smithi081.stdout:6/525: dwrite d3/da/de/d31/fa9 [0,4194304] 0 2024-03-22T07:18:38.778 INFO:tasks.workunit.client.0.smithi081.stdout:6/526: creat d3/da/de/fac x:0 0 0 2024-03-22T07:18:38.787 INFO:tasks.workunit.client.0.smithi081.stdout:7/653: write d9/d10/d53/d81/d59/d89/dd4/de0/d70/faa [2325916,2795] 0 2024-03-22T07:18:38.788 INFO:tasks.workunit.client.0.smithi081.stdout:7/654: mknod d9/d10/d16/d1e/d6f/ce1 0 2024-03-22T07:18:38.798 INFO:tasks.workunit.client.0.smithi081.stdout:7/655: dread d9/f75 [0,4194304] 0 2024-03-22T07:18:38.798 INFO:tasks.workunit.client.0.smithi081.stdout:6/527: dwrite d3/da/de/d31/d40/f99 [0,4194304] 0 2024-03-22T07:18:38.799 INFO:tasks.workunit.client.0.smithi081.stdout:7/656: creat d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/fe2 x:0 0 0 2024-03-22T07:18:38.799 INFO:tasks.workunit.client.0.smithi081.stdout:6/528: mknod d3/da/de/d31/cad 0 2024-03-22T07:18:38.800 INFO:tasks.workunit.client.0.smithi081.stdout:6/529: rmdir d3/da/de/d31/d40/d50/d63/d5e/d9d 39 2024-03-22T07:18:38.800 INFO:tasks.workunit.client.0.smithi081.stdout:6/530: chown d3/f6 22165 1 2024-03-22T07:18:38.800 INFO:tasks.workunit.client.0.smithi081.stdout:6/531: fsync d3/da/de/d31/d40/d50/f7a 0 2024-03-22T07:18:38.801 INFO:tasks.workunit.client.0.smithi081.stdout:6/532: creat d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/d9c/fae x:0 0 0 2024-03-22T07:18:38.802 INFO:tasks.workunit.client.0.smithi081.stdout:6/533: mknod d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/caf 0 2024-03-22T07:18:38.802 INFO:tasks.workunit.client.0.smithi081.stdout:6/534: creat d3/da/de/d31/d40/d50/d63/d26/d3f/d5a/d9c/fb0 x:0 0 0 2024-03-22T07:18:38.804 INFO:tasks.workunit.client.0.smithi081.stdout:6/535: rename d3/da/de/d31/d40/d50/d63/d26/d52/l94 to d3/da/de/d31/d40/d50/d63/d5e/d9d/da6/lb1 0 2024-03-22T07:18:38.810 INFO:tasks.workunit.client.0.smithi081.stdout:7/657: dread d9/d10/f2e [0,4194304] 0 2024-03-22T07:18:38.810 INFO:tasks.workunit.client.0.smithi081.stdout:7/658: write d9/d3f/d52/d83/f8e [1009873,5187] 0 2024-03-22T07:18:38.811 INFO:tasks.workunit.client.0.smithi081.stdout:6/536: write d3/da/de/d31/d40/d50/d63/f37 [1637821,55325] 0 2024-03-22T07:18:38.823 INFO:tasks.workunit.client.0.smithi081.stdout:6/537: rename d3/da/de/d31 to d3/d67/db2 0 2024-03-22T07:18:38.824 INFO:tasks.workunit.client.0.smithi081.stdout:6/538: dread d3/d67/db2/d40/d50/d63/d26/f71 [0,4194304] 0 2024-03-22T07:18:38.825 INFO:tasks.workunit.client.0.smithi081.stdout:6/539: symlink d3/d67/db2/d40/d50/d63/d26/d3f/lb3 0 2024-03-22T07:18:38.827 INFO:tasks.workunit.client.0.smithi081.stdout:6/540: mkdir d3/d67/db2/d40/d50/d63/db4 0 2024-03-22T07:18:38.828 INFO:tasks.workunit.client.0.smithi081.stdout:7/659: dwrite d9/d10/f2e [0,4194304] 0 2024-03-22T07:18:38.829 INFO:tasks.workunit.client.0.smithi081.stdout:6/541: mkdir d3/da/de/db5 0 2024-03-22T07:18:38.831 INFO:tasks.workunit.client.0.smithi081.stdout:6/542: dread d3/da/f53 [0,4194304] 0 2024-03-22T07:18:38.833 INFO:tasks.workunit.client.0.smithi081.stdout:6/543: symlink d3/d67/db2/d40/d50/d63/d26/d3f/d5a/d9c/lb6 0 2024-03-22T07:18:38.833 INFO:tasks.workunit.client.0.smithi081.stdout:6/544: fdatasync d3/d67/db2/d40/d50/d63/d26/d3f/d5a/f7e 0 2024-03-22T07:18:38.834 INFO:tasks.workunit.client.0.smithi081.stdout:6/545: creat d3/d67/db2/d40/d50/d63/d5e/d9d/da6/fb7 x:0 0 0 2024-03-22T07:18:38.834 INFO:tasks.workunit.client.0.smithi081.stdout:6/546: chown d3/d67/db2/d40/d50/d63/d26/f44 780723 1 2024-03-22T07:18:38.834 INFO:tasks.workunit.client.0.smithi081.stdout:6/547: chown d3/d67/db2/d40/d50/d63/d26/l74 1504 1 2024-03-22T07:18:38.835 INFO:tasks.workunit.client.0.smithi081.stdout:6/548: fdatasync d3/d67/db2/d40/d50/d63/d26/f84 0 2024-03-22T07:18:38.835 INFO:tasks.workunit.client.0.smithi081.stdout:6/549: creat d3/d67/db2/fb8 x:0 0 0 2024-03-22T07:18:38.845 INFO:tasks.workunit.client.0.smithi081.stdout:7/660: dwrite d9/d10/d9e/fce [0,4194304] 0 2024-03-22T07:18:38.845 INFO:tasks.workunit.client.0.smithi081.stdout:7/661: chown d9/d10/d53/d81/d59/d89 8563 1 2024-03-22T07:18:38.845 INFO:tasks.workunit.client.0.smithi081.stdout:7/662: dread - d9/d10/d53/d81/d59/fcf zero size 2024-03-22T07:18:38.853 INFO:tasks.workunit.client.0.smithi081.stdout:6/550: dwrite d3/d67/db2/d40/d50/f76 [0,4194304] 0 2024-03-22T07:18:38.854 INFO:tasks.workunit.client.0.smithi081.stdout:7/663: unlink d9/d3f/d52/l69 0 2024-03-22T07:18:38.856 INFO:tasks.workunit.client.0.smithi081.stdout:7/664: dread - d9/d14/f54 zero size 2024-03-22T07:18:38.859 INFO:tasks.workunit.client.0.smithi081.stdout:5/701: sync 2024-03-22T07:18:38.859 INFO:tasks.workunit.client.0.smithi081.stdout:8/560: sync 2024-03-22T07:18:38.859 INFO:tasks.workunit.client.0.smithi081.stdout:0/550: sync 2024-03-22T07:18:38.860 INFO:tasks.workunit.client.0.smithi081.stdout:0/551: truncate d3/d32/f8f 305521 0 2024-03-22T07:18:38.860 INFO:tasks.workunit.client.0.smithi081.stdout:7/665: dread d9/d3f/fa0 [0,4194304] 0 2024-03-22T07:18:38.862 INFO:tasks.workunit.client.0.smithi081.stdout:6/551: rename d3/d67/db2/d40/d50 to d3/d67/db2/d40/db9 0 2024-03-22T07:18:38.863 INFO:tasks.workunit.client.0.smithi081.stdout:8/561: rmdir d2/d3/d19/d1e/d13/d22/d2f/d71/d7c 39 2024-03-22T07:18:38.863 INFO:tasks.workunit.client.0.smithi081.stdout:8/562: write d2/d3/d19/d1e/d13/d22/d38/f44 [2464994,102564] 0 2024-03-22T07:18:38.864 INFO:tasks.workunit.client.0.smithi081.stdout:8/563: fsync d2/d3/d19/d1e/d63/da0/d9c/fb1 0 2024-03-22T07:18:38.864 INFO:tasks.workunit.client.0.smithi081.stdout:0/552: mkdir d3/dc/d40/d69/dbf 0 2024-03-22T07:18:38.864 INFO:tasks.workunit.client.0.smithi081.stdout:0/553: write d3/dc/d2b/fba [176132,92113] 0 2024-03-22T07:18:38.865 INFO:tasks.workunit.client.0.smithi081.stdout:6/552: truncate d3/f9 9341492 0 2024-03-22T07:18:38.866 INFO:tasks.workunit.client.0.smithi081.stdout:7/666: symlink d9/d10/d16/d1e/le3 0 2024-03-22T07:18:38.866 INFO:tasks.workunit.client.0.smithi081.stdout:7/667: write d9/d10/d16/fcb [1069892,86129] 0 2024-03-22T07:18:38.866 INFO:tasks.workunit.client.0.smithi081.stdout:5/702: dread d4/d75/d98/f9c [0,4194304] 0 2024-03-22T07:18:38.867 INFO:tasks.workunit.client.0.smithi081.stdout:8/564: truncate d2/d3/d28/d2b/f2e 602816 0 2024-03-22T07:18:38.870 INFO:tasks.workunit.client.0.smithi081.stdout:7/668: unlink d9/d10/d53/d81/d59/d89/dd4/de0/d6d/laf 0 2024-03-22T07:18:38.870 INFO:tasks.workunit.client.0.smithi081.stdout:7/669: dread - d9/d10/fb7 zero size 2024-03-22T07:18:38.871 INFO:tasks.workunit.client.0.smithi081.stdout:8/565: mkdir d2/d3/d19/d1e/d13/d66/dc1 0 2024-03-22T07:18:38.871 INFO:tasks.workunit.client.0.smithi081.stdout:0/554: dread d3/dc/d33/f7d [0,4194304] 0 2024-03-22T07:18:38.871 INFO:tasks.workunit.client.0.smithi081.stdout:0/555: write d3/d32/f84 [984631,117526] 0 2024-03-22T07:18:38.874 INFO:tasks.workunit.client.0.smithi081.stdout:0/556: creat d3/dc/d40/d69/dbf/fc0 x:0 0 0 2024-03-22T07:18:38.875 INFO:tasks.workunit.client.0.smithi081.stdout:8/566: rmdir d2/d3/d19/d1e/d63 39 2024-03-22T07:18:38.875 INFO:tasks.workunit.client.0.smithi081.stdout:8/567: read d2/d3/d19/d1e/d13/d22/f3a [34760,112382] 0 2024-03-22T07:18:38.875 INFO:tasks.workunit.client.0.smithi081.stdout:8/568: dread - d2/d3/d19/d1e/d13/d22/d38/f91 zero size 2024-03-22T07:18:38.876 INFO:tasks.workunit.client.0.smithi081.stdout:0/557: stat d3/dc/d33/f47 0 2024-03-22T07:18:38.878 INFO:tasks.workunit.client.0.smithi081.stdout:8/569: mknod d2/d3/d19/d1e/d13/d22/db8/cc2 0 2024-03-22T07:18:38.881 INFO:tasks.workunit.client.0.smithi081.stdout:0/558: rmdir d3/d9b 39 2024-03-22T07:18:38.881 INFO:tasks.workunit.client.0.smithi081.stdout:0/559: chown d3/dc/f8d 6544 1 2024-03-22T07:18:38.883 INFO:tasks.workunit.client.0.smithi081.stdout:8/570: rename d2/d3/d19/d1e/d13/c29 to d2/cc3 0 2024-03-22T07:18:38.883 INFO:tasks.workunit.client.0.smithi081.stdout:8/571: dread - d2/d3/d28/d2b/d61/f74 zero size 2024-03-22T07:18:38.885 INFO:tasks.workunit.client.0.smithi081.stdout:0/560: rename d3/d32/d35 to d3/dc/d40/d69/d92/dc1 0 2024-03-22T07:18:38.885 INFO:tasks.workunit.client.0.smithi081.stdout:0/561: chown d3/dc/d33/d62 951692 1 2024-03-22T07:18:38.885 INFO:tasks.workunit.client.0.smithi081.stdout:0/562: creat d3/d6a/fc2 x:0 0 0 2024-03-22T07:18:38.886 INFO:tasks.workunit.client.0.smithi081.stdout:0/563: dread - d3/dc/d2b/d39/f66 zero size 2024-03-22T07:18:38.891 INFO:tasks.workunit.client.0.smithi081.stdout:8/572: write d2/d3/d28/f30 [16736,52223] 0 2024-03-22T07:18:38.891 INFO:tasks.workunit.client.0.smithi081.stdout:8/573: chown d2/d3/d19/d1e/f9d 23 1 2024-03-22T07:18:38.893 INFO:tasks.workunit.client.0.smithi081.stdout:8/574: creat d2/d3/d19/d1e/d13/d22/d2f/d71/d7c/fc4 x:0 0 0 2024-03-22T07:18:38.893 INFO:tasks.workunit.client.0.smithi081.stdout:8/575: stat d2/d3/d19/d1e/d13/d22/d2f/d71/d7c/f81 0 2024-03-22T07:18:38.895 INFO:tasks.workunit.client.0.smithi081.stdout:6/553: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/fb0 [0,4194304] 0 2024-03-22T07:18:38.895 INFO:tasks.workunit.client.0.smithi081.stdout:5/703: dwrite d4/d10/f21 [4194304,4194304] 0 2024-03-22T07:18:38.895 INFO:tasks.workunit.client.0.smithi081.stdout:5/704: stat d4/d75/d98 0 2024-03-22T07:18:38.895 INFO:tasks.workunit.client.0.smithi081.stdout:8/576: creat d2/d3/d19/d1e/d13/d22/d38/fc5 x:0 0 0 2024-03-22T07:18:38.895 INFO:tasks.workunit.client.0.smithi081.stdout:5/705: write f3 [3420907,118148] 0 2024-03-22T07:18:38.895 INFO:tasks.workunit.client.0.smithi081.stdout:8/577: fdatasync d2/d3/d28/f82 0 2024-03-22T07:18:38.895 INFO:tasks.workunit.client.0.smithi081.stdout:5/706: fdatasync d4/df/d22/d4d/f63 0 2024-03-22T07:18:38.899 INFO:tasks.workunit.client.0.smithi081.stdout:0/564: dread d3/f24 [0,4194304] 0 2024-03-22T07:18:38.903 INFO:tasks.workunit.client.0.smithi081.stdout:6/554: rename d3/d67/db2/d40/db9/d63/f37 to d3/d67/db2/d40/db9/d63/d5e/d9d/fba 0 2024-03-22T07:18:38.903 INFO:tasks.workunit.client.0.smithi081.stdout:6/555: chown d3/d67/db2/d40/la5 31 1 2024-03-22T07:18:38.903 INFO:tasks.workunit.client.0.smithi081.stdout:6/556: chown d3/da/d48 17 1 2024-03-22T07:18:38.909 INFO:tasks.workunit.client.0.smithi081.stdout:6/557: symlink d3/d67/db2/d40/db9/d63/db4/lbb 0 2024-03-22T07:18:38.910 INFO:tasks.workunit.client.0.smithi081.stdout:8/578: mkdir d2/d3/d28/d2b/d61/dc6 0 2024-03-22T07:18:38.911 INFO:tasks.workunit.client.0.smithi081.stdout:6/558: creat d3/da/d48/fbc x:0 0 0 2024-03-22T07:18:38.911 INFO:tasks.workunit.client.0.smithi081.stdout:6/559: fsync d3/da/f57 0 2024-03-22T07:18:38.911 INFO:tasks.workunit.client.0.smithi081.stdout:8/579: stat d2/d3/d19/d1e/d13/d22/c45 0 2024-03-22T07:18:38.912 INFO:tasks.workunit.client.0.smithi081.stdout:5/707: dwrite d4/d10/d20/d25/da3/fa7 [0,4194304] 0 2024-03-22T07:18:38.913 INFO:tasks.workunit.client.0.smithi081.stdout:6/560: rename d3/d67/db2/l79 to d3/d67/db2/d40/db9/d63/d5e/d9d/da6/lbd 0 2024-03-22T07:18:38.919 INFO:tasks.workunit.client.0.smithi081.stdout:6/561: unlink d3/d67/db2/d40/db9/d63/c35 0 2024-03-22T07:18:38.920 INFO:tasks.workunit.client.0.smithi081.stdout:6/562: dread - d3/da/d48/fbc zero size 2024-03-22T07:18:38.920 INFO:tasks.workunit.client.0.smithi081.stdout:6/563: dread - d3/d67/db2/d40/db9/d63/d26/d3f/f98 zero size 2024-03-22T07:18:38.920 INFO:tasks.workunit.client.0.smithi081.stdout:0/565: write d3/dc/d2b/d39/f75 [875781,54108] 0 2024-03-22T07:18:38.921 INFO:tasks.workunit.client.0.smithi081.stdout:8/580: read d2/d3/d19/d1e/d13/f47 [2690663,42159] 0 2024-03-22T07:18:38.921 INFO:tasks.workunit.client.0.smithi081.stdout:6/564: readlink d3/d67/db2/d40/l51 0 2024-03-22T07:18:38.923 INFO:tasks.workunit.client.0.smithi081.stdout:0/566: symlink d3/dc/d2b/d4d/lc3 0 2024-03-22T07:18:38.927 INFO:tasks.workunit.client.0.smithi081.stdout:5/708: dwrite d4/d9/f2c [4194304,4194304] 0 2024-03-22T07:18:38.927 INFO:tasks.workunit.client.0.smithi081.stdout:6/565: truncate d3/da/de/f41 3880056 0 2024-03-22T07:18:38.927 INFO:tasks.workunit.client.0.smithi081.stdout:6/566: fsync d3/f3c 0 2024-03-22T07:18:38.932 INFO:tasks.workunit.client.0.smithi081.stdout:6/567: creat d3/d67/db2/d40/db9/d63/d26/d52/fbe x:0 0 0 2024-03-22T07:18:38.944 INFO:tasks.workunit.client.0.smithi081.stdout:8/581: dwrite d2/d3/d19/d1e/d13/f60 [0,4194304] 0 2024-03-22T07:18:38.946 INFO:tasks.workunit.client.0.smithi081.stdout:8/582: mknod d2/d3/d19/d1e/d13/dba/cc7 0 2024-03-22T07:18:38.948 INFO:tasks.workunit.client.0.smithi081.stdout:8/583: truncate d2/d3/d19/d1e/d13/fa3 626898 0 2024-03-22T07:18:38.949 INFO:tasks.workunit.client.0.smithi081.stdout:8/584: fsync d2/d3/d19/d1e/d13/f60 0 2024-03-22T07:18:38.949 INFO:tasks.workunit.client.0.smithi081.stdout:8/585: write d2/d3/fae [195414,102566] 0 2024-03-22T07:18:38.953 INFO:tasks.workunit.client.0.smithi081.stdout:8/586: rename d2/d3/d19/d1e/d13/d22/d2f/d71 to d2/d3/d19/d1e/d63/da0/db7/dc8 0 2024-03-22T07:18:38.958 INFO:tasks.workunit.client.0.smithi081.stdout:0/567: dwrite d3/dc/d33/f96 [0,4194304] 0 2024-03-22T07:18:38.958 INFO:tasks.workunit.client.0.smithi081.stdout:8/587: mkdir d2/d3/d9e/dc9 0 2024-03-22T07:18:38.958 INFO:tasks.workunit.client.0.smithi081.stdout:8/588: write d2/d3/d28/f30 [2545436,15234] 0 2024-03-22T07:18:38.959 INFO:tasks.workunit.client.0.smithi081.stdout:6/568: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/f98 [0,4194304] 0 2024-03-22T07:18:38.959 INFO:tasks.workunit.client.0.smithi081.stdout:8/589: mkdir d2/d3/d9e/dc9/dca 0 2024-03-22T07:18:38.959 INFO:tasks.workunit.client.0.smithi081.stdout:6/569: stat d3/d67/db2/d40/db9/d63/d7d/l88 0 2024-03-22T07:18:38.965 INFO:tasks.workunit.client.0.smithi081.stdout:8/590: dread d2/d3/d23/f53 [0,4194304] 0 2024-03-22T07:18:38.968 INFO:tasks.workunit.client.0.smithi081.stdout:0/568: symlink d3/dc/d2b/lc4 0 2024-03-22T07:18:38.968 INFO:tasks.workunit.client.0.smithi081.stdout:0/569: fdatasync d3/f44 0 2024-03-22T07:18:38.968 INFO:tasks.workunit.client.0.smithi081.stdout:0/570: write d3/dc/d33/f97 [5027292,45975] 0 2024-03-22T07:18:38.969 INFO:tasks.workunit.client.0.smithi081.stdout:8/591: unlink d2/d3/d19/d1e/d63/da0/d9c/f9f 0 2024-03-22T07:18:38.971 INFO:tasks.workunit.client.0.smithi081.stdout:0/571: creat d3/dc/d40/d69/d92/dc1/d91/fc5 x:0 0 0 2024-03-22T07:18:38.972 INFO:tasks.workunit.client.0.smithi081.stdout:6/570: mknod d3/d67/db2/d40/db9/d63/d26/cbf 0 2024-03-22T07:18:38.972 INFO:tasks.workunit.client.0.smithi081.stdout:6/571: write d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f6b [6006074,117881] 0 2024-03-22T07:18:38.974 INFO:tasks.workunit.client.0.smithi081.stdout:0/572: dread d3/dc/d2b/d39/f75 [0,4194304] 0 2024-03-22T07:18:38.975 INFO:tasks.workunit.client.0.smithi081.stdout:6/572: truncate d3/d67/db2/d40/db9/d63/d26/d3f/f29 1182041 0 2024-03-22T07:18:38.975 INFO:tasks.workunit.client.0.smithi081.stdout:6/573: fdatasync d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f78 0 2024-03-22T07:18:38.976 INFO:tasks.workunit.client.0.smithi081.stdout:8/592: rmdir d2/d3/d19/d1e/d13/d22/d38 39 2024-03-22T07:18:38.979 INFO:tasks.workunit.client.0.smithi081.stdout:8/593: rename d2/d3/ce to d2/d3/d19/d1e/d13/d22/d38/dbb/ccb 0 2024-03-22T07:18:38.980 INFO:tasks.workunit.client.0.smithi081.stdout:0/573: mkdir d3/dc/dc6 0 2024-03-22T07:18:38.986 INFO:tasks.workunit.client.0.smithi081.stdout:6/574: dread d3/d67/db2/d40/db9/d63/f68 [0,4194304] 0 2024-03-22T07:18:38.986 INFO:tasks.workunit.client.0.smithi081.stdout:6/575: truncate d3/d67/db2/d40/db9/d63/faa 86799 0 2024-03-22T07:18:38.986 INFO:tasks.workunit.client.0.smithi081.stdout:6/576: write d3/da/fd [36045,13580] 0 2024-03-22T07:18:38.987 INFO:tasks.workunit.client.0.smithi081.stdout:6/577: read d3/d67/db2/d40/f6d [13914,72329] 0 2024-03-22T07:18:39.041 INFO:tasks.workunit.client.0.smithi081.stdout:0/574: dwrite d3/f22 [0,4194304] 0 2024-03-22T07:18:39.041 INFO:tasks.workunit.client.0.smithi081.stdout:0/575: read - d3/dc/d40/fb6 zero size 2024-03-22T07:18:39.041 INFO:tasks.workunit.client.0.smithi081.stdout:0/576: write d3/d6a/fc2 [164882,103258] 0 2024-03-22T07:18:39.041 INFO:tasks.workunit.client.0.smithi081.stdout:0/577: truncate d3/d6a/f8e 4660992 0 2024-03-22T07:18:39.149 INFO:tasks.workunit.client.0.smithi081.stdout:0/578: dwrite d3/dc/f8d [0,4194304] 0 2024-03-22T07:18:39.155 INFO:tasks.workunit.client.0.smithi081.stdout:0/579: dread d3/f24 [0,4194304] 0 2024-03-22T07:18:39.155 INFO:tasks.workunit.client.0.smithi081.stdout:0/580: chown d3/f44 282 1 2024-03-22T07:18:39.156 INFO:tasks.workunit.client.0.smithi081.stdout:0/581: symlink d3/dc/d33/d62/lc7 0 2024-03-22T07:18:39.157 INFO:tasks.workunit.client.0.smithi081.stdout:0/582: creat d3/dc/d40/d69/d92/dc1/fc8 x:0 0 0 2024-03-22T07:18:39.161 INFO:tasks.workunit.client.0.smithi081.stdout:0/583: dread d3/dc/d2b/d39/f8c [0,4194304] 0 2024-03-22T07:18:39.169 INFO:tasks.workunit.client.0.smithi081.stdout:4/763: sync 2024-03-22T07:18:39.170 INFO:tasks.workunit.client.0.smithi081.stdout:9/626: sync 2024-03-22T07:18:39.170 INFO:tasks.workunit.client.0.smithi081.stdout:9/627: dread - d2/d5/d9d/f91 zero size 2024-03-22T07:18:39.170 INFO:tasks.workunit.client.0.smithi081.stdout:4/764: symlink da/d18/d1b/d47/lfa 0 2024-03-22T07:18:39.171 INFO:tasks.workunit.client.0.smithi081.stdout:4/765: write da/df/d38/dd8/f9a [275410,128004] 0 2024-03-22T07:18:39.171 INFO:tasks.workunit.client.0.smithi081.stdout:4/766: chown da/d18/d1b/d47/lfa 346046180 1 2024-03-22T07:18:39.172 INFO:tasks.workunit.client.0.smithi081.stdout:9/628: mknod d2/d5/d36/cce 0 2024-03-22T07:18:39.172 INFO:tasks.workunit.client.0.smithi081.stdout:9/629: write d2/d10/d30/d57/f62 [3066962,40343] 0 2024-03-22T07:18:39.173 INFO:tasks.workunit.client.0.smithi081.stdout:4/767: rename da/d18/d1b/lae to da/d57/lfb 0 2024-03-22T07:18:39.182 INFO:tasks.workunit.client.0.smithi081.stdout:0/584: dwrite d3/dc/f28 [0,4194304] 0 2024-03-22T07:18:39.244 INFO:tasks.workunit.client.0.smithi081.stdout:9/630: unlink d2/d5d/d5a/f63 0 2024-03-22T07:18:39.244 INFO:tasks.workunit.client.0.smithi081.stdout:9/631: chown d2/d5/d65 1049331628 1 2024-03-22T07:18:39.246 INFO:tasks.workunit.client.0.smithi081.stdout:4/768: dread da/d18/d1b/d26/d3e/f6f [0,4194304] 0 2024-03-22T07:18:39.246 INFO:tasks.workunit.client.0.smithi081.stdout:4/769: write da/fb [3970721,73284] 0 2024-03-22T07:18:39.249 INFO:tasks.workunit.client.0.smithi081.stdout:9/632: rename d2/d5d/l6f to d2/d73/lcf 0 2024-03-22T07:18:39.250 INFO:tasks.workunit.client.0.smithi081.stdout:4/770: truncate da/df/d38/dd8/d4f/d7c/d90/fc5 3282535 0 2024-03-22T07:18:39.253 INFO:tasks.workunit.client.0.smithi081.stdout:0/585: mkdir d3/dc/d2b/d39/dc9 0 2024-03-22T07:18:39.253 INFO:tasks.workunit.client.0.smithi081.stdout:0/586: write d3/dc/d40/d69/d92/dc1/d91/fc5 [814801,75924] 0 2024-03-22T07:18:39.256 INFO:tasks.workunit.client.0.smithi081.stdout:4/771: unlink da/d18/d1b/fdd 0 2024-03-22T07:18:39.258 INFO:tasks.workunit.client.0.smithi081.stdout:4/772: symlink da/d18/d1b/d1e/d37/lfc 0 2024-03-22T07:18:39.258 INFO:tasks.workunit.client.0.smithi081.stdout:0/587: write d3/dc/d33/f97 [1448336,54252] 0 2024-03-22T07:18:39.260 INFO:tasks.workunit.client.0.smithi081.stdout:4/773: rename da/d18/d1b/d1e/d37/d1c to da/d18/d1b/d26/dce/dfd 0 2024-03-22T07:18:39.260 INFO:tasks.workunit.client.0.smithi081.stdout:4/774: write f3 [5321766,72581] 0 2024-03-22T07:18:39.260 INFO:tasks.workunit.client.0.smithi081.stdout:4/775: readlink da/df/l8f 0 2024-03-22T07:18:39.260 INFO:tasks.workunit.client.0.smithi081.stdout:4/776: dread - da/de6/db8/ff0 zero size 2024-03-22T07:18:39.265 INFO:tasks.workunit.client.0.smithi081.stdout:0/588: write d3/dc/d33/f5b [5485545,119004] 0 2024-03-22T07:18:39.265 INFO:tasks.workunit.client.0.smithi081.stdout:0/589: fdatasync d3/dc/d33/f7d 0 2024-03-22T07:18:39.265 INFO:tasks.workunit.client.0.smithi081.stdout:9/633: dwrite d2/f11 [0,4194304] 0 2024-03-22T07:18:39.266 INFO:tasks.workunit.client.0.smithi081.stdout:0/590: creat d3/dc/d2b/d4d/fca x:0 0 0 2024-03-22T07:18:39.269 INFO:tasks.workunit.client.0.smithi081.stdout:0/591: rename d3/d4e/c56 to d3/d32/ccb 0 2024-03-22T07:18:39.269 INFO:tasks.workunit.client.0.smithi081.stdout:0/592: mknod d3/dc/d40/d69/d92/dc1/d91/ccc 0 2024-03-22T07:18:39.270 INFO:tasks.workunit.client.0.smithi081.stdout:0/593: creat d3/dc/d40/d69/d92/dc1/fcd x:0 0 0 2024-03-22T07:18:39.270 INFO:tasks.workunit.client.0.smithi081.stdout:0/594: truncate d3/dc/d2b/fba 382461 0 2024-03-22T07:18:39.271 INFO:tasks.workunit.client.0.smithi081.stdout:4/777: write da/df/d38/dd8/d4f/f58 [7721590,79703] 0 2024-03-22T07:18:39.273 INFO:tasks.workunit.client.0.smithi081.stdout:4/778: symlink da/d18/d1b/d26/dce/dfd/d74/lfe 0 2024-03-22T07:18:39.273 INFO:tasks.workunit.client.0.smithi081.stdout:4/779: chown da/df/d38/dd8/d4a/ce5 11308 1 2024-03-22T07:18:39.273 INFO:tasks.workunit.client.0.smithi081.stdout:4/780: creat da/df/d38/dc9/fff x:0 0 0 2024-03-22T07:18:39.274 INFO:tasks.workunit.client.0.smithi081.stdout:9/634: dread d2/d5/d36/f8c [0,4194304] 0 2024-03-22T07:18:39.276 INFO:tasks.workunit.client.0.smithi081.stdout:4/781: rename da/de6/fa4 to da/f100 0 2024-03-22T07:18:39.276 INFO:tasks.workunit.client.0.smithi081.stdout:4/782: write da/df/d38/dd8/d4f/daf/ff9 [743798,32100] 0 2024-03-22T07:18:39.276 INFO:tasks.workunit.client.0.smithi081.stdout:0/595: getdents d3/d32 0 2024-03-22T07:18:39.277 INFO:tasks.workunit.client.0.smithi081.stdout:9/635: creat d2/d5d/d58/dc2/fd0 x:0 0 0 2024-03-22T07:18:39.277 INFO:tasks.workunit.client.0.smithi081.stdout:9/636: readlink d2/d10/d30/d57/d66/d75/d94/dae/lbf 0 2024-03-22T07:18:39.278 INFO:tasks.workunit.client.0.smithi081.stdout:4/783: mkdir da/df/d38/dd8/d4f/daf/d101 0 2024-03-22T07:18:39.278 INFO:tasks.workunit.client.0.smithi081.stdout:4/784: chown da/df/lb4 411 1 2024-03-22T07:18:39.279 INFO:tasks.workunit.client.0.smithi081.stdout:0/596: mknod d3/dc/dc6/cce 0 2024-03-22T07:18:39.279 INFO:tasks.workunit.client.0.smithi081.stdout:0/597: write d3/f44 [1713349,3842] 0 2024-03-22T07:18:39.282 INFO:tasks.workunit.client.0.smithi081.stdout:4/785: link da/d18/d6e/fa8 da/df/df4/f102 0 2024-03-22T07:18:39.282 INFO:tasks.workunit.client.0.smithi081.stdout:4/786: fdatasync da/df/d38/f54 0 2024-03-22T07:18:39.283 INFO:tasks.workunit.client.0.smithi081.stdout:0/598: mkdir d3/dc/d40/d69/d92/dcf 0 2024-03-22T07:18:39.284 INFO:tasks.workunit.client.0.smithi081.stdout:0/599: symlink d3/d9b/da4/ld0 0 2024-03-22T07:18:39.284 INFO:tasks.workunit.client.0.smithi081.stdout:0/600: chown d3/dc/d40/d69/d92/dc1/d91/d9d 5 1 2024-03-22T07:18:39.297 INFO:tasks.workunit.client.0.smithi081.stdout:4/787: write da/d18/d1b/d26/d3e/f6f [314129,95194] 0 2024-03-22T07:18:39.302 INFO:tasks.workunit.client.0.smithi081.stdout:9/637: dwrite d2/d2b/d49/d86/f9a [0,4194304] 0 2024-03-22T07:18:39.302 INFO:tasks.workunit.client.0.smithi081.stdout:9/638: fsync d2/d2b/f2d 0 2024-03-22T07:18:39.304 INFO:tasks.workunit.client.0.smithi081.stdout:0/601: dwrite d3/dc/d2b/d4d/fb2 [0,4194304] 0 2024-03-22T07:18:39.304 INFO:tasks.workunit.client.0.smithi081.stdout:0/602: fsync d3/dc/d2b/fa0 0 2024-03-22T07:18:39.307 INFO:tasks.workunit.client.0.smithi081.stdout:0/603: dread d3/dc/d33/f47 [0,4194304] 0 2024-03-22T07:18:39.312 INFO:tasks.workunit.client.0.smithi081.stdout:0/604: write d3/dc/d2b/d39/f3c [5173988,30502] 0 2024-03-22T07:18:39.314 INFO:tasks.workunit.client.0.smithi081.stdout:9/639: dread d2/d5/fd [0,4194304] 0 2024-03-22T07:18:39.316 INFO:tasks.workunit.client.0.smithi081.stdout:9/640: rename d2/d73/d85/d7a/f88 to d2/d73/fd1 0 2024-03-22T07:18:39.316 INFO:tasks.workunit.client.0.smithi081.stdout:9/641: chown d2/d2b/d49/f4f 978 1 2024-03-22T07:18:39.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:39 smithi081 ceph-mon[113965]: pgmap v215: 65 pgs: 65 active+clean; 1.1 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 29 MiB/s rd, 38 MiB/s wr, 752 op/s 2024-03-22T07:18:39.322 INFO:tasks.workunit.client.0.smithi081.stdout:9/642: write d2/d10/d30/d57/f9e [7537299,122263] 0 2024-03-22T07:18:39.322 INFO:tasks.workunit.client.0.smithi081.stdout:4/788: dwrite da/d18/d6e/fa8 [0,4194304] 0 2024-03-22T07:18:39.325 INFO:tasks.workunit.client.0.smithi081.stdout:4/789: symlink da/d18/ddc/l103 0 2024-03-22T07:18:39.334 INFO:tasks.workunit.client.0.smithi081.stdout:4/790: getdents da/d18/ddc 0 2024-03-22T07:18:39.334 INFO:tasks.workunit.client.0.smithi081.stdout:4/791: write da/d18/d1b/d26/d3e/f6f [1257280,113533] 0 2024-03-22T07:18:39.336 INFO:tasks.workunit.client.0.smithi081.stdout:4/792: getdents da/d57 0 2024-03-22T07:18:39.337 INFO:tasks.workunit.client.0.smithi081.stdout:0/605: dwrite d3/dc/d40/d69/d92/dc1/fc8 [0,4194304] 0 2024-03-22T07:18:39.338 INFO:tasks.workunit.client.0.smithi081.stdout:4/793: unlink da/df/d38/dd8/fb0 0 2024-03-22T07:18:39.338 INFO:tasks.workunit.client.0.smithi081.stdout:4/794: stat da/d18/d1b/d26/dce/dfd/d74/lfe 0 2024-03-22T07:18:39.342 INFO:tasks.workunit.client.0.smithi081.stdout:4/795: getdents da/d18/d1b/d34 0 2024-03-22T07:18:39.342 INFO:tasks.workunit.client.0.smithi081.stdout:4/796: chown da/d18/d1b/d26/fde 1027109779 1 2024-03-22T07:18:39.344 INFO:tasks.workunit.client.0.smithi081.stdout:0/606: mkdir d3/dc/d33/dd1 0 2024-03-22T07:18:39.344 INFO:tasks.workunit.client.0.smithi081.stdout:0/607: creat d3/d6a/fd2 x:0 0 0 2024-03-22T07:18:39.344 INFO:tasks.workunit.client.0.smithi081.stdout:0/608: stat d3/dc/d77 0 2024-03-22T07:18:39.345 INFO:tasks.workunit.client.0.smithi081.stdout:0/609: truncate d3/dc/d2b/d39/f75 86727 0 2024-03-22T07:18:39.347 INFO:tasks.workunit.client.0.smithi081.stdout:9/643: dwrite d2/d73/d85/f18 [0,4194304] 0 2024-03-22T07:18:39.347 INFO:tasks.workunit.client.0.smithi081.stdout:9/644: readlink d2/d10/l2c 0 2024-03-22T07:18:39.348 INFO:tasks.workunit.client.0.smithi081.stdout:9/645: mknod d2/d78/d93/cd2 0 2024-03-22T07:18:39.348 INFO:tasks.workunit.client.0.smithi081.stdout:9/646: chown d2/d10/d30/d57/dad/lb5 13152 1 2024-03-22T07:18:39.349 INFO:tasks.workunit.client.0.smithi081.stdout:9/647: chown d2/d73/d85/f4c 29 1 2024-03-22T07:18:39.349 INFO:tasks.workunit.client.0.smithi081.stdout:9/648: chown d2/d10/d30/c96 32513 1 2024-03-22T07:18:39.349 INFO:tasks.workunit.client.0.smithi081.stdout:9/649: creat d2/d73/fd3 x:0 0 0 2024-03-22T07:18:39.350 INFO:tasks.workunit.client.0.smithi081.stdout:9/650: creat d2/d10/d30/d57/d66/d75/dca/fd4 x:0 0 0 2024-03-22T07:18:39.350 INFO:tasks.workunit.client.0.smithi081.stdout:9/651: stat d2/d2b/c2a 0 2024-03-22T07:18:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:39 smithi149 ceph-mon[83447]: pgmap v215: 65 pgs: 65 active+clean; 1.1 GiB data, 3.5 GiB used, 533 GiB / 536 GiB avail; 29 MiB/s rd, 38 MiB/s wr, 752 op/s 2024-03-22T07:18:39.353 INFO:tasks.workunit.client.0.smithi081.stdout:9/652: mknod d2/d10/d30/cd5 0 2024-03-22T07:18:39.353 INFO:tasks.workunit.client.0.smithi081.stdout:9/653: write d2/d5d/d58/dc2/fd0 [354589,47751] 0 2024-03-22T07:18:39.353 INFO:tasks.workunit.client.0.smithi081.stdout:9/654: fdatasync d2/d5d/d5a/fcd 0 2024-03-22T07:18:39.353 INFO:tasks.workunit.client.0.smithi081.stdout:9/655: write d2/d73/d85/f4a [78962,92884] 0 2024-03-22T07:18:39.353 INFO:tasks.workunit.client.0.smithi081.stdout:9/656: truncate d2/d10/d30/d57/fb8 341361 0 2024-03-22T07:18:39.357 INFO:tasks.workunit.client.0.smithi081.stdout:0/610: write d3/dc/f2a [6931238,17268] 0 2024-03-22T07:18:39.357 INFO:tasks.workunit.client.0.smithi081.stdout:0/611: creat d3/dc/fd3 x:0 0 0 2024-03-22T07:18:39.360 INFO:tasks.workunit.client.0.smithi081.stdout:4/797: dwrite da/d18/d1b/d26/d32/f91 [0,4194304] 0 2024-03-22T07:18:39.361 INFO:tasks.workunit.client.0.smithi081.stdout:0/612: dread d3/d6a/fc2 [0,4194304] 0 2024-03-22T07:18:39.362 INFO:tasks.workunit.client.0.smithi081.stdout:0/613: rename d3/d6a/l82 to d3/d9b/ld4 0 2024-03-22T07:18:39.363 INFO:tasks.workunit.client.0.smithi081.stdout:0/614: mkdir d3/d6a/dd5 0 2024-03-22T07:18:39.366 INFO:tasks.workunit.client.0.smithi081.stdout:0/615: write d3/dc/d2b/d39/f3c [4298214,101333] 0 2024-03-22T07:18:39.368 INFO:tasks.workunit.client.0.smithi081.stdout:4/798: dread da/d18/d1b/d34/f4c [0,4194304] 0 2024-03-22T07:18:39.368 INFO:tasks.workunit.client.0.smithi081.stdout:4/799: dread - da/df/df4/ff2 zero size 2024-03-22T07:18:39.368 INFO:tasks.workunit.client.0.smithi081.stdout:0/616: mkdir d3/dc/d40/d69/d92/dcf/dd6 0 2024-03-22T07:18:39.368 INFO:tasks.workunit.client.0.smithi081.stdout:4/800: mknod da/d57/c104 0 2024-03-22T07:18:39.369 INFO:tasks.workunit.client.0.smithi081.stdout:4/801: truncate da/d18/d1b/d26/dce/dfd/d74/fbf 1836660 0 2024-03-22T07:18:39.370 INFO:tasks.workunit.client.0.smithi081.stdout:4/802: truncate da/d18/d1b/d26/dce/dfd/d74/fd3 522662 0 2024-03-22T07:18:39.371 INFO:tasks.workunit.client.0.smithi081.stdout:0/617: rename d3/d9b/la7 to d3/d4e/ld7 0 2024-03-22T07:18:39.371 INFO:tasks.workunit.client.0.smithi081.stdout:4/803: readlink da/d18/d1b/d26/d3e/l72 0 2024-03-22T07:18:39.371 INFO:tasks.workunit.client.0.smithi081.stdout:4/804: fdatasync da/d18/d1b/d26/dce/fd6 0 2024-03-22T07:18:39.375 INFO:tasks.workunit.client.0.smithi081.stdout:0/618: mkdir d3/dc/d40/d69/dd8 0 2024-03-22T07:18:39.376 INFO:tasks.workunit.client.0.smithi081.stdout:9/657: dwrite d2/f5f [0,4194304] 0 2024-03-22T07:18:39.376 INFO:tasks.workunit.client.0.smithi081.stdout:4/805: mknod da/d18/d1b/d26/c105 0 2024-03-22T07:18:39.377 INFO:tasks.workunit.client.0.smithi081.stdout:0/619: link d3/dc/f43 d3/dc/dc6/fd9 0 2024-03-22T07:18:39.385 INFO:tasks.workunit.client.0.smithi081.stdout:4/806: read da/d18/d1b/d26/fcb [664292,24157] 0 2024-03-22T07:18:39.385 INFO:tasks.workunit.client.0.smithi081.stdout:4/807: write da/d18/d1b/d26/dce/fd6 [993408,6534] 0 2024-03-22T07:18:39.387 INFO:tasks.workunit.client.0.smithi081.stdout:4/808: read f3 [4278199,106105] 0 2024-03-22T07:18:39.388 INFO:tasks.workunit.client.0.smithi081.stdout:9/658: truncate d2/f89 2736899 0 2024-03-22T07:18:39.388 INFO:tasks.workunit.client.0.smithi081.stdout:9/659: truncate d2/d2b/d49/d86/fb7 856352 0 2024-03-22T07:18:39.393 INFO:tasks.workunit.client.0.smithi081.stdout:4/809: rename da/d18/ddc/fea to da/df/d38/dc9/f106 0 2024-03-22T07:18:39.395 INFO:tasks.workunit.client.0.smithi081.stdout:4/810: rename da/df/d38/dd8/f99 to da/de6/f107 0 2024-03-22T07:18:39.396 INFO:tasks.workunit.client.0.smithi081.stdout:9/660: getdents d2/d10/d30/d57/d66/d75/dca 0 2024-03-22T07:18:39.400 INFO:tasks.workunit.client.0.smithi081.stdout:3/379: sync 2024-03-22T07:18:39.400 INFO:tasks.workunit.client.0.smithi081.stdout:3/380: write db/d38/f6c [17564,33231] 0 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:1/449: sync 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:5/709: sync 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:6/578: sync 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:6/579: dread - d3/d67/db2/fb8 zero size 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:6/580: chown d3/d67/db2/d40/db9/d63/d26/d3f/fa1 50 1 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:7/670: sync 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:2/504: sync 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:8/594: sync 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:4/811: dread da/d18/d1b/d26/f73 [0,4194304] 0 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:1/450: write d6/d18/d1f/d24/d48/d6e/f75 [643555,24667] 0 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:0/620: dwrite d3/dc/d40/f4a [4194304,4194304] 0 2024-03-22T07:18:39.403 INFO:tasks.workunit.client.0.smithi081.stdout:3/381: getdents db/de 0 2024-03-22T07:18:39.404 INFO:tasks.workunit.client.0.smithi081.stdout:1/451: mknod d6/d18/d1f/d24/d48/d6e/c97 0 2024-03-22T07:18:39.405 INFO:tasks.workunit.client.0.smithi081.stdout:6/581: truncate d3/da/de/f27 1668563 0 2024-03-22T07:18:39.406 INFO:tasks.workunit.client.0.smithi081.stdout:2/505: rename da/db/d39/d47/d3b/d41/d50/d6a/d70/c48 to da/d11/d21/d71/ca6 0 2024-03-22T07:18:39.408 INFO:tasks.workunit.client.0.smithi081.stdout:8/595: chown d2/d3/d19/d1e/d13/l36 1961 1 2024-03-22T07:18:39.410 INFO:tasks.workunit.client.0.smithi081.stdout:0/621: creat d3/d6a/dd5/fda x:0 0 0 2024-03-22T07:18:39.411 INFO:tasks.workunit.client.0.smithi081.stdout:5/710: mkdir d4/df/de6 0 2024-03-22T07:18:39.413 INFO:tasks.workunit.client.0.smithi081.stdout:7/671: mkdir d9/de4 0 2024-03-22T07:18:39.414 INFO:tasks.workunit.client.0.smithi081.stdout:8/596: rmdir d2/d3/d19/d1e/d63/da0 39 2024-03-22T07:18:39.414 INFO:tasks.workunit.client.0.smithi081.stdout:8/597: write d2/d3/d23/fa2 [553484,99903] 0 2024-03-22T07:18:39.414 INFO:tasks.workunit.client.0.smithi081.stdout:4/812: getdents da/df/d38 0 2024-03-22T07:18:39.416 INFO:tasks.workunit.client.0.smithi081.stdout:5/711: stat d4/df/d22/d49/l5e 0 2024-03-22T07:18:39.417 INFO:tasks.workunit.client.0.smithi081.stdout:5/712: dread d4/d10/d20/d4f/f78 [0,4194304] 0 2024-03-22T07:18:39.417 INFO:tasks.workunit.client.0.smithi081.stdout:5/713: stat d4/d10/f8d 0 2024-03-22T07:18:39.418 INFO:tasks.workunit.client.0.smithi081.stdout:3/382: write db/de/d4b/d6a/d70/f40 [3921534,106984] 0 2024-03-22T07:18:39.419 INFO:tasks.workunit.client.0.smithi081.stdout:2/506: rename da/db/d39/d47/d3b/f55 to da/db/d39/d46/fa7 0 2024-03-22T07:18:39.419 INFO:tasks.workunit.client.0.smithi081.stdout:8/598: creat d2/d3/d28/fcc x:0 0 0 2024-03-22T07:18:39.419 INFO:tasks.workunit.client.0.smithi081.stdout:8/599: stat d2/d3/d19/d1e/c37 0 2024-03-22T07:18:39.420 INFO:tasks.workunit.client.0.smithi081.stdout:4/813: creat da/d3d/f108 x:0 0 0 2024-03-22T07:18:39.420 INFO:tasks.workunit.client.0.smithi081.stdout:4/814: dread - da/df/f65 zero size 2024-03-22T07:18:39.422 INFO:tasks.workunit.client.0.smithi081.stdout:7/672: truncate d9/d10/d16/d1e/f32 754569 0 2024-03-22T07:18:39.423 INFO:tasks.workunit.client.0.smithi081.stdout:3/383: unlink db/f78 0 2024-03-22T07:18:39.424 INFO:tasks.workunit.client.0.smithi081.stdout:2/507: unlink da/db/d6d/c75 0 2024-03-22T07:18:39.425 INFO:tasks.workunit.client.0.smithi081.stdout:9/661: dwrite d2/d2b/d49/d86/fb7 [0,4194304] 0 2024-03-22T07:18:39.425 INFO:tasks.workunit.client.0.smithi081.stdout:1/452: dwrite d6/d18/d1f/d24/d48/d5c/f73 [0,4194304] 0 2024-03-22T07:18:39.426 INFO:tasks.workunit.client.0.smithi081.stdout:1/453: fdatasync d6/d18/f21 0 2024-03-22T07:18:39.426 INFO:tasks.workunit.client.0.smithi081.stdout:8/600: unlink d2/d3/d19/d1e/d13/fa3 0 2024-03-22T07:18:39.427 INFO:tasks.workunit.client.0.smithi081.stdout:5/714: symlink d4/d75/le7 0 2024-03-22T07:18:39.427 INFO:tasks.workunit.client.0.smithi081.stdout:5/715: fdatasync d4/d75/fbd 0 2024-03-22T07:18:39.431 INFO:tasks.workunit.client.0.smithi081.stdout:4/815: dread da/df/d38/f46 [0,4194304] 0 2024-03-22T07:18:39.431 INFO:tasks.workunit.client.0.smithi081.stdout:4/816: write da/df/f5a [3054849,78038] 0 2024-03-22T07:18:39.431 INFO:tasks.workunit.client.0.smithi081.stdout:4/817: dread - da/d18/d1b/d26/fde zero size 2024-03-22T07:18:39.431 INFO:tasks.workunit.client.0.smithi081.stdout:6/582: write d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f97 [92090,82324] 0 2024-03-22T07:18:39.435 INFO:tasks.workunit.client.0.smithi081.stdout:9/662: rename d2/d73/d85/fb6 to d2/d10/d30/d57/d66/d75/d94/fd6 0 2024-03-22T07:18:39.435 INFO:tasks.workunit.client.0.smithi081.stdout:9/663: dread - d2/d5d/d5a/fa5 zero size 2024-03-22T07:18:39.438 INFO:tasks.workunit.client.0.smithi081.stdout:5/716: link d4/d10/d20/d4f/l61 d4/df/d73/d8c/dc0/dc9/le8 0 2024-03-22T07:18:39.440 INFO:tasks.workunit.client.0.smithi081.stdout:3/384: symlink db/d23/l82 0 2024-03-22T07:18:39.442 INFO:tasks.workunit.client.0.smithi081.stdout:2/508: mkdir da/d11/da8 0 2024-03-22T07:18:39.444 INFO:tasks.workunit.client.0.smithi081.stdout:4/818: creat da/d18/d1b/d26/dce/dfd/f109 x:0 0 0 2024-03-22T07:18:39.447 INFO:tasks.workunit.client.0.smithi081.stdout:9/664: creat d2/d2b/fd7 x:0 0 0 2024-03-22T07:18:39.450 INFO:tasks.workunit.client.0.smithi081.stdout:3/385: creat db/de/d4b/d6a/d70/d3a/f83 x:0 0 0 2024-03-22T07:18:39.452 INFO:tasks.workunit.client.0.smithi081.stdout:4/819: rename da/d18/ddc/l103 to da/df/d38/dd8/d4f/daf/ded/l10a 0 2024-03-22T07:18:39.452 INFO:tasks.workunit.client.0.smithi081.stdout:4/820: chown da/df/d38/f54 2867689 1 2024-03-22T07:18:39.453 INFO:tasks.workunit.client.0.smithi081.stdout:5/717: mknod d4/df/d22/d4d/ce9 0 2024-03-22T07:18:39.454 INFO:tasks.workunit.client.0.smithi081.stdout:6/583: mkdir d3/d67/db2/d40/db9/d63/d5e/dc0 0 2024-03-22T07:18:39.455 INFO:tasks.workunit.client.0.smithi081.stdout:7/673: link d9/d10/d2a/d73/l88 d9/dc/le5 0 2024-03-22T07:18:39.457 INFO:tasks.workunit.client.0.smithi081.stdout:3/386: creat db/de/d4b/f84 x:0 0 0 2024-03-22T07:18:39.457 INFO:tasks.workunit.client.0.smithi081.stdout:8/601: dwrite d2/d3/d19/d1e/d13/d22/f51 [0,4194304] 0 2024-03-22T07:18:39.459 INFO:tasks.workunit.client.0.smithi081.stdout:0/622: dwrite d3/dc/d2b/f31 [0,4194304] 0 2024-03-22T07:18:39.459 INFO:tasks.workunit.client.0.smithi081.stdout:6/584: readlink d3/da/de/l49 0 2024-03-22T07:18:39.462 INFO:tasks.workunit.client.0.smithi081.stdout:8/602: link f0 d2/d3/d28/d2b/d61/dc6/fcd 0 2024-03-22T07:18:39.462 INFO:tasks.workunit.client.0.smithi081.stdout:8/603: write d2/f15 [1192012,51053] 0 2024-03-22T07:18:39.463 INFO:tasks.workunit.client.0.smithi081.stdout:4/821: mknod da/d18/d1b/d26/dce/dfd/c10b 0 2024-03-22T07:18:39.464 INFO:tasks.workunit.client.0.smithi081.stdout:0/623: getdents d3/d32 0 2024-03-22T07:18:39.465 INFO:tasks.workunit.client.0.smithi081.stdout:0/624: fdatasync d3/dc/d40/f9a 0 2024-03-22T07:18:39.465 INFO:tasks.workunit.client.0.smithi081.stdout:0/625: write d3/dc/d40/d69/d92/dc1/d74/fb7 [672695,94384] 0 2024-03-22T07:18:39.466 INFO:tasks.workunit.client.0.smithi081.stdout:6/585: creat d3/d67/db2/d40/db9/d63/d26/d52/fc1 x:0 0 0 2024-03-22T07:18:39.466 INFO:tasks.workunit.client.0.smithi081.stdout:6/586: fsync d3/d67/db2/d40/db9/d63/d26/f71 0 2024-03-22T07:18:39.466 INFO:tasks.workunit.client.0.smithi081.stdout:6/587: chown d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f6e 59 1 2024-03-22T07:18:39.466 INFO:tasks.workunit.client.0.smithi081.stdout:3/387: getdents db/d38 0 2024-03-22T07:18:39.466 INFO:tasks.workunit.client.0.smithi081.stdout:3/388: truncate db/de/d4b/d6a/d70/d3a/f46 893215 0 2024-03-22T07:18:39.468 INFO:tasks.workunit.client.0.smithi081.stdout:8/604: creat d2/fce x:0 0 0 2024-03-22T07:18:39.469 INFO:tasks.workunit.client.0.smithi081.stdout:7/674: dread d9/dc/f1f [0,4194304] 0 2024-03-22T07:18:39.469 INFO:tasks.workunit.client.0.smithi081.stdout:0/626: mknod d3/d6a/cdb 0 2024-03-22T07:18:39.472 INFO:tasks.workunit.client.0.smithi081.stdout:8/605: mknod d2/d3/d19/d1e/d13/d66/d70/ccf 0 2024-03-22T07:18:39.472 INFO:tasks.workunit.client.0.smithi081.stdout:8/606: fsync d2/d3/d19/d1e/f9d 0 2024-03-22T07:18:39.473 INFO:tasks.workunit.client.0.smithi081.stdout:2/509: dwrite da/d11/d21/f4e [0,4194304] 0 2024-03-22T07:18:39.473 INFO:tasks.workunit.client.0.smithi081.stdout:2/510: fdatasync da/f28 0 2024-03-22T07:18:39.474 INFO:tasks.workunit.client.0.smithi081.stdout:6/588: mknod d3/da/de/cc2 0 2024-03-22T07:18:39.475 INFO:tasks.workunit.client.0.smithi081.stdout:8/607: readlink d2/d3/d28/d2b/l4f 0 2024-03-22T07:18:39.476 INFO:tasks.workunit.client.0.smithi081.stdout:8/608: chown d2/d3/d19/d1e/d13/f47 9621 1 2024-03-22T07:18:39.478 INFO:tasks.workunit.client.0.smithi081.stdout:2/511: getdents da/db/d39/d47/d6e 0 2024-03-22T07:18:39.479 INFO:tasks.workunit.client.0.smithi081.stdout:6/589: truncate d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f78 2423279 0 2024-03-22T07:18:39.481 INFO:tasks.workunit.client.0.smithi081.stdout:4/822: read da/d18/d6e/fa8 [156115,130921] 0 2024-03-22T07:18:39.482 INFO:tasks.workunit.client.0.smithi081.stdout:8/609: rename d2/d3/d28/d2b/f2e to d2/d3/d19/d1e/d13/d66/d70/fd0 0 2024-03-22T07:18:39.483 INFO:tasks.workunit.client.0.smithi081.stdout:6/590: rmdir d3/d67/db2/d40/db9/d63/d26/d3f/d5a 39 2024-03-22T07:18:39.483 INFO:tasks.workunit.client.0.smithi081.stdout:8/610: chown d2/d3/d19/d1e/d13/d66/d70/fd0 0 1 2024-03-22T07:18:39.483 INFO:tasks.workunit.client.0.smithi081.stdout:8/611: chown d2/d3/d19/d1e/d13/d66/d70/d86/l8d 197615765 1 2024-03-22T07:18:39.483 INFO:tasks.workunit.client.0.smithi081.stdout:8/612: readlink d2/d3/d19/d1e/d13/d66/d70/l75 0 2024-03-22T07:18:39.483 INFO:tasks.workunit.client.0.smithi081.stdout:4/823: mknod da/d18/d1b/d26/dce/c10c 0 2024-03-22T07:18:39.484 INFO:tasks.workunit.client.0.smithi081.stdout:4/824: truncate da/d18/d6e/f9e 277796 0 2024-03-22T07:18:39.485 INFO:tasks.workunit.client.0.smithi081.stdout:6/591: mkdir d3/da/dc3 0 2024-03-22T07:18:39.486 INFO:tasks.workunit.client.0.smithi081.stdout:4/825: mkdir da/df/d38/dd8/d4f/d7c/d90/d10d 0 2024-03-22T07:18:39.488 INFO:tasks.workunit.client.0.smithi081.stdout:8/613: dread f1 [0,4194304] 0 2024-03-22T07:18:39.490 INFO:tasks.workunit.client.0.smithi081.stdout:4/826: mknod da/df/d38/dd8/d4f/d7c/d90/d10d/c10e 0 2024-03-22T07:18:39.491 INFO:tasks.workunit.client.0.smithi081.stdout:2/512: write da/db/d39/d47/d3b/d41/d50/d6a/d70/f40 [3817465,3450] 0 2024-03-22T07:18:39.491 INFO:tasks.workunit.client.0.smithi081.stdout:2/513: readlink da/db/d39/d47/d3b/d41/d50/d6a/d70/l84 0 2024-03-22T07:18:39.493 INFO:tasks.workunit.client.0.smithi081.stdout:8/614: creat d2/d3/d19/d1e/d13/d22/d2f/fd1 x:0 0 0 2024-03-22T07:18:39.493 INFO:tasks.workunit.client.0.smithi081.stdout:4/827: symlink da/d57/l10f 0 2024-03-22T07:18:39.502 INFO:tasks.workunit.client.0.smithi081.stdout:9/665: dwrite d2/d78/d93/f9f [0,4194304] 0 2024-03-22T07:18:39.504 INFO:tasks.workunit.client.0.smithi081.stdout:5/718: dwrite d4/d10/d20/db2/d2a/fde [0,4194304] 0 2024-03-22T07:18:39.504 INFO:tasks.workunit.client.0.smithi081.stdout:2/514: dread da/d11/f1f [0,4194304] 0 2024-03-22T07:18:39.508 INFO:tasks.workunit.client.0.smithi081.stdout:9/666: symlink d2/d10/d30/d57/d66/d75/d94/dae/dc1/ld8 0 2024-03-22T07:18:39.508 INFO:tasks.workunit.client.0.smithi081.stdout:9/667: chown d2/d73/lcf 36 1 2024-03-22T07:18:39.511 INFO:tasks.workunit.client.0.smithi081.stdout:1/454: dwrite d6/d18/d1f/f37 [4194304,4194304] 0 2024-03-22T07:18:39.511 INFO:tasks.workunit.client.0.smithi081.stdout:1/455: read d6/d18/d1f/d24/f39 [391935,31670] 0 2024-03-22T07:18:39.511 INFO:tasks.workunit.client.0.smithi081.stdout:1/456: truncate d6/d18/d41/d49/d56/f3e 918046 0 2024-03-22T07:18:39.511 INFO:tasks.workunit.client.0.smithi081.stdout:1/457: chown d6/d18/d41/d49/d56/d30/f34 2 1 2024-03-22T07:18:39.511 INFO:tasks.workunit.client.0.smithi081.stdout:3/389: dwrite db/f4f [0,4194304] 0 2024-03-22T07:18:39.511 INFO:tasks.workunit.client.0.smithi081.stdout:7/675: dwrite d9/d10/d16/f74 [0,4194304] 0 2024-03-22T07:18:39.511 INFO:tasks.workunit.client.0.smithi081.stdout:0/627: dwrite d3/dc/d2b/fba [0,4194304] 0 2024-03-22T07:18:39.512 INFO:tasks.workunit.client.0.smithi081.stdout:1/458: stat d6/d18/d1f 0 2024-03-22T07:18:39.512 INFO:tasks.workunit.client.0.smithi081.stdout:1/459: write f1 [2577925,19156] 0 2024-03-22T07:18:39.515 INFO:tasks.workunit.client.0.smithi081.stdout:3/390: truncate db/f13 3114736 0 2024-03-22T07:18:39.516 INFO:tasks.workunit.client.0.smithi081.stdout:3/391: chown db/de/f68 1901373848 1 2024-03-22T07:18:39.516 INFO:tasks.workunit.client.0.smithi081.stdout:3/392: fsync db/d38/f47 0 2024-03-22T07:18:39.516 INFO:tasks.workunit.client.0.smithi081.stdout:9/668: read d2/f89 [1381407,49723] 0 2024-03-22T07:18:39.522 INFO:tasks.workunit.client.0.smithi081.stdout:5/719: write d4/d10/d20/db2/f28 [3964578,68031] 0 2024-03-22T07:18:39.525 INFO:tasks.workunit.client.0.smithi081.stdout:1/460: creat d6/d18/d41/d49/d56/d30/d93/f98 x:0 0 0 2024-03-22T07:18:39.526 INFO:tasks.workunit.client.0.smithi081.stdout:7/676: dread d9/d10/d9e/fce [0,4194304] 0 2024-03-22T07:18:39.530 INFO:tasks.workunit.client.0.smithi081.stdout:5/720: mkdir d4/d10/d20/dd2/dea 0 2024-03-22T07:18:39.530 INFO:tasks.workunit.client.0.smithi081.stdout:4/828: rmdir da/d18/d1b/d26/dce 39 2024-03-22T07:18:39.532 INFO:tasks.workunit.client.0.smithi081.stdout:9/669: dread d2/d73/d85/fb1 [0,4194304] 0 2024-03-22T07:18:39.532 INFO:tasks.workunit.client.0.smithi081.stdout:9/670: dread - d2/d5d/d5a/fcd zero size 2024-03-22T07:18:39.534 INFO:tasks.workunit.client.0.smithi081.stdout:1/461: rename d6/d18/d1f/d24/c63 to d6/d18/c99 0 2024-03-22T07:18:39.534 INFO:tasks.workunit.client.0.smithi081.stdout:3/393: symlink db/de/d4b/d6a/d70/l85 0 2024-03-22T07:18:39.535 INFO:tasks.workunit.client.0.smithi081.stdout:6/592: write d3/da/de/f11 [4862284,122228] 0 2024-03-22T07:18:39.536 INFO:tasks.workunit.client.0.smithi081.stdout:4/829: symlink da/d3d/def/l110 0 2024-03-22T07:18:39.537 INFO:tasks.workunit.client.0.smithi081.stdout:6/593: write d3/da/f57 [468953,104735] 0 2024-03-22T07:18:39.537 INFO:tasks.workunit.client.0.smithi081.stdout:6/594: write d3/da/de/f11 [5095182,115410] 0 2024-03-22T07:18:39.541 INFO:tasks.workunit.client.0.smithi081.stdout:9/671: mkdir d2/d73/d74/dd9 0 2024-03-22T07:18:39.542 INFO:tasks.workunit.client.0.smithi081.stdout:7/677: mkdir d9/d10/d4b/de6 0 2024-03-22T07:18:39.542 INFO:tasks.workunit.client.0.smithi081.stdout:1/462: dread d6/d18/d41/d49/d56/d30/f83 [0,4194304] 0 2024-03-22T07:18:39.545 INFO:tasks.workunit.client.0.smithi081.stdout:4/830: creat da/d18/f111 x:0 0 0 2024-03-22T07:18:39.547 INFO:tasks.workunit.client.0.smithi081.stdout:9/672: mkdir d2/d73/d85/d7a/d97/dda 0 2024-03-22T07:18:39.547 INFO:tasks.workunit.client.0.smithi081.stdout:9/673: write d2/d2b/fd7 [287426,13139] 0 2024-03-22T07:18:39.547 INFO:tasks.workunit.client.0.smithi081.stdout:9/674: write d2/d2b/f81 [967333,23166] 0 2024-03-22T07:18:39.548 INFO:tasks.workunit.client.0.smithi081.stdout:7/678: creat d9/d10/d53/d81/d59/d89/dd4/de0/fe7 x:0 0 0 2024-03-22T07:18:39.549 INFO:tasks.workunit.client.0.smithi081.stdout:6/595: creat d3/d67/db2/d40/db9/d63/d5e/fc4 x:0 0 0 2024-03-22T07:18:39.550 INFO:tasks.workunit.client.0.smithi081.stdout:1/463: creat d6/d18/d41/d49/d56/d30/f9a x:0 0 0 2024-03-22T07:18:39.550 INFO:tasks.workunit.client.0.smithi081.stdout:1/464: write d6/d18/d41/d49/d56/f3e [844703,97410] 0 2024-03-22T07:18:39.551 INFO:tasks.workunit.client.0.smithi081.stdout:3/394: mknod db/de/d4b/d6a/d70/c86 0 2024-03-22T07:18:39.551 INFO:tasks.workunit.client.0.smithi081.stdout:4/831: truncate da/d18/d6e/fa8 241615 0 2024-03-22T07:18:39.552 INFO:tasks.workunit.client.0.smithi081.stdout:9/675: creat d2/d73/d74/dd9/fdb x:0 0 0 2024-03-22T07:18:39.552 INFO:tasks.workunit.client.0.smithi081.stdout:9/676: write d2/d78/d93/fc6 [210446,63905] 0 2024-03-22T07:18:39.552 INFO:tasks.workunit.client.0.smithi081.stdout:9/677: dread - d2/d5d/d58/f90 zero size 2024-03-22T07:18:39.553 INFO:tasks.workunit.client.0.smithi081.stdout:7/679: creat d9/d10/d16/db6/dd9/fe8 x:0 0 0 2024-03-22T07:18:39.555 INFO:tasks.workunit.client.0.smithi081.stdout:3/395: mknod db/de/c87 0 2024-03-22T07:18:39.557 INFO:tasks.workunit.client.0.smithi081.stdout:1/465: rename d6/d18/d1f/c33 to d6/d18/d1f/d24/c9b 0 2024-03-22T07:18:39.557 INFO:tasks.workunit.client.0.smithi081.stdout:1/466: fsync d6/f67 0 2024-03-22T07:18:39.558 INFO:tasks.workunit.client.0.smithi081.stdout:9/678: rename d2/d2b/d49/cc8 to d2/d10/d30/d57/dad/cdc 0 2024-03-22T07:18:39.558 INFO:tasks.workunit.client.0.smithi081.stdout:9/679: chown d2/d5/f3f 9 1 2024-03-22T07:18:39.558 INFO:tasks.workunit.client.0.smithi081.stdout:9/680: write d2/d10/d30/f45 [725027,90370] 0 2024-03-22T07:18:39.558 INFO:tasks.workunit.client.0.smithi081.stdout:9/681: stat d2/d2b/l8d 0 2024-03-22T07:18:39.558 INFO:tasks.workunit.client.0.smithi081.stdout:9/682: fsync d2/f1e 0 2024-03-22T07:18:39.602 INFO:tasks.workunit.client.0.smithi081.stdout:2/515: dwrite da/db/d39/d47/d3b/d64/f8f [0,4194304] 0 2024-03-22T07:18:39.605 INFO:tasks.workunit.client.0.smithi081.stdout:3/396: rename db/f12 to db/de/d4b/f88 0 2024-03-22T07:18:39.605 INFO:tasks.workunit.client.0.smithi081.stdout:3/397: truncate db/d23/f61 814841 0 2024-03-22T07:18:39.607 INFO:tasks.workunit.client.0.smithi081.stdout:9/683: mkdir d2/d73/d85/ddd 0 2024-03-22T07:18:39.607 INFO:tasks.workunit.client.0.smithi081.stdout:9/684: creat d2/d78/d93/fde x:0 0 0 2024-03-22T07:18:39.607 INFO:tasks.workunit.client.0.smithi081.stdout:9/685: creat d2/d78/d93/fdf x:0 0 0 2024-03-22T07:18:39.607 INFO:tasks.workunit.client.0.smithi081.stdout:9/686: creat d2/d73/d74/fe0 x:0 0 0 2024-03-22T07:18:39.611 INFO:tasks.workunit.client.0.smithi081.stdout:2/516: creat da/fa9 x:0 0 0 2024-03-22T07:18:39.611 INFO:tasks.workunit.client.0.smithi081.stdout:2/517: fdatasync da/db/d39/d47/f43 0 2024-03-22T07:18:39.611 INFO:tasks.workunit.client.0.smithi081.stdout:2/518: chown da/db/d39/d46/d42/d67 322 1 2024-03-22T07:18:39.611 INFO:tasks.workunit.client.0.smithi081.stdout:3/398: rmdir db/de/d4b/d6a/d70 39 2024-03-22T07:18:39.611 INFO:tasks.workunit.client.0.smithi081.stdout:3/399: creat db/de/d4b/d6a/f89 x:0 0 0 2024-03-22T07:18:39.612 INFO:tasks.workunit.client.0.smithi081.stdout:3/400: chown db/de/d4b/l72 13 1 2024-03-22T07:18:39.616 INFO:tasks.workunit.client.0.smithi081.stdout:3/401: creat db/de/d4b/d6a/d70/d3a/f8a x:0 0 0 2024-03-22T07:18:39.616 INFO:tasks.workunit.client.0.smithi081.stdout:3/402: dread - db/de/d4b/d6a/d70/f55 zero size 2024-03-22T07:18:39.618 INFO:tasks.workunit.client.0.smithi081.stdout:2/519: mknod da/db/caa 0 2024-03-22T07:18:39.623 INFO:tasks.workunit.client.0.smithi081.stdout:9/687: read d2/f8 [1154497,12926] 0 2024-03-22T07:18:39.623 INFO:tasks.workunit.client.0.smithi081.stdout:9/688: write d2/d2b/f70 [517163,107689] 0 2024-03-22T07:18:39.623 INFO:tasks.workunit.client.0.smithi081.stdout:9/689: dread - d2/d73/d74/fa4 zero size 2024-03-22T07:18:39.623 INFO:tasks.workunit.client.0.smithi081.stdout:3/403: truncate db/fc 41204 0 2024-03-22T07:18:39.623 INFO:tasks.workunit.client.0.smithi081.stdout:4/832: truncate da/d18/d1b/d26/dce/fd6 982106 0 2024-03-22T07:18:39.623 INFO:tasks.workunit.client.0.smithi081.stdout:4/833: fsync da/d18/d1b/d1e/f62 0 2024-03-22T07:18:39.623 INFO:tasks.workunit.client.0.smithi081.stdout:6/596: dwrite d3/f9f [0,4194304] 0 2024-03-22T07:18:39.623 INFO:tasks.workunit.client.0.smithi081.stdout:2/520: rmdir da/db/d39/d45 39 2024-03-22T07:18:39.624 INFO:tasks.workunit.client.0.smithi081.stdout:2/521: fsync da/db/d39/d46/fa7 0 2024-03-22T07:18:39.624 INFO:tasks.workunit.client.0.smithi081.stdout:2/522: chown da/db/d39/d47/d3b/d41/d50 864390 1 2024-03-22T07:18:39.624 INFO:tasks.workunit.client.0.smithi081.stdout:8/615: dwrite d2/d3/d28/f4a [0,4194304] 0 2024-03-22T07:18:39.625 INFO:tasks.workunit.client.0.smithi081.stdout:7/680: dwrite d9/d10/d53/d81/d59/d89/dd4/de0/fe7 [0,4194304] 0 2024-03-22T07:18:39.629 INFO:tasks.workunit.client.0.smithi081.stdout:7/681: creat d9/d10/d16/d1e/fe9 x:0 0 0 2024-03-22T07:18:39.630 INFO:tasks.workunit.client.0.smithi081.stdout:9/690: dread d2/d73/d85/f2e [4194304,4194304] 0 2024-03-22T07:18:39.630 INFO:tasks.workunit.client.0.smithi081.stdout:9/691: stat d2/d2b/f2d 0 2024-03-22T07:18:39.631 INFO:tasks.workunit.client.0.smithi081.stdout:8/616: mknod d2/d3/d19/d1e/cd2 0 2024-03-22T07:18:39.636 INFO:tasks.workunit.client.0.smithi081.stdout:8/617: symlink d2/d3/d19/d1e/ld3 0 2024-03-22T07:18:39.640 INFO:tasks.workunit.client.0.smithi081.stdout:8/618: rename d2/d3/d19/d1e/f9a to d2/d3/d19/d1e/d13/dba/fd4 0 2024-03-22T07:18:39.644 INFO:tasks.workunit.client.0.smithi081.stdout:2/523: truncate da/db/d39/d47/d3b/d64/f8f 1294981 0 2024-03-22T07:18:39.645 INFO:tasks.workunit.client.0.smithi081.stdout:2/524: chown da/l1d 1 1 2024-03-22T07:18:39.646 INFO:tasks.workunit.client.0.smithi081.stdout:2/525: read da/d56/f58 [669633,59803] 0 2024-03-22T07:18:39.649 INFO:tasks.workunit.client.0.smithi081.stdout:2/526: mkdir da/db/d39/d47/d3b/d41/dab 0 2024-03-22T07:18:39.649 INFO:tasks.workunit.client.0.smithi081.stdout:2/527: chown da/fa9 151 1 2024-03-22T07:18:39.649 INFO:tasks.workunit.client.0.smithi081.stdout:2/528: write da/db/d39/d45/f4c [1852786,30403] 0 2024-03-22T07:18:39.651 INFO:tasks.workunit.client.0.smithi081.stdout:2/529: mkdir da/db/d39/d47/d3b/d41/dab/dac 0 2024-03-22T07:18:39.652 INFO:tasks.workunit.client.0.smithi081.stdout:2/530: chown f6 83 1 2024-03-22T07:18:39.655 INFO:tasks.workunit.client.0.smithi081.stdout:8/619: read d2/d3/d28/d2b/f4b [3843900,23227] 0 2024-03-22T07:18:39.655 INFO:tasks.workunit.client.0.smithi081.stdout:8/620: dread - d2/d3/d23/f7f zero size 2024-03-22T07:18:39.655 INFO:tasks.workunit.client.0.smithi081.stdout:8/621: stat d2/d3/d19/d1e/d13/d22/f51 0 2024-03-22T07:18:39.655 INFO:tasks.workunit.client.0.smithi081.stdout:8/622: fdatasync d2/d3/d28/d2b/f7a 0 2024-03-22T07:18:39.660 INFO:tasks.workunit.client.0.smithi081.stdout:9/692: fsync d2/d10/d30/f45 0 2024-03-22T07:18:39.660 INFO:tasks.workunit.client.0.smithi081.stdout:3/404: dwrite db/d23/f67 [0,4194304] 0 2024-03-22T07:18:39.661 INFO:tasks.workunit.client.0.smithi081.stdout:6/597: dwrite d3/da/de/f28 [0,4194304] 0 2024-03-22T07:18:39.661 INFO:tasks.workunit.client.0.smithi081.stdout:2/531: read da/d11/f1b [1258574,73541] 0 2024-03-22T07:18:39.661 INFO:tasks.workunit.client.0.smithi081.stdout:9/693: dread d2/d2b/fd7 [0,4194304] 0 2024-03-22T07:18:39.661 INFO:tasks.workunit.client.0.smithi081.stdout:2/532: dread - da/fa9 zero size 2024-03-22T07:18:39.662 INFO:tasks.workunit.client.0.smithi081.stdout:4/834: dwrite da/f7b [0,4194304] 0 2024-03-22T07:18:39.662 INFO:tasks.workunit.client.0.smithi081.stdout:2/533: stat da/db/d39/d47/d3b/d41/d50/d6a/d70/f40 0 2024-03-22T07:18:39.665 INFO:tasks.workunit.client.0.smithi081.stdout:6/598: mkdir d3/d67/db2/d40/db9/d63/db4/dc5 0 2024-03-22T07:18:39.668 INFO:tasks.workunit.client.0.smithi081.stdout:9/694: symlink d2/d5d/d5a/le1 0 2024-03-22T07:18:39.669 INFO:tasks.workunit.client.0.smithi081.stdout:2/534: creat da/db/d39/d47/d3b/d41/dab/fad x:0 0 0 2024-03-22T07:18:39.671 INFO:tasks.workunit.client.0.smithi081.stdout:3/405: rename db/l27 to db/de/d4b/d6a/d70/l8b 0 2024-03-22T07:18:39.671 INFO:tasks.workunit.client.0.smithi081.stdout:3/406: chown db/f77 926378483 1 2024-03-22T07:18:39.671 INFO:tasks.workunit.client.0.smithi081.stdout:3/407: write db/d38/f47 [1377914,20965] 0 2024-03-22T07:18:39.673 INFO:tasks.workunit.client.0.smithi081.stdout:9/695: unlink d2/d10/d30/d57/f62 0 2024-03-22T07:18:39.673 INFO:tasks.workunit.client.0.smithi081.stdout:9/696: write d2/d73/fd1 [1379618,40072] 0 2024-03-22T07:18:39.675 INFO:tasks.workunit.client.0.smithi081.stdout:3/408: mknod db/de/d4b/d6a/d70/d3a/d5d/c8c 0 2024-03-22T07:18:39.677 INFO:tasks.workunit.client.0.smithi081.stdout:9/697: mkdir d2/d10/d30/d57/de2 0 2024-03-22T07:18:39.678 INFO:tasks.workunit.client.0.smithi081.stdout:3/409: mkdir db/d38/d8d 0 2024-03-22T07:18:39.680 INFO:tasks.workunit.client.0.smithi081.stdout:3/410: symlink db/l8e 0 2024-03-22T07:18:39.681 INFO:tasks.workunit.client.0.smithi081.stdout:3/411: creat db/de/d4b/d6a/d70/d3a/f8f x:0 0 0 2024-03-22T07:18:39.682 INFO:tasks.workunit.client.0.smithi081.stdout:6/599: write d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/fb0 [795330,109601] 0 2024-03-22T07:18:39.682 INFO:tasks.workunit.client.0.smithi081.stdout:6/600: chown d3/d67/db2/d40/db9/d63/d5e/dc0 14 1 2024-03-22T07:18:39.682 INFO:tasks.workunit.client.0.smithi081.stdout:3/412: write db/d23/f61 [683320,115520] 0 2024-03-22T07:18:39.684 INFO:tasks.workunit.client.0.smithi081.stdout:3/413: symlink db/de/d4b/d6a/d70/d3a/d5d/l90 0 2024-03-22T07:18:39.686 INFO:tasks.workunit.client.0.smithi081.stdout:4/835: dwrite da/df/d38/f68 [0,4194304] 0 2024-03-22T07:18:39.687 INFO:tasks.workunit.client.0.smithi081.stdout:7/682: dwrite d9/d3f/f65 [0,4194304] 0 2024-03-22T07:18:39.690 INFO:tasks.workunit.client.0.smithi081.stdout:7/683: creat d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/fea x:0 0 0 2024-03-22T07:18:39.691 INFO:tasks.workunit.client.0.smithi081.stdout:7/684: truncate d9/d10/d16/f74 1411470 0 2024-03-22T07:18:39.691 INFO:tasks.workunit.client.0.smithi081.stdout:7/685: chown d9/d10/d9e 1 1 2024-03-22T07:18:39.694 INFO:tasks.workunit.client.0.smithi081.stdout:7/686: unlink d9/d10/d4b/d9b/fbf 0 2024-03-22T07:18:39.694 INFO:tasks.workunit.client.0.smithi081.stdout:7/687: fdatasync d9/d10/d16/d1e/f58 0 2024-03-22T07:18:39.696 INFO:tasks.workunit.client.0.smithi081.stdout:7/688: link d9/d3f/d52/d83/lab d9/d3f/leb 0 2024-03-22T07:18:39.698 INFO:tasks.workunit.client.0.smithi081.stdout:7/689: mknod d9/d10/d53/d81/d59/cec 0 2024-03-22T07:18:39.700 INFO:tasks.workunit.client.0.smithi081.stdout:7/690: rename d9/d10/d16/d1e/f4e to d9/d10/d16/db6/dd9/fed 0 2024-03-22T07:18:39.702 INFO:tasks.workunit.client.0.smithi081.stdout:7/691: truncate d9/dc/f38 7589259 0 2024-03-22T07:18:39.702 INFO:tasks.workunit.client.0.smithi081.stdout:7/692: write d9/d10/d16/f7a [423927,5597] 0 2024-03-22T07:18:39.702 INFO:tasks.workunit.client.0.smithi081.stdout:3/414: getdents db 0 2024-03-22T07:18:39.703 INFO:tasks.workunit.client.0.smithi081.stdout:2/535: dwrite da/d11/d21/f33 [4194304,4194304] 0 2024-03-22T07:18:39.704 INFO:tasks.workunit.client.0.smithi081.stdout:6/601: write d3/d67/db2/d40/db9/d63/d26/f1c [957637,87318] 0 2024-03-22T07:18:39.705 INFO:tasks.workunit.client.0.smithi081.stdout:6/602: readlink d3/l30 0 2024-03-22T07:18:39.705 INFO:tasks.workunit.client.0.smithi081.stdout:2/536: creat da/db/d39/fae x:0 0 0 2024-03-22T07:18:39.706 INFO:tasks.workunit.client.0.smithi081.stdout:6/603: mkdir d3/d67/db2/d40/db9/d63/d7d/dc6 0 2024-03-22T07:18:39.708 INFO:tasks.workunit.client.0.smithi081.stdout:2/537: truncate da/db/f35 607018 0 2024-03-22T07:18:39.709 INFO:tasks.workunit.client.0.smithi081.stdout:6/604: chown d3/da/de/f4b 0 1 2024-03-22T07:18:39.709 INFO:tasks.workunit.client.0.smithi081.stdout:6/605: truncate d3/da/d48/fbc 751503 0 2024-03-22T07:18:39.709 INFO:tasks.workunit.client.0.smithi081.stdout:6/606: stat d3/d67 0 2024-03-22T07:18:39.711 INFO:tasks.workunit.client.0.smithi081.stdout:6/607: truncate d3/f3c 1552888 0 2024-03-22T07:18:39.715 INFO:tasks.workunit.client.0.smithi081.stdout:6/608: creat d3/d67/db2/d40/d75/fc7 x:0 0 0 2024-03-22T07:18:39.715 INFO:tasks.workunit.client.0.smithi081.stdout:1/467: link f1 d6/d18/d1f/d4c/d70/f9c 0 2024-03-22T07:18:39.724 INFO:tasks.workunit.client.0.smithi081.stdout:3/415: dwrite db/de/f29 [0,4194304] 0 2024-03-22T07:18:39.724 INFO:tasks.workunit.client.0.smithi081.stdout:9/698: dwrite d2/d73/d74/dd9/fdb [0,4194304] 0 2024-03-22T07:18:39.725 INFO:tasks.workunit.client.0.smithi081.stdout:6/609: mknod d3/d67/db2/d40/db9/d63/d5e/cc8 0 2024-03-22T07:18:39.737 INFO:tasks.workunit.client.0.smithi081.stdout:7/693: dwrite d9/dc/f38 [4194304,4194304] 0 2024-03-22T07:18:39.743 INFO:tasks.workunit.client.0.smithi081.stdout:9/699: unlink d2/d5/f68 0 2024-03-22T07:18:39.743 INFO:tasks.workunit.client.0.smithi081.stdout:9/700: creat d2/d10/d30/d57/d66/fe3 x:0 0 0 2024-03-22T07:18:39.743 INFO:tasks.workunit.client.0.smithi081.stdout:9/701: fdatasync d2/d73/d85/fb1 0 2024-03-22T07:18:39.744 INFO:tasks.workunit.client.0.smithi081.stdout:7/694: stat d9/d10/d2a/l43 0 2024-03-22T07:18:39.746 INFO:tasks.workunit.client.0.smithi081.stdout:3/416: mknod db/de/d4b/d6a/d70/d3a/c91 0 2024-03-22T07:18:39.747 INFO:tasks.workunit.client.0.smithi081.stdout:3/417: write db/de/f64 [4208599,72806] 0 2024-03-22T07:18:39.748 INFO:tasks.workunit.client.0.smithi081.stdout:9/702: link d2/d5d/c6b d2/d73/d85/ddd/ce4 0 2024-03-22T07:18:39.751 INFO:tasks.workunit.client.0.smithi081.stdout:3/418: unlink db/de/d4b/d6a/d70/d3a/f59 0 2024-03-22T07:18:39.753 INFO:tasks.workunit.client.0.smithi081.stdout:6/610: write d3/f6 [3016829,72274] 0 2024-03-22T07:18:39.753 INFO:tasks.workunit.client.0.smithi081.stdout:1/468: rename d6/d18/d1f/d24/f39 to d6/d6a/f9d 0 2024-03-22T07:18:39.755 INFO:tasks.workunit.client.0.smithi081.stdout:6/611: creat d3/d67/db2/d40/db9/d63/db4/dc5/fc9 x:0 0 0 2024-03-22T07:18:39.755 INFO:tasks.workunit.client.0.smithi081.stdout:6/612: fsync d3/d67/db2/d40/db9/f5d 0 2024-03-22T07:18:39.755 INFO:tasks.workunit.client.0.smithi081.stdout:7/695: dread d9/d10/d2a/fa9 [0,4194304] 0 2024-03-22T07:18:39.755 INFO:tasks.workunit.client.0.smithi081.stdout:6/613: fsync d3/d67/db2/f66 0 2024-03-22T07:18:39.755 INFO:tasks.workunit.client.0.smithi081.stdout:6/614: write d3/da/de/f7f [428643,124061] 0 2024-03-22T07:18:39.757 INFO:tasks.workunit.client.0.smithi081.stdout:1/469: creat d6/d18/d41/d49/d56/f9e x:0 0 0 2024-03-22T07:18:39.758 INFO:tasks.workunit.client.0.smithi081.stdout:9/703: rename d2/d10/d30/d57/d66/d75/l95 to d2/d5/le5 0 2024-03-22T07:18:39.759 INFO:tasks.workunit.client.0.smithi081.stdout:9/704: write d2/d10/f29 [4356170,93863] 0 2024-03-22T07:18:39.760 INFO:tasks.workunit.client.0.smithi081.stdout:9/705: mkdir d2/d10/d30/d57/d66/d75/d94/dae/de6 0 2024-03-22T07:18:39.761 INFO:tasks.workunit.client.0.smithi081.stdout:3/419: rename db/f4f to db/de/d4b/d6a/d70/f92 0 2024-03-22T07:18:39.764 INFO:tasks.workunit.client.0.smithi081.stdout:6/615: dread d3/da/de/f5b [0,4194304] 0 2024-03-22T07:18:39.767 INFO:tasks.workunit.client.0.smithi081.stdout:3/420: symlink db/de/d4b/d6a/d70/l93 0 2024-03-22T07:18:39.768 INFO:tasks.workunit.client.0.smithi081.stdout:3/421: symlink db/d38/l94 0 2024-03-22T07:18:39.768 INFO:tasks.workunit.client.0.smithi081.stdout:3/422: write db/de/d4b/f84 [297703,87213] 0 2024-03-22T07:18:39.768 INFO:tasks.workunit.client.0.smithi081.stdout:3/423: mknod db/d23/c95 0 2024-03-22T07:18:39.769 INFO:tasks.workunit.client.0.smithi081.stdout:3/424: symlink db/de/d4b/d6a/d70/d3a/l96 0 2024-03-22T07:18:39.770 INFO:tasks.workunit.client.0.smithi081.stdout:3/425: mknod db/d38/c97 0 2024-03-22T07:18:39.771 INFO:tasks.workunit.client.0.smithi081.stdout:3/426: mknod db/d23/c98 0 2024-03-22T07:18:39.772 INFO:tasks.workunit.client.0.smithi081.stdout:3/427: mknod db/de/d4b/d6a/d70/c99 0 2024-03-22T07:18:39.773 INFO:tasks.workunit.client.0.smithi081.stdout:9/706: rmdir d2/d73 39 2024-03-22T07:18:39.773 INFO:tasks.workunit.client.0.smithi081.stdout:9/707: stat d2/d5/d9d 0 2024-03-22T07:18:39.775 INFO:tasks.workunit.client.0.smithi081.stdout:9/708: creat d2/d5d/fe7 x:0 0 0 2024-03-22T07:18:39.775 INFO:tasks.workunit.client.0.smithi081.stdout:9/709: write d2/d10/d30/d57/f9e [5215125,122726] 0 2024-03-22T07:18:39.775 INFO:tasks.workunit.client.0.smithi081.stdout:9/710: chown d2/d2b/f81 6109 1 2024-03-22T07:18:39.775 INFO:tasks.workunit.client.0.smithi081.stdout:9/711: fsync d2/d2b/f2d 0 2024-03-22T07:18:39.776 INFO:tasks.workunit.client.0.smithi081.stdout:6/616: read d3/d67/db2/d40/db9/f6f [2259778,71068] 0 2024-03-22T07:18:39.776 INFO:tasks.workunit.client.0.smithi081.stdout:6/617: chown d3/da/f59 294112 1 2024-03-22T07:18:39.776 INFO:tasks.workunit.client.0.smithi081.stdout:3/428: dread db/d38/f47 [0,4194304] 0 2024-03-22T07:18:39.783 INFO:tasks.workunit.client.0.smithi081.stdout:1/470: dwrite d6/d18/f4f [4194304,4194304] 0 2024-03-22T07:18:39.783 INFO:tasks.workunit.client.0.smithi081.stdout:1/471: dread - d6/d18/d41/d49/d56/d30/d93/f98 zero size 2024-03-22T07:18:39.789 INFO:tasks.workunit.client.0.smithi081.stdout:3/429: dread db/f5c [0,4194304] 0 2024-03-22T07:18:39.824 INFO:tasks.workunit.client.0.smithi081.stdout:9/712: rename d2/d2b/d49/c55 to d2/d5d/d58/ce8 0 2024-03-22T07:18:39.827 INFO:tasks.workunit.client.0.smithi081.stdout:3/430: mkdir db/de/d4b/d9a 0 2024-03-22T07:18:39.827 INFO:tasks.workunit.client.0.smithi081.stdout:3/431: chown db/de/l2c 2115813 1 2024-03-22T07:18:39.834 INFO:tasks.workunit.client.0.smithi081.stdout:3/432: mknod db/d38/c9b 0 2024-03-22T07:18:39.834 INFO:tasks.workunit.client.0.smithi081.stdout:3/433: dread - db/de/f7d zero size 2024-03-22T07:18:39.834 INFO:tasks.workunit.client.0.smithi081.stdout:3/434: creat db/de/d4b/d6a/f9c x:0 0 0 2024-03-22T07:18:39.834 INFO:tasks.workunit.client.0.smithi081.stdout:3/435: write db/de/f68 [299365,11040] 0 2024-03-22T07:18:39.834 INFO:tasks.workunit.client.0.smithi081.stdout:3/436: fdatasync db/de/f24 0 2024-03-22T07:18:39.835 INFO:tasks.workunit.client.0.smithi081.stdout:6/618: rename d3/d67/db2/d40/db9/f7a to d3/d67/db2/d40/fca 0 2024-03-22T07:18:39.836 INFO:tasks.workunit.client.0.smithi081.stdout:9/713: symlink d2/le9 0 2024-03-22T07:18:39.839 INFO:tasks.workunit.client.0.smithi081.stdout:3/437: read db/f13 [1787229,103008] 0 2024-03-22T07:18:39.841 INFO:tasks.workunit.client.0.smithi081.stdout:3/438: symlink db/de/d4b/d6a/d70/l9d 0 2024-03-22T07:18:39.842 INFO:tasks.workunit.client.0.smithi081.stdout:3/439: read db/d23/f61 [170257,71846] 0 2024-03-22T07:18:39.842 INFO:tasks.workunit.client.0.smithi081.stdout:3/440: creat db/d38/f9e x:0 0 0 2024-03-22T07:18:39.843 INFO:tasks.workunit.client.0.smithi081.stdout:3/441: truncate db/de/f4e 2553768 0 2024-03-22T07:18:39.846 INFO:tasks.workunit.client.0.smithi081.stdout:3/442: creat db/de/d4b/d6a/d70/d3a/d5d/f9f x:0 0 0 2024-03-22T07:18:39.856 INFO:tasks.workunit.client.0.smithi081.stdout:6/619: dwrite d3/da/d48/fbc [0,4194304] 0 2024-03-22T07:18:39.858 INFO:tasks.workunit.client.0.smithi081.stdout:1/472: rename d6/d18/d41/d49/d56/d30/f55 to d6/d18/d1f/d4c/f9f 0 2024-03-22T07:18:39.859 INFO:tasks.workunit.client.0.smithi081.stdout:1/473: creat d6/d18/d1f/d92/fa0 x:0 0 0 2024-03-22T07:18:39.861 INFO:tasks.workunit.client.0.smithi081.stdout:9/714: rename d2/d2b/d49/d86/l8f to d2/d5/d36/lea 0 2024-03-22T07:18:39.861 INFO:tasks.workunit.client.0.smithi081.stdout:9/715: write d2/d5/d9d/f91 [742448,219] 0 2024-03-22T07:18:39.862 INFO:tasks.workunit.client.0.smithi081.stdout:6/620: rmdir d3/d67/db2/d40/db9 39 2024-03-22T07:18:39.866 INFO:tasks.workunit.client.0.smithi081.stdout:6/621: link d3/d67/db2/d40/db9/d63/db4/dc5/fc9 d3/d67/db2/d40/db9/d63/d5e/d9d/fcb 0 2024-03-22T07:18:39.869 INFO:tasks.workunit.client.0.smithi081.stdout:1/474: dread d6/d18/d41/d49/d56/d30/f3d [0,4194304] 0 2024-03-22T07:18:39.871 INFO:tasks.workunit.client.0.smithi081.stdout:1/475: rmdir d6/d18/d78 39 2024-03-22T07:18:39.871 INFO:tasks.workunit.client.0.smithi081.stdout:1/476: stat d6/l64 0 2024-03-22T07:18:39.871 INFO:tasks.workunit.client.0.smithi081.stdout:1/477: fsync d6/f35 0 2024-03-22T07:18:39.871 INFO:tasks.workunit.client.0.smithi081.stdout:1/478: fdatasync d6/d18/d1f/d24/f2b 0 2024-03-22T07:18:39.883 INFO:tasks.workunit.client.0.smithi081.stdout:1/479: dread d6/f67 [0,4194304] 0 2024-03-22T07:18:39.884 INFO:tasks.workunit.client.0.smithi081.stdout:1/480: unlink d6/d18/d1f/d32/f4b 0 2024-03-22T07:18:39.886 INFO:tasks.workunit.client.0.smithi081.stdout:1/481: rename d6/d18/d1f/c5e to d6/d18/d1f/d92/ca1 0 2024-03-22T07:18:39.888 INFO:tasks.workunit.client.0.smithi081.stdout:1/482: rename d6/d18/d41/d49/c51 to d6/d18/d1f/ca2 0 2024-03-22T07:18:39.896 INFO:tasks.workunit.client.0.smithi081.stdout:9/716: dwrite d2/d5/d9d/f7b [0,4194304] 0 2024-03-22T07:18:39.897 INFO:tasks.workunit.client.0.smithi081.stdout:9/717: rmdir d2/d10/d30/d57/d66/d75/dca 39 2024-03-22T07:18:39.897 INFO:tasks.workunit.client.0.smithi081.stdout:9/718: fdatasync d2/d73/d74/fa4 0 2024-03-22T07:18:39.899 INFO:tasks.workunit.client.0.smithi081.stdout:3/443: dwrite db/fc [0,4194304] 0 2024-03-22T07:18:39.899 INFO:tasks.workunit.client.0.smithi081.stdout:3/444: write db/d38/f6c [425730,90216] 0 2024-03-22T07:18:39.900 INFO:tasks.workunit.client.0.smithi081.stdout:6/622: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f6e [0,4194304] 0 2024-03-22T07:18:39.900 INFO:tasks.workunit.client.0.smithi081.stdout:6/623: stat d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f6e 0 2024-03-22T07:18:39.900 INFO:tasks.workunit.client.0.smithi081.stdout:6/624: chown d3/d67/db2/d40/d75/fc7 0 1 2024-03-22T07:18:39.900 INFO:tasks.workunit.client.0.smithi081.stdout:3/445: chown db/c1d 6 1 2024-03-22T07:18:39.902 INFO:tasks.workunit.client.0.smithi081.stdout:3/446: symlink db/d38/la0 0 2024-03-22T07:18:39.902 INFO:tasks.workunit.client.0.smithi081.stdout:3/447: chown db/de/d4b/d6a/d70/f3e 9 1 2024-03-22T07:18:39.903 INFO:tasks.workunit.client.0.smithi081.stdout:6/625: mkdir d3/dcc 0 2024-03-22T07:18:39.903 INFO:tasks.workunit.client.0.smithi081.stdout:3/448: mkdir db/d38/da1 0 2024-03-22T07:18:39.909 INFO:tasks.workunit.client.0.smithi081.stdout:1/483: dwrite d6/d18/d1f/d4c/f9f [0,4194304] 0 2024-03-22T07:18:39.912 INFO:tasks.workunit.client.0.smithi081.stdout:3/449: dread db/f77 [0,4194304] 0 2024-03-22T07:18:39.916 INFO:tasks.workunit.client.0.smithi081.stdout:3/450: getdents db/de/d4b/d6a/d70/d3a 0 2024-03-22T07:18:39.916 INFO:tasks.workunit.client.0.smithi081.stdout:3/451: dread - db/d38/f80 zero size 2024-03-22T07:18:39.919 INFO:tasks.workunit.client.0.smithi081.stdout:3/452: getdents db/d38 0 2024-03-22T07:18:39.920 INFO:tasks.workunit.client.0.smithi081.stdout:3/453: symlink db/de/d4b/d6a/d70/d3a/la2 0 2024-03-22T07:18:39.920 INFO:tasks.workunit.client.0.smithi081.stdout:3/454: chown db/de/d4b/d6a/d70/d3a/l96 68 1 2024-03-22T07:18:39.923 INFO:tasks.workunit.client.0.smithi081.stdout:6/626: dwrite d3/f9f [0,4194304] 0 2024-03-22T07:18:39.924 INFO:tasks.workunit.client.0.smithi081.stdout:6/627: readlink d3/d67/db2/d40/db9/d63/d5e/d9d/da6/lb1 0 2024-03-22T07:18:39.925 INFO:tasks.workunit.client.0.smithi081.stdout:1/484: dwrite d6/d6a/f8f [0,4194304] 0 2024-03-22T07:18:39.925 INFO:tasks.workunit.client.0.smithi081.stdout:1/485: truncate d6/f17 4629475 0 2024-03-22T07:18:39.925 INFO:tasks.workunit.client.0.smithi081.stdout:1/486: chown d6/d18/d41/d49/d56/f3e 13 1 2024-03-22T07:18:39.925 INFO:tasks.workunit.client.0.smithi081.stdout:1/487: write d6/d18/d1f/d76/f8d [321007,118688] 0 2024-03-22T07:18:39.931 INFO:tasks.workunit.client.0.smithi081.stdout:6/628: rmdir d3/d67/db2/d40/db9/d63/d5e 39 2024-03-22T07:18:39.932 INFO:tasks.workunit.client.0.smithi081.stdout:6/629: chown d3/d67/db2/d40/db9/d63/d26/l74 8814 1 2024-03-22T07:18:39.932 INFO:tasks.workunit.client.0.smithi081.stdout:6/630: fdatasync d3/d67/db2/d40/db9/d63/d26/d3f/d5a/fa7 0 2024-03-22T07:18:39.932 INFO:tasks.workunit.client.0.smithi081.stdout:0/628: sync 2024-03-22T07:18:39.934 INFO:tasks.workunit.client.0.smithi081.stdout:0/629: link d3/dc/d2b/d39/l9e d3/d9b/ldc 0 2024-03-22T07:18:39.934 INFO:tasks.workunit.client.0.smithi081.stdout:0/630: read d3/dc/f2a [5628900,104148] 0 2024-03-22T07:18:39.949 INFO:tasks.workunit.client.0.smithi081.stdout:3/455: dwrite db/d38/f6c [0,4194304] 0 2024-03-22T07:18:39.949 INFO:tasks.workunit.client.0.smithi081.stdout:3/456: dread - db/de/f7d zero size 2024-03-22T07:18:39.950 INFO:tasks.workunit.client.0.smithi081.stdout:6/631: dwrite d3/da/f83 [0,4194304] 0 2024-03-22T07:18:39.950 INFO:tasks.workunit.client.0.smithi081.stdout:1/488: link d6/d18/d41/d49/d56/d30/f7f d6/d18/d78/fa3 0 2024-03-22T07:18:39.950 INFO:tasks.workunit.client.0.smithi081.stdout:1/489: write d6/d18/d1f/d24/f6f [1930098,87272] 0 2024-03-22T07:18:39.950 INFO:tasks.workunit.client.0.smithi081.stdout:1/490: stat d6/d18/d41/d49/d56/d30/f61 0 2024-03-22T07:18:39.953 INFO:tasks.workunit.client.0.smithi081.stdout:3/457: unlink db/de/f68 0 2024-03-22T07:18:39.955 INFO:tasks.workunit.client.0.smithi081.stdout:3/458: mknod db/ca3 0 2024-03-22T07:18:39.955 INFO:tasks.workunit.client.0.smithi081.stdout:1/491: unlink d6/d18/d1f/d24/d48/d6e/c91 0 2024-03-22T07:18:39.957 INFO:tasks.workunit.client.0.smithi081.stdout:3/459: symlink db/de/d4b/d6a/d70/la4 0 2024-03-22T07:18:39.957 INFO:tasks.workunit.client.0.smithi081.stdout:3/460: write db/d38/f9e [997215,65619] 0 2024-03-22T07:18:39.957 INFO:tasks.workunit.client.0.smithi081.stdout:3/461: write db/d38/f80 [6848,78966] 0 2024-03-22T07:18:39.958 INFO:tasks.workunit.client.0.smithi081.stdout:1/492: mknod d6/d18/d41/d49/d56/ca4 0 2024-03-22T07:18:39.958 INFO:tasks.workunit.client.0.smithi081.stdout:1/493: chown d6/f17 23001201 1 2024-03-22T07:18:39.961 INFO:tasks.workunit.client.0.smithi081.stdout:6/632: dread d3/da/f59 [0,4194304] 0 2024-03-22T07:18:39.961 INFO:tasks.workunit.client.0.smithi081.stdout:6/633: truncate d3/d67/db2/fb8 552909 0 2024-03-22T07:18:39.969 INFO:tasks.workunit.client.0.smithi081.stdout:1/494: read d6/d18/d1f/f25 [3049330,76487] 0 2024-03-22T07:18:39.971 INFO:tasks.workunit.client.0.smithi081.stdout:1/495: mkdir d6/d18/da5 0 2024-03-22T07:18:39.972 INFO:tasks.workunit.client.0.smithi081.stdout:3/462: dwrite db/de/d4b/d6a/d70/f44 [0,4194304] 0 2024-03-22T07:18:39.975 INFO:tasks.workunit.client.0.smithi081.stdout:3/463: link l8 db/de/la5 0 2024-03-22T07:18:39.990 INFO:tasks.workunit.client.0.smithi081.stdout:1/496: dwrite d6/d18/d1f/d24/f5f [4194304,4194304] 0 2024-03-22T07:18:39.990 INFO:tasks.workunit.client.0.smithi081.stdout:6/634: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/f29 [0,4194304] 0 2024-03-22T07:18:39.992 INFO:tasks.workunit.client.0.smithi081.stdout:1/497: mkdir d6/d18/d1f/d24/da6 0 2024-03-22T07:18:39.993 INFO:tasks.workunit.client.0.smithi081.stdout:1/498: mknod d6/d18/d41/d49/ca7 0 2024-03-22T07:18:39.995 INFO:tasks.workunit.client.0.smithi081.stdout:1/499: mkdir d6/d18/d41/d49/d56/d30/da8 0 2024-03-22T07:18:39.996 INFO:tasks.workunit.client.0.smithi081.stdout:1/500: mknod d6/d18/d41/ca9 0 2024-03-22T07:18:39.998 INFO:tasks.workunit.client.0.smithi081.stdout:6/635: dread d3/da/de/f28 [0,4194304] 0 2024-03-22T07:18:39.998 INFO:tasks.workunit.client.0.smithi081.stdout:6/636: dread - d3/d67/db2/d40/d75/fc7 zero size 2024-03-22T07:18:39.999 INFO:tasks.workunit.client.0.smithi081.stdout:6/637: mknod d3/da/de/db5/ccd 0 2024-03-22T07:18:39.999 INFO:tasks.workunit.client.0.smithi081.stdout:6/638: truncate d3/d67/db2/d40/db9/f85 4750090 0 2024-03-22T07:18:40.000 INFO:tasks.workunit.client.0.smithi081.stdout:6/639: creat d3/d67/db2/d40/db9/d63/d26/d3f/d5a/fce x:0 0 0 2024-03-22T07:18:40.000 INFO:tasks.workunit.client.0.smithi081.stdout:6/640: write d3/d67/db2/d40/db9/d63/d5e/f82 [4769859,27692] 0 2024-03-22T07:18:40.001 INFO:tasks.workunit.client.0.smithi081.stdout:6/641: creat d3/da/de/db5/fcf x:0 0 0 2024-03-22T07:18:40.002 INFO:tasks.workunit.client.0.smithi081.stdout:6/642: mkdir d3/d67/db2/dd0 0 2024-03-22T07:18:40.002 INFO:tasks.workunit.client.0.smithi081.stdout:6/643: readlink d3/d67/db2/d40/db9/d63/d26/d3f/d4c/l89 0 2024-03-22T07:18:40.002 INFO:tasks.workunit.client.0.smithi081.stdout:6/644: creat d3/d67/db2/d40/d75/fd1 x:0 0 0 2024-03-22T07:18:40.002 INFO:tasks.workunit.client.0.smithi081.stdout:1/501: dread d6/d6a/f9d [0,4194304] 0 2024-03-22T07:18:40.006 INFO:tasks.workunit.client.0.smithi081.stdout:1/502: mkdir d6/d18/d1f/d24/daa 0 2024-03-22T07:18:40.007 INFO:tasks.workunit.client.0.smithi081.stdout:1/503: symlink d6/d18/d1f/d24/da6/lab 0 2024-03-22T07:18:40.008 INFO:tasks.workunit.client.0.smithi081.stdout:1/504: truncate d6/d18/d1f/d24/d48/d5c/f73 1640408 0 2024-03-22T07:18:40.009 INFO:tasks.workunit.client.0.smithi081.stdout:1/505: dread - d6/d18/d41/d49/d56/f9e zero size 2024-03-22T07:18:40.009 INFO:tasks.workunit.client.0.smithi081.stdout:1/506: truncate d6/d18/d1f/d24/d48/d5c/f77 158715 0 2024-03-22T07:18:40.019 INFO:tasks.workunit.client.0.smithi081.stdout:6/645: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f6e [0,4194304] 0 2024-03-22T07:18:40.019 INFO:tasks.workunit.client.0.smithi081.stdout:6/646: write d3/da/f57 [1904882,19841] 0 2024-03-22T07:18:40.021 INFO:tasks.workunit.client.0.smithi081.stdout:6/647: mkdir d3/d67/db2/d40/db9/d63/dd2 0 2024-03-22T07:18:40.023 INFO:tasks.workunit.client.0.smithi081.stdout:1/507: rename d6/d18/d41/d49/d56 to d6/d18/d1f/d32/dac 0 2024-03-22T07:18:40.024 INFO:tasks.workunit.client.0.smithi081.stdout:1/508: creat d6/d18/d1f/d24/d48/fad x:0 0 0 2024-03-22T07:18:40.025 INFO:tasks.workunit.client.0.smithi081.stdout:1/509: getdents d6/d18/d1f 0 2024-03-22T07:18:40.025 INFO:tasks.workunit.client.0.smithi081.stdout:1/510: fdatasync d6/d18/d1f/d24/d48/f69 0 2024-03-22T07:18:40.025 INFO:tasks.workunit.client.0.smithi081.stdout:1/511: readlink d6/d18/d1f/l6b 0 2024-03-22T07:18:40.030 INFO:tasks.workunit.client.0.smithi081.stdout:1/512: creat d6/d18/d1f/d24/d48/d6e/fae x:0 0 0 2024-03-22T07:18:40.030 INFO:tasks.workunit.client.0.smithi081.stdout:1/513: truncate d6/f5b 301719 0 2024-03-22T07:18:40.030 INFO:tasks.workunit.client.0.smithi081.stdout:6/648: write d3/d67/db2/fa9 [2073193,41389] 0 2024-03-22T07:18:40.032 INFO:tasks.workunit.client.0.smithi081.stdout:1/514: write f1 [2726571,49183] 0 2024-03-22T07:18:40.032 INFO:tasks.workunit.client.0.smithi081.stdout:6/649: rename d3/d67/db2/d40/d75 to d3/d67/dd3 0 2024-03-22T07:18:40.034 INFO:tasks.workunit.client.0.smithi081.stdout:6/650: unlink d3/da/l4a 0 2024-03-22T07:18:40.041 INFO:tasks.workunit.client.0.smithi081.stdout:1/515: dread d6/d18/d78/f89 [0,4194304] 0 2024-03-22T07:18:40.045 INFO:tasks.workunit.client.0.smithi081.stdout:6/651: dread d3/d67/db2/d40/db9/d63/d26/d3f/f29 [0,4194304] 0 2024-03-22T07:18:40.047 INFO:tasks.workunit.client.0.smithi081.stdout:6/652: link d3/d67/db2/d40/db9/d63/d5e/d9d/da6/lbd d3/d67/db2/dd0/ld4 0 2024-03-22T07:18:40.051 INFO:tasks.workunit.client.0.smithi081.stdout:6/653: rename d3/d67/db2/d40/db9/d63/d26/d3f/lb3 to d3/d67/ld5 0 2024-03-22T07:18:40.053 INFO:tasks.workunit.client.0.smithi081.stdout:6/654: symlink d3/d67/db2/d40/db9/d63/ld6 0 2024-03-22T07:18:40.055 INFO:tasks.workunit.client.0.smithi081.stdout:1/516: dwrite d6/d18/f74 [0,4194304] 0 2024-03-22T07:18:40.055 INFO:tasks.workunit.client.0.smithi081.stdout:1/517: write d6/d18/d1f/d24/d48/d5c/f77 [867409,28140] 0 2024-03-22T07:18:40.057 INFO:tasks.workunit.client.0.smithi081.stdout:6/655: getdents d3/d67/db2/d40 0 2024-03-22T07:18:40.059 INFO:tasks.workunit.client.0.smithi081.stdout:6/656: unlink d3/f9f 0 2024-03-22T07:18:40.059 INFO:tasks.workunit.client.0.smithi081.stdout:6/657: readlink d3/da/de/l55 0 2024-03-22T07:18:40.064 INFO:tasks.workunit.client.0.smithi081.stdout:1/518: dread d6/d18/d1f/d4c/d70/f9c [0,4194304] 0 2024-03-22T07:18:40.066 INFO:tasks.workunit.client.0.smithi081.stdout:1/519: creat d6/d18/d41/d49/faf x:0 0 0 2024-03-22T07:18:40.066 INFO:tasks.workunit.client.0.smithi081.stdout:1/520: dread - d6/d18/d1f/d32/dac/f9e zero size 2024-03-22T07:18:40.066 INFO:tasks.workunit.client.0.smithi081.stdout:1/521: write d6/d18/d78/f89 [2505021,29720] 0 2024-03-22T07:18:40.067 INFO:tasks.workunit.client.0.smithi081.stdout:1/522: rmdir d6/d18/d60 39 2024-03-22T07:18:40.067 INFO:tasks.workunit.client.0.smithi081.stdout:1/523: truncate d6/f2a 1016168 0 2024-03-22T07:18:40.069 INFO:tasks.workunit.client.0.smithi081.stdout:1/524: mkdir d6/d18/d1f/db0 0 2024-03-22T07:18:40.069 INFO:tasks.workunit.client.0.smithi081.stdout:1/525: readlink d6/d18/l53 0 2024-03-22T07:18:40.076 INFO:tasks.workunit.client.0.smithi081.stdout:6/658: dwrite d3/d67/db2/d40/db9/f5d [0,4194304] 0 2024-03-22T07:18:40.080 INFO:tasks.workunit.client.0.smithi081.stdout:1/526: link d6/d6a/d85/l8e d6/d18/d1f/db0/lb1 0 2024-03-22T07:18:40.081 INFO:tasks.workunit.client.0.smithi081.stdout:1/527: rmdir d6/d18/d1f/d24 39 2024-03-22T07:18:40.081 INFO:tasks.workunit.client.0.smithi081.stdout:1/528: fsync d6/d18/d1f/d32/dac/d30/f9a 0 2024-03-22T07:18:40.081 INFO:tasks.workunit.client.0.smithi081.stdout:1/529: write d6/d18/d1f/d32/dac/d30/f61 [315218,22951] 0 2024-03-22T07:18:40.082 INFO:tasks.workunit.client.0.smithi081.stdout:1/530: getdents d6/d18/d1f/d4c/d70 0 2024-03-22T07:18:40.082 INFO:tasks.workunit.client.0.smithi081.stdout:1/531: creat d6/d6a/fb2 x:0 0 0 2024-03-22T07:18:40.082 INFO:tasks.workunit.client.0.smithi081.stdout:1/532: dread - d6/d18/d1f/f87 zero size 2024-03-22T07:18:40.088 INFO:tasks.workunit.client.0.smithi081.stdout:6/659: dread d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f5f [0,4194304] 0 2024-03-22T07:18:40.089 INFO:tasks.workunit.client.0.smithi081.stdout:1/533: dread d6/d18/d78/fa3 [0,4194304] 0 2024-03-22T07:18:40.092 INFO:tasks.workunit.client.0.smithi081.stdout:1/534: mknod d6/d18/d1f/d24/d48/cb3 0 2024-03-22T07:18:40.094 INFO:tasks.workunit.client.0.smithi081.stdout:1/535: unlink l5 0 2024-03-22T07:18:40.094 INFO:tasks.workunit.client.0.smithi081.stdout:1/536: write d6/d18/d1f/d4c/d70/d7d/f88 [4201887,122198] 0 2024-03-22T07:18:40.095 INFO:tasks.workunit.client.0.smithi081.stdout:1/537: link d6/d18/d41/f46 d6/d18/d1f/db0/fb4 0 2024-03-22T07:18:40.096 INFO:tasks.workunit.client.0.smithi081.stdout:1/538: dread - d6/d18/d1f/d32/dac/d30/f9a zero size 2024-03-22T07:18:40.102 INFO:tasks.workunit.client.0.smithi081.stdout:1/539: write d6/d18/d1f/d4c/d70/d7d/f88 [298074,89741] 0 2024-03-22T07:18:40.103 INFO:tasks.workunit.client.0.smithi081.stdout:1/540: symlink d6/lb5 0 2024-03-22T07:18:40.104 INFO:tasks.workunit.client.0.smithi081.stdout:1/541: truncate d6/d18/d1f/d32/dac/f3c 1103825 0 2024-03-22T07:18:40.111 INFO:tasks.workunit.client.0.smithi081.stdout:5/721: sync 2024-03-22T07:18:40.112 INFO:tasks.workunit.client.0.smithi081.stdout:5/722: truncate d4/d10/d20/d4f/f78 71797 0 2024-03-22T07:18:40.112 INFO:tasks.workunit.client.0.smithi081.stdout:1/542: write d6/d18/f27 [3248791,25257] 0 2024-03-22T07:18:40.113 INFO:tasks.workunit.client.0.smithi081.stdout:5/723: symlink d4/d10/d20/d25/db8/de0/leb 0 2024-03-22T07:18:40.114 INFO:tasks.workunit.client.0.smithi081.stdout:5/724: mkdir d4/df/de6/dec 0 2024-03-22T07:18:40.116 INFO:tasks.workunit.client.0.smithi081.stdout:5/725: creat d4/d10/d20/dd2/ddf/fed x:0 0 0 2024-03-22T07:18:40.117 INFO:tasks.workunit.client.0.smithi081.stdout:5/726: symlink d4/d10/d20/d25/lee 0 2024-03-22T07:18:40.118 INFO:tasks.workunit.client.0.smithi081.stdout:5/727: rmdir d4/df/d73/d8c/dc0/dc9 39 2024-03-22T07:18:40.118 INFO:tasks.workunit.client.0.smithi081.stdout:5/728: chown d4/df/d73/f9d 3 1 2024-03-22T07:18:40.119 INFO:tasks.workunit.client.0.smithi081.stdout:5/729: creat d4/d75/fef x:0 0 0 2024-03-22T07:18:40.119 INFO:tasks.workunit.client.0.smithi081.stdout:5/730: chown d4/d10/d20/db2/d2f/f92 9604440 1 2024-03-22T07:18:40.119 INFO:tasks.workunit.client.0.smithi081.stdout:1/543: dread d6/d18/d1f/d4c/d70/f9c [0,4194304] 0 2024-03-22T07:18:40.120 INFO:tasks.workunit.client.0.smithi081.stdout:5/731: rmdir d4/d10/d20/d25 39 2024-03-22T07:18:40.122 INFO:tasks.workunit.client.0.smithi081.stdout:5/732: creat d4/df/d73/d8c/ff0 x:0 0 0 2024-03-22T07:18:40.123 INFO:tasks.workunit.client.0.smithi081.stdout:1/544: rmdir d6/d18/d60 39 2024-03-22T07:18:40.124 INFO:tasks.workunit.client.0.smithi081.stdout:5/733: rmdir d4/df/d73 39 2024-03-22T07:18:40.125 INFO:tasks.workunit.client.0.smithi081.stdout:1/545: symlink d6/d18/d41/lb6 0 2024-03-22T07:18:40.125 INFO:tasks.workunit.client.0.smithi081.stdout:1/546: fdatasync d6/d18/d1f/d32/dac/f3e 0 2024-03-22T07:18:40.125 INFO:tasks.workunit.client.0.smithi081.stdout:1/547: creat d6/d18/d1f/d24/d48/d5c/fb7 x:0 0 0 2024-03-22T07:18:40.126 INFO:tasks.workunit.client.0.smithi081.stdout:1/548: mkdir d6/d6a/d85/db8 0 2024-03-22T07:18:40.126 INFO:tasks.workunit.client.0.smithi081.stdout:1/549: chown d6/d18/d1f/d24/f3b 1565019 1 2024-03-22T07:18:40.138 INFO:tasks.workunit.client.0.smithi081.stdout:1/550: dwrite d6/d18/d1f/d32/dac/d30/f9a [0,4194304] 0 2024-03-22T07:18:40.138 INFO:tasks.workunit.client.0.smithi081.stdout:1/551: creat d6/d6a/fb9 x:0 0 0 2024-03-22T07:18:40.140 INFO:tasks.workunit.client.0.smithi081.stdout:1/552: link d6/f2a d6/d18/d1f/d32/dac/d30/da8/fba 0 2024-03-22T07:18:40.143 INFO:tasks.workunit.client.0.smithi081.stdout:1/553: mknod d6/d18/d1f/d32/cbb 0 2024-03-22T07:18:40.144 INFO:tasks.workunit.client.0.smithi081.stdout:1/554: write d6/d6a/fb9 [586068,8525] 0 2024-03-22T07:18:40.144 INFO:tasks.workunit.client.0.smithi081.stdout:1/555: dread - d6/d18/d1f/f87 zero size 2024-03-22T07:18:40.160 INFO:tasks.workunit.client.0.smithi081.stdout:1/556: dwrite d6/d18/d1f/d32/dac/d30/f34 [0,4194304] 0 2024-03-22T07:18:40.165 INFO:tasks.workunit.client.0.smithi081.stdout:1/557: read d6/d18/d1f/d32/dac/f3c [806802,26307] 0 2024-03-22T07:18:40.165 INFO:tasks.workunit.client.0.smithi081.stdout:1/558: write d6/d18/d1f/d24/d48/d5c/fb7 [276371,55297] 0 2024-03-22T07:18:40.176 INFO:tasks.workunit.client.0.smithi081.stdout:1/559: rename d6/d18/d78 to d6/d18/d1f/d32/dac/d30/d93/dbc 0 2024-03-22T07:18:40.191 INFO:tasks.workunit.client.0.smithi081.stdout:1/560: dwrite d6/d18/d1f/d32/dac/f3c [0,4194304] 0 2024-03-22T07:18:40.207 INFO:tasks.workunit.client.0.smithi081.stdout:1/561: dwrite d6/d18/d1f/d32/dac/f3e [0,4194304] 0 2024-03-22T07:18:40.219 INFO:tasks.workunit.client.0.smithi081.stdout:1/562: dwrite d6/d18/d1f/d24/d48/d5c/f77 [0,4194304] 0 2024-03-22T07:18:40.221 INFO:tasks.workunit.client.0.smithi081.stdout:1/563: rename d6/d18/d1f/d32/dac/d30/d93/f98 to d6/d6a/d85/fbd 0 2024-03-22T07:18:40.227 INFO:tasks.workunit.client.0.smithi081.stdout:1/564: dread d6/d18/d1f/d32/dac/d30/f68 [0,4194304] 0 2024-03-22T07:18:40.228 INFO:tasks.workunit.client.0.smithi081.stdout:1/565: rmdir d6/d18/d1f/d24/da6 39 2024-03-22T07:18:40.228 INFO:tasks.workunit.client.0.smithi081.stdout:1/566: chown d6/d18/d1f/d24/f3b 229772923 1 2024-03-22T07:18:40.228 INFO:tasks.workunit.client.0.smithi081.stdout:1/567: truncate d6/d6a/d85/fbd 173966 0 2024-03-22T07:18:40.234 INFO:tasks.workunit.client.0.smithi081.stdout:1/568: rename d6/d18/d1f/d92 to d6/d18/d1f/d32/dbe 0 2024-03-22T07:18:40.234 INFO:tasks.workunit.client.0.smithi081.stdout:1/569: symlink d6/d18/d1f/d32/dac/d30/da8/lbf 0 2024-03-22T07:18:40.245 INFO:tasks.workunit.client.0.smithi081.stdout:1/570: dwrite d6/d18/d1f/d24/d48/d6e/f75 [0,4194304] 0 2024-03-22T07:18:40.247 INFO:tasks.workunit.client.0.smithi081.stdout:1/571: truncate d6/d18/d1f/d24/f6f 678816 0 2024-03-22T07:18:40.249 INFO:tasks.workunit.client.0.smithi081.stdout:1/572: rmdir d6 39 2024-03-22T07:18:40.266 INFO:tasks.workunit.client.0.smithi081.stdout:1/573: dwrite d6/d18/d1f/d32/dac/d30/d93/dbc/fa3 [0,4194304] 0 2024-03-22T07:18:40.266 INFO:tasks.workunit.client.0.smithi081.stdout:1/574: creat d6/d6a/d85/fc0 x:0 0 0 2024-03-22T07:18:40.267 INFO:tasks.workunit.client.0.smithi081.stdout:1/575: mknod d6/d18/d1f/d32/dac/d30/cc1 0 2024-03-22T07:18:40.267 INFO:tasks.workunit.client.0.smithi081.stdout:1/576: write d6/d18/f27 [3251189,5318] 0 2024-03-22T07:18:40.274 INFO:tasks.workunit.client.0.smithi081.stdout:1/577: write d6/f7b [1864403,76867] 0 2024-03-22T07:18:40.291 INFO:tasks.workunit.client.0.smithi081.stdout:1/578: dwrite d6/d6a/fb9 [0,4194304] 0 2024-03-22T07:18:40.293 INFO:tasks.workunit.client.0.smithi081.stdout:1/579: mknod d6/d18/cc2 0 2024-03-22T07:18:40.294 INFO:tasks.workunit.client.0.smithi081.stdout:1/580: symlink d6/d18/d1f/d32/dac/d30/lc3 0 2024-03-22T07:18:40.301 INFO:tasks.workunit.client.0.smithi081.stdout:1/581: dread d6/f67 [0,4194304] 0 2024-03-22T07:18:40.302 INFO:tasks.workunit.client.0.smithi081.stdout:1/582: mkdir d6/d18/d1f/d32/dbe/dc4 0 2024-03-22T07:18:40.302 INFO:tasks.workunit.client.0.smithi081.stdout:1/583: truncate d6/d18/d1f/d32/dac/d30/f9a 4982487 0 2024-03-22T07:18:40.303 INFO:tasks.workunit.client.0.smithi081.stdout:1/584: fdatasync d6/d18/d1f/db0/fb4 0 2024-03-22T07:18:40.315 INFO:tasks.workunit.client.0.smithi081.stdout:1/585: dwrite d6/d18/d1f/d24/f2b [4194304,4194304] 0 2024-03-22T07:18:40.315 INFO:tasks.workunit.client.0.smithi081.stdout:1/586: chown d6/d18/d1f/d24/d48/cb3 4190 1 2024-03-22T07:18:40.316 INFO:tasks.workunit.client.0.smithi081.stdout:1/587: creat d6/d18/d1f/d32/dbe/dc4/fc5 x:0 0 0 2024-03-22T07:18:40.316 INFO:tasks.workunit.client.0.smithi081.stdout:1/588: truncate d6/d18/d1f/d24/d48/d6e/fae 312698 0 2024-03-22T07:18:40.317 INFO:tasks.workunit.client.0.smithi081.stdout:1/589: mkdir d6/dc6 0 2024-03-22T07:18:40.323 INFO:tasks.workunit.client.0.smithi081.stdout:8/623: sync 2024-03-22T07:18:40.327 INFO:tasks.workunit.client.0.smithi081.stdout:1/590: rename d6/d18/d1f/f37 to d6/d18/d1f/d32/dac/d30/da8/fc7 0 2024-03-22T07:18:40.329 INFO:tasks.workunit.client.0.smithi081.stdout:1/591: link d6/d18/d1f/c96 d6/d18/d1f/d32/dbe/dc4/cc8 0 2024-03-22T07:18:40.329 INFO:tasks.workunit.client.0.smithi081.stdout:1/592: write d6/d18/d1f/d32/dbe/fa0 [652067,54338] 0 2024-03-22T07:18:40.330 INFO:tasks.workunit.client.0.smithi081.stdout:1/593: stat d6/d18/d60 0 2024-03-22T07:18:40.333 INFO:tasks.workunit.client.0.smithi081.stdout:8/624: write d2/d3/d19/f6e [2530363,122750] 0 2024-03-22T07:18:40.334 INFO:tasks.workunit.client.0.smithi081.stdout:8/625: write d2/d3/d19/d1e/d63/da0/d9c/fb1 [2604035,114993] 0 2024-03-22T07:18:40.355 INFO:tasks.workunit.client.0.smithi081.stdout:8/626: dwrite d2/d3/d28/d2b/f4b [0,4194304] 0 2024-03-22T07:18:40.355 INFO:tasks.workunit.client.0.smithi081.stdout:8/627: fdatasync d2/d3/d19/d1e/f5f 0 2024-03-22T07:18:40.355 INFO:tasks.workunit.client.0.smithi081.stdout:8/628: fdatasync d2/d3/d19/d1e/d13/d22/d38/f91 0 2024-03-22T07:18:40.355 INFO:tasks.workunit.client.0.smithi081.stdout:8/629: write d2/f5e [4818143,92125] 0 2024-03-22T07:18:40.355 INFO:tasks.workunit.client.0.smithi081.stdout:1/594: dwrite d6/d18/d1f/f87 [0,4194304] 0 2024-03-22T07:18:40.355 INFO:tasks.workunit.client.0.smithi081.stdout:1/595: truncate d6/d18/d1f/d4c/f9f 4869596 0 2024-03-22T07:18:40.362 INFO:tasks.workunit.client.0.smithi081.stdout:1/596: write f1 [2303647,48078] 0 2024-03-22T07:18:40.364 INFO:tasks.workunit.client.0.smithi081.stdout:1/597: dread d6/d18/d1f/d24/f6f [0,4194304] 0 2024-03-22T07:18:40.372 INFO:tasks.workunit.client.0.smithi081.stdout:8/630: dwrite d2/f15 [0,4194304] 0 2024-03-22T07:18:40.372 INFO:tasks.workunit.client.0.smithi081.stdout:8/631: fsync d2/d3/d23/f53 0 2024-03-22T07:18:40.374 INFO:tasks.workunit.client.0.smithi081.stdout:8/632: mknod d2/d3/d23/cd5 0 2024-03-22T07:18:40.374 INFO:tasks.workunit.client.0.smithi081.stdout:1/598: link d6/d18/d1f/d32/dac/f3c d6/d18/d60/fc9 0 2024-03-22T07:18:40.375 INFO:tasks.workunit.client.0.smithi081.stdout:8/633: symlink d2/d3/d19/d1e/d13/d22/d38/dbb/ld6 0 2024-03-22T07:18:40.375 INFO:tasks.workunit.client.0.smithi081.stdout:8/634: fdatasync d2/d3/d19/d1e/f5f 0 2024-03-22T07:18:40.376 INFO:tasks.workunit.client.0.smithi081.stdout:8/635: chown f0 82611167 1 2024-03-22T07:18:40.376 INFO:tasks.workunit.client.0.smithi081.stdout:1/599: mknod d6/d18/d60/cca 0 2024-03-22T07:18:40.380 INFO:tasks.workunit.client.0.smithi081.stdout:1/600: truncate d6/d18/d1f/d32/dac/f3c 3600756 0 2024-03-22T07:18:40.380 INFO:tasks.workunit.client.0.smithi081.stdout:1/601: mknod d6/d18/d1f/d32/dbe/dc4/ccb 0 2024-03-22T07:18:40.383 INFO:tasks.workunit.client.0.smithi081.stdout:1/602: unlink d6/l64 0 2024-03-22T07:18:40.383 INFO:tasks.workunit.client.0.smithi081.stdout:1/603: write d6/d18/d1f/d24/d48/d6e/fae [1041935,123037] 0 2024-03-22T07:18:40.394 INFO:tasks.workunit.client.0.smithi081.stdout:1/604: dread d6/d18/f28 [0,4194304] 0 2024-03-22T07:18:40.396 INFO:tasks.workunit.client.0.smithi081.stdout:8/636: dwrite d2/d3/d19/d1e/d63/da0/d9c/fb1 [4194304,4194304] 0 2024-03-22T07:18:40.396 INFO:tasks.workunit.client.0.smithi081.stdout:8/637: write d2/d3/f1f [3295242,8749] 0 2024-03-22T07:18:40.398 INFO:tasks.workunit.client.0.smithi081.stdout:1/605: dread d6/d18/d1f/d4c/f6c [0,4194304] 0 2024-03-22T07:18:40.398 INFO:tasks.workunit.client.0.smithi081.stdout:1/606: truncate d6/d18/d1f/d32/dac/d30/f9a 5073458 0 2024-03-22T07:18:40.401 INFO:tasks.workunit.client.0.smithi081.stdout:8/638: mkdir d2/d3/d28/d2b/d61/dd7 0 2024-03-22T07:18:40.402 INFO:tasks.workunit.client.0.smithi081.stdout:8/639: creat d2/d3/d28/d2b/d61/dd7/fd8 x:0 0 0 2024-03-22T07:18:40.402 INFO:tasks.workunit.client.0.smithi081.stdout:8/640: chown d2/d3/d23/f3b 28438075 1 2024-03-22T07:18:40.403 INFO:tasks.workunit.client.0.smithi081.stdout:8/641: link d2/d3/d23/ca6 d2/d3/d28/d2b/cd9 0 2024-03-22T07:18:40.404 INFO:tasks.workunit.client.0.smithi081.stdout:8/642: chown d2/d3/d19/d1e/c3f 3587584 1 2024-03-22T07:18:40.404 INFO:tasks.workunit.client.0.smithi081.stdout:8/643: write d2/d3/d19/d1e/d13/d22/d38/fc5 [716561,5264] 0 2024-03-22T07:18:40.404 INFO:tasks.workunit.client.0.smithi081.stdout:8/644: write d2/d3/d19/d1e/d63/da0/d9c/fb1 [9326834,16682] 0 2024-03-22T07:18:40.404 INFO:tasks.workunit.client.0.smithi081.stdout:8/645: mkdir d2/d3/d28/d2b/d61/d7e/dda 0 2024-03-22T07:18:40.406 INFO:tasks.workunit.client.0.smithi081.stdout:8/646: getdents d2/d3/d19/d1e/d13 0 2024-03-22T07:18:40.406 INFO:tasks.workunit.client.0.smithi081.stdout:8/647: chown d2/d3/d19/d1e/d13/d22/d38/f44 633 1 2024-03-22T07:18:40.406 INFO:tasks.workunit.client.0.smithi081.stdout:8/648: fdatasync d2/d3/d23/fa2 0 2024-03-22T07:18:40.407 INFO:tasks.workunit.client.0.smithi081.stdout:8/649: mkdir d2/d3/d28/d2b/d61/ddb 0 2024-03-22T07:18:40.407 INFO:tasks.workunit.client.0.smithi081.stdout:8/650: getdents d2/d3/d19/d1e/d63/da0/db7 0 2024-03-22T07:18:40.416 INFO:tasks.workunit.client.0.smithi081.stdout:1/607: dwrite d6/d18/d1f/d4c/f6c [0,4194304] 0 2024-03-22T07:18:40.417 INFO:tasks.workunit.client.0.smithi081.stdout:1/608: rmdir d6/d18/d1f/d32/dac/d30/d93 39 2024-03-22T07:18:40.418 INFO:tasks.workunit.client.0.smithi081.stdout:1/609: creat d6/d18/d1f/d24/daa/fcc x:0 0 0 2024-03-22T07:18:40.419 INFO:tasks.workunit.client.0.smithi081.stdout:1/610: creat d6/d18/d1f/d32/dac/fcd x:0 0 0 2024-03-22T07:18:40.420 INFO:tasks.workunit.client.0.smithi081.stdout:1/611: symlink d6/d18/d1f/d32/lce 0 2024-03-22T07:18:40.421 INFO:tasks.workunit.client.0.smithi081.stdout:1/612: creat d6/d18/d1f/fcf x:0 0 0 2024-03-22T07:18:40.422 INFO:tasks.workunit.client.0.smithi081.stdout:1/613: stat d6/d18/d1f/d32/dac/d30/f3d 0 2024-03-22T07:18:40.422 INFO:tasks.workunit.client.0.smithi081.stdout:1/614: chown d6/d18/d1f/d24/l94 3 1 2024-03-22T07:18:40.424 INFO:tasks.workunit.client.0.smithi081.stdout:4/836: sync 2024-03-22T07:18:40.425 INFO:tasks.workunit.client.0.smithi081.stdout:4/837: truncate da/f100 1497531 0 2024-03-22T07:18:40.426 INFO:tasks.workunit.client.0.smithi081.stdout:4/838: mkdir da/d18/d112 0 2024-03-22T07:18:40.427 INFO:tasks.workunit.client.0.smithi081.stdout:4/839: creat da/d18/d1b/d26/d3e/f113 x:0 0 0 2024-03-22T07:18:40.428 INFO:tasks.workunit.client.0.smithi081.stdout:8/651: dwrite d2/d3/d19/d1e/d63/da0/db7/dc8/d7c/f81 [0,4194304] 0 2024-03-22T07:18:40.428 INFO:tasks.workunit.client.0.smithi081.stdout:8/652: fdatasync d2/d3/d19/d1e/d13/d22/d2f/fd1 0 2024-03-22T07:18:40.431 INFO:tasks.workunit.client.0.smithi081.stdout:8/653: symlink d2/d3/d28/d2b/d61/d7e/dda/ldc 0 2024-03-22T07:18:40.432 INFO:tasks.workunit.client.0.smithi081.stdout:8/654: creat d2/d3/d19/d1e/d13/d22/d2f/fdd x:0 0 0 2024-03-22T07:18:40.432 INFO:tasks.workunit.client.0.smithi081.stdout:8/655: readlink d2/d3/d28/d2b/l62 0 2024-03-22T07:18:40.433 INFO:tasks.workunit.client.0.smithi081.stdout:8/656: mknod d2/d3/d19/d1e/d13/d22/d2f/d73/cde 0 2024-03-22T07:18:40.433 INFO:tasks.workunit.client.0.smithi081.stdout:8/657: chown d2/d3/d19/d1e/d13/dba/fd4 52595 1 2024-03-22T07:18:40.437 INFO:tasks.workunit.client.0.smithi081.stdout:1/615: rename d6/d18/d1f/db0/lb1 to d6/d18/d60/ld0 0 2024-03-22T07:18:40.437 INFO:tasks.workunit.client.0.smithi081.stdout:1/616: stat d6/d18/d1f/d32/dac/ca4 0 2024-03-22T07:18:40.437 INFO:tasks.workunit.client.0.smithi081.stdout:1/617: creat d6/d18/d41/fd1 x:0 0 0 2024-03-22T07:18:40.438 INFO:tasks.workunit.client.0.smithi081.stdout:1/618: stat d6/d18/d1f/d32/dbe/ca1 0 2024-03-22T07:18:40.438 INFO:tasks.workunit.client.0.smithi081.stdout:1/619: write d6/d18/d1f/d24/daa/fcc [1000395,66004] 0 2024-03-22T07:18:40.438 INFO:tasks.workunit.client.0.smithi081.stdout:1/620: truncate d6/d18/f27 3718849 0 2024-03-22T07:18:40.442 INFO:tasks.workunit.client.0.smithi081.stdout:1/621: mkdir d6/d18/d1f/d4c/d70/dd2 0 2024-03-22T07:18:40.444 INFO:tasks.workunit.client.0.smithi081.stdout:8/658: write d2/f24 [901992,67620] 0 2024-03-22T07:18:40.445 INFO:tasks.workunit.client.0.smithi081.stdout:8/659: dread d2/d3/d19/d1e/d63/f65 [0,4194304] 0 2024-03-22T07:18:40.446 INFO:tasks.workunit.client.0.smithi081.stdout:4/840: dwrite da/de6/f107 [0,4194304] 0 2024-03-22T07:18:40.446 INFO:tasks.workunit.client.0.smithi081.stdout:4/841: creat da/de6/db8/f114 x:0 0 0 2024-03-22T07:18:40.452 INFO:tasks.workunit.client.0.smithi081.stdout:4/842: symlink da/de6/l115 0 2024-03-22T07:18:40.452 INFO:tasks.workunit.client.0.smithi081.stdout:4/843: rename da/df to da/df/d38/dc9/d116 22 2024-03-22T07:18:40.452 INFO:tasks.workunit.client.0.smithi081.stdout:4/844: truncate da/d18/d1b/d26/d3e/fd1 723028 0 2024-03-22T07:18:40.453 INFO:tasks.workunit.client.0.smithi081.stdout:8/660: mkdir d2/d3/d28/d2b/d61/ddf 0 2024-03-22T07:18:40.453 INFO:tasks.workunit.client.0.smithi081.stdout:8/661: write d2/d3/d19/f6e [4865377,57322] 0 2024-03-22T07:18:40.453 INFO:tasks.workunit.client.0.smithi081.stdout:8/662: chown d2/d3/d19/d1e/d13/d66/d70/l75 9 1 2024-03-22T07:18:40.456 INFO:tasks.workunit.client.0.smithi081.stdout:8/663: link d2/d3/d19/d1e/d13/d22/f34 d2/d3/d28/d2b/d61/fe0 0 2024-03-22T07:18:40.457 INFO:tasks.workunit.client.0.smithi081.stdout:8/664: symlink d2/d3/d19/d1e/d63/da0/le1 0 2024-03-22T07:18:40.458 INFO:tasks.workunit.client.0.smithi081.stdout:1/622: dwrite d6/d6a/f8f [0,4194304] 0 2024-03-22T07:18:40.458 INFO:tasks.workunit.client.0.smithi081.stdout:1/623: chown d6/d6a/d85/fbd 57999515 1 2024-03-22T07:18:40.458 INFO:tasks.workunit.client.0.smithi081.stdout:8/665: symlink d2/d3/d19/d1e/d13/d66/d70/le2 0 2024-03-22T07:18:40.459 INFO:tasks.workunit.client.0.smithi081.stdout:8/666: write d2/d3/d23/f53 [3349054,98392] 0 2024-03-22T07:18:40.463 INFO:tasks.workunit.client.0.smithi081.stdout:8/667: truncate d2/d3/d28/d2b/f58 3122497 0 2024-03-22T07:18:40.465 INFO:tasks.workunit.client.0.smithi081.stdout:8/668: getdents d2/d3/d19/d1e/d13/d22/d2f 0 2024-03-22T07:18:40.467 INFO:tasks.workunit.client.0.smithi081.stdout:8/669: symlink d2/d3/d9e/dc9/le3 0 2024-03-22T07:18:40.469 INFO:tasks.workunit.client.0.smithi081.stdout:8/670: creat d2/d3/d19/d1e/d63/da0/d9c/fe4 x:0 0 0 2024-03-22T07:18:40.470 INFO:tasks.workunit.client.0.smithi081.stdout:8/671: getdents d2/d3/d19/d1e/d63/da0/db7/dc8/dc0 0 2024-03-22T07:18:40.472 INFO:tasks.workunit.client.0.smithi081.stdout:4/845: dwrite da/d18/d1b/d1e/d37/ff3 [0,4194304] 0 2024-03-22T07:18:40.477 INFO:tasks.workunit.client.0.smithi081.stdout:1/624: dwrite d6/d18/d1f/d32/dac/d30/f7f [0,4194304] 0 2024-03-22T07:18:40.477 INFO:tasks.workunit.client.0.smithi081.stdout:4/846: creat da/df/d38/dd8/d4f/daf/d101/f117 x:0 0 0 2024-03-22T07:18:40.477 INFO:tasks.workunit.client.0.smithi081.stdout:4/847: chown da/d18/d1b/d1e/d37/l14 1188354500 1 2024-03-22T07:18:40.478 INFO:tasks.workunit.client.0.smithi081.stdout:1/625: mknod d6/d18/d41/cd3 0 2024-03-22T07:18:40.479 INFO:tasks.workunit.client.0.smithi081.stdout:1/626: mknod d6/d18/d1f/d32/dac/d30/d93/cd4 0 2024-03-22T07:18:40.480 INFO:tasks.workunit.client.0.smithi081.stdout:1/627: truncate d6/d18/f74 2745384 0 2024-03-22T07:18:40.482 INFO:tasks.workunit.client.0.smithi081.stdout:1/628: creat d6/d18/d1f/d32/dac/d30/d93/fd5 x:0 0 0 2024-03-22T07:18:40.485 INFO:tasks.workunit.client.0.smithi081.stdout:8/672: dwrite d2/d3/d23/f3b [0,4194304] 0 2024-03-22T07:18:40.487 INFO:tasks.workunit.client.0.smithi081.stdout:4/848: dread da/d18/d1b/d26/fb5 [0,4194304] 0 2024-03-22T07:18:40.488 INFO:tasks.workunit.client.0.smithi081.stdout:8/673: rename d2/d3/d19/d1e/d63/da0/db7/dc8/d7c/f81 to d2/d3/d28/d2b/d61/ddf/fe5 0 2024-03-22T07:18:40.488 INFO:tasks.workunit.client.0.smithi081.stdout:8/674: chown d2/d3/d19/d1e/f8a 431311 1 2024-03-22T07:18:40.491 INFO:tasks.workunit.client.0.smithi081.stdout:3/464: sync 2024-03-22T07:18:40.491 INFO:tasks.workunit.client.0.smithi081.stdout:5/734: sync 2024-03-22T07:18:40.492 INFO:tasks.workunit.client.0.smithi081.stdout:9/719: sync 2024-03-22T07:18:40.492 INFO:tasks.workunit.client.0.smithi081.stdout:0/631: sync 2024-03-22T07:18:40.492 INFO:tasks.workunit.client.0.smithi081.stdout:2/538: sync 2024-03-22T07:18:40.492 INFO:tasks.workunit.client.0.smithi081.stdout:7/696: sync 2024-03-22T07:18:40.493 INFO:tasks.workunit.client.0.smithi081.stdout:6/660: sync 2024-03-22T07:18:40.496 INFO:tasks.workunit.client.0.smithi081.stdout:8/675: mkdir d2/d3/d28/d2b/d61/dc6/de6 0 2024-03-22T07:18:40.496 INFO:tasks.workunit.client.0.smithi081.stdout:4/849: unlink da/d18/d1b/d26/dce/dfd/d74/fd3 0 2024-03-22T07:18:40.498 INFO:tasks.workunit.client.0.smithi081.stdout:3/465: getdents db 0 2024-03-22T07:18:40.502 INFO:tasks.workunit.client.0.smithi081.stdout:6/661: mknod d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/cd7 0 2024-03-22T07:18:40.505 INFO:tasks.workunit.client.0.smithi081.stdout:4/850: rename da/df/f65 to da/de6/f118 0 2024-03-22T07:18:40.506 INFO:tasks.workunit.client.0.smithi081.stdout:9/720: mkdir d2/d10/d30/deb 0 2024-03-22T07:18:40.508 INFO:tasks.workunit.client.0.smithi081.stdout:8/676: getdents d2/d3/d23 0 2024-03-22T07:18:40.508 INFO:tasks.workunit.client.0.smithi081.stdout:8/677: dread - d2/d3/d19/d1e/d13/d22/d38/f91 zero size 2024-03-22T07:18:40.509 INFO:tasks.workunit.client.0.smithi081.stdout:4/851: creat da/df/d38/dd8/f119 x:0 0 0 2024-03-22T07:18:40.510 INFO:tasks.workunit.client.0.smithi081.stdout:6/662: symlink d3/d67/db2/d40/db9/d63/d26/d52/ld8 0 2024-03-22T07:18:40.510 INFO:tasks.workunit.client.0.smithi081.stdout:6/663: chown d3/d67/db2/d40/db9/d63/d5e/d9d/da6/lbd 1137 1 2024-03-22T07:18:40.511 INFO:tasks.workunit.client.0.smithi081.stdout:6/664: fdatasync d3/d67/db2/d40/db9/d63/d26/d3f/f98 0 2024-03-22T07:18:40.512 INFO:tasks.workunit.client.0.smithi081.stdout:8/678: creat d2/d3/d19/d1e/d13/da5/fe7 x:0 0 0 2024-03-22T07:18:40.512 INFO:tasks.workunit.client.0.smithi081.stdout:3/466: write db/de/f24 [643279,119390] 0 2024-03-22T07:18:40.514 INFO:tasks.workunit.client.0.smithi081.stdout:6/665: rmdir d3/da/dc3 0 2024-03-22T07:18:40.514 INFO:tasks.workunit.client.0.smithi081.stdout:6/666: getdents d3/d67/db2/d40/db9/d63/d7d/dc6 0 2024-03-22T07:18:40.515 INFO:tasks.workunit.client.0.smithi081.stdout:3/467: creat db/fa6 x:0 0 0 2024-03-22T07:18:40.523 INFO:tasks.workunit.client.0.smithi081.stdout:4/852: write da/d18/d1b/d26/f43 [2699209,31772] 0 2024-03-22T07:18:40.524 INFO:tasks.workunit.client.0.smithi081.stdout:4/853: creat da/d3d/f11a x:0 0 0 2024-03-22T07:18:40.526 INFO:tasks.workunit.client.0.smithi081.stdout:4/854: link da/f87 da/d18/d1b/d1e/d5f/f11b 0 2024-03-22T07:18:40.528 INFO:tasks.workunit.client.0.smithi081.stdout:4/855: unlink da/df/d38/dd8/c85 0 2024-03-22T07:18:40.528 INFO:tasks.workunit.client.0.smithi081.stdout:6/667: write d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f6e [2405105,97440] 0 2024-03-22T07:18:40.530 INFO:tasks.workunit.client.0.smithi081.stdout:6/668: unlink d3/d67/db2/d40/db9/f5d 0 2024-03-22T07:18:40.531 INFO:tasks.workunit.client.0.smithi081.stdout:6/669: fdatasync d3/d67/db2/d40/db9/d63/d5e/d9d/fcb 0 2024-03-22T07:18:40.531 INFO:tasks.workunit.client.0.smithi081.stdout:6/670: chown d3/d67/db2/d40/db9/d63/d26/d3f/d5a 116288 1 2024-03-22T07:18:40.533 INFO:tasks.workunit.client.0.smithi081.stdout:4/856: mknod da/df/d38/dd8/d4f/c11c 0 2024-03-22T07:18:40.533 INFO:tasks.workunit.client.0.smithi081.stdout:2/539: dwrite da/d11/d21/f4e [0,4194304] 0 2024-03-22T07:18:40.533 INFO:tasks.workunit.client.0.smithi081.stdout:1/629: dwrite d6/d18/d1f/d24/daa/fcc [0,4194304] 0 2024-03-22T07:18:40.533 INFO:tasks.workunit.client.0.smithi081.stdout:2/540: chown da/db/d39/d46/d42 25479 1 2024-03-22T07:18:40.534 INFO:tasks.workunit.client.0.smithi081.stdout:6/671: creat d3/da/de/db5/fd9 x:0 0 0 2024-03-22T07:18:40.534 INFO:tasks.workunit.client.0.smithi081.stdout:0/632: dwrite d3/d32/f84 [0,4194304] 0 2024-03-22T07:18:40.535 INFO:tasks.workunit.client.0.smithi081.stdout:2/541: truncate da/db/d39/d45/f8c 2036181 0 2024-03-22T07:18:40.536 INFO:tasks.workunit.client.0.smithi081.stdout:2/542: write da/db/d39/d47/d3b/d41/d50/f62 [620555,95327] 0 2024-03-22T07:18:40.536 INFO:tasks.workunit.client.0.smithi081.stdout:5/735: dwrite d4/f89 [0,4194304] 0 2024-03-22T07:18:40.538 INFO:tasks.workunit.client.0.smithi081.stdout:2/543: rmdir da/d11 39 2024-03-22T07:18:40.538 INFO:tasks.workunit.client.0.smithi081.stdout:2/544: stat da/f13 0 2024-03-22T07:18:40.539 INFO:tasks.workunit.client.0.smithi081.stdout:5/736: truncate d4/d10/d20/d25/f8b 7427216 0 2024-03-22T07:18:40.540 INFO:tasks.workunit.client.0.smithi081.stdout:0/633: mknod d3/dc/d77/cdd 0 2024-03-22T07:18:40.541 INFO:tasks.workunit.client.0.smithi081.stdout:5/737: creat d4/df/d73/d8c/dc0/dc9/ff1 x:0 0 0 2024-03-22T07:18:40.542 INFO:tasks.workunit.client.0.smithi081.stdout:3/468: dwrite db/d38/f51 [0,4194304] 0 2024-03-22T07:18:40.546 INFO:tasks.workunit.client.0.smithi081.stdout:3/469: creat db/d23/fa7 x:0 0 0 2024-03-22T07:18:40.554 INFO:tasks.workunit.client.0.smithi081.stdout:4/857: dread da/d18/d1b/d26/f66 [0,4194304] 0 2024-03-22T07:18:40.554 INFO:tasks.workunit.client.0.smithi081.stdout:4/858: dread - da/ff7 zero size 2024-03-22T07:18:40.554 INFO:tasks.workunit.client.0.smithi081.stdout:4/859: write da/df/d38/dd8/f9a [705002,10056] 0 2024-03-22T07:18:40.555 INFO:tasks.workunit.client.0.smithi081.stdout:6/672: dread d3/d67/db2/d40/db9/d63/d26/d3f/f98 [0,4194304] 0 2024-03-22T07:18:40.558 INFO:tasks.workunit.client.0.smithi081.stdout:4/860: symlink da/d3d/l11d 0 2024-03-22T07:18:40.560 INFO:tasks.workunit.client.0.smithi081.stdout:6/673: mknod d3/d67/dd3/cda 0 2024-03-22T07:18:40.561 INFO:tasks.workunit.client.0.smithi081.stdout:4/861: unlink da/d18/d1b/d1e/d37/f6c 0 2024-03-22T07:18:40.563 INFO:tasks.workunit.client.0.smithi081.stdout:2/545: dwrite f6 [0,4194304] 0 2024-03-22T07:18:40.563 INFO:tasks.workunit.client.0.smithi081.stdout:2/546: chown da/d56/c79 13967 1 2024-03-22T07:18:40.566 INFO:tasks.workunit.client.0.smithi081.stdout:6/674: unlink d3/d67/db2/d40/db9/d63/db4/dc5/fc9 0 2024-03-22T07:18:40.567 INFO:tasks.workunit.client.0.smithi081.stdout:4/862: link da/df/f5a da/df/d38/dd8/d4f/daf/f11e 0 2024-03-22T07:18:40.570 INFO:tasks.workunit.client.0.smithi081.stdout:2/547: unlink da/db/d39/d47/d3b/d41/d50/l78 0 2024-03-22T07:18:40.570 INFO:tasks.workunit.client.0.smithi081.stdout:2/548: chown da/db/d39/d46/fa7 6 1 2024-03-22T07:18:40.571 INFO:tasks.workunit.client.0.smithi081.stdout:4/863: creat da/df/d38/dd8/d4f/d7c/d90/dd7/f11f x:0 0 0 2024-03-22T07:18:40.574 INFO:tasks.workunit.client.0.smithi081.stdout:4/864: unlink da/d18/d1b/d1e/f3b 0 2024-03-22T07:18:40.575 INFO:tasks.workunit.client.0.smithi081.stdout:4/865: creat da/d18/d6e/f120 x:0 0 0 2024-03-22T07:18:40.576 INFO:tasks.workunit.client.0.smithi081.stdout:6/675: dread d3/d67/db2/d40/db9/d63/d26/f92 [0,4194304] 0 2024-03-22T07:18:40.576 INFO:tasks.workunit.client.0.smithi081.stdout:4/866: creat da/df/d38/dd8/d4f/d7c/d90/dd7/f121 x:0 0 0 2024-03-22T07:18:40.576 INFO:tasks.workunit.client.0.smithi081.stdout:6/676: write d3/d67/db2/d40/fca [4693494,59383] 0 2024-03-22T07:18:40.576 INFO:tasks.workunit.client.0.smithi081.stdout:4/867: write da/df/d38/dd8/d4f/d7c/d90/dd7/f11f [945750,20370] 0 2024-03-22T07:18:40.576 INFO:tasks.workunit.client.0.smithi081.stdout:4/868: fsync da/d3d/f9b 0 2024-03-22T07:18:40.576 INFO:tasks.workunit.client.0.smithi081.stdout:5/738: dwrite d4/f3e [0,4194304] 0 2024-03-22T07:18:40.579 INFO:tasks.workunit.client.0.smithi081.stdout:6/677: truncate d3/d67/db2/d40/db9/f87 2890188 0 2024-03-22T07:18:40.580 INFO:tasks.workunit.client.0.smithi081.stdout:6/678: fdatasync d3/da/f8f 0 2024-03-22T07:18:40.581 INFO:tasks.workunit.client.0.smithi081.stdout:1/630: rename d6/d18/d1f/d32/dac/f3e to d6/d18/d1f/d24/d48/d6e/fd6 0 2024-03-22T07:18:40.584 INFO:tasks.workunit.client.0.smithi081.stdout:2/549: dread da/d11/d21/fa2 [0,4194304] 0 2024-03-22T07:18:40.585 INFO:tasks.workunit.client.0.smithi081.stdout:5/739: symlink d4/df/de6/lf2 0 2024-03-22T07:18:40.588 INFO:tasks.workunit.client.0.smithi081.stdout:3/470: rename db/de/d4b/d6a/d70/d3a/f8a to db/de/d4b/d6a/d70/d3a/d5d/fa8 0 2024-03-22T07:18:40.588 INFO:tasks.workunit.client.0.smithi081.stdout:3/471: stat db/de/d4b/d6a/d70/f40 0 2024-03-22T07:18:40.588 INFO:tasks.workunit.client.0.smithi081.stdout:3/472: creat db/fa9 x:0 0 0 2024-03-22T07:18:40.590 INFO:tasks.workunit.client.0.smithi081.stdout:1/631: rmdir d6/d18/d1f/d24 39 2024-03-22T07:18:40.591 INFO:tasks.workunit.client.0.smithi081.stdout:1/632: chown d6/d18/d1f/d24/f3b 451 1 2024-03-22T07:18:40.591 INFO:tasks.workunit.client.0.smithi081.stdout:1/633: write d6/d18/d1f/db0/fb4 [1234579,79657] 0 2024-03-22T07:18:40.593 INFO:tasks.workunit.client.0.smithi081.stdout:0/634: dwrite d3/dc/f36 [4194304,4194304] 0 2024-03-22T07:18:40.593 INFO:tasks.workunit.client.0.smithi081.stdout:2/550: mknod da/db/d39/d45/caf 0 2024-03-22T07:18:40.595 INFO:tasks.workunit.client.0.smithi081.stdout:5/740: mkdir d4/df3 0 2024-03-22T07:18:40.595 INFO:tasks.workunit.client.0.smithi081.stdout:5/741: chown d4/d10/d20/d25/db8/de0/leb 32343 1 2024-03-22T07:18:40.596 INFO:tasks.workunit.client.0.smithi081.stdout:3/473: symlink db/de/laa 0 2024-03-22T07:18:40.596 INFO:tasks.workunit.client.0.smithi081.stdout:3/474: dread - db/de/d4b/d6a/d70/f55 zero size 2024-03-22T07:18:40.597 INFO:tasks.workunit.client.0.smithi081.stdout:1/634: mkdir d6/d18/d1f/d32/dac/d30/d93/dd7 0 2024-03-22T07:18:40.599 INFO:tasks.workunit.client.0.smithi081.stdout:1/635: unlink d6/d18/d41/d49/faf 0 2024-03-22T07:18:40.599 INFO:tasks.workunit.client.0.smithi081.stdout:1/636: getdents d6/d18/d1f/d24/da6 0 2024-03-22T07:18:40.600 INFO:tasks.workunit.client.0.smithi081.stdout:2/551: symlink da/db/d39/d47/d3b/d41/d50/d6a/d70/lb0 0 2024-03-22T07:18:40.600 INFO:tasks.workunit.client.0.smithi081.stdout:2/552: readlink da/db/d6d/d72/l7a 0 2024-03-22T07:18:40.600 INFO:tasks.workunit.client.0.smithi081.stdout:4/869: dwrite da/df/d38/dd8/d4f/daf/ff9 [0,4194304] 0 2024-03-22T07:18:40.601 INFO:tasks.workunit.client.0.smithi081.stdout:6/679: dwrite d3/d67/db2/d40/db9/d63/d26/f1c [0,4194304] 0 2024-03-22T07:18:40.601 INFO:tasks.workunit.client.0.smithi081.stdout:5/742: mkdir d4/df3/df4 0 2024-03-22T07:18:40.601 INFO:tasks.workunit.client.0.smithi081.stdout:1/637: link d6/d18/d1f/d24/d48/d6e/fae d6/d18/d60/fd8 0 2024-03-22T07:18:40.602 INFO:tasks.workunit.client.0.smithi081.stdout:1/638: symlink d6/d18/d1f/d4c/d70/dd2/ld9 0 2024-03-22T07:18:40.602 INFO:tasks.workunit.client.0.smithi081.stdout:1/639: creat d6/d18/d1f/d24/d48/d5c/fda x:0 0 0 2024-03-22T07:18:40.602 INFO:tasks.workunit.client.0.smithi081.stdout:1/640: dread - d6/d18/d1f/d32/dbe/dc4/fc5 zero size 2024-03-22T07:18:40.603 INFO:tasks.workunit.client.0.smithi081.stdout:5/743: dread d4/fb1 [0,4194304] 0 2024-03-22T07:18:40.604 INFO:tasks.workunit.client.0.smithi081.stdout:0/635: symlink d3/dc/d40/lde 0 2024-03-22T07:18:40.604 INFO:tasks.workunit.client.0.smithi081.stdout:0/636: chown d3/d9b/ld4 84498 1 2024-03-22T07:18:40.604 INFO:tasks.workunit.client.0.smithi081.stdout:0/637: chown d3/dc/f36 840 1 2024-03-22T07:18:40.607 INFO:tasks.workunit.client.0.smithi081.stdout:2/553: link da/db/d39/d45/l52 da/db/d39/d47/d3b/d41/dab/dac/lb1 0 2024-03-22T07:18:40.612 INFO:tasks.workunit.client.0.smithi081.stdout:4/870: creat da/d18/d1b/d1e/d5f/f122 x:0 0 0 2024-03-22T07:18:40.612 INFO:tasks.workunit.client.0.smithi081.stdout:4/871: fsync da/d18/d1b/d26/fde 0 2024-03-22T07:18:40.618 INFO:tasks.workunit.client.0.smithi081.stdout:3/475: write db/de/d4b/d6a/d70/d3a/f46 [1154968,24410] 0 2024-03-22T07:18:40.618 INFO:tasks.workunit.client.0.smithi081.stdout:3/476: truncate db/d38/f69 408021 0 2024-03-22T07:18:40.618 INFO:tasks.workunit.client.0.smithi081.stdout:2/554: write da/d11/d21/f90 [1622278,48626] 0 2024-03-22T07:18:40.621 INFO:tasks.workunit.client.0.smithi081.stdout:6/680: mkdir d3/ddb 0 2024-03-22T07:18:40.622 INFO:tasks.workunit.client.0.smithi081.stdout:6/681: dread d3/da/f53 [0,4194304] 0 2024-03-22T07:18:40.623 INFO:tasks.workunit.client.0.smithi081.stdout:6/682: chown d3/d67/db2/d40/db9/d63/d5e/d9d/fba 0 1 2024-03-22T07:18:40.625 INFO:tasks.workunit.client.0.smithi081.stdout:6/683: chown d3/da/de/f27 28 1 2024-03-22T07:18:40.626 INFO:tasks.workunit.client.0.smithi081.stdout:2/555: dread da/db/d39/d46/f36 [0,4194304] 0 2024-03-22T07:18:40.626 INFO:tasks.workunit.client.0.smithi081.stdout:0/638: dwrite d3/dc/d33/d62/f68 [0,4194304] 0 2024-03-22T07:18:40.628 INFO:tasks.workunit.client.0.smithi081.stdout:6/684: creat d3/d67/db2/d40/db9/d63/fdc x:0 0 0 2024-03-22T07:18:40.628 INFO:tasks.workunit.client.0.smithi081.stdout:6/685: readlink d3/d67/db2/dd0/ld4 0 2024-03-22T07:18:40.630 INFO:tasks.workunit.client.0.smithi081.stdout:5/744: dwrite d4/d10/d20/d25/f8b [4194304,4194304] 0 2024-03-22T07:18:40.632 INFO:tasks.workunit.client.0.smithi081.stdout:6/686: dread d3/d67/db2/d40/db9/d63/d26/f92 [0,4194304] 0 2024-03-22T07:18:40.633 INFO:tasks.workunit.client.0.smithi081.stdout:4/872: dwrite da/df/d38/dd8/d4f/d7c/d90/fc5 [0,4194304] 0 2024-03-22T07:18:40.634 INFO:tasks.workunit.client.0.smithi081.stdout:1/641: write d6/d6a/f9d [1352694,100964] 0 2024-03-22T07:18:40.635 INFO:tasks.workunit.client.0.smithi081.stdout:2/556: rename da/d11/da8 to da/db/d39/d47/d3b/d41/d50/d6a/db2 0 2024-03-22T07:18:40.636 INFO:tasks.workunit.client.0.smithi081.stdout:1/642: symlink d6/d18/d1f/d24/daa/ldb 0 2024-03-22T07:18:40.636 INFO:tasks.workunit.client.0.smithi081.stdout:1/643: chown d6/d18/d41/cd3 14083 1 2024-03-22T07:18:40.636 INFO:tasks.workunit.client.0.smithi081.stdout:1/644: chown d6/d18/d1f/d24/d48/d6e/c97 3106903 1 2024-03-22T07:18:40.636 INFO:tasks.workunit.client.0.smithi081.stdout:1/645: write d6/d18/d1f/d24/d48/fad [714583,62200] 0 2024-03-22T07:18:40.636 INFO:tasks.workunit.client.0.smithi081.stdout:5/745: mkdir d4/df/d73/df5 0 2024-03-22T07:18:40.637 INFO:tasks.workunit.client.0.smithi081.stdout:5/746: creat d4/d10/d20/dd2/ddf/ff6 x:0 0 0 2024-03-22T07:18:40.637 INFO:tasks.workunit.client.0.smithi081.stdout:5/747: fdatasync d4/d10/d20/db2/d2f/fc4 0 2024-03-22T07:18:40.637 INFO:tasks.workunit.client.0.smithi081.stdout:5/748: fdatasync d4/d10/d20/db2/d2f/fc4 0 2024-03-22T07:18:40.638 INFO:tasks.workunit.client.0.smithi081.stdout:5/749: dread - d4/d75/fef zero size 2024-03-22T07:18:40.638 INFO:tasks.workunit.client.0.smithi081.stdout:6/687: mknod d3/dcc/cdd 0 2024-03-22T07:18:40.642 INFO:tasks.workunit.client.0.smithi081.stdout:2/557: read da/db/f27 [259009,16413] 0 2024-03-22T07:18:40.647 INFO:tasks.workunit.client.0.smithi081.stdout:6/688: truncate d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f5f 5013352 0 2024-03-22T07:18:40.647 INFO:tasks.workunit.client.0.smithi081.stdout:6/689: creat d3/da/d48/fde x:0 0 0 2024-03-22T07:18:40.647 INFO:tasks.workunit.client.0.smithi081.stdout:6/690: write d3/d67/db2/d40/db9/d63/d5e/fc4 [508305,8973] 0 2024-03-22T07:18:40.647 INFO:tasks.workunit.client.0.smithi081.stdout:0/639: rename d3/dc/l83 to d3/ldf 0 2024-03-22T07:18:40.647 INFO:tasks.workunit.client.0.smithi081.stdout:0/640: write d3/dc/d2b/fa0 [579987,18291] 0 2024-03-22T07:18:40.649 INFO:tasks.workunit.client.0.smithi081.stdout:5/750: write d4/df/d73/d8c/f87 [3874149,20243] 0 2024-03-22T07:18:40.651 INFO:tasks.workunit.client.0.smithi081.stdout:2/558: mknod da/db/d39/cb3 0 2024-03-22T07:18:40.651 INFO:tasks.workunit.client.0.smithi081.stdout:2/559: fdatasync da/db/d39/d45/f8c 0 2024-03-22T07:18:40.653 INFO:tasks.workunit.client.0.smithi081.stdout:0/641: dread d3/f44 [0,4194304] 0 2024-03-22T07:18:40.662 INFO:tasks.workunit.client.0.smithi081.stdout:1/646: dwrite d6/d18/d1f/fcf [0,4194304] 0 2024-03-22T07:18:40.662 INFO:tasks.workunit.client.0.smithi081.stdout:1/647: chown d6/d18/d1f/c81 4902073 1 2024-03-22T07:18:40.662 INFO:tasks.workunit.client.0.smithi081.stdout:2/560: chown da/d11/ca5 0 1 2024-03-22T07:18:40.662 INFO:tasks.workunit.client.0.smithi081.stdout:5/751: rmdir d4/d75/d98 39 2024-03-22T07:18:40.663 INFO:tasks.workunit.client.0.smithi081.stdout:5/752: write d4/d75/da5/fd7 [88040,52666] 0 2024-03-22T07:18:40.666 INFO:tasks.workunit.client.0.smithi081.stdout:0/642: dread d3/dc/d2b/f2f [0,4194304] 0 2024-03-22T07:18:40.666 INFO:tasks.workunit.client.0.smithi081.stdout:2/561: mkdir da/db/d39/d46/d42/db4 0 2024-03-22T07:18:40.667 INFO:tasks.workunit.client.0.smithi081.stdout:6/691: dwrite d3/d67/db2/d40/db9/d63/d26/d52/f69 [0,4194304] 0 2024-03-22T07:18:40.667 INFO:tasks.workunit.client.0.smithi081.stdout:6/692: chown d3/d67/db2/d40/db9/d63/d26/d3f/l3e 420 1 2024-03-22T07:18:40.667 INFO:tasks.workunit.client.0.smithi081.stdout:5/753: mkdir d4/d9/df7 0 2024-03-22T07:18:40.668 INFO:tasks.workunit.client.0.smithi081.stdout:6/693: dread d3/d67/db2/d40/db9/d63/faa [0,4194304] 0 2024-03-22T07:18:40.668 INFO:tasks.workunit.client.0.smithi081.stdout:0/643: getdents d3/dc/d40/d69/d92 0 2024-03-22T07:18:40.668 INFO:tasks.workunit.client.0.smithi081.stdout:6/694: fdatasync d3/f3c 0 2024-03-22T07:18:40.668 INFO:tasks.workunit.client.0.smithi081.stdout:6/695: stat d3/d67/db2/d40/db9/d63/f68 0 2024-03-22T07:18:40.669 INFO:tasks.workunit.client.0.smithi081.stdout:5/754: truncate d4/d10/d20/d25/db8/f3b 2805375 0 2024-03-22T07:18:40.669 INFO:tasks.workunit.client.0.smithi081.stdout:0/644: truncate d3/dc/f28 1322052 0 2024-03-22T07:18:40.670 INFO:tasks.workunit.client.0.smithi081.stdout:6/696: mkdir d3/d67/db2/d40/db9/d63/d5e/dc0/ddf 0 2024-03-22T07:18:40.672 INFO:tasks.workunit.client.0.smithi081.stdout:0/645: unlink d3/dc/d2b/d4d/c87 0 2024-03-22T07:18:40.673 INFO:tasks.workunit.client.0.smithi081.stdout:6/697: getdents d3 0 2024-03-22T07:18:40.674 INFO:tasks.workunit.client.0.smithi081.stdout:0/646: mkdir d3/d6a/de0 0 2024-03-22T07:18:40.674 INFO:tasks.workunit.client.0.smithi081.stdout:0/647: chown d3/dc/d40/lde 132942 1 2024-03-22T07:18:40.674 INFO:tasks.workunit.client.0.smithi081.stdout:0/648: chown d3/dc/d77 0 1 2024-03-22T07:18:40.676 INFO:tasks.workunit.client.0.smithi081.stdout:6/698: mkdir d3/d67/db2/d40/db9/d63/d26/d3f/d4c/de0 0 2024-03-22T07:18:40.678 INFO:tasks.workunit.client.0.smithi081.stdout:5/755: read d4/d10/f1d [408015,73650] 0 2024-03-22T07:18:40.678 INFO:tasks.workunit.client.0.smithi081.stdout:5/756: truncate d4/d10/d20/d25/da3/fd6 855494 0 2024-03-22T07:18:40.681 INFO:tasks.workunit.client.0.smithi081.stdout:6/699: rename d3/d67/db2/d40/db9/d63/d26/d3f/d5a/caf to d3/d67/db2/d40/db9/d63/d26/d3f/d4c/ce1 0 2024-03-22T07:18:40.681 INFO:tasks.workunit.client.0.smithi081.stdout:6/700: truncate d3/d67/db2/d40/db9/f85 4939971 0 2024-03-22T07:18:40.682 INFO:tasks.workunit.client.0.smithi081.stdout:6/701: read - d3/d67/db2/d40/db9/d63/d26/d3f/d5a/fce zero size 2024-03-22T07:18:40.682 INFO:tasks.workunit.client.0.smithi081.stdout:6/702: fsync d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/fb0 0 2024-03-22T07:18:40.683 INFO:tasks.workunit.client.0.smithi081.stdout:1/648: dread d6/d18/d1f/d32/dac/d30/f3d [0,4194304] 0 2024-03-22T07:18:40.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/649: link d3/dc/d40/d69/d92/dc1/d91/ccc d3/dc/d2b/ce1 0 2024-03-22T07:18:40.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/650: getdents d3/dc/d2b/d39/dc9 0 2024-03-22T07:18:40.684 INFO:tasks.workunit.client.0.smithi081.stdout:6/703: symlink d3/d67/le2 0 2024-03-22T07:18:40.685 INFO:tasks.workunit.client.0.smithi081.stdout:0/651: link d3/dc/d33/d62/l81 d3/d6a/le2 0 2024-03-22T07:18:40.686 INFO:tasks.workunit.client.0.smithi081.stdout:2/562: dwrite da/d11/d21/f4e [0,4194304] 0 2024-03-22T07:18:40.690 INFO:tasks.workunit.client.0.smithi081.stdout:5/757: read f1 [2239859,73207] 0 2024-03-22T07:18:40.690 INFO:tasks.workunit.client.0.smithi081.stdout:6/704: rename d3/d67/db2/d40/db9/d63/d5e/cc8 to d3/d67/db2/d40/db9/d63/d7d/ce3 0 2024-03-22T07:18:40.693 INFO:tasks.workunit.client.0.smithi081.stdout:5/758: readlink d4/d9/l1c 0 2024-03-22T07:18:40.697 INFO:tasks.workunit.client.0.smithi081.stdout:0/652: link d3/dc/d33/f96 d3/dc/d40/fe3 0 2024-03-22T07:18:40.700 INFO:tasks.workunit.client.0.smithi081.stdout:2/563: rename da/db/d39/d47/d3b/d41/d50/d6a/d70/lb0 to da/db/d39/lb5 0 2024-03-22T07:18:40.700 INFO:tasks.workunit.client.0.smithi081.stdout:2/564: write da/db/d39/d47/d3b/d41/d50/f9d [469182,78056] 0 2024-03-22T07:18:40.700 INFO:tasks.workunit.client.0.smithi081.stdout:0/653: creat d3/d9b/fe4 x:0 0 0 2024-03-22T07:18:40.700 INFO:tasks.workunit.client.0.smithi081.stdout:0/654: truncate d3/dc/d33/f97 5593530 0 2024-03-22T07:18:40.702 INFO:tasks.workunit.client.0.smithi081.stdout:6/705: write d3/d67/db2/d40/db9/d63/d26/f92 [2596607,49379] 0 2024-03-22T07:18:40.703 INFO:tasks.workunit.client.0.smithi081.stdout:2/565: link da/f13 da/db/d39/d46/fb6 0 2024-03-22T07:18:40.706 INFO:tasks.workunit.client.0.smithi081.stdout:6/706: unlink d3/d67/db2/d40/db9/d63/d26/d52/f69 0 2024-03-22T07:18:40.706 INFO:tasks.workunit.client.0.smithi081.stdout:6/707: chown d3/d67/db2/d40/db9/d63/d26/d3f/d5a/c91 7767 1 2024-03-22T07:18:40.706 INFO:tasks.workunit.client.0.smithi081.stdout:6/708: write d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f6e [1262414,29258] 0 2024-03-22T07:18:40.708 INFO:tasks.workunit.client.0.smithi081.stdout:2/566: symlink da/db/d39/d47/d3b/d41/d50/d6a/db2/lb7 0 2024-03-22T07:18:40.709 INFO:tasks.workunit.client.0.smithi081.stdout:6/709: rmdir d3/d67/db2/d40/db9/d63/d26/d3f/d4c 39 2024-03-22T07:18:40.710 INFO:tasks.workunit.client.0.smithi081.stdout:6/710: write d3/d67/db2/d40/db9/d63/d5e/d9d/fcb [503217,18337] 0 2024-03-22T07:18:40.710 INFO:tasks.workunit.client.0.smithi081.stdout:6/711: creat d3/da/d48/fe4 x:0 0 0 2024-03-22T07:18:40.710 INFO:tasks.workunit.client.0.smithi081.stdout:6/712: chown d3/d67/dd3/cda 184222 1 2024-03-22T07:18:40.710 INFO:tasks.workunit.client.0.smithi081.stdout:6/713: chown d3/d67/db2/cad 774 1 2024-03-22T07:18:40.710 INFO:tasks.workunit.client.0.smithi081.stdout:6/714: chown d3/d67/db2/dd0/ld4 3676878 1 2024-03-22T07:18:40.724 INFO:tasks.workunit.client.0.smithi081.stdout:1/649: dwrite d6/d18/d60/fc9 [0,4194304] 0 2024-03-22T07:18:40.724 INFO:tasks.workunit.client.0.smithi081.stdout:1/650: stat d6/d18/d1f/l6b 0 2024-03-22T07:18:40.725 INFO:tasks.workunit.client.0.smithi081.stdout:1/651: getdents d6/d18/d1f/d4c 0 2024-03-22T07:18:40.726 INFO:tasks.workunit.client.0.smithi081.stdout:0/655: dwrite d3/dc/d40/f9a [0,4194304] 0 2024-03-22T07:18:40.730 INFO:tasks.workunit.client.0.smithi081.stdout:5/759: dwrite d4/d10/d20/d4f/fb6 [0,4194304] 0 2024-03-22T07:18:40.739 INFO:tasks.workunit.client.0.smithi081.stdout:0/656: symlink d3/dc/d40/d69/db3/le5 0 2024-03-22T07:18:40.740 INFO:tasks.workunit.client.0.smithi081.stdout:5/760: creat d4/df/d73/d8c/dc0/de1/ff8 x:0 0 0 2024-03-22T07:18:40.741 INFO:tasks.workunit.client.0.smithi081.stdout:0/657: rename d3/d32/fa9 to d3/d9b/da4/fe6 0 2024-03-22T07:18:40.741 INFO:tasks.workunit.client.0.smithi081.stdout:0/658: fdatasync d3/d6a/fd2 0 2024-03-22T07:18:40.743 INFO:tasks.workunit.client.0.smithi081.stdout:0/659: creat d3/dc/d40/d69/d92/dcf/dd6/fe7 x:0 0 0 2024-03-22T07:18:40.743 INFO:tasks.workunit.client.0.smithi081.stdout:0/660: getdents d3/dc/d33/dd1 0 2024-03-22T07:18:40.750 INFO:tasks.workunit.client.0.smithi081.stdout:6/715: dwrite d3/d67/db2/d40/f99 [0,4194304] 0 2024-03-22T07:18:40.750 INFO:tasks.workunit.client.0.smithi081.stdout:6/716: write d3/d67/db2/d40/db9/d63/d26/d3f/f29 [4431220,120043] 0 2024-03-22T07:18:40.752 INFO:tasks.workunit.client.0.smithi081.stdout:6/717: write d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f9a [245278,83247] 0 2024-03-22T07:18:40.752 INFO:tasks.workunit.client.0.smithi081.stdout:6/718: stat d3/d67/db2/d40/db9/d63/d26/d3f/f29 0 2024-03-22T07:18:40.752 INFO:tasks.workunit.client.0.smithi081.stdout:1/652: dread d6/f6d [0,4194304] 0 2024-03-22T07:18:40.752 INFO:tasks.workunit.client.0.smithi081.stdout:2/567: dwrite da/fa9 [0,4194304] 0 2024-03-22T07:18:40.753 INFO:tasks.workunit.client.0.smithi081.stdout:1/653: symlink d6/d18/d1f/d32/dbe/dc4/ldc 0 2024-03-22T07:18:40.753 INFO:tasks.workunit.client.0.smithi081.stdout:6/719: dread d3/d67/db2/d40/db9/d63/f1e [0,4194304] 0 2024-03-22T07:18:40.755 INFO:tasks.workunit.client.0.smithi081.stdout:2/568: mknod da/db/d39/d47/d3b/d64/cb8 0 2024-03-22T07:18:40.756 INFO:tasks.workunit.client.0.smithi081.stdout:6/720: rmdir d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c 39 2024-03-22T07:18:40.759 INFO:tasks.workunit.client.0.smithi081.stdout:5/761: dwrite d4/df/d22/d4d/f99 [0,4194304] 0 2024-03-22T07:18:40.759 INFO:tasks.workunit.client.0.smithi081.stdout:6/721: fsync d3/da/de/f11 0 2024-03-22T07:18:40.760 INFO:tasks.workunit.client.0.smithi081.stdout:2/569: symlink da/lb9 0 2024-03-22T07:18:40.760 INFO:tasks.workunit.client.0.smithi081.stdout:2/570: chown da/d11/d21/d71/ca6 0 1 2024-03-22T07:18:40.763 INFO:tasks.workunit.client.0.smithi081.stdout:2/571: mkdir da/db/d39/d47/d3b/d41/d50/d6a/d70/dba 0 2024-03-22T07:18:40.774 INFO:tasks.workunit.client.0.smithi081.stdout:0/661: dwrite d3/dc/f2a [4194304,4194304] 0 2024-03-22T07:18:40.775 INFO:tasks.workunit.client.0.smithi081.stdout:0/662: symlink d3/dc/le8 0 2024-03-22T07:18:40.786 INFO:tasks.workunit.client.0.smithi081.stdout:0/663: read d3/dc/d2b/d39/f3c [3563837,114275] 0 2024-03-22T07:18:40.787 INFO:tasks.workunit.client.0.smithi081.stdout:0/664: symlink d3/le9 0 2024-03-22T07:18:40.787 INFO:tasks.workunit.client.0.smithi081.stdout:0/665: mkdir d3/dc/d40/d69/d92/dc1/d74/dea 0 2024-03-22T07:18:40.788 INFO:tasks.workunit.client.0.smithi081.stdout:6/722: dwrite d3/da/f83 [0,4194304] 0 2024-03-22T07:18:40.789 INFO:tasks.workunit.client.0.smithi081.stdout:2/572: dwrite da/db/d39/d47/fa1 [0,4194304] 0 2024-03-22T07:18:40.789 INFO:tasks.workunit.client.0.smithi081.stdout:5/762: dwrite d4/df/f6d [0,4194304] 0 2024-03-22T07:18:40.790 INFO:tasks.workunit.client.0.smithi081.stdout:6/723: mknod d3/da/ce5 0 2024-03-22T07:18:40.791 INFO:tasks.workunit.client.0.smithi081.stdout:0/666: dread d3/dc/d40/d69/d92/dc1/d74/fb7 [0,4194304] 0 2024-03-22T07:18:40.794 INFO:tasks.workunit.client.0.smithi081.stdout:6/724: read - d3/da/de/fac zero size 2024-03-22T07:18:40.801 INFO:tasks.workunit.client.0.smithi081.stdout:0/667: rmdir d3/dc/d40/d69/d92/dc1/d74 39 2024-03-22T07:18:40.802 INFO:tasks.workunit.client.0.smithi081.stdout:2/573: rmdir da 39 2024-03-22T07:18:40.803 INFO:tasks.workunit.client.0.smithi081.stdout:5/763: symlink d4/df/lf9 0 2024-03-22T07:18:40.804 INFO:tasks.workunit.client.0.smithi081.stdout:6/725: mknod d3/d67/db2/ce6 0 2024-03-22T07:18:40.805 INFO:tasks.workunit.client.0.smithi081.stdout:0/668: chown d3/dc/l16 185149932 1 2024-03-22T07:18:40.805 INFO:tasks.workunit.client.0.smithi081.stdout:2/574: mkdir da/db/d39/d46/d42/db4/dbb 0 2024-03-22T07:18:40.807 INFO:tasks.workunit.client.0.smithi081.stdout:0/669: creat d3/dc/d40/d69/dd8/feb x:0 0 0 2024-03-22T07:18:40.810 INFO:tasks.workunit.client.0.smithi081.stdout:0/670: unlink d3/c5 0 2024-03-22T07:18:40.810 INFO:tasks.workunit.client.0.smithi081.stdout:0/671: readlink d3/dc/d33/d62/l98 0 2024-03-22T07:18:40.811 INFO:tasks.workunit.client.0.smithi081.stdout:0/672: dread d3/dc/f43 [0,4194304] 0 2024-03-22T07:18:40.811 INFO:tasks.workunit.client.0.smithi081.stdout:0/673: chown d3/dc/c80 120854 1 2024-03-22T07:18:40.811 INFO:tasks.workunit.client.0.smithi081.stdout:0/674: dread - d3/d6a/fd2 zero size 2024-03-22T07:18:40.812 INFO:tasks.workunit.client.0.smithi081.stdout:0/675: mkdir d3/dc/d33/dd1/dec 0 2024-03-22T07:18:40.815 INFO:tasks.workunit.client.0.smithi081.stdout:0/676: mknod d3/ced 0 2024-03-22T07:18:40.815 INFO:tasks.workunit.client.0.smithi081.stdout:2/575: read da/d11/d21/f4e [554026,92765] 0 2024-03-22T07:18:40.815 INFO:tasks.workunit.client.0.smithi081.stdout:2/576: dread - da/db/d39/fae zero size 2024-03-22T07:18:40.817 INFO:tasks.workunit.client.0.smithi081.stdout:0/677: mkdir d3/d4e/dee 0 2024-03-22T07:18:40.818 INFO:tasks.workunit.client.0.smithi081.stdout:5/764: dwrite d4/fb1 [0,4194304] 0 2024-03-22T07:18:40.818 INFO:tasks.workunit.client.0.smithi081.stdout:5/765: truncate d4/d10/d20/db2/d2a/fde 5220348 0 2024-03-22T07:18:40.819 INFO:tasks.workunit.client.0.smithi081.stdout:6/726: write d3/da/de/f1f [1273536,60489] 0 2024-03-22T07:18:40.821 INFO:tasks.workunit.client.0.smithi081.stdout:5/766: rename d4/d75/d98/f9c to d4/d9/d90/ffa 0 2024-03-22T07:18:40.823 INFO:tasks.workunit.client.0.smithi081.stdout:6/727: link d3/da/de/l7c d3/da/de/db5/le7 0 2024-03-22T07:18:40.827 INFO:tasks.workunit.client.0.smithi081.stdout:0/678: dread d3/dc/f1c [0,4194304] 0 2024-03-22T07:18:40.834 INFO:tasks.workunit.client.0.smithi081.stdout:5/767: dread d4/d75/da5/fcb [0,4194304] 0 2024-03-22T07:18:40.842 INFO:tasks.workunit.client.0.smithi081.stdout:0/679: dread d3/dc/d33/f97 [0,4194304] 0 2024-03-22T07:18:40.843 INFO:tasks.workunit.client.0.smithi081.stdout:0/680: creat d3/d6a/fef x:0 0 0 2024-03-22T07:18:40.843 INFO:tasks.workunit.client.0.smithi081.stdout:2/577: dwrite da/f13 [0,4194304] 0 2024-03-22T07:18:40.844 INFO:tasks.workunit.client.0.smithi081.stdout:6/728: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f97 [0,4194304] 0 2024-03-22T07:18:40.844 INFO:tasks.workunit.client.0.smithi081.stdout:0/681: unlink d3/dc/d2b/ce1 0 2024-03-22T07:18:40.846 INFO:tasks.workunit.client.0.smithi081.stdout:6/729: link d3/d67/db2/d40/db9/d63/d26/d3f/d5a/fa3 d3/d67/db2/d40/db9/d63/d26/d3f/d4c/fe8 0 2024-03-22T07:18:40.847 INFO:tasks.workunit.client.0.smithi081.stdout:6/730: creat d3/d67/db2/d40/da8/fe9 x:0 0 0 2024-03-22T07:18:40.848 INFO:tasks.workunit.client.0.smithi081.stdout:6/731: symlink d3/da/de/db5/lea 0 2024-03-22T07:18:40.850 INFO:tasks.workunit.client.0.smithi081.stdout:6/732: rename d3/da/l70 to d3/d67/db2/d40/db9/d63/d26/d52/leb 0 2024-03-22T07:18:40.854 INFO:tasks.workunit.client.0.smithi081.stdout:0/682: dread d3/dc/d40/d69/d92/dc1/f38 [0,4194304] 0 2024-03-22T07:18:40.862 INFO:tasks.workunit.client.0.smithi081.stdout:6/733: dread d3/d67/db2/d40/db9/d63/d26/f92 [0,4194304] 0 2024-03-22T07:18:40.863 INFO:tasks.workunit.client.0.smithi081.stdout:6/734: chown d3/d67/db2/l4e 2680318 1 2024-03-22T07:18:40.863 INFO:tasks.workunit.client.0.smithi081.stdout:6/735: getdents d3/d67/db2/d40/db9/d63/d26/d3f/d4c/de0 0 2024-03-22T07:18:40.863 INFO:tasks.workunit.client.0.smithi081.stdout:2/578: dread da/db/f29 [0,4194304] 0 2024-03-22T07:18:40.863 INFO:tasks.workunit.client.0.smithi081.stdout:2/579: truncate da/db/d39/fae 66290 0 2024-03-22T07:18:40.863 INFO:tasks.workunit.client.0.smithi081.stdout:5/768: dwrite d4/d10/d20/dd2/ddf/ff6 [0,4194304] 0 2024-03-22T07:18:40.864 INFO:tasks.workunit.client.0.smithi081.stdout:6/736: symlink d3/d67/db2/d40/lec 0 2024-03-22T07:18:40.864 INFO:tasks.workunit.client.0.smithi081.stdout:6/737: readlink d3/d67/db2/d40/db9/d63/d26/d3f/d4c/l89 0 2024-03-22T07:18:40.872 INFO:tasks.workunit.client.0.smithi081.stdout:6/738: rename d3/d67/db2/d40/fca to d3/dcc/fed 0 2024-03-22T07:18:40.872 INFO:tasks.workunit.client.0.smithi081.stdout:6/739: stat d3/d67/db2/d40/db9/d63/d5e/f82 0 2024-03-22T07:18:40.873 INFO:tasks.workunit.client.0.smithi081.stdout:5/769: rmdir d4/df/d73/d8c 39 2024-03-22T07:18:40.873 INFO:tasks.workunit.client.0.smithi081.stdout:6/740: dread d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f7e [0,4194304] 0 2024-03-22T07:18:40.874 INFO:tasks.workunit.client.0.smithi081.stdout:5/770: rename d4/df/d22/d4d/ce9 to d4/df/de6/cfb 0 2024-03-22T07:18:40.876 INFO:tasks.workunit.client.0.smithi081.stdout:5/771: creat d4/df3/ffc x:0 0 0 2024-03-22T07:18:40.876 INFO:tasks.workunit.client.0.smithi081.stdout:0/683: dwrite d3/f1d [0,4194304] 0 2024-03-22T07:18:40.877 INFO:tasks.workunit.client.0.smithi081.stdout:6/741: symlink d3/d67/db2/d40/db9/d63/d26/d52/lee 0 2024-03-22T07:18:40.878 INFO:tasks.workunit.client.0.smithi081.stdout:5/772: creat d4/d10/d20/ffd x:0 0 0 2024-03-22T07:18:40.880 INFO:tasks.workunit.client.0.smithi081.stdout:0/684: creat d3/dc/d2b/d39/dc9/ff0 x:0 0 0 2024-03-22T07:18:40.880 INFO:tasks.workunit.client.0.smithi081.stdout:6/742: truncate d3/d67/db2/d40/db9/d63/d5e/d9d/fcb 175959 0 2024-03-22T07:18:40.881 INFO:tasks.workunit.client.0.smithi081.stdout:0/685: dread d3/dc/d2b/d39/f75 [0,4194304] 0 2024-03-22T07:18:40.882 INFO:tasks.workunit.client.0.smithi081.stdout:5/773: truncate d4/df/d22/f57 11282 0 2024-03-22T07:18:40.882 INFO:tasks.workunit.client.0.smithi081.stdout:5/774: fsync d4/d10/d20/d4f/f78 0 2024-03-22T07:18:40.882 INFO:tasks.workunit.client.0.smithi081.stdout:5/775: write d4/d10/d20/d25/db8/f31 [650028,94763] 0 2024-03-22T07:18:40.885 INFO:tasks.workunit.client.0.smithi081.stdout:0/686: creat d3/dc/d33/dd1/dec/ff1 x:0 0 0 2024-03-22T07:18:40.886 INFO:tasks.workunit.client.0.smithi081.stdout:5/776: mkdir d4/d10/daf/dfe 0 2024-03-22T07:18:40.886 INFO:tasks.workunit.client.0.smithi081.stdout:5/777: stat d4/df/d22/d4d 0 2024-03-22T07:18:40.887 INFO:tasks.workunit.client.0.smithi081.stdout:0/687: link d3/dc/f43 d3/dc/d40/d69/d92/dcf/dd6/ff2 0 2024-03-22T07:18:40.888 INFO:tasks.workunit.client.0.smithi081.stdout:5/778: creat d4/df/d22/d4d/d82/fff x:0 0 0 2024-03-22T07:18:40.890 INFO:tasks.workunit.client.0.smithi081.stdout:0/688: rmdir d3/dc/d40/d69/d92/dc1/d74/dea 0 2024-03-22T07:18:40.891 INFO:tasks.workunit.client.0.smithi081.stdout:0/689: getdents d3/dc/d40/d69/d92/dcf 0 2024-03-22T07:18:40.904 INFO:tasks.workunit.client.0.smithi081.stdout:6/743: dwrite d3/da/de/f5b [4194304,4194304] 0 2024-03-22T07:18:40.906 INFO:tasks.workunit.client.0.smithi081.stdout:6/744: creat d3/d67/db2/d40/db9/d63/fef x:0 0 0 2024-03-22T07:18:40.909 INFO:tasks.workunit.client.0.smithi081.stdout:6/745: chown d3/da/c47 4888588 1 2024-03-22T07:18:40.910 INFO:tasks.workunit.client.0.smithi081.stdout:6/746: creat d3/d67/db2/d40/db9/d63/db4/ff0 x:0 0 0 2024-03-22T07:18:40.912 INFO:tasks.workunit.client.0.smithi081.stdout:5/779: dwrite d4/d10/d20/d25/f43 [0,4194304] 0 2024-03-22T07:18:40.916 INFO:tasks.workunit.client.0.smithi081.stdout:6/747: read d3/f3c [779968,91679] 0 2024-03-22T07:18:40.936 INFO:tasks.workunit.client.0.smithi081.stdout:0/690: dwrite d3/dc/d33/f47 [0,4194304] 0 2024-03-22T07:18:40.941 INFO:tasks.workunit.client.0.smithi081.stdout:7/697: sync 2024-03-22T07:18:40.943 INFO:tasks.workunit.client.0.smithi081.stdout:7/698: mknod d9/cee 0 2024-03-22T07:18:40.948 INFO:tasks.workunit.client.0.smithi081.stdout:5/780: dwrite d4/df/d73/fab [0,4194304] 0 2024-03-22T07:18:40.949 INFO:tasks.workunit.client.0.smithi081.stdout:5/781: stat d4/df/lf9 0 2024-03-22T07:18:40.953 INFO:tasks.workunit.client.0.smithi081.stdout:5/782: mknod d4/d10/daf/c100 0 2024-03-22T07:18:40.953 INFO:tasks.workunit.client.0.smithi081.stdout:5/783: write d4/df/d73/d8c/dc0/de1/ff8 [1015323,92256] 0 2024-03-22T07:18:40.953 INFO:tasks.workunit.client.0.smithi081.stdout:5/784: fdatasync d4/d10/d20/dd2/ddf/ff6 0 2024-03-22T07:18:40.953 INFO:tasks.workunit.client.0.smithi081.stdout:5/785: fdatasync d4/d10/d20/dd2/ddf/fed 0 2024-03-22T07:18:40.953 INFO:tasks.workunit.client.0.smithi081.stdout:5/786: write d4/df/d22/d4d/f63 [1507766,59096] 0 2024-03-22T07:18:40.953 INFO:tasks.workunit.client.0.smithi081.stdout:5/787: creat d4/d10/d20/d4f/f101 x:0 0 0 2024-03-22T07:18:40.953 INFO:tasks.workunit.client.0.smithi081.stdout:5/788: fdatasync d4/f89 0 2024-03-22T07:18:40.954 INFO:tasks.workunit.client.0.smithi081.stdout:5/789: truncate d4/d10/d20/d25/f9e 4201410 0 2024-03-22T07:18:40.954 INFO:tasks.workunit.client.0.smithi081.stdout:6/748: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f7e [0,4194304] 0 2024-03-22T07:18:40.954 INFO:tasks.workunit.client.0.smithi081.stdout:6/749: readlink d3/d67/db2/d40/db9/d63/d7d/l88 0 2024-03-22T07:18:40.954 INFO:tasks.workunit.client.0.smithi081.stdout:5/790: creat d4/d9/f102 x:0 0 0 2024-03-22T07:18:40.955 INFO:tasks.workunit.client.0.smithi081.stdout:5/791: stat d4/d10/d20/db2/d2f/d8e/lda 0 2024-03-22T07:18:40.972 INFO:tasks.workunit.client.0.smithi081.stdout:0/691: dwrite d3/d9b/da4/fe6 [0,4194304] 0 2024-03-22T07:18:40.973 INFO:tasks.workunit.client.0.smithi081.stdout:0/692: mkdir d3/d32/df3 0 2024-03-22T07:18:40.974 INFO:tasks.workunit.client.0.smithi081.stdout:7/699: dwrite d9/d10/d2a/d73/f8b [0,4194304] 0 2024-03-22T07:18:40.974 INFO:tasks.workunit.client.0.smithi081.stdout:7/700: write d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/fd6 [85627,43317] 0 2024-03-22T07:18:40.974 INFO:tasks.workunit.client.0.smithi081.stdout:7/701: truncate d9/d10/d4b/d9b/fdc 426660 0 2024-03-22T07:18:40.974 INFO:tasks.workunit.client.0.smithi081.stdout:7/702: stat d9/d3f 0 2024-03-22T07:18:40.974 INFO:tasks.workunit.client.0.smithi081.stdout:7/703: fdatasync d9/d10/d2a/fcd 0 2024-03-22T07:18:40.974 INFO:tasks.workunit.client.0.smithi081.stdout:7/704: truncate d9/d10/d16/f74 1955670 0 2024-03-22T07:18:40.974 INFO:tasks.workunit.client.0.smithi081.stdout:7/705: chown d9/d10/d4b 1356953 1 2024-03-22T07:18:40.975 INFO:tasks.workunit.client.0.smithi081.stdout:0/693: unlink d3/dc/d33/lbd 0 2024-03-22T07:18:40.976 INFO:tasks.workunit.client.0.smithi081.stdout:7/706: creat d9/d10/d4b/d9b/fef x:0 0 0 2024-03-22T07:18:40.976 INFO:tasks.workunit.client.0.smithi081.stdout:7/707: write d9/d10/d53/d81/d59/d89/dd4/de0/d70/f8d [528646,107301] 0 2024-03-22T07:18:40.978 INFO:tasks.workunit.client.0.smithi081.stdout:7/708: creat d9/d3f/d52/ff0 x:0 0 0 2024-03-22T07:18:40.979 INFO:tasks.workunit.client.0.smithi081.stdout:6/750: dwrite d3/d67/db2/fb8 [0,4194304] 0 2024-03-22T07:18:40.980 INFO:tasks.workunit.client.0.smithi081.stdout:7/709: symlink d9/d3f/d52/lf1 0 2024-03-22T07:18:40.981 INFO:tasks.workunit.client.0.smithi081.stdout:7/710: fsync d9/d10/d16/f7a 0 2024-03-22T07:18:40.987 INFO:tasks.workunit.client.0.smithi081.stdout:7/711: unlink d9/d10/d53/d81/d59/d89/dd4/fa4 0 2024-03-22T07:18:40.989 INFO:tasks.workunit.client.0.smithi081.stdout:7/712: mknod d9/d10/d16/d1e/cf2 0 2024-03-22T07:18:40.996 INFO:tasks.workunit.client.0.smithi081.stdout:0/694: dwrite d3/dc/d40/fe3 [0,4194304] 0 2024-03-22T07:18:40.997 INFO:tasks.workunit.client.0.smithi081.stdout:0/695: read d3/dc/d40/d69/d92/dc1/d74/fb7 [685445,124886] 0 2024-03-22T07:18:41.003 INFO:tasks.workunit.client.0.smithi081.stdout:7/713: dread d9/dc/f90 [0,4194304] 0 2024-03-22T07:18:41.004 INFO:tasks.workunit.client.0.smithi081.stdout:7/714: mknod d9/d10/d16/d1e/cf3 0 2024-03-22T07:18:41.004 INFO:tasks.workunit.client.0.smithi081.stdout:7/715: write d9/d10/d16/db6/dd9/fe8 [705357,11662] 0 2024-03-22T07:18:41.005 INFO:tasks.workunit.client.0.smithi081.stdout:6/751: dwrite d3/dcc/fed [0,4194304] 0 2024-03-22T07:18:41.005 INFO:tasks.workunit.client.0.smithi081.stdout:6/752: dread - d3/d67/db2/d40/db9/d63/d26/d3f/d5a/fa7 zero size 2024-03-22T07:18:41.006 INFO:tasks.workunit.client.0.smithi081.stdout:6/753: link d3/d67/db2/c39 d3/d67/db2/d40/db9/d63/db4/dc5/cf1 0 2024-03-22T07:18:41.006 INFO:tasks.workunit.client.0.smithi081.stdout:6/754: readlink d3/d67/db2/d40/db9/d63/d26/d3f/l3e 0 2024-03-22T07:18:41.010 INFO:tasks.workunit.client.0.smithi081.stdout:6/755: dread d3/d67/db2/d40/db9/f87 [0,4194304] 0 2024-03-22T07:18:41.011 INFO:tasks.workunit.client.0.smithi081.stdout:6/756: chown d3/d67/db2/d40/db9/d63/f2c 700 1 2024-03-22T07:18:41.011 INFO:tasks.workunit.client.0.smithi081.stdout:7/716: dread d9/d10/d53/d81/d59/d89/dd4/de0/f57 [0,4194304] 0 2024-03-22T07:18:41.013 INFO:tasks.workunit.client.0.smithi081.stdout:7/717: symlink d9/d3f/d52/lf4 0 2024-03-22T07:18:41.013 INFO:tasks.workunit.client.0.smithi081.stdout:0/696: write d3/dc/f2a [7631364,71068] 0 2024-03-22T07:18:41.014 INFO:tasks.workunit.client.0.smithi081.stdout:7/718: mknod d9/d10/d53/d81/d59/d89/cf5 0 2024-03-22T07:18:41.014 INFO:tasks.workunit.client.0.smithi081.stdout:0/697: mkdir d3/dc/d40/d69/d92/df4 0 2024-03-22T07:18:41.015 INFO:tasks.workunit.client.0.smithi081.stdout:0/698: creat d3/dc/d33/d62/ff5 x:0 0 0 2024-03-22T07:18:41.016 INFO:tasks.workunit.client.0.smithi081.stdout:7/719: mkdir d9/dc/df6 0 2024-03-22T07:18:41.016 INFO:tasks.workunit.client.0.smithi081.stdout:6/757: dread d3/d67/db2/fa9 [0,4194304] 0 2024-03-22T07:18:41.018 INFO:tasks.workunit.client.0.smithi081.stdout:7/720: creat d9/dc/ff7 x:0 0 0 2024-03-22T07:18:41.018 INFO:tasks.workunit.client.0.smithi081.stdout:7/721: truncate d9/d10/d2a/fcd 943069 0 2024-03-22T07:18:41.018 INFO:tasks.workunit.client.0.smithi081.stdout:7/722: readlink d9/d3f/d52/d83/lab 0 2024-03-22T07:18:41.018 INFO:tasks.workunit.client.0.smithi081.stdout:7/723: chown d9/d3f/d52/lf4 52 1 2024-03-22T07:18:41.018 INFO:tasks.workunit.client.0.smithi081.stdout:0/699: mkdir d3/dc/d40/d69/d92/dc1/df6 0 2024-03-22T07:18:41.020 INFO:tasks.workunit.client.0.smithi081.stdout:7/724: read d9/d3f/f79 [751961,25242] 0 2024-03-22T07:18:41.020 INFO:tasks.workunit.client.0.smithi081.stdout:7/725: stat d9/d3f/d52/d83/fba 0 2024-03-22T07:18:41.020 INFO:tasks.workunit.client.0.smithi081.stdout:7/726: chown d9/d10/d53/d81/d59/d89/dd4/c64 3974798 1 2024-03-22T07:18:41.022 INFO:tasks.workunit.client.0.smithi081.stdout:7/727: creat d9/d3f/db3/ff8 x:0 0 0 2024-03-22T07:18:41.022 INFO:tasks.workunit.client.0.smithi081.stdout:7/728: creat d9/d10/d53/ff9 x:0 0 0 2024-03-22T07:18:41.024 INFO:tasks.workunit.client.0.smithi081.stdout:7/729: creat d9/d10/d53/d81/d59/d89/ffa x:0 0 0 2024-03-22T07:18:41.026 INFO:tasks.workunit.client.0.smithi081.stdout:7/730: fdatasync d9/d10/d53/d81/d59/d89/dd4/f7f 0 2024-03-22T07:18:41.027 INFO:tasks.workunit.client.0.smithi081.stdout:0/700: write d3/f24 [3308434,739] 0 2024-03-22T07:18:41.028 INFO:tasks.workunit.client.0.smithi081.stdout:0/701: dread d3/d4e/f54 [0,4194304] 0 2024-03-22T07:18:41.028 INFO:tasks.workunit.client.0.smithi081.stdout:7/731: link d9/d10/d9e/fce d9/d10/d16/d1e/d6f/ffb 0 2024-03-22T07:18:41.028 INFO:tasks.workunit.client.0.smithi081.stdout:0/702: fsync d3/d32/f5f 0 2024-03-22T07:18:41.033 INFO:tasks.workunit.client.0.smithi081.stdout:7/732: mkdir d9/d10/d16/d1e/d80/da1/dfc 0 2024-03-22T07:18:41.033 INFO:tasks.workunit.client.0.smithi081.stdout:7/733: write d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/fe2 [951651,107534] 0 2024-03-22T07:18:41.033 INFO:tasks.workunit.client.0.smithi081.stdout:7/734: fsync d9/d10/d53/d81/d59/d89/dd4/de0/d6d/fa5 0 2024-03-22T07:18:41.033 INFO:tasks.workunit.client.0.smithi081.stdout:0/703: mkdir d3/dc/d40/df7 0 2024-03-22T07:18:41.033 INFO:tasks.workunit.client.0.smithi081.stdout:0/704: chown d3/dc/d40/cb5 12 1 2024-03-22T07:18:41.033 INFO:tasks.workunit.client.0.smithi081.stdout:6/758: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d4c/fe8 [0,4194304] 0 2024-03-22T07:18:41.034 INFO:tasks.workunit.client.0.smithi081.stdout:6/759: chown d3/da/de/f4b 64721168 1 2024-03-22T07:18:41.037 INFO:tasks.workunit.client.0.smithi081.stdout:7/735: link d9/d10/d2a/l36 d9/d10/d53/d81/d59/d89/lfd 0 2024-03-22T07:18:41.038 INFO:tasks.workunit.client.0.smithi081.stdout:0/705: mknod d3/dc/d2b/d39/cf8 0 2024-03-22T07:18:41.039 INFO:tasks.workunit.client.0.smithi081.stdout:0/706: dread d3/d4e/f54 [0,4194304] 0 2024-03-22T07:18:41.040 INFO:tasks.workunit.client.0.smithi081.stdout:0/707: mkdir d3/dc/d33/d62/df9 0 2024-03-22T07:18:41.042 INFO:tasks.workunit.client.0.smithi081.stdout:0/708: rename d3/dc/d2b/f49 to d3/dc/ffa 0 2024-03-22T07:18:41.043 INFO:tasks.workunit.client.0.smithi081.stdout:0/709: stat d3/dc/d40/d69/d92/dc1/d91/ccc 0 2024-03-22T07:18:41.045 INFO:tasks.workunit.client.0.smithi081.stdout:0/710: rmdir d3/d9b 39 2024-03-22T07:18:41.048 INFO:tasks.workunit.client.0.smithi081.stdout:0/711: symlink d3/dc/d33/dd1/lfb 0 2024-03-22T07:18:41.048 INFO:tasks.workunit.client.0.smithi081.stdout:0/712: creat d3/dc/d2b/d39/dc9/ffc x:0 0 0 2024-03-22T07:18:41.048 INFO:tasks.workunit.client.0.smithi081.stdout:0/713: creat d3/dc/d40/d69/d92/dcf/dd6/ffd x:0 0 0 2024-03-22T07:18:41.049 INFO:tasks.workunit.client.0.smithi081.stdout:7/736: dread d9/d10/d9e/fc3 [0,4194304] 0 2024-03-22T07:18:41.055 INFO:tasks.workunit.client.0.smithi081.stdout:7/737: fdatasync d9/f75 0 2024-03-22T07:18:41.055 INFO:tasks.workunit.client.0.smithi081.stdout:7/738: creat d9/d10/d16/d1e/d80/ffe x:0 0 0 2024-03-22T07:18:41.055 INFO:tasks.workunit.client.0.smithi081.stdout:7/739: stat d9/d10/d16/d1e/c31 0 2024-03-22T07:18:41.055 INFO:tasks.workunit.client.0.smithi081.stdout:6/760: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/f7 [4194304,4194304] 0 2024-03-22T07:18:41.056 INFO:tasks.workunit.client.0.smithi081.stdout:7/740: creat d9/d10/d53/d81/d59/dbe/fff x:0 0 0 2024-03-22T07:18:41.056 INFO:tasks.workunit.client.0.smithi081.stdout:6/761: creat d3/d67/db2/d40/db9/d63/d26/d52/ff2 x:0 0 0 2024-03-22T07:18:41.058 INFO:tasks.workunit.client.0.smithi081.stdout:6/762: mkdir d3/d67/db2/d40/db9/d63/d7d/df3 0 2024-03-22T07:18:41.058 INFO:tasks.workunit.client.0.smithi081.stdout:7/741: mknod d9/d10/d2a/c100 0 2024-03-22T07:18:41.069 INFO:tasks.workunit.client.0.smithi081.stdout:7/742: dread d9/d10/d53/d81/d59/d89/dd4/de0/d6d/fa5 [0,4194304] 0 2024-03-22T07:18:41.069 INFO:tasks.workunit.client.0.smithi081.stdout:7/743: write d9/d3f/d52/fc8 [111849,120533] 0 2024-03-22T07:18:41.069 INFO:tasks.workunit.client.0.smithi081.stdout:0/714: dwrite d3/dc/f8d [4194304,4194304] 0 2024-03-22T07:18:41.073 INFO:tasks.workunit.client.0.smithi081.stdout:7/744: rename d9/d10/d16/db6/dd9/fed to d9/d3f/d52/d83/f101 0 2024-03-22T07:18:41.074 INFO:tasks.workunit.client.0.smithi081.stdout:6/763: dread d3/da/de/f28 [0,4194304] 0 2024-03-22T07:18:41.074 INFO:tasks.workunit.client.0.smithi081.stdout:6/764: write d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/fb0 [807956,58338] 0 2024-03-22T07:18:41.074 INFO:tasks.workunit.client.0.smithi081.stdout:6/765: chown d3/d67/db2/d40/db9/d63/d5e/dc0 3 1 2024-03-22T07:18:41.075 INFO:tasks.workunit.client.0.smithi081.stdout:0/715: rename d3/dc/d40/la8 to d3/dc/d40/d69/d92/dc1/d74/lfe 0 2024-03-22T07:18:41.077 INFO:tasks.workunit.client.0.smithi081.stdout:6/766: mkdir d3/d67/dd3/df4 0 2024-03-22T07:18:41.079 INFO:tasks.workunit.client.0.smithi081.stdout:0/716: write d3/f4 [2879070,37342] 0 2024-03-22T07:18:41.080 INFO:tasks.workunit.client.0.smithi081.stdout:6/767: creat d3/d67/db2/ff5 x:0 0 0 2024-03-22T07:18:41.082 INFO:tasks.workunit.client.0.smithi081.stdout:7/745: truncate d9/d3f/d52/d99/f9a 849206 0 2024-03-22T07:18:41.082 INFO:tasks.workunit.client.0.smithi081.stdout:7/746: chown d9/d10/d4b/d9b/fef 88 1 2024-03-22T07:18:41.082 INFO:tasks.workunit.client.0.smithi081.stdout:7/747: dread - d9/d3f/d52/ff0 zero size 2024-03-22T07:18:41.082 INFO:tasks.workunit.client.0.smithi081.stdout:7/748: symlink d9/d10/d16/db6/dd9/l102 0 2024-03-22T07:18:41.084 INFO:tasks.workunit.client.0.smithi081.stdout:7/749: symlink d9/d10/d4b/l103 0 2024-03-22T07:18:41.084 INFO:tasks.workunit.client.0.smithi081.stdout:7/750: chown d9/l44 28719530 1 2024-03-22T07:18:41.085 INFO:tasks.workunit.client.0.smithi081.stdout:7/751: fsync d9/d3f/db3/fc6 0 2024-03-22T07:18:41.085 INFO:tasks.workunit.client.0.smithi081.stdout:7/752: chown d9/c34 1770 1 2024-03-22T07:18:41.085 INFO:tasks.workunit.client.0.smithi081.stdout:7/753: readlink d9/d10/d16/db6/dd9/l102 0 2024-03-22T07:18:41.088 INFO:tasks.workunit.client.0.smithi081.stdout:0/717: dread d3/dc/d33/f7d [0,4194304] 0 2024-03-22T07:18:41.089 INFO:tasks.workunit.client.0.smithi081.stdout:0/718: truncate d3/dc/d2b/d39/f3c 2484804 0 2024-03-22T07:18:41.090 INFO:tasks.workunit.client.0.smithi081.stdout:0/719: dread - d3/dc/d40/d69/d92/dc1/fcd zero size 2024-03-22T07:18:41.091 INFO:tasks.workunit.client.0.smithi081.stdout:0/720: link d3/f2e d3/d6a/dd5/fff 0 2024-03-22T07:18:41.092 INFO:tasks.workunit.client.0.smithi081.stdout:7/754: dread d9/f11 [4194304,4194304] 0 2024-03-22T07:18:41.102 INFO:tasks.workunit.client.0.smithi081.stdout:6/768: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/f7 [8388608,4194304] 0 2024-03-22T07:18:41.102 INFO:tasks.workunit.client.0.smithi081.stdout:6/769: creat d3/d67/db2/d40/db9/d63/d26/d52/ff6 x:0 0 0 2024-03-22T07:18:41.105 INFO:tasks.workunit.client.0.smithi081.stdout:6/770: mkdir d3/d67/db2/df7 0 2024-03-22T07:18:41.105 INFO:tasks.workunit.client.0.smithi081.stdout:6/771: stat d3/d67/db2/d40/f99 0 2024-03-22T07:18:41.116 INFO:tasks.workunit.client.0.smithi081.stdout:0/721: dwrite d3/dc/dc6/fd9 [0,4194304] 0 2024-03-22T07:18:41.117 INFO:tasks.workunit.client.0.smithi081.stdout:0/722: creat d3/dc/d33/dd1/dec/f100 x:0 0 0 2024-03-22T07:18:41.117 INFO:tasks.workunit.client.0.smithi081.stdout:0/723: write d3/d6a/dd5/fda [843131,123301] 0 2024-03-22T07:18:41.117 INFO:tasks.workunit.client.0.smithi081.stdout:0/724: dread - d3/d32/f59 zero size 2024-03-22T07:18:41.122 INFO:tasks.workunit.client.0.smithi081.stdout:0/725: dread d3/dc/fe [0,4194304] 0 2024-03-22T07:18:41.122 INFO:tasks.workunit.client.0.smithi081.stdout:7/755: dwrite d9/d10/d16/fda [0,4194304] 0 2024-03-22T07:18:41.123 INFO:tasks.workunit.client.0.smithi081.stdout:6/772: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d4c/fe8 [0,4194304] 0 2024-03-22T07:18:41.123 INFO:tasks.workunit.client.0.smithi081.stdout:6/773: write d3/d67/db2/d40/db9/d63/d5e/d9d/fcb [854077,74334] 0 2024-03-22T07:18:41.134 INFO:tasks.workunit.client.0.smithi081.stdout:7/756: write f1 [4250997,105491] 0 2024-03-22T07:18:41.134 INFO:tasks.workunit.client.0.smithi081.stdout:7/757: write d9/d10/d2a/fcd [702099,54387] 0 2024-03-22T07:18:41.137 INFO:tasks.workunit.client.0.smithi081.stdout:6/774: creat d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/ff8 x:0 0 0 2024-03-22T07:18:41.141 INFO:tasks.workunit.client.0.smithi081.stdout:0/726: dwrite d3/dc/d40/d69/d92/dc1/d74/fa6 [0,4194304] 0 2024-03-22T07:18:41.141 INFO:tasks.workunit.client.0.smithi081.stdout:0/727: dread - d3/dc/d40/d69/d92/dcf/dd6/ffd zero size 2024-03-22T07:18:41.141 INFO:tasks.workunit.client.0.smithi081.stdout:0/728: creat d3/dc/d40/d69/d92/dcf/dd6/f101 x:0 0 0 2024-03-22T07:18:41.141 INFO:tasks.workunit.client.0.smithi081.stdout:6/775: creat d3/d67/db2/d40/db9/d63/d5e/dc0/ff9 x:0 0 0 2024-03-22T07:18:41.142 INFO:tasks.workunit.client.0.smithi081.stdout:7/758: rmdir d9/d10/d16/db6/dd9 39 2024-03-22T07:18:41.147 INFO:tasks.workunit.client.0.smithi081.stdout:7/759: symlink d9/d10/d53/d81/d59/dbe/l104 0 2024-03-22T07:18:41.147 INFO:tasks.workunit.client.0.smithi081.stdout:6/776: mkdir d3/d67/db2/d40/db9/d63/d26/dfa 0 2024-03-22T07:18:41.149 INFO:tasks.workunit.client.0.smithi081.stdout:6/777: symlink d3/ddb/lfb 0 2024-03-22T07:18:41.149 INFO:tasks.workunit.client.0.smithi081.stdout:6/778: fdatasync d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/ff8 0 2024-03-22T07:18:41.149 INFO:tasks.workunit.client.0.smithi081.stdout:6/779: fsync d3/d67/db2/d40/f6d 0 2024-03-22T07:18:41.149 INFO:tasks.workunit.client.0.smithi081.stdout:6/780: chown d3/l30 0 1 2024-03-22T07:18:41.149 INFO:tasks.workunit.client.0.smithi081.stdout:7/760: mkdir d9/d3f/d105 0 2024-03-22T07:18:41.150 INFO:tasks.workunit.client.0.smithi081.stdout:7/761: stat d9/dc/df6 0 2024-03-22T07:18:41.151 INFO:tasks.workunit.client.0.smithi081.stdout:7/762: mknod d9/d3f/db3/c106 0 2024-03-22T07:18:41.151 INFO:tasks.workunit.client.0.smithi081.stdout:7/763: write d9/d3f/d52/fc8 [710979,74467] 0 2024-03-22T07:18:41.151 INFO:tasks.workunit.client.0.smithi081.stdout:7/764: creat d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/f107 x:0 0 0 2024-03-22T07:18:41.165 INFO:tasks.workunit.client.0.smithi081.stdout:7/765: dread d9/d10/d53/d81/d59/d89/dd4/de0/d6d/fa5 [0,4194304] 0 2024-03-22T07:18:41.165 INFO:tasks.workunit.client.0.smithi081.stdout:7/766: creat d9/d10/d53/d81/d59/d89/dd4/de0/f108 x:0 0 0 2024-03-22T07:18:41.167 INFO:tasks.workunit.client.0.smithi081.stdout:7/767: dread d9/d3f/fa0 [0,4194304] 0 2024-03-22T07:18:41.168 INFO:tasks.workunit.client.0.smithi081.stdout:9/721: sync 2024-03-22T07:18:41.168 INFO:tasks.workunit.client.0.smithi081.stdout:9/722: fdatasync d2/d73/d74/fe0 0 2024-03-22T07:18:41.169 INFO:tasks.workunit.client.0.smithi081.stdout:9/723: stat d2/d5d/lbb 0 2024-03-22T07:18:41.169 INFO:tasks.workunit.client.0.smithi081.stdout:9/724: write d2/d73/d74/fa4 [587802,6166] 0 2024-03-22T07:18:41.169 INFO:tasks.workunit.client.0.smithi081.stdout:9/725: write d2/f1e [375866,111211] 0 2024-03-22T07:18:41.169 INFO:tasks.workunit.client.0.smithi081.stdout:9/726: stat d2/d10 0 2024-03-22T07:18:41.169 INFO:tasks.workunit.client.0.smithi081.stdout:7/768: mknod d9/d10/d53/d81/d59/d89/dd4/de0/c109 0 2024-03-22T07:18:41.171 INFO:tasks.workunit.client.0.smithi081.stdout:9/727: rename d2/d2b/c39 to d2/d73/d85/d7a/cec 0 2024-03-22T07:18:41.173 INFO:tasks.workunit.client.0.smithi081.stdout:6/781: dwrite d3/da/de/db5/fcf [0,4194304] 0 2024-03-22T07:18:41.173 INFO:tasks.workunit.client.0.smithi081.stdout:9/728: rename d2/d73/d85/f4c to d2/d73/fed 0 2024-03-22T07:18:41.174 INFO:tasks.workunit.client.0.smithi081.stdout:9/729: read - d2/d78/d93/fde zero size 2024-03-22T07:18:41.183 INFO:tasks.workunit.client.0.smithi081.stdout:6/782: write d3/d67/db2/d40/db9/d63/d26/d3f/f7 [5497748,7847] 0 2024-03-22T07:18:41.185 INFO:tasks.workunit.client.0.smithi081.stdout:7/769: dwrite d9/d10/d9e/fc3 [4194304,4194304] 0 2024-03-22T07:18:41.185 INFO:tasks.workunit.client.0.smithi081.stdout:7/770: write d9/d10/f2e [4760878,93726] 0 2024-03-22T07:18:41.186 INFO:tasks.workunit.client.0.smithi081.stdout:6/783: unlink d3/da/de/l6c 0 2024-03-22T07:18:41.188 INFO:tasks.workunit.client.0.smithi081.stdout:7/771: mkdir d9/d14/d10a 0 2024-03-22T07:18:41.189 INFO:tasks.workunit.client.0.smithi081.stdout:9/730: dwrite d2/f8 [0,4194304] 0 2024-03-22T07:18:41.189 INFO:tasks.workunit.client.0.smithi081.stdout:9/731: dread - d2/d10/fb0 zero size 2024-03-22T07:18:41.193 INFO:tasks.workunit.client.0.smithi081.stdout:9/732: link d2/d2b/f77 d2/d10/d30/d57/d66/d75/d94/dae/fee 0 2024-03-22T07:18:41.196 INFO:tasks.workunit.client.0.smithi081.stdout:9/733: link d2/d5d/c6b d2/d10/d30/d57/dad/cef 0 2024-03-22T07:18:41.197 INFO:tasks.workunit.client.0.smithi081.stdout:9/734: getdents d2/d73 0 2024-03-22T07:18:41.198 INFO:tasks.workunit.client.0.smithi081.stdout:9/735: symlink d2/lf0 0 2024-03-22T07:18:41.198 INFO:tasks.workunit.client.0.smithi081.stdout:7/772: dread d9/d10/d9e/fc3 [4194304,4194304] 0 2024-03-22T07:18:41.198 INFO:tasks.workunit.client.0.smithi081.stdout:7/773: chown d9/d10/d16/l6a 0 1 2024-03-22T07:18:41.200 INFO:tasks.workunit.client.0.smithi081.stdout:6/784: dwrite d3/da/de/f7f [0,4194304] 0 2024-03-22T07:18:41.201 INFO:tasks.workunit.client.0.smithi081.stdout:7/774: dread d9/d10/d16/f6b [0,4194304] 0 2024-03-22T07:18:41.202 INFO:tasks.workunit.client.0.smithi081.stdout:7/775: read - d9/dc/ff7 zero size 2024-03-22T07:18:41.204 INFO:tasks.workunit.client.0.smithi081.stdout:7/776: mknod d9/d10/d4b/d9b/c10b 0 2024-03-22T07:18:41.211 INFO:tasks.workunit.client.0.smithi081.stdout:6/785: dread d3/d67/db2/d40/db9/d63/d26/f92 [0,4194304] 0 2024-03-22T07:18:41.211 INFO:tasks.workunit.client.0.smithi081.stdout:6/786: creat d3/d67/db2/d40/da8/ffc x:0 0 0 2024-03-22T07:18:41.213 INFO:tasks.workunit.client.0.smithi081.stdout:6/787: mkdir d3/da/de/dfd 0 2024-03-22T07:18:41.213 INFO:tasks.workunit.client.0.smithi081.stdout:6/788: write d3/f9 [2870439,40878] 0 2024-03-22T07:18:41.213 INFO:tasks.workunit.client.0.smithi081.stdout:6/789: fdatasync d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/fae 0 2024-03-22T07:18:41.214 INFO:tasks.workunit.client.0.smithi081.stdout:6/790: rmdir d3/da/de/db5 39 2024-03-22T07:18:41.214 INFO:tasks.workunit.client.0.smithi081.stdout:6/791: truncate d3/d67/dd3/fd1 579132 0 2024-03-22T07:18:41.215 INFO:tasks.workunit.client.0.smithi081.stdout:6/792: unlink d3/da/de/f4b 0 2024-03-22T07:18:41.224 INFO:tasks.workunit.client.0.smithi081.stdout:6/793: dread d3/da/f59 [0,4194304] 0 2024-03-22T07:18:41.225 INFO:tasks.workunit.client.0.smithi081.stdout:6/794: dread - d3/da/de/f9b zero size 2024-03-22T07:18:41.226 INFO:tasks.workunit.client.0.smithi081.stdout:6/795: getdents d3/d67/db2/d40/db9 0 2024-03-22T07:18:41.226 INFO:tasks.workunit.client.0.smithi081.stdout:6/796: fdatasync d3/d67/db2/d40/db9/d63/d26/d3f/f98 0 2024-03-22T07:18:41.226 INFO:tasks.workunit.client.0.smithi081.stdout:6/797: dread d3/d67/db2/d40/db9/d63/f2c [0,4194304] 0 2024-03-22T07:18:41.227 INFO:tasks.workunit.client.0.smithi081.stdout:6/798: mkdir d3/da/de/db5/dfe 0 2024-03-22T07:18:41.227 INFO:tasks.workunit.client.0.smithi081.stdout:6/799: truncate d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/fae 829349 0 2024-03-22T07:18:41.227 INFO:tasks.workunit.client.0.smithi081.stdout:6/800: write d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f86 [316918,54787] 0 2024-03-22T07:18:41.229 INFO:tasks.workunit.client.0.smithi081.stdout:7/777: dwrite d9/d10/d16/d1e/f58 [0,4194304] 0 2024-03-22T07:18:41.232 INFO:tasks.workunit.client.0.smithi081.stdout:7/778: mkdir d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d10c 0 2024-03-22T07:18:41.270 INFO:tasks.workunit.client.0.smithi081.stdout:7/779: dread d9/d10/d2a/d73/f8b [4194304,4194304] 0 2024-03-22T07:18:41.271 INFO:tasks.workunit.client.0.smithi081.stdout:7/780: rename d9/d10/d4b/l103 to d9/dc/df6/l10d 0 2024-03-22T07:18:41.274 INFO:tasks.workunit.client.0.smithi081.stdout:6/801: write d3/dcc/fed [1012210,123139] 0 2024-03-22T07:18:41.274 INFO:tasks.workunit.client.0.smithi081.stdout:9/736: dwrite d2/d73/d85/f6d [0,4194304] 0 2024-03-22T07:18:41.274 INFO:tasks.workunit.client.0.smithi081.stdout:9/737: write d2/d73/d85/f4a [787489,113379] 0 2024-03-22T07:18:41.276 INFO:tasks.workunit.client.0.smithi081.stdout:6/802: link d3/dcc/fed d3/d67/db2/d40/db9/d63/db4/fff 0 2024-03-22T07:18:41.277 INFO:tasks.workunit.client.0.smithi081.stdout:9/738: mknod d2/d10/d30/d57/d66/d75/d94/dae/de6/cf1 0 2024-03-22T07:18:41.278 INFO:tasks.workunit.client.0.smithi081.stdout:7/781: mknod d9/d10/d53/d81/d59/d89/dd4/de0/d6d/c10e 0 2024-03-22T07:18:41.307 INFO:tasks.workunit.client.0.smithi081.stdout:6/803: dwrite d3/d67/db2/d40/db9/d63/d26/f1c [0,4194304] 0 2024-03-22T07:18:41.307 INFO:tasks.workunit.client.0.smithi081.stdout:6/804: fdatasync d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f78 0 2024-03-22T07:18:41.307 INFO:tasks.workunit.client.0.smithi081.stdout:7/782: dwrite d9/d10/d2a/fcd [0,4194304] 0 2024-03-22T07:18:41.310 INFO:tasks.workunit.client.0.smithi081.stdout:6/805: mkdir d3/d67/db2/d40/db9/d63/d5e/d9d/da6/d100 0 2024-03-22T07:18:41.312 INFO:tasks.workunit.client.0.smithi081.stdout:7/783: unlink d9/dc/c13 0 2024-03-22T07:18:41.312 INFO:tasks.workunit.client.0.smithi081.stdout:7/784: dread - d9/d10/d53/d81/d59/d89/dd4/de0/f108 zero size 2024-03-22T07:18:41.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:41 smithi081 ceph-mon[113965]: pgmap v216: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 69 MiB/s rd, 86 MiB/s wr, 786 op/s 2024-03-22T07:18:41.331 INFO:tasks.workunit.client.0.smithi081.stdout:6/806: dwrite d3/da/de/f27 [0,4194304] 0 2024-03-22T07:18:41.332 INFO:tasks.workunit.client.0.smithi081.stdout:6/807: write d3/da/fd [655477,58954] 0 2024-03-22T07:18:41.332 INFO:tasks.workunit.client.0.smithi081.stdout:6/808: dread - d3/d67/db2/d40/da8/fe9 zero size 2024-03-22T07:18:41.332 INFO:tasks.workunit.client.0.smithi081.stdout:7/785: dwrite d9/dc/f38 [4194304,4194304] 0 2024-03-22T07:18:41.333 INFO:tasks.workunit.client.0.smithi081.stdout:7/786: mknod d9/c10f 0 2024-03-22T07:18:41.334 INFO:tasks.workunit.client.0.smithi081.stdout:7/787: getdents d9/d10/d2a/d73 0 2024-03-22T07:18:41.334 INFO:tasks.workunit.client.0.smithi081.stdout:7/788: creat d9/d10/d2a/f110 x:0 0 0 2024-03-22T07:18:41.334 INFO:tasks.workunit.client.0.smithi081.stdout:7/789: chown d9/de4 91521 1 2024-03-22T07:18:41.343 INFO:tasks.workunit.client.0.smithi081.stdout:6/809: write d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f5f [2044153,105010] 0 2024-03-22T07:18:41.345 INFO:tasks.workunit.client.0.smithi081.stdout:7/790: write d9/d10/d53/d81/d59/d89/dd4/de0/fe7 [3196345,35798] 0 2024-03-22T07:18:41.345 INFO:tasks.workunit.client.0.smithi081.stdout:6/810: creat d3/d67/db2/df7/f101 x:0 0 0 2024-03-22T07:18:41.346 INFO:tasks.workunit.client.0.smithi081.stdout:7/791: rename d9/d10/cad to d9/d3f/d52/c111 0 2024-03-22T07:18:41.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:41 smithi149 ceph-mon[83447]: pgmap v216: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 69 MiB/s rd, 86 MiB/s wr, 786 op/s 2024-03-22T07:18:41.355 INFO:tasks.workunit.client.0.smithi081.stdout:7/792: write d9/d3f/db3/fc6 [375103,45924] 0 2024-03-22T07:18:41.357 INFO:tasks.workunit.client.0.smithi081.stdout:7/793: mknod d9/d10/d16/d1e/d6f/c112 0 2024-03-22T07:18:41.357 INFO:tasks.workunit.client.0.smithi081.stdout:7/794: stat d9/dc/f1a 0 2024-03-22T07:18:41.359 INFO:tasks.workunit.client.0.smithi081.stdout:7/795: link d9/d10/d16/c2f d9/d10/d16/c113 0 2024-03-22T07:18:41.361 INFO:tasks.workunit.client.0.smithi081.stdout:7/796: rename d9/d10/d53/d81/c51 to d9/d10/d16/d1e/c114 0 2024-03-22T07:18:41.362 INFO:tasks.workunit.client.0.smithi081.stdout:7/797: creat d9/de4/f115 x:0 0 0 2024-03-22T07:18:41.362 INFO:tasks.workunit.client.0.smithi081.stdout:7/798: chown d9/f75 9776490 1 2024-03-22T07:18:41.363 INFO:tasks.workunit.client.0.smithi081.stdout:6/811: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f6b [0,4194304] 0 2024-03-22T07:18:41.363 INFO:tasks.workunit.client.0.smithi081.stdout:7/799: mknod d9/d10/d53/d81/d59/d89/dd4/de0/d70/c116 0 2024-03-22T07:18:41.365 INFO:tasks.workunit.client.0.smithi081.stdout:6/812: creat d3/d67/db2/d40/f102 x:0 0 0 2024-03-22T07:18:41.365 INFO:tasks.workunit.client.0.smithi081.stdout:6/813: creat d3/d67/db2/d40/f103 x:0 0 0 2024-03-22T07:18:41.367 INFO:tasks.workunit.client.0.smithi081.stdout:6/814: creat d3/d67/db2/d40/db9/d63/f104 x:0 0 0 2024-03-22T07:18:41.368 INFO:tasks.workunit.client.0.smithi081.stdout:6/815: truncate d3/da/f8f 707452 0 2024-03-22T07:18:41.383 INFO:tasks.workunit.client.0.smithi081.stdout:8/679: sync 2024-03-22T07:18:41.383 INFO:tasks.workunit.client.0.smithi081.stdout:8/680: stat d2/d3 0 2024-03-22T07:18:41.386 INFO:tasks.workunit.client.0.smithi081.stdout:6/816: dwrite d3/d67/db2/d40/db9/d63/d26/f84 [0,4194304] 0 2024-03-22T07:18:41.386 INFO:tasks.workunit.client.0.smithi081.stdout:6/817: read - d3/da/de/fac zero size 2024-03-22T07:18:41.399 INFO:tasks.workunit.client.0.smithi081.stdout:6/818: dwrite d3/d67/db2/d40/db9/d63/faa [0,4194304] 0 2024-03-22T07:18:41.400 INFO:tasks.workunit.client.0.smithi081.stdout:6/819: dread - d3/d67/db2/d40/db9/d63/d26/d3f/fa1 zero size 2024-03-22T07:18:41.400 INFO:tasks.workunit.client.0.smithi081.stdout:6/820: write d3/d67/f93 [478338,107560] 0 2024-03-22T07:18:41.401 INFO:tasks.workunit.client.0.smithi081.stdout:6/821: mknod d3/da/c105 0 2024-03-22T07:18:41.402 INFO:tasks.workunit.client.0.smithi081.stdout:6/822: mkdir d3/ddb/d106 0 2024-03-22T07:18:41.402 INFO:tasks.workunit.client.0.smithi081.stdout:6/823: truncate d3/d67/db2/d40/db9/d63/d5e/f8c 674679 0 2024-03-22T07:18:41.417 INFO:tasks.workunit.client.0.smithi081.stdout:3/477: sync 2024-03-22T07:18:41.417 INFO:tasks.workunit.client.0.smithi081.stdout:4/873: sync 2024-03-22T07:18:41.418 INFO:tasks.workunit.client.0.smithi081.stdout:6/824: mkdir d3/da/de/db5/dfe/d107 0 2024-03-22T07:18:41.418 INFO:tasks.workunit.client.0.smithi081.stdout:6/825: write d3/d67/db2/d40/db9/d63/fef [871471,10447] 0 2024-03-22T07:18:41.418 INFO:tasks.workunit.client.0.smithi081.stdout:6/826: getdents d3/d67/db2/d40/db9/d63/d26/dfa 0 2024-03-22T07:18:41.418 INFO:tasks.workunit.client.0.smithi081.stdout:6/827: fsync d3/d67/db2/d40/db9/d63/d26/f84 0 2024-03-22T07:18:41.419 INFO:tasks.workunit.client.0.smithi081.stdout:3/478: rename db/de/d4b/d6a/d70/d3a/d5d/c5f to db/cab 0 2024-03-22T07:18:41.420 INFO:tasks.workunit.client.0.smithi081.stdout:6/828: mkdir d3/d67/db2/d40/db9/d63/d26/dfa/d108 0 2024-03-22T07:18:41.420 INFO:tasks.workunit.client.0.smithi081.stdout:6/829: readlink d3/d67/db2/d40/db9/d63/d26/d52/leb 0 2024-03-22T07:18:41.420 INFO:tasks.workunit.client.0.smithi081.stdout:6/830: mknod d3/da/de/db5/dfe/c109 0 2024-03-22T07:18:41.426 INFO:tasks.workunit.client.0.smithi081.stdout:4/874: write da/df/d38/f46 [2118116,47361] 0 2024-03-22T07:18:41.426 INFO:tasks.workunit.client.0.smithi081.stdout:4/875: write da/d18/f111 [110230,79482] 0 2024-03-22T07:18:41.430 INFO:tasks.workunit.client.0.smithi081.stdout:3/479: write db/f54 [3187618,50908] 0 2024-03-22T07:18:41.430 INFO:tasks.workunit.client.0.smithi081.stdout:3/480: fsync db/de/d4b/d6a/d70/d3a/d5d/fa8 0 2024-03-22T07:18:41.447 INFO:tasks.workunit.client.0.smithi081.stdout:3/481: dwrite db/d38/f69 [0,4194304] 0 2024-03-22T07:18:41.490 INFO:tasks.workunit.client.0.smithi081.stdout:6/831: link d3/da/c90 d3/d67/db2/d40/db9/d63/d7d/df3/c10a 0 2024-03-22T07:18:41.490 INFO:tasks.workunit.client.0.smithi081.stdout:6/832: chown d3/d67/db2/d40/db9/d63/d7d 1711353 1 2024-03-22T07:18:41.491 INFO:tasks.workunit.client.0.smithi081.stdout:4/876: getdents da/d18/d1b/d26/dce/dfd 0 2024-03-22T07:18:41.491 INFO:tasks.workunit.client.0.smithi081.stdout:4/877: readlink da/d18/d1b/d26/d3e/l72 0 2024-03-22T07:18:41.491 INFO:tasks.workunit.client.0.smithi081.stdout:4/878: chown da/d18/d1b/d26/l5d 11779 1 2024-03-22T07:18:41.491 INFO:tasks.workunit.client.0.smithi081.stdout:4/879: write da/d18/d1b/d1e/d5f/f11b [911841,96393] 0 2024-03-22T07:18:41.493 INFO:tasks.workunit.client.0.smithi081.stdout:3/482: mknod db/d38/d8d/cac 0 2024-03-22T07:18:41.501 INFO:tasks.workunit.client.0.smithi081.stdout:4/880: creat da/d18/d1b/d47/f123 x:0 0 0 2024-03-22T07:18:41.501 INFO:tasks.workunit.client.0.smithi081.stdout:4/881: fdatasync da/de6/db8/fbb 0 2024-03-22T07:18:41.501 INFO:tasks.workunit.client.0.smithi081.stdout:3/483: rmdir db/d38/d8d 39 2024-03-22T07:18:41.502 INFO:tasks.workunit.client.0.smithi081.stdout:6/833: symlink d3/d67/db2/d40/db9/d63/d26/d52/l10b 0 2024-03-22T07:18:41.503 INFO:tasks.workunit.client.0.smithi081.stdout:6/834: rename d3/d67/db2/d40/db9/f87 to d3/d67/db2/d40/db9/d63/db4/dc5/f10c 0 2024-03-22T07:18:41.506 INFO:tasks.workunit.client.0.smithi081.stdout:6/835: mknod d3/d67/db2/d40/db9/d63/d7d/c10d 0 2024-03-22T07:18:41.507 INFO:tasks.workunit.client.0.smithi081.stdout:6/836: write d3/da/de/f8e [536300,56427] 0 2024-03-22T07:18:41.515 INFO:tasks.workunit.client.0.smithi081.stdout:3/484: dwrite db/f1e [0,4194304] 0 2024-03-22T07:18:41.530 INFO:tasks.workunit.client.0.smithi081.stdout:3/485: dwrite db/f18 [0,4194304] 0 2024-03-22T07:18:41.530 INFO:tasks.workunit.client.0.smithi081.stdout:3/486: write db/de/d4b/d6a/d70/d3a/f7a [1019754,57985] 0 2024-03-22T07:18:41.530 INFO:tasks.workunit.client.0.smithi081.stdout:3/487: write db/de/f4c [722521,113475] 0 2024-03-22T07:18:41.530 INFO:tasks.workunit.client.0.smithi081.stdout:3/488: fdatasync db/d23/f52 0 2024-03-22T07:18:41.530 INFO:tasks.workunit.client.0.smithi081.stdout:3/489: getdents db/de/d4b/d6a/d70 0 2024-03-22T07:18:41.531 INFO:tasks.workunit.client.0.smithi081.stdout:3/490: chown db/d23/f3f 742133 1 2024-03-22T07:18:41.532 INFO:tasks.workunit.client.0.smithi081.stdout:3/491: mknod db/de/d4b/d6a/d70/cad 0 2024-03-22T07:18:41.546 INFO:tasks.workunit.client.0.smithi081.stdout:3/492: dwrite db/de/d4b/d6a/d70/f44 [0,4194304] 0 2024-03-22T07:18:41.567 INFO:tasks.workunit.client.0.smithi081.stdout:3/493: dwrite db/de/f41 [0,4194304] 0 2024-03-22T07:18:41.568 INFO:tasks.workunit.client.0.smithi081.stdout:3/494: write db/de/f4e [3033299,61187] 0 2024-03-22T07:18:41.568 INFO:tasks.workunit.client.0.smithi081.stdout:3/495: truncate db/de/d4b/d6a/d70/f32 1447342 0 2024-03-22T07:18:41.569 INFO:tasks.workunit.client.0.smithi081.stdout:3/496: mknod db/de/d4b/d6a/d70/cae 0 2024-03-22T07:18:41.576 INFO:tasks.workunit.client.0.smithi081.stdout:3/497: dread db/de/d4b/d6a/d70/f44 [0,4194304] 0 2024-03-22T07:18:41.576 INFO:tasks.workunit.client.0.smithi081.stdout:3/498: chown db/f54 252230513 1 2024-03-22T07:18:41.577 INFO:tasks.workunit.client.0.smithi081.stdout:3/499: rename db/de/d4b/d6a/d70/f55 to db/d38/d8d/faf 0 2024-03-22T07:18:41.577 INFO:tasks.workunit.client.0.smithi081.stdout:3/500: fsync db/de/f7d 0 2024-03-22T07:18:41.578 INFO:tasks.workunit.client.0.smithi081.stdout:3/501: mkdir db/de/d4b/db0 0 2024-03-22T07:18:41.578 INFO:tasks.workunit.client.0.smithi081.stdout:3/502: stat db/de/la5 0 2024-03-22T07:18:41.579 INFO:tasks.workunit.client.0.smithi081.stdout:3/503: truncate db/de/d4b/d6a/d70/f40 1182438 0 2024-03-22T07:18:41.579 INFO:tasks.workunit.client.0.smithi081.stdout:3/504: symlink db/de/d4b/db0/lb1 0 2024-03-22T07:18:41.589 INFO:tasks.workunit.client.0.smithi081.stdout:3/505: read db/d38/f69 [3013300,37700] 0 2024-03-22T07:18:41.594 INFO:tasks.workunit.client.0.smithi081.stdout:3/506: dread db/f13 [0,4194304] 0 2024-03-22T07:18:41.594 INFO:tasks.workunit.client.0.smithi081.stdout:3/507: chown db/d38/d8d/cac 3902344 1 2024-03-22T07:18:41.595 INFO:tasks.workunit.client.0.smithi081.stdout:3/508: creat db/d23/fb2 x:0 0 0 2024-03-22T07:18:41.597 INFO:tasks.workunit.client.0.smithi081.stdout:3/509: dread db/d23/f61 [0,4194304] 0 2024-03-22T07:18:41.597 INFO:tasks.workunit.client.0.smithi081.stdout:3/510: mknod db/cb3 0 2024-03-22T07:18:41.600 INFO:tasks.workunit.client.0.smithi081.stdout:3/511: dread db/d23/f28 [0,4194304] 0 2024-03-22T07:18:41.603 INFO:tasks.workunit.client.0.smithi081.stdout:3/512: dread db/de/d4b/d6a/d70/d3a/f46 [0,4194304] 0 2024-03-22T07:18:41.604 INFO:tasks.workunit.client.0.smithi081.stdout:3/513: dread db/de/d4b/d6a/d70/f32 [0,4194304] 0 2024-03-22T07:18:41.605 INFO:tasks.workunit.client.0.smithi081.stdout:3/514: mkdir db/d38/d8d/db4 0 2024-03-22T07:18:41.605 INFO:tasks.workunit.client.0.smithi081.stdout:3/515: dread - db/d38/d8d/faf zero size 2024-03-22T07:18:41.605 INFO:tasks.workunit.client.0.smithi081.stdout:3/516: getdents db/de/d4b/d6a/d70 0 2024-03-22T07:18:41.608 INFO:tasks.workunit.client.0.smithi081.stdout:3/517: rename db/d38/f9e to db/de/d4b/d6a/d70/d3a/d5d/fb5 0 2024-03-22T07:18:41.608 INFO:tasks.workunit.client.0.smithi081.stdout:3/518: mknod db/de/d4b/d6a/d70/d3a/cb6 0 2024-03-22T07:18:41.608 INFO:tasks.workunit.client.0.smithi081.stdout:3/519: truncate db/de/f29 4880672 0 2024-03-22T07:18:41.608 INFO:tasks.workunit.client.0.smithi081.stdout:3/520: write db/de/d4b/d6a/f89 [855206,67577] 0 2024-03-22T07:18:41.614 INFO:tasks.workunit.client.0.smithi081.stdout:3/521: dread db/d38/f6c [0,4194304] 0 2024-03-22T07:18:41.614 INFO:tasks.workunit.client.0.smithi081.stdout:3/522: mkdir db/d38/da1/db7 0 2024-03-22T07:18:41.619 INFO:tasks.workunit.client.0.smithi081.stdout:3/523: getdents db/d38 0 2024-03-22T07:18:41.619 INFO:tasks.workunit.client.0.smithi081.stdout:3/524: write db/de/d4b/d6a/f7c [585307,127173] 0 2024-03-22T07:18:41.620 INFO:tasks.workunit.client.0.smithi081.stdout:3/525: mkdir db/de/db8 0 2024-03-22T07:18:41.623 INFO:tasks.workunit.client.0.smithi081.stdout:1/654: sync 2024-03-22T07:18:41.624 INFO:tasks.workunit.client.0.smithi081.stdout:2/580: sync 2024-03-22T07:18:41.624 INFO:tasks.workunit.client.0.smithi081.stdout:1/655: symlink d6/d18/d1f/d32/ldd 0 2024-03-22T07:18:41.628 INFO:tasks.workunit.client.0.smithi081.stdout:2/581: mknod da/db/d39/d47/d3b/d41/cbc 0 2024-03-22T07:18:41.628 INFO:tasks.workunit.client.0.smithi081.stdout:1/656: mkdir d6/d18/d1f/d24/dde 0 2024-03-22T07:18:41.629 INFO:tasks.workunit.client.0.smithi081.stdout:1/657: mknod d6/d18/d1f/d76/cdf 0 2024-03-22T07:18:41.630 INFO:tasks.workunit.client.0.smithi081.stdout:5/792: sync 2024-03-22T07:18:41.630 INFO:tasks.workunit.client.0.smithi081.stdout:5/793: chown d4/d10/d20/d25/db8/f31 0 1 2024-03-22T07:18:41.630 INFO:tasks.workunit.client.0.smithi081.stdout:5/794: truncate d4/d9/d90/ffa 5140963 0 2024-03-22T07:18:41.638 INFO:tasks.workunit.client.0.smithi081.stdout:2/582: write da/db/d39/d46/fb6 [2926882,47206] 0 2024-03-22T07:18:41.640 INFO:tasks.workunit.client.0.smithi081.stdout:2/583: link da/d11/d21/f53 da/db/d39/d46/d42/db4/fbd 0 2024-03-22T07:18:41.642 INFO:tasks.workunit.client.0.smithi081.stdout:3/526: dwrite db/de/f4e [0,4194304] 0 2024-03-22T07:18:41.642 INFO:tasks.workunit.client.0.smithi081.stdout:5/795: write d4/d10/d20/dd2/ddf/ff6 [1403404,51677] 0 2024-03-22T07:18:41.642 INFO:tasks.workunit.client.0.smithi081.stdout:5/796: chown d4/df/d73/d8c/la0 5 1 2024-03-22T07:18:41.643 INFO:tasks.workunit.client.0.smithi081.stdout:2/584: dread da/d11/d21/fa2 [0,4194304] 0 2024-03-22T07:18:41.643 INFO:tasks.workunit.client.0.smithi081.stdout:3/527: rename db/d23/c95 to db/de/d4b/cb9 0 2024-03-22T07:18:41.644 INFO:tasks.workunit.client.0.smithi081.stdout:1/658: dwrite d6/d18/d1f/d32/f59 [0,4194304] 0 2024-03-22T07:18:41.645 INFO:tasks.workunit.client.0.smithi081.stdout:5/797: mkdir d4/d75/d98/d103 0 2024-03-22T07:18:41.646 INFO:tasks.workunit.client.0.smithi081.stdout:1/659: getdents d6/d18/d1f/d4c/d70/d7d 0 2024-03-22T07:18:41.646 INFO:tasks.workunit.client.0.smithi081.stdout:3/528: truncate db/d38/f69 231652 0 2024-03-22T07:18:41.648 INFO:tasks.workunit.client.0.smithi081.stdout:1/660: getdents d6/d18/d1f/d24/d48 0 2024-03-22T07:18:41.653 INFO:tasks.workunit.client.0.smithi081.stdout:1/661: dread d6/f2a [0,4194304] 0 2024-03-22T07:18:41.653 INFO:tasks.workunit.client.0.smithi081.stdout:2/585: write da/f28 [2715219,8740] 0 2024-03-22T07:18:41.654 INFO:tasks.workunit.client.0.smithi081.stdout:1/662: truncate d6/d18/d1f/d24/d48/d6e/f75 1224905 0 2024-03-22T07:18:41.657 INFO:tasks.workunit.client.0.smithi081.stdout:1/663: link d6/d18/d41/lb6 d6/d18/d41/le0 0 2024-03-22T07:18:41.657 INFO:tasks.workunit.client.0.smithi081.stdout:1/664: chown d6/d18/d1f/d24/dde 198242737 1 2024-03-22T07:18:41.657 INFO:tasks.workunit.client.0.smithi081.stdout:3/529: dread db/de/f24 [8388608,4194304] 0 2024-03-22T07:18:41.658 INFO:tasks.workunit.client.0.smithi081.stdout:3/530: creat db/d38/d8d/db4/fba x:0 0 0 2024-03-22T07:18:41.658 INFO:tasks.workunit.client.0.smithi081.stdout:1/665: creat d6/d18/da5/fe1 x:0 0 0 2024-03-22T07:18:41.658 INFO:tasks.workunit.client.0.smithi081.stdout:1/666: chown d6/d18/d1f/d32/dac/d30/f34 30 1 2024-03-22T07:18:41.661 INFO:tasks.workunit.client.0.smithi081.stdout:3/531: rmdir db/de 39 2024-03-22T07:18:41.667 INFO:tasks.workunit.client.0.smithi081.stdout:2/586: dread da/db/d39/d46/f44 [0,4194304] 0 2024-03-22T07:18:41.667 INFO:tasks.workunit.client.0.smithi081.stdout:2/587: write da/d11/d21/f90 [382042,116688] 0 2024-03-22T07:18:41.668 INFO:tasks.workunit.client.0.smithi081.stdout:2/588: creat da/d11/d21/d71/fbe x:0 0 0 2024-03-22T07:18:41.687 INFO:tasks.workunit.client.0.smithi081.stdout:1/667: rmdir d6/d18/d41 39 2024-03-22T07:18:41.690 INFO:tasks.workunit.client.0.smithi081.stdout:1/668: rmdir d6/d6a/d85/db8 0 2024-03-22T07:18:41.692 INFO:tasks.workunit.client.0.smithi081.stdout:1/669: link d6/d18/d1f/d24/daa/ldb d6/d18/le2 0 2024-03-22T07:18:41.692 INFO:tasks.workunit.client.0.smithi081.stdout:1/670: readlink d6/d18/d60/ld0 0 2024-03-22T07:18:41.694 INFO:tasks.workunit.client.0.smithi081.stdout:1/671: mkdir d6/d18/de3 0 2024-03-22T07:18:41.696 INFO:tasks.workunit.client.0.smithi081.stdout:3/532: dwrite db/f5c [0,4194304] 0 2024-03-22T07:18:41.696 INFO:tasks.workunit.client.0.smithi081.stdout:3/533: mknod db/d38/da1/db7/cbb 0 2024-03-22T07:18:41.697 INFO:tasks.workunit.client.0.smithi081.stdout:3/534: creat db/de/d4b/d6a/d70/d3a/d5d/fbc x:0 0 0 2024-03-22T07:18:41.700 INFO:tasks.workunit.client.0.smithi081.stdout:2/589: dwrite da/db/d39/d46/fa7 [0,4194304] 0 2024-03-22T07:18:41.701 INFO:tasks.workunit.client.0.smithi081.stdout:2/590: dread - da/d56/f6b zero size 2024-03-22T07:18:41.704 INFO:tasks.workunit.client.0.smithi081.stdout:1/672: link d6/d18/d1f/d24/d48/c50 d6/d18/d1f/d32/dac/d30/d93/ce4 0 2024-03-22T07:18:41.705 INFO:tasks.workunit.client.0.smithi081.stdout:1/673: chown d6/d18/d60/cca 0 1 2024-03-22T07:18:41.708 INFO:tasks.workunit.client.0.smithi081.stdout:1/674: link d6/d18/d1f/d24/d48/d5c/f7e d6/fe5 0 2024-03-22T07:18:41.708 INFO:tasks.workunit.client.0.smithi081.stdout:1/675: dread - d6/d6a/fb2 zero size 2024-03-22T07:18:41.708 INFO:tasks.workunit.client.0.smithi081.stdout:1/676: chown d6/d6a/f8f 14 1 2024-03-22T07:18:41.708 INFO:tasks.workunit.client.0.smithi081.stdout:1/677: truncate d6/d18/d1f/f72 874292 0 2024-03-22T07:18:41.709 INFO:tasks.workunit.client.0.smithi081.stdout:1/678: truncate d6/f67 159754 0 2024-03-22T07:18:41.709 INFO:tasks.workunit.client.0.smithi081.stdout:1/679: readlink d6/d18/d41/d49/l58 0 2024-03-22T07:18:41.709 INFO:tasks.workunit.client.0.smithi081.stdout:1/680: creat d6/d18/d1f/d76/fe6 x:0 0 0 2024-03-22T07:18:41.712 INFO:tasks.workunit.client.0.smithi081.stdout:1/681: mkdir d6/d18/d1f/d24/d48/d6e/de7 0 2024-03-22T07:18:41.712 INFO:tasks.workunit.client.0.smithi081.stdout:1/682: dread - d6/d18/d41/fd1 zero size 2024-03-22T07:18:41.712 INFO:tasks.workunit.client.0.smithi081.stdout:1/683: write d6/f7b [822277,33593] 0 2024-03-22T07:18:41.720 INFO:tasks.workunit.client.0.smithi081.stdout:2/591: dwrite da/d11/d21/f4e [0,4194304] 0 2024-03-22T07:18:41.722 INFO:tasks.workunit.client.0.smithi081.stdout:1/684: dread d6/d18/d1f/d32/dac/d30/f34 [0,4194304] 0 2024-03-22T07:18:41.722 INFO:tasks.workunit.client.0.smithi081.stdout:1/685: getdents d6/d18/d1f/d24/d48/d6e/de7 0 2024-03-22T07:18:41.730 INFO:tasks.workunit.client.0.smithi081.stdout:1/686: write d6/fc [2879643,129136] 0 2024-03-22T07:18:41.739 INFO:tasks.workunit.client.0.smithi081.stdout:2/592: dwrite da/d11/f4d [0,4194304] 0 2024-03-22T07:18:41.739 INFO:tasks.workunit.client.0.smithi081.stdout:2/593: write da/db/d39/d47/d3b/d41/d50/f62 [2025820,54273] 0 2024-03-22T07:18:41.739 INFO:tasks.workunit.client.0.smithi081.stdout:2/594: write da/f13 [2181541,13981] 0 2024-03-22T07:18:41.743 INFO:tasks.workunit.client.0.smithi081.stdout:1/687: link d6/d6a/f9d d6/d18/d1f/d4c/fe8 0 2024-03-22T07:18:41.743 INFO:tasks.workunit.client.0.smithi081.stdout:1/688: readlink d6/d18/d41/le0 0 2024-03-22T07:18:41.743 INFO:tasks.workunit.client.0.smithi081.stdout:1/689: readlink d6/d18/d41/lb6 0 2024-03-22T07:18:41.747 INFO:tasks.workunit.client.0.smithi081.stdout:2/595: rename da/db/d39/d47/d3b/d41/d50/d6a/db2/lb7 to da/db/d39/lbf 0 2024-03-22T07:18:41.749 INFO:tasks.workunit.client.0.smithi081.stdout:2/596: unlink da/db/c7b 0 2024-03-22T07:18:41.749 INFO:tasks.workunit.client.0.smithi081.stdout:2/597: fdatasync da/db/d39/d47/f3d 0 2024-03-22T07:18:41.749 INFO:tasks.workunit.client.0.smithi081.stdout:2/598: truncate f6 4498079 0 2024-03-22T07:18:41.750 INFO:tasks.workunit.client.0.smithi081.stdout:2/599: rename da/db/d39/d46/d7e/c87 to da/db/d39/d47/d3b/d64/cc0 0 2024-03-22T07:18:41.752 INFO:tasks.workunit.client.0.smithi081.stdout:2/600: mknod da/db/d39/cc1 0 2024-03-22T07:18:41.754 INFO:tasks.workunit.client.0.smithi081.stdout:2/601: mkdir da/db/d39/d47/d3b/d41/dab/dac/dc2 0 2024-03-22T07:18:41.765 INFO:tasks.workunit.client.0.smithi081.stdout:1/690: dwrite d6/d18/d1f/d4c/f9f [0,4194304] 0 2024-03-22T07:18:41.765 INFO:tasks.workunit.client.0.smithi081.stdout:1/691: truncate d6/d18/d1f/d24/d48/d5c/fb7 500761 0 2024-03-22T07:18:41.777 INFO:tasks.workunit.client.0.smithi081.stdout:2/602: dwrite da/d11/f1b [0,4194304] 0 2024-03-22T07:18:41.779 INFO:tasks.workunit.client.0.smithi081.stdout:2/603: mkdir da/d11/d21/d71/dc3 0 2024-03-22T07:18:41.781 INFO:tasks.workunit.client.0.smithi081.stdout:2/604: truncate da/d11/f1f 2942725 0 2024-03-22T07:18:41.781 INFO:tasks.workunit.client.0.smithi081.stdout:2/605: chown da/db/d39/d47/d3b/d41/dab/dac/lb1 7 1 2024-03-22T07:18:41.788 INFO:tasks.workunit.client.0.smithi081.stdout:1/692: dwrite d6/f6d [4194304,4194304] 0 2024-03-22T07:18:41.788 INFO:tasks.workunit.client.0.smithi081.stdout:1/693: creat d6/d18/d1f/d32/dac/d30/d93/fe9 x:0 0 0 2024-03-22T07:18:41.788 INFO:tasks.workunit.client.0.smithi081.stdout:1/694: readlink d6/d18/d1f/d4c/d70/dd2/ld9 0 2024-03-22T07:18:41.788 INFO:tasks.workunit.client.0.smithi081.stdout:1/695: fsync d6/d18/d1f/d24/f5f 0 2024-03-22T07:18:41.789 INFO:tasks.workunit.client.0.smithi081.stdout:1/696: truncate d6/d18/d1f/d24/d48/d5c/fda 50846 0 2024-03-22T07:18:41.790 INFO:tasks.workunit.client.0.smithi081.stdout:1/697: symlink d6/d18/d1f/d32/dac/lea 0 2024-03-22T07:18:41.791 INFO:tasks.workunit.client.0.smithi081.stdout:1/698: creat d6/d18/d41/feb x:0 0 0 2024-03-22T07:18:41.792 INFO:tasks.workunit.client.0.smithi081.stdout:1/699: rmdir d6/d18/d1f/d24/d48 39 2024-03-22T07:18:41.809 INFO:tasks.workunit.client.0.smithi081.stdout:1/700: dwrite d6/d18/d1f/d76/fe6 [0,4194304] 0 2024-03-22T07:18:41.809 INFO:tasks.workunit.client.0.smithi081.stdout:1/701: mkdir d6/d6a/d85/dec 0 2024-03-22T07:18:41.811 INFO:tasks.workunit.client.0.smithi081.stdout:1/702: mknod d6/d18/d1f/d32/dac/ced 0 2024-03-22T07:18:41.812 INFO:tasks.workunit.client.0.smithi081.stdout:1/703: creat d6/d18/d1f/d24/d48/d6e/de7/fee x:0 0 0 2024-03-22T07:18:41.813 INFO:tasks.workunit.client.0.smithi081.stdout:1/704: rmdir d6/d18/d1f/d24 39 2024-03-22T07:18:41.813 INFO:tasks.workunit.client.0.smithi081.stdout:1/705: fsync d6/d6a/fb2 0 2024-03-22T07:18:41.815 INFO:tasks.workunit.client.0.smithi081.stdout:1/706: unlink d6/d18/d1f/d24/f2b 0 2024-03-22T07:18:41.817 INFO:tasks.workunit.client.0.smithi081.stdout:1/707: truncate d6/d18/d60/fc9 599208 0 2024-03-22T07:18:41.817 INFO:tasks.workunit.client.0.smithi081.stdout:0/729: sync 2024-03-22T07:18:41.819 INFO:tasks.workunit.client.0.smithi081.stdout:1/708: unlink d6/d18/d1f/fcf 0 2024-03-22T07:18:41.819 INFO:tasks.workunit.client.0.smithi081.stdout:0/730: rename d3/dc/d40/c6f to d3/dc/d40/d69/d92/df4/c102 0 2024-03-22T07:18:41.825 INFO:tasks.workunit.client.0.smithi081.stdout:1/709: dread d6/fc [0,4194304] 0 2024-03-22T07:18:41.827 INFO:tasks.workunit.client.0.smithi081.stdout:0/731: dread d3/d6a/dd5/fff [0,4194304] 0 2024-03-22T07:18:41.828 INFO:tasks.workunit.client.0.smithi081.stdout:0/732: creat d3/dc/d2b/d4d/f103 x:0 0 0 2024-03-22T07:18:41.832 INFO:tasks.workunit.client.0.smithi081.stdout:0/733: rename d3/d32/f8f to d3/dc/d77/f104 0 2024-03-22T07:18:41.833 INFO:tasks.workunit.client.0.smithi081.stdout:0/734: unlink d3/dc/d33/d62/lc7 0 2024-03-22T07:18:41.833 INFO:tasks.workunit.client.0.smithi081.stdout:0/735: chown d3/dc/d40/fe3 173104 1 2024-03-22T07:18:41.835 INFO:tasks.workunit.client.0.smithi081.stdout:0/736: read d3/dc/d40/d69/d92/dc1/d74/fb7 [634230,43983] 0 2024-03-22T07:18:41.835 INFO:tasks.workunit.client.0.smithi081.stdout:0/737: mknod d3/dc/d40/d69/d92/dcf/c105 0 2024-03-22T07:18:41.836 INFO:tasks.workunit.client.0.smithi081.stdout:0/738: read d3/dc/f73 [220336,62206] 0 2024-03-22T07:18:41.846 INFO:tasks.workunit.client.0.smithi081.stdout:1/710: link d6/d18/d1f/d4c/fe8 d6/d18/d1f/d32/fef 0 2024-03-22T07:18:41.847 INFO:tasks.workunit.client.0.smithi081.stdout:1/711: symlink d6/d6a/lf0 0 2024-03-22T07:18:41.847 INFO:tasks.workunit.client.0.smithi081.stdout:0/739: write d3/dc/d2b/d4d/f5e [464977,128625] 0 2024-03-22T07:18:41.848 INFO:tasks.workunit.client.0.smithi081.stdout:0/740: symlink d3/d9b/da4/l106 0 2024-03-22T07:18:41.850 INFO:tasks.workunit.client.0.smithi081.stdout:0/741: rename d3/dc/d40/d69/l6e to d3/dc/d2b/d4d/l107 0 2024-03-22T07:18:41.851 INFO:tasks.workunit.client.0.smithi081.stdout:0/742: mknod d3/dc/d40/d69/db3/c108 0 2024-03-22T07:18:41.852 INFO:tasks.workunit.client.0.smithi081.stdout:0/743: symlink d3/d6a/dd5/l109 0 2024-03-22T07:18:41.852 INFO:tasks.workunit.client.0.smithi081.stdout:0/744: getdents d3/dc/d40/d69/d92/dcf/dd6 0 2024-03-22T07:18:41.855 INFO:tasks.workunit.client.0.smithi081.stdout:1/712: write d6/d18/d1f/d24/d48/d6e/fd6 [2512956,71771] 0 2024-03-22T07:18:41.855 INFO:tasks.workunit.client.0.smithi081.stdout:1/713: dread - d6/d18/d1f/d32/dac/d30/d93/fd5 zero size 2024-03-22T07:18:41.857 INFO:tasks.workunit.client.0.smithi081.stdout:1/714: creat d6/d18/de3/ff1 x:0 0 0 2024-03-22T07:18:41.857 INFO:tasks.workunit.client.0.smithi081.stdout:1/715: write d6/d18/d1f/d32/dac/d30/d93/fe9 [189905,68061] 0 2024-03-22T07:18:41.864 INFO:tasks.workunit.client.0.smithi081.stdout:0/745: dread d3/d9b/da4/fe6 [0,4194304] 0 2024-03-22T07:18:41.866 INFO:tasks.workunit.client.0.smithi081.stdout:0/746: rename d3/dc/d2b/fba to d3/dc/d40/d69/d92/dc1/df6/f10a 0 2024-03-22T07:18:41.873 INFO:tasks.workunit.client.0.smithi081.stdout:1/716: dwrite d6/d18/d1f/db0/fb4 [0,4194304] 0 2024-03-22T07:18:41.879 INFO:tasks.workunit.client.0.smithi081.stdout:0/747: write d3/dc/d2b/f2f [1269556,6056] 0 2024-03-22T07:18:41.879 INFO:tasks.workunit.client.0.smithi081.stdout:1/717: write d6/d18/d1f/d32/dac/d30/da8/fc7 [4567039,80383] 0 2024-03-22T07:18:41.879 INFO:tasks.workunit.client.0.smithi081.stdout:1/718: chown d6/d18/d1f/d4c/d70/dd2 1802 1 2024-03-22T07:18:41.879 INFO:tasks.workunit.client.0.smithi081.stdout:1/719: fsync d6/d18/d1f/d24/f3b 0 2024-03-22T07:18:41.881 INFO:tasks.workunit.client.0.smithi081.stdout:0/748: mkdir d3/d9b/d10b 0 2024-03-22T07:18:41.881 INFO:tasks.workunit.client.0.smithi081.stdout:0/749: creat d3/dc/f10c x:0 0 0 2024-03-22T07:18:41.882 INFO:tasks.workunit.client.0.smithi081.stdout:1/720: mkdir d6/d18/d1f/d32/dac/d30/df2 0 2024-03-22T07:18:41.882 INFO:tasks.workunit.client.0.smithi081.stdout:1/721: readlink d6/d6a/d85/l8e 0 2024-03-22T07:18:41.883 INFO:tasks.workunit.client.0.smithi081.stdout:1/722: mknod d6/d18/d1f/d32/dac/d30/d93/dbc/cf3 0 2024-03-22T07:18:41.883 INFO:tasks.workunit.client.0.smithi081.stdout:1/723: truncate d6/d6a/fb9 1499135 0 2024-03-22T07:18:41.884 INFO:tasks.workunit.client.0.smithi081.stdout:1/724: mknod d6/d18/d1f/db0/cf4 0 2024-03-22T07:18:41.886 INFO:tasks.workunit.client.0.smithi081.stdout:0/750: write d3/dc/d33/f47 [1934478,13938] 0 2024-03-22T07:18:41.892 INFO:tasks.workunit.client.0.smithi081.stdout:1/725: write d6/d18/d1f/d4c/f6c [1782149,55553] 0 2024-03-22T07:18:41.892 INFO:tasks.workunit.client.0.smithi081.stdout:1/726: chown d6/d6a/d85/f90 1280845 1 2024-03-22T07:18:41.894 INFO:tasks.workunit.client.0.smithi081.stdout:1/727: dread d6/d18/d1f/d24/d48/d6e/fae [0,4194304] 0 2024-03-22T07:18:41.898 INFO:tasks.workunit.client.0.smithi081.stdout:9/739: sync 2024-03-22T07:18:41.900 INFO:tasks.workunit.client.0.smithi081.stdout:9/740: link d2/d5d/d5a/c72 d2/d10/d30/d57/dad/cf2 0 2024-03-22T07:18:41.900 INFO:tasks.workunit.client.0.smithi081.stdout:9/741: truncate d2/d2b/d49/d86/fb7 5081901 0 2024-03-22T07:18:41.900 INFO:tasks.workunit.client.0.smithi081.stdout:9/742: truncate d2/d73/d85/f4a 1825453 0 2024-03-22T07:18:41.900 INFO:tasks.workunit.client.0.smithi081.stdout:9/743: chown d2/d5/d9d/f7b 2590486 1 2024-03-22T07:18:41.912 INFO:tasks.workunit.client.0.smithi081.stdout:1/728: dread d6/f6d [4194304,4194304] 0 2024-03-22T07:18:41.928 INFO:tasks.workunit.client.0.smithi081.stdout:0/751: dwrite d3/d4e/f54 [0,4194304] 0 2024-03-22T07:18:41.928 INFO:tasks.workunit.client.0.smithi081.stdout:1/729: dwrite d6/d18/d41/f46 [0,4194304] 0 2024-03-22T07:18:41.930 INFO:tasks.workunit.client.0.smithi081.stdout:0/752: rename d3/dc/d40/d69/d92/dc1/lb8 to d3/dc/d33/dd1/l10d 0 2024-03-22T07:18:41.931 INFO:tasks.workunit.client.0.smithi081.stdout:0/753: stat d3/dc/l26 0 2024-03-22T07:18:41.931 INFO:tasks.workunit.client.0.smithi081.stdout:0/754: symlink d3/dc/d40/d69/d92/dcf/l10e 0 2024-03-22T07:18:41.932 INFO:tasks.workunit.client.0.smithi081.stdout:0/755: mknod d3/dc/d2b/d39/dc9/c10f 0 2024-03-22T07:18:41.932 INFO:tasks.workunit.client.0.smithi081.stdout:0/756: write d3/d9b/fe4 [666523,129649] 0 2024-03-22T07:18:41.934 INFO:tasks.workunit.client.0.smithi081.stdout:6/837: sync 2024-03-22T07:18:41.934 INFO:tasks.workunit.client.0.smithi081.stdout:8/681: sync 2024-03-22T07:18:41.934 INFO:tasks.workunit.client.0.smithi081.stdout:2/606: sync 2024-03-22T07:18:41.934 INFO:tasks.workunit.client.0.smithi081.stdout:7/800: sync 2024-03-22T07:18:41.934 INFO:tasks.workunit.client.0.smithi081.stdout:3/535: sync 2024-03-22T07:18:41.934 INFO:tasks.workunit.client.0.smithi081.stdout:5/798: sync 2024-03-22T07:18:41.934 INFO:tasks.workunit.client.0.smithi081.stdout:6/838: stat d3/d67/db2/d40/db9/d63/d5e/d9d/da6 0 2024-03-22T07:18:41.935 INFO:tasks.workunit.client.0.smithi081.stdout:4/882: sync 2024-03-22T07:18:41.935 INFO:tasks.workunit.client.0.smithi081.stdout:2/607: write da/db/d39/d47/d3b/d41/d50/d6a/d70/f40 [9386621,128249] 0 2024-03-22T07:18:41.935 INFO:tasks.workunit.client.0.smithi081.stdout:7/801: creat d9/d10/d53/d81/d59/dbe/f117 x:0 0 0 2024-03-22T07:18:41.935 INFO:tasks.workunit.client.0.smithi081.stdout:3/536: rename db/d23/f67 to db/de/d4b/d6a/fbd 0 2024-03-22T07:18:41.937 INFO:tasks.workunit.client.0.smithi081.stdout:5/799: creat d4/d10/d20/db2/f104 x:0 0 0 2024-03-22T07:18:41.937 INFO:tasks.workunit.client.0.smithi081.stdout:5/800: write d4/d75/fbd [385837,9832] 0 2024-03-22T07:18:41.939 INFO:tasks.workunit.client.0.smithi081.stdout:6/839: unlink d3/da/de/f28 0 2024-03-22T07:18:41.940 INFO:tasks.workunit.client.0.smithi081.stdout:4/883: creat da/de6/f124 x:0 0 0 2024-03-22T07:18:41.941 INFO:tasks.workunit.client.0.smithi081.stdout:2/608: mknod da/db/d39/d46/d42/db4/dbb/cc4 0 2024-03-22T07:18:41.942 INFO:tasks.workunit.client.0.smithi081.stdout:2/609: readlink da/d11/d21/l4f 0 2024-03-22T07:18:41.943 INFO:tasks.workunit.client.0.smithi081.stdout:1/730: dread d6/d18/d1f/f87 [0,4194304] 0 2024-03-22T07:18:41.945 INFO:tasks.workunit.client.0.smithi081.stdout:8/682: rename d2/fa9 to d2/d3/d19/d1e/d13/d22/db8/fe8 0 2024-03-22T07:18:41.945 INFO:tasks.workunit.client.0.smithi081.stdout:3/537: mknod db/d38/cbe 0 2024-03-22T07:18:41.945 INFO:tasks.workunit.client.0.smithi081.stdout:3/538: fdatasync db/d23/f61 0 2024-03-22T07:18:41.945 INFO:tasks.workunit.client.0.smithi081.stdout:5/801: creat d4/d10/daf/dfe/f105 x:0 0 0 2024-03-22T07:18:41.946 INFO:tasks.workunit.client.0.smithi081.stdout:2/610: rmdir da/d11 39 2024-03-22T07:18:41.947 INFO:tasks.workunit.client.0.smithi081.stdout:2/611: getdents da/db/d39/d47/d3b/d41/d50/d9a 0 2024-03-22T07:18:41.947 INFO:tasks.workunit.client.0.smithi081.stdout:7/802: symlink d9/d10/d4b/l118 0 2024-03-22T07:18:41.949 INFO:tasks.workunit.client.0.smithi081.stdout:1/731: mknod d6/d18/d1f/d32/dac/d30/df2/cf5 0 2024-03-22T07:18:41.949 INFO:tasks.workunit.client.0.smithi081.stdout:8/683: creat d2/d3/d19/fe9 x:0 0 0 2024-03-22T07:18:41.950 INFO:tasks.workunit.client.0.smithi081.stdout:3/539: mknod db/d38/d8d/db4/cbf 0 2024-03-22T07:18:41.951 INFO:tasks.workunit.client.0.smithi081.stdout:3/540: stat db/de/d4b/d6a/d70/d3a/d5d/fa8 0 2024-03-22T07:18:41.953 INFO:tasks.workunit.client.0.smithi081.stdout:5/802: creat d4/d10/d20/d25/db8/f106 x:0 0 0 2024-03-22T07:18:41.954 INFO:tasks.workunit.client.0.smithi081.stdout:4/884: truncate da/d18/d1b/d26/dce/dfd/f51 2511912 0 2024-03-22T07:18:41.956 INFO:tasks.workunit.client.0.smithi081.stdout:7/803: mkdir d9/d10/d119 0 2024-03-22T07:18:41.957 INFO:tasks.workunit.client.0.smithi081.stdout:4/885: mknod da/d18/d1b/d34/c125 0 2024-03-22T07:18:41.960 INFO:tasks.workunit.client.0.smithi081.stdout:5/803: unlink d4/d10/l1e 0 2024-03-22T07:18:41.960 INFO:tasks.workunit.client.0.smithi081.stdout:5/804: readlink d4/d10/d20/lc6 0 2024-03-22T07:18:41.961 INFO:tasks.workunit.client.0.smithi081.stdout:7/804: creat d9/d10/d53/d81/d59/d89/dd4/de0/d70/f11a x:0 0 0 2024-03-22T07:18:41.962 INFO:tasks.workunit.client.0.smithi081.stdout:0/757: dwrite d3/dc/d33/dd1/dec/f100 [0,4194304] 0 2024-03-22T07:18:41.964 INFO:tasks.workunit.client.0.smithi081.stdout:8/684: dread d2/d3/d23/f3b [0,4194304] 0 2024-03-22T07:18:41.964 INFO:tasks.workunit.client.0.smithi081.stdout:2/612: fsync da/db/d39/d47/d3b/d41/d50/d6a/d70/f95 0 2024-03-22T07:18:41.964 INFO:tasks.workunit.client.0.smithi081.stdout:5/805: read d4/d10/d20/d25/da3/fd6 [755820,39257] 0 2024-03-22T07:18:41.966 INFO:tasks.workunit.client.0.smithi081.stdout:4/886: rename da/d18/d1b/d26/dce/dfd/c10b to da/df/d38/dc9/c126 0 2024-03-22T07:18:41.970 INFO:tasks.workunit.client.0.smithi081.stdout:6/840: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/f98 [0,4194304] 0 2024-03-22T07:18:41.972 INFO:tasks.workunit.client.0.smithi081.stdout:0/758: dread d3/dc/d2b/d39/f3c [0,4194304] 0 2024-03-22T07:18:41.973 INFO:tasks.workunit.client.0.smithi081.stdout:3/541: dwrite db/fa6 [0,4194304] 0 2024-03-22T07:18:41.974 INFO:tasks.workunit.client.0.smithi081.stdout:8/685: truncate d2/f15 3652436 0 2024-03-22T07:18:41.983 INFO:tasks.workunit.client.0.smithi081.stdout:8/686: truncate d2/d3/f4 453895 0 2024-03-22T07:18:41.984 INFO:tasks.workunit.client.0.smithi081.stdout:2/613: mkdir da/db/d39/d47/d3b/dc5 0 2024-03-22T07:18:41.985 INFO:tasks.workunit.client.0.smithi081.stdout:5/806: rename d4/d10/d20/d4f/l61 to d4/d10/d20/l107 0 2024-03-22T07:18:41.986 INFO:tasks.workunit.client.0.smithi081.stdout:8/687: creat d2/d3/d28/d2b/d61/fea x:0 0 0 2024-03-22T07:18:41.987 INFO:tasks.workunit.client.0.smithi081.stdout:8/688: chown d2/d3/d9e/fbd 229 1 2024-03-22T07:18:41.990 INFO:tasks.workunit.client.0.smithi081.stdout:0/759: rename d3/dc/d2b/d39/cf8 to d3/dc/d40/d69/d92/dc1/d74/c110 0 2024-03-22T07:18:41.990 INFO:tasks.workunit.client.0.smithi081.stdout:0/760: write d3/dc/d33/f7d [2847324,27203] 0 2024-03-22T07:18:41.992 INFO:tasks.workunit.client.0.smithi081.stdout:5/807: link d4/d10/d20/db2/d2f/l3f d4/df/d73/d8c/l108 0 2024-03-22T07:18:41.992 INFO:tasks.workunit.client.0.smithi081.stdout:5/808: truncate d4/d10/d20/dd2/ddf/fed 720998 0 2024-03-22T07:18:41.992 INFO:tasks.workunit.client.0.smithi081.stdout:5/809: creat d4/d10/daf/dfe/f109 x:0 0 0 2024-03-22T07:18:41.993 INFO:tasks.workunit.client.0.smithi081.stdout:0/761: dread d3/d32/f70 [0,4194304] 0 2024-03-22T07:18:41.993 INFO:tasks.workunit.client.0.smithi081.stdout:0/762: fdatasync d3/d6a/fa1 0 2024-03-22T07:18:41.993 INFO:tasks.workunit.client.0.smithi081.stdout:0/763: stat d3/d4e/f88 0 2024-03-22T07:18:41.993 INFO:tasks.workunit.client.0.smithi081.stdout:7/805: dwrite d9/d10/d4b/d9b/fef [0,4194304] 0 2024-03-22T07:18:41.994 INFO:tasks.workunit.client.0.smithi081.stdout:2/614: rename da/db/d39/d45/c8b to da/db/d39/d46/d42/db4/dbb/cc6 0 2024-03-22T07:18:41.995 INFO:tasks.workunit.client.0.smithi081.stdout:1/732: dwrite d6/d18/d1f/d32/dac/d30/f7f [0,4194304] 0 2024-03-22T07:18:41.995 INFO:tasks.workunit.client.0.smithi081.stdout:1/733: write d6/d18/d1f/d24/d48/d5c/f73 [3830199,67362] 0 2024-03-22T07:18:42.005 INFO:tasks.workunit.client.0.smithi081.stdout:8/689: rename d2/d3/d28/fcc to d2/d3/d19/d1e/d13/d22/d2f/d73/feb 0 2024-03-22T07:18:42.008 INFO:tasks.workunit.client.0.smithi081.stdout:0/764: mkdir d3/dc/d40/d69/d111 0 2024-03-22T07:18:42.009 INFO:tasks.workunit.client.0.smithi081.stdout:3/542: dwrite db/f5c [0,4194304] 0 2024-03-22T07:18:42.009 INFO:tasks.workunit.client.0.smithi081.stdout:4/887: dwrite da/df/d38/f68 [0,4194304] 0 2024-03-22T07:18:42.009 INFO:tasks.workunit.client.0.smithi081.stdout:4/888: chown da/d18/d1b/d26/dce 41605518 1 2024-03-22T07:18:42.009 INFO:tasks.workunit.client.0.smithi081.stdout:7/806: symlink d9/d14/l11b 0 2024-03-22T07:18:42.009 INFO:tasks.workunit.client.0.smithi081.stdout:6/841: dwrite d3/d67/db2/d40/db9/f6f [0,4194304] 0 2024-03-22T07:18:42.011 INFO:tasks.workunit.client.0.smithi081.stdout:6/842: dread d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f86 [0,4194304] 0 2024-03-22T07:18:42.011 INFO:tasks.workunit.client.0.smithi081.stdout:5/810: rename d4/df/d73/d8c/f62 to d4/d10/d20/dd2/dea/f10a 0 2024-03-22T07:18:42.013 INFO:tasks.workunit.client.0.smithi081.stdout:0/765: mkdir d3/d4e/dee/d112 0 2024-03-22T07:18:42.013 INFO:tasks.workunit.client.0.smithi081.stdout:0/766: chown d3/dc/d2b/d39/f3c 879 1 2024-03-22T07:18:42.015 INFO:tasks.workunit.client.0.smithi081.stdout:3/543: symlink db/d38/d8d/db4/lc0 0 2024-03-22T07:18:42.019 INFO:tasks.workunit.client.0.smithi081.stdout:4/889: link da/d3d/f11a da/d3d/def/f127 0 2024-03-22T07:18:42.019 INFO:tasks.workunit.client.0.smithi081.stdout:4/890: creat da/df/d38/dd8/d4f/daf/f128 x:0 0 0 2024-03-22T07:18:42.022 INFO:tasks.workunit.client.0.smithi081.stdout:6/843: rmdir d3/d67/db2/d40/db9/d63/d5e/dc0/ddf 0 2024-03-22T07:18:42.023 INFO:tasks.workunit.client.0.smithi081.stdout:0/767: write d3/dc/f8d [6975130,28266] 0 2024-03-22T07:18:42.025 INFO:tasks.workunit.client.0.smithi081.stdout:5/811: creat d4/d10/d20/d25/da3/db5/f10b x:0 0 0 2024-03-22T07:18:42.026 INFO:tasks.workunit.client.0.smithi081.stdout:8/690: dwrite d2/d3/d19/d1e/f43 [0,4194304] 0 2024-03-22T07:18:42.026 INFO:tasks.workunit.client.0.smithi081.stdout:3/544: symlink db/d38/da1/db7/lc1 0 2024-03-22T07:18:42.031 INFO:tasks.workunit.client.0.smithi081.stdout:6/844: truncate d3/da/f59 4035068 0 2024-03-22T07:18:42.031 INFO:tasks.workunit.client.0.smithi081.stdout:4/891: dread da/d18/d1b/d26/f43 [0,4194304] 0 2024-03-22T07:18:42.034 INFO:tasks.workunit.client.0.smithi081.stdout:7/807: rename d9/d10/d16/d1e/c31 to d9/d3f/d52/d83/c11c 0 2024-03-22T07:18:42.037 INFO:tasks.workunit.client.0.smithi081.stdout:8/691: mknod d2/d3/d19/d1e/d63/da0/db7/cec 0 2024-03-22T07:18:42.037 INFO:tasks.workunit.client.0.smithi081.stdout:8/692: getdents d2/d3/d19/d1e/d63/da0/db7/dc8/dc0 0 2024-03-22T07:18:42.038 INFO:tasks.workunit.client.0.smithi081.stdout:3/545: mknod db/de/d4b/d6a/d70/d3a/cc2 0 2024-03-22T07:18:42.039 INFO:tasks.workunit.client.0.smithi081.stdout:3/546: chown db/de/l81 62 1 2024-03-22T07:18:42.042 INFO:tasks.workunit.client.0.smithi081.stdout:8/693: mkdir d2/d3/d28/d2b/d61/d7e/ded 0 2024-03-22T07:18:42.044 INFO:tasks.workunit.client.0.smithi081.stdout:3/547: link db/de/f4c db/d38/da1/fc3 0 2024-03-22T07:18:42.046 INFO:tasks.workunit.client.0.smithi081.stdout:6/845: mkdir d3/da/d10e 0 2024-03-22T07:18:42.046 INFO:tasks.workunit.client.0.smithi081.stdout:6/846: chown d3/d67/db2/d40/db9/d63/db4/dc5 3743 1 2024-03-22T07:18:42.046 INFO:tasks.workunit.client.0.smithi081.stdout:7/808: mkdir d9/d10/d16/d11d 0 2024-03-22T07:18:42.047 INFO:tasks.workunit.client.0.smithi081.stdout:8/694: creat d2/d3/d19/d64/fee x:0 0 0 2024-03-22T07:18:42.047 INFO:tasks.workunit.client.0.smithi081.stdout:8/695: truncate d2/d3/d19/fe9 273602 0 2024-03-22T07:18:42.048 INFO:tasks.workunit.client.0.smithi081.stdout:5/812: dwrite d4/d9/fc2 [0,4194304] 0 2024-03-22T07:18:42.048 INFO:tasks.workunit.client.0.smithi081.stdout:5/813: getdents d4/df/d22/d49 0 2024-03-22T07:18:42.049 INFO:tasks.workunit.client.0.smithi081.stdout:6/847: mkdir d3/d67/db2/d40/db9/d10f 0 2024-03-22T07:18:42.050 INFO:tasks.workunit.client.0.smithi081.stdout:6/848: read d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/fae [530200,115439] 0 2024-03-22T07:18:42.054 INFO:tasks.workunit.client.0.smithi081.stdout:7/809: creat d9/d10/d16/d1e/d6f/f11e x:0 0 0 2024-03-22T07:18:42.054 INFO:tasks.workunit.client.0.smithi081.stdout:0/768: dwrite d3/dc/fd3 [0,4194304] 0 2024-03-22T07:18:42.054 INFO:tasks.workunit.client.0.smithi081.stdout:0/769: readlink d3/d9b/ldc 0 2024-03-22T07:18:42.056 INFO:tasks.workunit.client.0.smithi081.stdout:8/696: symlink d2/d3/d19/d1e/d13/d22/d2f/d73/lef 0 2024-03-22T07:18:42.056 INFO:tasks.workunit.client.0.smithi081.stdout:0/770: write d3/f44 [314968,77673] 0 2024-03-22T07:18:42.056 INFO:tasks.workunit.client.0.smithi081.stdout:3/548: read db/f77 [2616882,5614] 0 2024-03-22T07:18:42.056 INFO:tasks.workunit.client.0.smithi081.stdout:3/549: write db/de/f4c [1092117,17500] 0 2024-03-22T07:18:42.056 INFO:tasks.workunit.client.0.smithi081.stdout:5/814: mkdir d4/d10/d20/d25/db8/dcf/d10c 0 2024-03-22T07:18:42.059 INFO:tasks.workunit.client.0.smithi081.stdout:3/550: dread db/de/d4b/d6a/d70/d3a/f6b [0,4194304] 0 2024-03-22T07:18:42.059 INFO:tasks.workunit.client.0.smithi081.stdout:8/697: symlink d2/d3/d19/d1e/d13/d22/d2f/d73/lf0 0 2024-03-22T07:18:42.061 INFO:tasks.workunit.client.0.smithi081.stdout:5/815: mkdir d4/d9/d90/d10d 0 2024-03-22T07:18:42.063 INFO:tasks.workunit.client.0.smithi081.stdout:3/551: rename db/de/d4b/d6a/f9c to db/de/d4b/d6a/d70/fc4 0 2024-03-22T07:18:42.065 INFO:tasks.workunit.client.0.smithi081.stdout:3/552: chown db/de/d4b/d6a/d70/c99 10982 1 2024-03-22T07:18:42.065 INFO:tasks.workunit.client.0.smithi081.stdout:1/734: dwrite d6/d18/d1f/d32/dac/d30/f7f [0,4194304] 0 2024-03-22T07:18:42.066 INFO:tasks.workunit.client.0.smithi081.stdout:8/698: unlink d2/d3/d28/d2b/d61/f6b 0 2024-03-22T07:18:42.067 INFO:tasks.workunit.client.0.smithi081.stdout:8/699: read d2/d3/d19/d1e/d13/d66/d70/fd0 [598059,2033] 0 2024-03-22T07:18:42.067 INFO:tasks.workunit.client.0.smithi081.stdout:8/700: write d2/d3/d19/d1e/f8a [497711,31767] 0 2024-03-22T07:18:42.067 INFO:tasks.workunit.client.0.smithi081.stdout:5/816: creat d4/d75/d98/d103/f10e x:0 0 0 2024-03-22T07:18:42.068 INFO:tasks.workunit.client.0.smithi081.stdout:7/810: read d9/d10/d16/d1e/d6f/ffb [1743871,82927] 0 2024-03-22T07:18:42.069 INFO:tasks.workunit.client.0.smithi081.stdout:0/771: dread d3/dc/fd3 [0,4194304] 0 2024-03-22T07:18:42.071 INFO:tasks.workunit.client.0.smithi081.stdout:8/701: unlink d2/d3/d19/lbf 0 2024-03-22T07:18:42.072 INFO:tasks.workunit.client.0.smithi081.stdout:5/817: truncate d4/d75/faa 604460 0 2024-03-22T07:18:42.079 INFO:tasks.workunit.client.0.smithi081.stdout:3/553: dread db/d38/f51 [0,4194304] 0 2024-03-22T07:18:42.079 INFO:tasks.workunit.client.0.smithi081.stdout:3/554: dread - db/d23/fb2 zero size 2024-03-22T07:18:42.081 INFO:tasks.workunit.client.0.smithi081.stdout:6/849: dwrite d3/da/f59 [0,4194304] 0 2024-03-22T07:18:42.082 INFO:tasks.workunit.client.0.smithi081.stdout:6/850: readlink d3/d67/db2/d40/db9/d63/l42 0 2024-03-22T07:18:42.083 INFO:tasks.workunit.client.0.smithi081.stdout:6/851: write d3/d67/db2/d40/db9/d63/d5e/f82 [4194542,89032] 0 2024-03-22T07:18:42.085 INFO:tasks.workunit.client.0.smithi081.stdout:6/852: link d3/d67/db2/c3d d3/d67/db2/d40/db9/d63/db4/c110 0 2024-03-22T07:18:42.085 INFO:tasks.workunit.client.0.smithi081.stdout:6/853: fsync d3/da/f57 0 2024-03-22T07:18:42.086 INFO:tasks.workunit.client.0.smithi081.stdout:6/854: mkdir d3/da/d111 0 2024-03-22T07:18:42.087 INFO:tasks.workunit.client.0.smithi081.stdout:6/855: chown d3/d67/db2/d40/db9/d63/c5c 16366 1 2024-03-22T07:18:42.088 INFO:tasks.workunit.client.0.smithi081.stdout:6/856: rmdir d3/dcc 39 2024-03-22T07:18:42.095 INFO:tasks.workunit.client.0.smithi081.stdout:7/811: write d9/d10/d53/d81/d59/d89/dd4/de0/f57 [2763491,19697] 0 2024-03-22T07:18:42.102 INFO:tasks.workunit.client.0.smithi081.stdout:6/857: write d3/da/de/db5/fcf [2063707,29738] 0 2024-03-22T07:18:42.123 INFO:tasks.workunit.client.0.smithi081.stdout:8/702: dwrite d2/d3/d28/f82 [4194304,4194304] 0 2024-03-22T07:18:42.123 INFO:tasks.workunit.client.0.smithi081.stdout:0/772: dwrite d3/dc/d40/fb6 [0,4194304] 0 2024-03-22T07:18:42.123 INFO:tasks.workunit.client.0.smithi081.stdout:0/773: truncate d3/dc/d77/f104 945724 0 2024-03-22T07:18:42.123 INFO:tasks.workunit.client.0.smithi081.stdout:5/818: dwrite d4/df/d22/d4d/fe3 [0,4194304] 0 2024-03-22T07:18:42.123 INFO:tasks.workunit.client.0.smithi081.stdout:5/819: write d4/d10/f3c [2490346,81187] 0 2024-03-22T07:18:42.123 INFO:tasks.workunit.client.0.smithi081.stdout:6/858: dwrite d3/d67/db2/fb8 [0,4194304] 0 2024-03-22T07:18:42.123 INFO:tasks.workunit.client.0.smithi081.stdout:6/859: fsync d3/d67/db2/d40/db9/d63/d26/d3f/d4c/fe8 0 2024-03-22T07:18:42.126 INFO:tasks.workunit.client.0.smithi081.stdout:0/774: getdents d3/dc/d77 0 2024-03-22T07:18:42.128 INFO:tasks.workunit.client.0.smithi081.stdout:3/555: dread db/de/f4c [0,4194304] 0 2024-03-22T07:18:42.131 INFO:tasks.workunit.client.0.smithi081.stdout:8/703: write d2/d3/d19/d1e/d13/d22/f34 [170671,122049] 0 2024-03-22T07:18:42.137 INFO:tasks.workunit.client.0.smithi081.stdout:1/735: rename d6/d18/d1f to d6/d6a/d85/df6 0 2024-03-22T07:18:42.137 INFO:tasks.workunit.client.0.smithi081.stdout:1/736: chown d6/fc 94630873 1 2024-03-22T07:18:42.138 INFO:tasks.workunit.client.0.smithi081.stdout:1/737: unlink d6/d6a/d85/df6/d32/dbe/dc4/cc8 0 2024-03-22T07:18:42.139 INFO:tasks.workunit.client.0.smithi081.stdout:1/738: chown d6/d6a/d85/df6/d32/dac/lea 4304 1 2024-03-22T07:18:42.139 INFO:tasks.workunit.client.0.smithi081.stdout:1/739: truncate d6/d6a/d85/df6/d76/fe6 2428522 0 2024-03-22T07:18:42.140 INFO:tasks.workunit.client.0.smithi081.stdout:1/740: dread - d6/d6a/d85/df6/d32/dac/f95 zero size 2024-03-22T07:18:42.145 INFO:tasks.workunit.client.0.smithi081.stdout:8/704: rmdir d2/d3/d19/d1e/d63/da0/db7 39 2024-03-22T07:18:42.145 INFO:tasks.workunit.client.0.smithi081.stdout:8/705: stat d2/d3/d19/d64/fee 0 2024-03-22T07:18:42.145 INFO:tasks.workunit.client.0.smithi081.stdout:5/820: mknod d4/df/c10f 0 2024-03-22T07:18:42.145 INFO:tasks.workunit.client.0.smithi081.stdout:5/821: stat d4/d10/d20/db2 0 2024-03-22T07:18:42.148 INFO:tasks.workunit.client.0.smithi081.stdout:8/706: symlink d2/d3/d19/d1e/d13/d66/d70/lf1 0 2024-03-22T07:18:42.150 INFO:tasks.workunit.client.0.smithi081.stdout:3/556: dwrite db/de/d4b/d6a/d70/d3a/d5d/fbc [0,4194304] 0 2024-03-22T07:18:42.152 INFO:tasks.workunit.client.0.smithi081.stdout:7/812: rename d9/d3f/d52/c93 to d9/d10/d16/d84/c11f 0 2024-03-22T07:18:42.156 INFO:tasks.workunit.client.0.smithi081.stdout:7/813: chown d9/d10/d16/l48 3264 1 2024-03-22T07:18:42.158 INFO:tasks.workunit.client.0.smithi081.stdout:6/860: rename d3/d67/db2/d40/db9/d63/d7d/ce3 to d3/da/de/db5/c112 0 2024-03-22T07:18:42.159 INFO:tasks.workunit.client.0.smithi081.stdout:3/557: dread db/de/d4b/d6a/d70/f92 [0,4194304] 0 2024-03-22T07:18:42.160 INFO:tasks.workunit.client.0.smithi081.stdout:7/814: creat d9/d10/d9e/f120 x:0 0 0 2024-03-22T07:18:42.160 INFO:tasks.workunit.client.0.smithi081.stdout:7/815: chown d9/d10/d16/f74 12448 1 2024-03-22T07:18:42.161 INFO:tasks.workunit.client.0.smithi081.stdout:6/861: rmdir d3/d67/db2/d40/db9/d63/d26/d3f 39 2024-03-22T07:18:42.164 INFO:tasks.workunit.client.0.smithi081.stdout:3/558: unlink db/de/d4b/d6a/d70/d3a/d5d/l75 0 2024-03-22T07:18:42.164 INFO:tasks.workunit.client.0.smithi081.stdout:3/559: fsync db/de/f4c 0 2024-03-22T07:18:42.164 INFO:tasks.workunit.client.0.smithi081.stdout:7/816: fdatasync d9/f3c 0 2024-03-22T07:18:42.165 INFO:tasks.workunit.client.0.smithi081.stdout:3/560: write db/de/d4b/d6a/f89 [196114,111668] 0 2024-03-22T07:18:42.165 INFO:tasks.workunit.client.0.smithi081.stdout:3/561: chown db/de/d4b/d6a 98902 1 2024-03-22T07:18:42.166 INFO:tasks.workunit.client.0.smithi081.stdout:6/862: creat d3/d67/db2/d40/db9/d63/d26/dfa/f113 x:0 0 0 2024-03-22T07:18:42.167 INFO:tasks.workunit.client.0.smithi081.stdout:7/817: mknod d9/d10/d53/d81/d59/dbe/c121 0 2024-03-22T07:18:42.169 INFO:tasks.workunit.client.0.smithi081.stdout:3/562: link db/de/d4b/d6a/d70/l93 db/de/d4b/d6a/d70/d3a/lc5 0 2024-03-22T07:18:42.169 INFO:tasks.workunit.client.0.smithi081.stdout:3/563: fsync db/f13 0 2024-03-22T07:18:42.170 INFO:tasks.workunit.client.0.smithi081.stdout:5/822: dwrite d4/d9/f91 [0,4194304] 0 2024-03-22T07:18:42.170 INFO:tasks.workunit.client.0.smithi081.stdout:5/823: chown d4/df/de6/lf2 0 1 2024-03-22T07:18:42.170 INFO:tasks.workunit.client.0.smithi081.stdout:5/824: read d4/d10/d20/dd2/ddf/ff6 [1756248,43172] 0 2024-03-22T07:18:42.173 INFO:tasks.workunit.client.0.smithi081.stdout:6/863: stat d3/da/de/l49 0 2024-03-22T07:18:42.182 INFO:tasks.workunit.client.0.smithi081.stdout:1/741: dwrite d6/d6a/d85/df6/d32/dac/d30/f3d [0,4194304] 0 2024-03-22T07:18:42.183 INFO:tasks.workunit.client.0.smithi081.stdout:8/707: dwrite d2/d3/d19/d1e/d63/da0/db7/dc8/d7c/fc4 [0,4194304] 0 2024-03-22T07:18:42.183 INFO:tasks.workunit.client.0.smithi081.stdout:8/708: read d2/d3/d19/d1e/d13/f47 [1174104,128404] 0 2024-03-22T07:18:42.183 INFO:tasks.workunit.client.0.smithi081.stdout:8/709: chown d2/d3/d19/d1e/d63/da0/db7 864962 1 2024-03-22T07:18:42.183 INFO:tasks.workunit.client.0.smithi081.stdout:8/710: write d2/d3/d28/d2b/fac [3397650,28069] 0 2024-03-22T07:18:42.198 INFO:tasks.workunit.client.0.smithi081.stdout:1/742: dread d6/f35 [0,4194304] 0 2024-03-22T07:18:42.198 INFO:tasks.workunit.client.0.smithi081.stdout:1/743: write d6/d6a/d85/fbd [699764,85185] 0 2024-03-22T07:18:42.199 INFO:tasks.workunit.client.0.smithi081.stdout:1/744: symlink d6/d18/de3/lf7 0 2024-03-22T07:18:42.199 INFO:tasks.workunit.client.0.smithi081.stdout:1/745: dread - d6/d18/d41/fd1 zero size 2024-03-22T07:18:42.200 INFO:tasks.workunit.client.0.smithi081.stdout:1/746: link d6/d6a/d85/df6/d32/dac/lea d6/d6a/lf8 0 2024-03-22T07:18:42.201 INFO:tasks.workunit.client.0.smithi081.stdout:1/747: mkdir d6/d6a/d85/df6/d32/df9 0 2024-03-22T07:18:42.203 INFO:tasks.workunit.client.0.smithi081.stdout:1/748: symlink d6/d6a/d85/df6/d32/dac/d30/lfa 0 2024-03-22T07:18:42.204 INFO:tasks.workunit.client.0.smithi081.stdout:1/749: unlink d6/d6a/d85/df6/d32/cbb 0 2024-03-22T07:18:42.204 INFO:tasks.workunit.client.0.smithi081.stdout:1/750: chown d6/f2a 21 1 2024-03-22T07:18:42.206 INFO:tasks.workunit.client.0.smithi081.stdout:1/751: creat d6/d6a/d85/df6/d32/dac/d30/da8/ffb x:0 0 0 2024-03-22T07:18:42.207 INFO:tasks.workunit.client.0.smithi081.stdout:7/818: dwrite f1 [4194304,4194304] 0 2024-03-22T07:18:42.207 INFO:tasks.workunit.client.0.smithi081.stdout:1/752: creat d6/d6a/d85/df6/d32/dac/d30/ffc x:0 0 0 2024-03-22T07:18:42.208 INFO:tasks.workunit.client.0.smithi081.stdout:7/819: chown d9/d10/c76 155509526 1 2024-03-22T07:18:42.210 INFO:tasks.workunit.client.0.smithi081.stdout:8/711: dwrite d2/d3/d19/d1e/d13/d22/db8/fbc [0,4194304] 0 2024-03-22T07:18:42.210 INFO:tasks.workunit.client.0.smithi081.stdout:8/712: fdatasync d2/d3/d23/f3b 0 2024-03-22T07:18:42.212 INFO:tasks.workunit.client.0.smithi081.stdout:1/753: truncate d6/f17 2466044 0 2024-03-22T07:18:42.212 INFO:tasks.workunit.client.0.smithi081.stdout:5/825: dwrite d4/d10/d20/d25/db8/f106 [0,4194304] 0 2024-03-22T07:18:42.212 INFO:tasks.workunit.client.0.smithi081.stdout:3/564: dwrite db/de/f20 [0,4194304] 0 2024-03-22T07:18:42.213 INFO:tasks.workunit.client.0.smithi081.stdout:6/864: dwrite d3/da/de/f8e [0,4194304] 0 2024-03-22T07:18:42.213 INFO:tasks.workunit.client.0.smithi081.stdout:6/865: readlink d3/d67/db2/d40/db9/d63/d5e/d9d/da6/lb1 0 2024-03-22T07:18:42.214 INFO:tasks.workunit.client.0.smithi081.stdout:1/754: dread d6/f67 [0,4194304] 0 2024-03-22T07:18:42.221 INFO:tasks.workunit.client.0.smithi081.stdout:8/713: dread d2/d3/d19/d1e/f3c [0,4194304] 0 2024-03-22T07:18:42.222 INFO:tasks.workunit.client.0.smithi081.stdout:3/565: mknod db/de/cc6 0 2024-03-22T07:18:42.224 INFO:tasks.workunit.client.0.smithi081.stdout:8/714: truncate d2/d3/d19/d1e/f3e 586375 0 2024-03-22T07:18:42.225 INFO:tasks.workunit.client.0.smithi081.stdout:5/826: read d4/d10/d20/d25/f8b [7874756,102305] 0 2024-03-22T07:18:42.227 INFO:tasks.workunit.client.0.smithi081.stdout:8/715: symlink d2/d3/d19/d1e/d13/dba/lf2 0 2024-03-22T07:18:42.229 INFO:tasks.workunit.client.0.smithi081.stdout:8/716: unlink d2/d3/d19/d1e/d13/d22/d38/f44 0 2024-03-22T07:18:42.230 INFO:tasks.workunit.client.0.smithi081.stdout:8/717: creat d2/d3/d19/d64/ff3 x:0 0 0 2024-03-22T07:18:42.232 INFO:tasks.workunit.client.0.smithi081.stdout:8/718: mkdir d2/d3/d9e/df4 0 2024-03-22T07:18:42.232 INFO:tasks.workunit.client.0.smithi081.stdout:8/719: truncate d2/d3/d23/fa2 900142 0 2024-03-22T07:18:42.234 INFO:tasks.workunit.client.0.smithi081.stdout:8/720: rmdir d2/d3/d19/d1e/d63/da0/db7/dc8/dc0 0 2024-03-22T07:18:42.242 INFO:tasks.workunit.client.0.smithi081.stdout:5/827: dread d4/d10/f3c [0,4194304] 0 2024-03-22T07:18:42.242 INFO:tasks.workunit.client.0.smithi081.stdout:5/828: dread - d4/d10/d20/d25/da3/db5/f10b zero size 2024-03-22T07:18:42.242 INFO:tasks.workunit.client.0.smithi081.stdout:5/829: dread - d4/df/d73/d8c/dc0/dc9/ff1 zero size 2024-03-22T07:18:42.243 INFO:tasks.workunit.client.0.smithi081.stdout:5/830: creat d4/d10/d20/d25/db8/dcf/d10c/f110 x:0 0 0 2024-03-22T07:18:42.247 INFO:tasks.workunit.client.0.smithi081.stdout:1/755: write d6/d6a/d85/df6/d32/dac/d30/da8/fc7 [8606820,46175] 0 2024-03-22T07:18:42.247 INFO:tasks.workunit.client.0.smithi081.stdout:7/820: dwrite d9/d10/d2a/d73/f8b [0,4194304] 0 2024-03-22T07:18:42.247 INFO:tasks.workunit.client.0.smithi081.stdout:6/866: dwrite d3/da/de/db5/fcf [0,4194304] 0 2024-03-22T07:18:42.248 INFO:tasks.workunit.client.0.smithi081.stdout:3/566: dwrite db/de/d4b/d6a/d70/fc4 [0,4194304] 0 2024-03-22T07:18:42.249 INFO:tasks.workunit.client.0.smithi081.stdout:7/821: link d9/d10/d2a/c100 d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d10c/c122 0 2024-03-22T07:18:42.249 INFO:tasks.workunit.client.0.smithi081.stdout:7/822: dread - d9/d10/d53/d81/d59/d89/dd4/de0/f108 zero size 2024-03-22T07:18:42.249 INFO:tasks.workunit.client.0.smithi081.stdout:7/823: creat d9/d10/d53/f123 x:0 0 0 2024-03-22T07:18:42.251 INFO:tasks.workunit.client.0.smithi081.stdout:8/721: write d2/d3/d19/d1e/f3e [475437,46858] 0 2024-03-22T07:18:42.254 INFO:tasks.workunit.client.0.smithi081.stdout:6/867: symlink d3/da/de/db5/dfe/d107/l114 0 2024-03-22T07:18:42.254 INFO:tasks.workunit.client.0.smithi081.stdout:6/868: truncate d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f86 1291183 0 2024-03-22T07:18:42.257 INFO:tasks.workunit.client.0.smithi081.stdout:7/824: truncate d9/d3f/d52/d83/fba 19824 0 2024-03-22T07:18:42.257 INFO:tasks.workunit.client.0.smithi081.stdout:7/825: dread - d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/fea zero size 2024-03-22T07:18:42.258 INFO:tasks.workunit.client.0.smithi081.stdout:5/831: dwrite d4/f89 [0,4194304] 0 2024-03-22T07:18:42.258 INFO:tasks.workunit.client.0.smithi081.stdout:5/832: truncate d4/d10/d20/db2/fa2 301261 0 2024-03-22T07:18:42.260 INFO:tasks.workunit.client.0.smithi081.stdout:3/567: mknod db/d23/cc7 0 2024-03-22T07:18:42.260 INFO:tasks.workunit.client.0.smithi081.stdout:3/568: stat db/d38/f51 0 2024-03-22T07:18:42.260 INFO:tasks.workunit.client.0.smithi081.stdout:3/569: fsync db/d38/f69 0 2024-03-22T07:18:42.262 INFO:tasks.workunit.client.0.smithi081.stdout:6/869: symlink d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c/l115 0 2024-03-22T07:18:42.265 INFO:tasks.workunit.client.0.smithi081.stdout:8/722: dread d2/d3/d19/d1e/d13/d22/f51 [0,4194304] 0 2024-03-22T07:18:42.265 INFO:tasks.workunit.client.0.smithi081.stdout:8/723: write d2/d3/d19/f6e [5307714,6808] 0 2024-03-22T07:18:42.265 INFO:tasks.workunit.client.0.smithi081.stdout:7/826: unlink f1 0 2024-03-22T07:18:42.267 INFO:tasks.workunit.client.0.smithi081.stdout:3/570: creat db/d38/d8d/fc8 x:0 0 0 2024-03-22T07:18:42.271 INFO:tasks.workunit.client.0.smithi081.stdout:7/827: rmdir d9/d10/d16/d11d 0 2024-03-22T07:18:42.272 INFO:tasks.workunit.client.0.smithi081.stdout:6/870: creat d3/d67/db2/d40/db9/d63/d5e/f116 x:0 0 0 2024-03-22T07:18:42.272 INFO:tasks.workunit.client.0.smithi081.stdout:6/871: fsync d3/d67/db2/d40/db9/f85 0 2024-03-22T07:18:42.274 INFO:tasks.workunit.client.0.smithi081.stdout:7/828: creat d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/f124 x:0 0 0 2024-03-22T07:18:42.276 INFO:tasks.workunit.client.0.smithi081.stdout:3/571: link db/de/d4b/d6a/d70/fc4 db/d23/fc9 0 2024-03-22T07:18:42.276 INFO:tasks.workunit.client.0.smithi081.stdout:3/572: chown db/de/f4e 96899 1 2024-03-22T07:18:42.276 INFO:tasks.workunit.client.0.smithi081.stdout:5/833: dwrite d4/df/d22/d4d/fe3 [4194304,4194304] 0 2024-03-22T07:18:42.278 INFO:tasks.workunit.client.0.smithi081.stdout:8/724: dread d2/d3/f1b [0,4194304] 0 2024-03-22T07:18:42.282 INFO:tasks.workunit.client.0.smithi081.stdout:5/834: creat d4/d10/f111 x:0 0 0 2024-03-22T07:18:42.282 INFO:tasks.workunit.client.0.smithi081.stdout:5/835: stat d4/d10/d20/d25/db8/l44 0 2024-03-22T07:18:42.282 INFO:tasks.workunit.client.0.smithi081.stdout:6/872: dread d3/d67/db2/d40/db9/d63/d26/f44 [0,4194304] 0 2024-03-22T07:18:42.282 INFO:tasks.workunit.client.0.smithi081.stdout:7/829: symlink d9/d10/d53/d81/d59/d89/dd4/de0/d6d/l125 0 2024-03-22T07:18:42.283 INFO:tasks.workunit.client.0.smithi081.stdout:8/725: creat d2/d3/d23/ff5 x:0 0 0 2024-03-22T07:18:42.286 INFO:tasks.workunit.client.0.smithi081.stdout:6/873: getdents d3/da/de/db5 0 2024-03-22T07:18:42.287 INFO:tasks.workunit.client.0.smithi081.stdout:7/830: symlink d9/d10/d53/d81/d59/d89/dd4/de0/d70/l126 0 2024-03-22T07:18:42.288 INFO:tasks.workunit.client.0.smithi081.stdout:8/726: creat d2/d3/d19/d1e/d13/ff6 x:0 0 0 2024-03-22T07:18:42.288 INFO:tasks.workunit.client.0.smithi081.stdout:8/727: write d2/d3/d19/d1e/d13/d22/d2f/fd1 [1034134,117412] 0 2024-03-22T07:18:42.289 INFO:tasks.workunit.client.0.smithi081.stdout:5/836: mkdir d4/d10/d112 0 2024-03-22T07:18:42.289 INFO:tasks.workunit.client.0.smithi081.stdout:5/837: fsync d4/f89 0 2024-03-22T07:18:42.289 INFO:tasks.workunit.client.0.smithi081.stdout:5/838: fdatasync d4/d10/d20/db2/f28 0 2024-03-22T07:18:42.297 INFO:tasks.workunit.client.0.smithi081.stdout:8/728: dread d2/d3/d19/d1e/d13/d22/f6f [0,4194304] 0 2024-03-22T07:18:42.305 INFO:tasks.workunit.client.0.smithi081.stdout:8/729: read d2/d3/d28/d2b/f7a [69449,1439] 0 2024-03-22T07:18:42.343 INFO:tasks.workunit.client.0.smithi081.stdout:7/831: readlink d9/d10/l21 0 2024-03-22T07:18:42.350 INFO:tasks.workunit.client.0.smithi081.stdout:6/874: rmdir d3/d67/db2/d40/db9/d63/db4/dc5 39 2024-03-22T07:18:42.351 INFO:tasks.workunit.client.0.smithi081.stdout:1/756: rename d6/d6a/d85/df6/d32/dac/d30/da8/fc7 to d6/d6a/d85/df6/d24/da6/ffd 0 2024-03-22T07:18:42.352 INFO:tasks.workunit.client.0.smithi081.stdout:1/757: write d6/d6a/d85/df6/d24/d48/d6e/fae [293809,16467] 0 2024-03-22T07:18:42.352 INFO:tasks.workunit.client.0.smithi081.stdout:1/758: stat d6/d6a/d85/df6/d32/dac/d30/d93/fd5 0 2024-03-22T07:18:42.352 INFO:tasks.workunit.client.0.smithi081.stdout:1/759: write d6/d18/d41/fd1 [128936,48816] 0 2024-03-22T07:18:42.353 INFO:tasks.workunit.client.0.smithi081.stdout:1/760: fsync d6/d6a/d85/df6/d76/f79 0 2024-03-22T07:18:42.353 INFO:tasks.workunit.client.0.smithi081.stdout:1/761: creat d6/d6a/d85/df6/d32/ffe x:0 0 0 2024-03-22T07:18:42.363 INFO:tasks.workunit.client.0.smithi081.stdout:6/875: dread d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f97 [0,4194304] 0 2024-03-22T07:18:42.408 INFO:tasks.workunit.client.0.smithi081.stdout:5/839: getdents d4/d75/d98/d103 0 2024-03-22T07:18:42.408 INFO:tasks.workunit.client.0.smithi081.stdout:5/840: chown d4/d10/d20/db2/lc5 63519284 1 2024-03-22T07:18:42.411 INFO:tasks.workunit.client.0.smithi081.stdout:3/573: rename db/d38/cbe to db/de/d4b/d6a/cca 0 2024-03-22T07:18:42.412 INFO:tasks.workunit.client.0.smithi081.stdout:3/574: dread db/de/d4b/d6a/d70/f5b [0,4194304] 0 2024-03-22T07:18:42.418 INFO:tasks.workunit.client.0.smithi081.stdout:3/575: mkdir db/de/db8/dcb 0 2024-03-22T07:18:42.421 INFO:tasks.workunit.client.0.smithi081.stdout:6/876: unlink d3/da/de/cc2 0 2024-03-22T07:18:42.422 INFO:tasks.workunit.client.0.smithi081.stdout:3/576: mknod db/de/d4b/d6a/d70/d3a/d5d/ccc 0 2024-03-22T07:18:42.423 INFO:tasks.workunit.client.0.smithi081.stdout:6/877: creat d3/d67/db2/d40/db9/d63/dd2/f117 x:0 0 0 2024-03-22T07:18:42.425 INFO:tasks.workunit.client.0.smithi081.stdout:3/577: creat db/de/d4b/d6a/d70/fcd x:0 0 0 2024-03-22T07:18:42.425 INFO:tasks.workunit.client.0.smithi081.stdout:3/578: write db/de/d4b/d6a/d70/d3a/f8f [139199,31596] 0 2024-03-22T07:18:42.426 INFO:tasks.workunit.client.0.smithi081.stdout:3/579: mknod db/de/db8/dcb/cce 0 2024-03-22T07:18:42.428 INFO:tasks.workunit.client.0.smithi081.stdout:7/832: dwrite d9/d10/d16/fcb [0,4194304] 0 2024-03-22T07:18:42.429 INFO:tasks.workunit.client.0.smithi081.stdout:7/833: fdatasync d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/f124 0 2024-03-22T07:18:42.431 INFO:tasks.workunit.client.0.smithi081.stdout:7/834: mknod d9/d10/d16/db6/dd9/c127 0 2024-03-22T07:18:42.435 INFO:tasks.workunit.client.0.smithi081.stdout:7/835: symlink d9/d10/d4b/d9b/l128 0 2024-03-22T07:18:42.446 INFO:tasks.workunit.client.0.smithi081.stdout:6/878: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d4c/fe8 [0,4194304] 0 2024-03-22T07:18:42.446 INFO:tasks.workunit.client.0.smithi081.stdout:1/762: dwrite d6/d6a/f9d [0,4194304] 0 2024-03-22T07:18:42.448 INFO:tasks.workunit.client.0.smithi081.stdout:1/763: symlink d6/d18/d60/lff 0 2024-03-22T07:18:42.448 INFO:tasks.workunit.client.0.smithi081.stdout:6/879: symlink d3/d67/db2/d40/db9/d63/d26/dfa/d108/l118 0 2024-03-22T07:18:42.449 INFO:tasks.workunit.client.0.smithi081.stdout:7/836: read f6 [7012476,117041] 0 2024-03-22T07:18:42.450 INFO:tasks.workunit.client.0.smithi081.stdout:6/880: mknod d3/da/de/db5/dfe/c119 0 2024-03-22T07:18:42.455 INFO:tasks.workunit.client.0.smithi081.stdout:5/841: creat d4/d75/f113 x:0 0 0 2024-03-22T07:18:42.456 INFO:tasks.workunit.client.0.smithi081.stdout:5/842: stat d4/d10/d20/db2/d2f 0 2024-03-22T07:18:42.457 INFO:tasks.workunit.client.0.smithi081.stdout:5/843: link d4/d10/d20/d25/db8/l70 d4/d75/d98/l114 0 2024-03-22T07:18:42.469 INFO:tasks.workunit.client.0.smithi081.stdout:3/580: rename db/de/f71 to db/d38/da1/fcf 0 2024-03-22T07:18:42.471 INFO:tasks.workunit.client.0.smithi081.stdout:1/764: dwrite d6/d6a/d85/df6/d24/d48/fad [0,4194304] 0 2024-03-22T07:18:42.471 INFO:tasks.workunit.client.0.smithi081.stdout:5/844: write d4/d10/d20/db2/d2a/fde [3985736,129177] 0 2024-03-22T07:18:42.471 INFO:tasks.workunit.client.0.smithi081.stdout:3/581: creat db/de/fd0 x:0 0 0 2024-03-22T07:18:42.472 INFO:tasks.workunit.client.0.smithi081.stdout:3/582: chown db/d23/c63 207734089 1 2024-03-22T07:18:42.472 INFO:tasks.workunit.client.0.smithi081.stdout:9/744: sync 2024-03-22T07:18:42.472 INFO:tasks.workunit.client.0.smithi081.stdout:7/837: rename d9/d3f/d52/c7b to d9/d10/d16/d1e/d80/da1/dfc/c129 0 2024-03-22T07:18:42.472 INFO:tasks.workunit.client.0.smithi081.stdout:1/765: dread d6/fc [0,4194304] 0 2024-03-22T07:18:42.474 INFO:tasks.workunit.client.0.smithi081.stdout:5/845: mkdir d4/df/d22/d49/d115 0 2024-03-22T07:18:42.474 INFO:tasks.workunit.client.0.smithi081.stdout:5/846: stat d4/d10/daf 0 2024-03-22T07:18:42.474 INFO:tasks.workunit.client.0.smithi081.stdout:5/847: dread - d4/d10/d20/d4f/f5f zero size 2024-03-22T07:18:42.474 INFO:tasks.workunit.client.0.smithi081.stdout:3/583: creat db/de/d4b/db0/fd1 x:0 0 0 2024-03-22T07:18:42.476 INFO:tasks.workunit.client.0.smithi081.stdout:9/745: creat d2/d73/d85/d7a/d97/dda/ff3 x:0 0 0 2024-03-22T07:18:42.479 INFO:tasks.workunit.client.0.smithi081.stdout:5/848: unlink d4/d10/d20/d25/lee 0 2024-03-22T07:18:42.481 INFO:tasks.workunit.client.0.smithi081.stdout:3/584: mkdir db/de/db8/dd2 0 2024-03-22T07:18:42.481 INFO:tasks.workunit.client.0.smithi081.stdout:3/585: chown db/d23/c98 4 1 2024-03-22T07:18:42.482 INFO:tasks.workunit.client.0.smithi081.stdout:9/746: creat d2/d2b/ff4 x:0 0 0 2024-03-22T07:18:42.484 INFO:tasks.workunit.client.0.smithi081.stdout:7/838: mknod d9/d10/d53/d81/d59/d89/dd4/de0/d6d/c12a 0 2024-03-22T07:18:42.485 INFO:tasks.workunit.client.0.smithi081.stdout:1/766: stat d6/d6a/d85/df6/d24/d48/d5c/f7e 0 2024-03-22T07:18:42.488 INFO:tasks.workunit.client.0.smithi081.stdout:3/586: link db/de/d4b/d6a/f7c db/de/d4b/d6a/d70/d3a/d5d/fd3 0 2024-03-22T07:18:42.488 INFO:tasks.workunit.client.0.smithi081.stdout:3/587: chown db/de/d4b/d6a/d70/d3a/d5d/c79 4809 1 2024-03-22T07:18:42.490 INFO:tasks.workunit.client.0.smithi081.stdout:6/881: rename d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d9c to d3/da/de/db5/dfe/d11a 0 2024-03-22T07:18:42.492 INFO:tasks.workunit.client.0.smithi081.stdout:1/767: creat d6/f100 x:0 0 0 2024-03-22T07:18:42.493 INFO:tasks.workunit.client.0.smithi081.stdout:3/588: creat db/de/fd4 x:0 0 0 2024-03-22T07:18:42.493 INFO:tasks.workunit.client.0.smithi081.stdout:3/589: chown db/de/d4b/d6a/d70/d3a/d5d/fbc 0 1 2024-03-22T07:18:42.494 INFO:tasks.workunit.client.0.smithi081.stdout:1/768: dread d6/d18/d60/fd8 [0,4194304] 0 2024-03-22T07:18:42.495 INFO:tasks.workunit.client.0.smithi081.stdout:1/769: mkdir d6/d18/d41/d49/d101 0 2024-03-22T07:18:42.496 INFO:tasks.workunit.client.0.smithi081.stdout:9/747: rename d2/d2b/d49/d86/f9a to d2/d5d/ff5 0 2024-03-22T07:18:42.496 INFO:tasks.workunit.client.0.smithi081.stdout:9/748: fdatasync d2/d5/fd 0 2024-03-22T07:18:42.497 INFO:tasks.workunit.client.0.smithi081.stdout:6/882: mknod d3/d67/db2/dd0/c11b 0 2024-03-22T07:18:42.497 INFO:tasks.workunit.client.0.smithi081.stdout:6/883: dread - d3/d67/db2/ff5 zero size 2024-03-22T07:18:42.497 INFO:tasks.workunit.client.0.smithi081.stdout:6/884: dread - d3/d67/db2/d40/db9/d63/f104 zero size 2024-03-22T07:18:42.498 INFO:tasks.workunit.client.0.smithi081.stdout:7/839: mknod d9/d10/d119/c12b 0 2024-03-22T07:18:42.498 INFO:tasks.workunit.client.0.smithi081.stdout:7/840: chown d9/d3f/fa0 515834 1 2024-03-22T07:18:42.500 INFO:tasks.workunit.client.0.smithi081.stdout:3/590: unlink db/de/l81 0 2024-03-22T07:18:42.502 INFO:tasks.workunit.client.0.smithi081.stdout:9/749: creat d2/d10/d30/d57/d66/d75/d94/dae/ff6 x:0 0 0 2024-03-22T07:18:42.504 INFO:tasks.workunit.client.0.smithi081.stdout:6/885: rename d3/da/ce5 to d3/d67/db2/d40/db9/c11c 0 2024-03-22T07:18:42.508 INFO:tasks.workunit.client.0.smithi081.stdout:7/841: unlink d9/d10/d53/d81/d59/dbe/ld2 0 2024-03-22T07:18:42.508 INFO:tasks.workunit.client.0.smithi081.stdout:9/750: dread d2/f11 [0,4194304] 0 2024-03-22T07:18:42.509 INFO:tasks.workunit.client.0.smithi081.stdout:3/591: link db/de/fd4 db/de/d4b/db0/fd5 0 2024-03-22T07:18:42.511 INFO:tasks.workunit.client.0.smithi081.stdout:9/751: getdents d2/d10/d30/d57/d66/d75/d94/dae/de6 0 2024-03-22T07:18:42.513 INFO:tasks.workunit.client.0.smithi081.stdout:6/886: read d3/da/de/db5/fcf [2841248,15695] 0 2024-03-22T07:18:42.514 INFO:tasks.workunit.client.0.smithi081.stdout:9/752: mknod d2/d5d/d58/dc2/cf7 0 2024-03-22T07:18:42.518 INFO:tasks.workunit.client.0.smithi081.stdout:6/887: mknod d3/da/de/db5/dfe/d11a/c11d 0 2024-03-22T07:18:42.542 INFO:tasks.workunit.client.0.smithi081.stdout:7/842: write d9/dc/f1f [2750384,28166] 0 2024-03-22T07:18:42.543 INFO:tasks.workunit.client.0.smithi081.stdout:3/592: dwrite db/d38/f6c [0,4194304] 0 2024-03-22T07:18:42.544 INFO:tasks.workunit.client.0.smithi081.stdout:3/593: mknod db/de/db8/dd2/cd6 0 2024-03-22T07:18:42.545 INFO:tasks.workunit.client.0.smithi081.stdout:6/888: dwrite d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f6b [8388608,4194304] 0 2024-03-22T07:18:42.545 INFO:tasks.workunit.client.0.smithi081.stdout:9/753: dwrite d2/d5d/d58/dc2/fd0 [0,4194304] 0 2024-03-22T07:18:42.558 INFO:tasks.workunit.client.0.smithi081.stdout:7/843: dwrite d9/d10/fb7 [0,4194304] 0 2024-03-22T07:18:42.558 INFO:tasks.workunit.client.0.smithi081.stdout:7/844: readlink d9/d10/d16/d1e/d80/ld1 0 2024-03-22T07:18:42.567 INFO:tasks.workunit.client.0.smithi081.stdout:7/845: write d9/d10/fb7 [618470,105316] 0 2024-03-22T07:18:42.648 INFO:tasks.workunit.client.0.smithi081.stdout:3/594: creat db/d38/da1/db7/fd7 x:0 0 0 2024-03-22T07:18:42.649 INFO:tasks.workunit.client.0.smithi081.stdout:6/889: mkdir d3/d67/db2/d40/db9/d63/d7d/dc6/d11e 0 2024-03-22T07:18:42.651 INFO:tasks.workunit.client.0.smithi081.stdout:9/754: creat d2/d10/ff8 x:0 0 0 2024-03-22T07:18:42.651 INFO:tasks.workunit.client.0.smithi081.stdout:9/755: chown d2/d10/d30/d57/d66/d75/d94/dae/ff6 288129258 1 2024-03-22T07:18:42.655 INFO:tasks.workunit.client.0.smithi081.stdout:2/615: sync 2024-03-22T07:18:42.655 INFO:tasks.workunit.client.0.smithi081.stdout:4/892: sync 2024-03-22T07:18:42.655 INFO:tasks.workunit.client.0.smithi081.stdout:2/616: write da/db/d39/d46/fb6 [2599131,112558] 0 2024-03-22T07:18:42.657 INFO:tasks.workunit.client.0.smithi081.stdout:3/595: write db/f1e [156191,124524] 0 2024-03-22T07:18:42.658 INFO:tasks.workunit.client.0.smithi081.stdout:9/756: truncate d2/d73/d85/fb1 2253319 0 2024-03-22T07:18:42.659 INFO:tasks.workunit.client.0.smithi081.stdout:4/893: symlink da/df/d38/dd8/d4f/d7c/d90/d10d/l129 0 2024-03-22T07:18:42.661 INFO:tasks.workunit.client.0.smithi081.stdout:3/596: symlink db/de/db8/ld8 0 2024-03-22T07:18:42.664 INFO:tasks.workunit.client.0.smithi081.stdout:2/617: dread da/db/d39/d45/f8c [0,4194304] 0 2024-03-22T07:18:42.664 INFO:tasks.workunit.client.0.smithi081.stdout:2/618: write da/db/d39/d47/d3b/d41/f86 [107557,88205] 0 2024-03-22T07:18:42.664 INFO:tasks.workunit.client.0.smithi081.stdout:2/619: read f5 [3379817,21055] 0 2024-03-22T07:18:42.664 INFO:tasks.workunit.client.0.smithi081.stdout:2/620: dread - da/db/d39/d47/d3b/d41/d50/f66 zero size 2024-03-22T07:18:42.664 INFO:tasks.workunit.client.0.smithi081.stdout:6/890: dwrite d3/d67/db2/d40/db9/d63/d5e/f82 [4194304,4194304] 0 2024-03-22T07:18:42.765 INFO:tasks.workunit.client.0.smithi081.stdout:9/757: creat d2/d10/d30/d57/d66/d75/dca/ff9 x:0 0 0 2024-03-22T07:18:42.767 INFO:tasks.workunit.client.0.smithi081.stdout:7/846: mknod d9/d10/d9e/c12c 0 2024-03-22T07:18:42.767 INFO:tasks.workunit.client.0.smithi081.stdout:7/847: chown d9/d10/d16/f6b 0 1 2024-03-22T07:18:42.767 INFO:tasks.workunit.client.0.smithi081.stdout:7/848: write d9/d10/d53/d81/d59/fcf [1028326,103335] 0 2024-03-22T07:18:42.768 INFO:tasks.workunit.client.0.smithi081.stdout:4/894: creat da/df/d38/dd8/d4a/f12a x:0 0 0 2024-03-22T07:18:42.771 INFO:tasks.workunit.client.0.smithi081.stdout:2/621: creat da/db/d39/d46/d42/fc7 x:0 0 0 2024-03-22T07:18:42.771 INFO:tasks.workunit.client.0.smithi081.stdout:2/622: stat da/db/d39/d47/d3b/d41/d50/f66 0 2024-03-22T07:18:42.771 INFO:tasks.workunit.client.0.smithi081.stdout:8/730: sync 2024-03-22T07:18:42.771 INFO:tasks.workunit.client.0.smithi081.stdout:0/775: sync 2024-03-22T07:18:42.771 INFO:tasks.workunit.client.0.smithi081.stdout:8/731: write d2/fce [1026898,17046] 0 2024-03-22T07:18:42.771 INFO:tasks.workunit.client.0.smithi081.stdout:8/732: fsync d2/d3/d19/d1e/d13/f60 0 2024-03-22T07:18:42.771 INFO:tasks.workunit.client.0.smithi081.stdout:8/733: chown d2/d3/d19/d1e/d13/d22/d2f/d73/c94 59 1 2024-03-22T07:18:42.771 INFO:tasks.workunit.client.0.smithi081.stdout:8/734: creat d2/d3/d19/d1e/d13/ff7 x:0 0 0 2024-03-22T07:18:42.772 INFO:tasks.workunit.client.0.smithi081.stdout:6/891: creat d3/da/de/db5/dfe/d107/f11f x:0 0 0 2024-03-22T07:18:42.774 INFO:tasks.workunit.client.0.smithi081.stdout:9/758: mkdir d2/d5d/d58/dc2/dfa 0 2024-03-22T07:18:42.774 INFO:tasks.workunit.client.0.smithi081.stdout:9/759: stat d2/d5/d9d/f7b 0 2024-03-22T07:18:42.780 INFO:tasks.workunit.client.0.smithi081.stdout:3/597: write db/de/d4b/d6a/d70/f32 [1082746,129740] 0 2024-03-22T07:18:42.780 INFO:tasks.workunit.client.0.smithi081.stdout:3/598: readlink db/de/d4b/d6a/d70/d3a/lc5 0 2024-03-22T07:18:42.780 INFO:tasks.workunit.client.0.smithi081.stdout:3/599: chown db/de/d4b/d6a/d70/d3a/l53 15903 1 2024-03-22T07:18:42.780 INFO:tasks.workunit.client.0.smithi081.stdout:6/892: chown d3/da/de/f41 0 1 2024-03-22T07:18:42.781 INFO:tasks.workunit.client.0.smithi081.stdout:9/760: mknod d2/d78/d93/cfb 0 2024-03-22T07:18:42.781 INFO:tasks.workunit.client.0.smithi081.stdout:9/761: chown d2/d10/d30/d57/d66/d75/lb9 69 1 2024-03-22T07:18:42.782 INFO:tasks.workunit.client.0.smithi081.stdout:7/849: rename d9/d14/d10a to d9/d3f/d12d 0 2024-03-22T07:18:42.783 INFO:tasks.workunit.client.0.smithi081.stdout:4/895: mkdir da/df/d38/dd8/d4f/d12b 0 2024-03-22T07:18:42.784 INFO:tasks.workunit.client.0.smithi081.stdout:3/600: creat db/de/d4b/fd9 x:0 0 0 2024-03-22T07:18:42.788 INFO:tasks.workunit.client.0.smithi081.stdout:7/850: mknod d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/dc1/c12e 0 2024-03-22T07:18:42.788 INFO:tasks.workunit.client.0.smithi081.stdout:7/851: readlink d9/d10/d9e/la8 0 2024-03-22T07:18:42.791 INFO:tasks.workunit.client.0.smithi081.stdout:3/601: truncate db/de/d4b/d6a/f89 174514 0 2024-03-22T07:18:42.792 INFO:tasks.workunit.client.0.smithi081.stdout:9/762: rename d2/d10/d30/d57/d66/d75/d94/dae to d2/d5d/d58/dfc 0 2024-03-22T07:18:42.793 INFO:tasks.workunit.client.0.smithi081.stdout:3/602: truncate db/f77 4165241 0 2024-03-22T07:18:42.793 INFO:tasks.workunit.client.0.smithi081.stdout:3/603: write db/de/d4b/db0/fd5 [712112,45327] 0 2024-03-22T07:18:42.795 INFO:tasks.workunit.client.0.smithi081.stdout:9/763: rename d2/d73/d85/f6d to d2/d78/ffd 0 2024-03-22T07:18:42.795 INFO:tasks.workunit.client.0.smithi081.stdout:3/604: mknod db/de/db8/cda 0 2024-03-22T07:18:42.796 INFO:tasks.workunit.client.0.smithi081.stdout:9/764: symlink d2/d73/d85/ddd/lfe 0 2024-03-22T07:18:42.796 INFO:tasks.workunit.client.0.smithi081.stdout:9/765: write d2/d73/d85/fb1 [136166,19549] 0 2024-03-22T07:18:42.796 INFO:tasks.workunit.client.0.smithi081.stdout:9/766: read d2/d2b/f60 [919239,55774] 0 2024-03-22T07:18:42.797 INFO:tasks.workunit.client.0.smithi081.stdout:9/767: read d2/d73/f7e [12280,123183] 0 2024-03-22T07:18:42.797 INFO:tasks.workunit.client.0.smithi081.stdout:9/768: write d2/d10/d30/d57/d66/d75/fa9 [145613,81175] 0 2024-03-22T07:18:42.797 INFO:tasks.workunit.client.0.smithi081.stdout:6/893: read d3/f6 [445104,66395] 0 2024-03-22T07:18:42.798 INFO:tasks.workunit.client.0.smithi081.stdout:3/605: mkdir db/de/ddb 0 2024-03-22T07:18:42.798 INFO:tasks.workunit.client.0.smithi081.stdout:3/606: write db/de/d4b/d6a/d70/d3a/d5d/fd3 [1157612,44531] 0 2024-03-22T07:18:42.798 INFO:tasks.workunit.client.0.smithi081.stdout:9/769: mkdir d2/d5d/d5a/dff 0 2024-03-22T07:18:42.800 INFO:tasks.workunit.client.0.smithi081.stdout:9/770: rmdir d2/d5d/d5a/dff 0 2024-03-22T07:18:42.803 INFO:tasks.workunit.client.0.smithi081.stdout:7/852: write d9/d10/d9e/fc3 [7637720,88949] 0 2024-03-22T07:18:42.813 INFO:tasks.workunit.client.0.smithi081.stdout:0/776: dwrite d3/d6a/dd5/fff [4194304,4194304] 0 2024-03-22T07:18:42.813 INFO:tasks.workunit.client.0.smithi081.stdout:0/777: fdatasync d3/d6a/fc2 0 2024-03-22T07:18:42.813 INFO:tasks.workunit.client.0.smithi081.stdout:0/778: write d3/dc/f10c [385178,37153] 0 2024-03-22T07:18:42.824 INFO:tasks.workunit.client.0.smithi081.stdout:4/896: dwrite da/df/df4/ff2 [0,4194304] 0 2024-03-22T07:18:42.825 INFO:tasks.workunit.client.0.smithi081.stdout:4/897: write da/de6/f124 [501727,67791] 0 2024-03-22T07:18:42.825 INFO:tasks.workunit.client.0.smithi081.stdout:4/898: fdatasync da/d18/d1b/d26/d32/f91 0 2024-03-22T07:18:42.829 INFO:tasks.workunit.client.0.smithi081.stdout:2/623: dwrite da/db/f3f [0,4194304] 0 2024-03-22T07:18:42.837 INFO:tasks.workunit.client.0.smithi081.stdout:8/735: dwrite d2/d3/f1f [0,4194304] 0 2024-03-22T07:18:42.837 INFO:tasks.workunit.client.0.smithi081.stdout:8/736: chown d2/d3/d28/d2b/cd9 191530734 1 2024-03-22T07:18:42.848 INFO:tasks.workunit.client.0.smithi081.stdout:0/779: truncate d3/f24 1110406 0 2024-03-22T07:18:42.848 INFO:tasks.workunit.client.0.smithi081.stdout:0/780: chown d3/d32/f4f 1914 1 2024-03-22T07:18:42.858 INFO:tasks.workunit.client.0.smithi081.stdout:8/737: rename d2/d3/d19/d1e/ld3 to d2/d3/d19/d1e/d63/da0/lf8 0 2024-03-22T07:18:42.860 INFO:tasks.workunit.client.0.smithi081.stdout:0/781: write d3/d6a/f8e [1340476,122293] 0 2024-03-22T07:18:42.863 INFO:tasks.workunit.client.0.smithi081.stdout:0/782: creat d3/dc/d40/d69/f113 x:0 0 0 2024-03-22T07:18:42.864 INFO:tasks.workunit.client.0.smithi081.stdout:0/783: mkdir d3/d6a/de0/d114 0 2024-03-22T07:18:42.864 INFO:tasks.workunit.client.0.smithi081.stdout:0/784: fdatasync d3/f24 0 2024-03-22T07:18:42.877 INFO:tasks.workunit.client.0.smithi081.stdout:8/738: dwrite d2/d3/d23/fb3 [0,4194304] 0 2024-03-22T07:18:42.879 INFO:tasks.workunit.client.0.smithi081.stdout:8/739: unlink d2/l1d 0 2024-03-22T07:18:42.880 INFO:tasks.workunit.client.0.smithi081.stdout:8/740: symlink d2/d3/d28/d2b/d61/dc6/lf9 0 2024-03-22T07:18:42.881 INFO:tasks.workunit.client.0.smithi081.stdout:8/741: symlink d2/d3/d9e/lfa 0 2024-03-22T07:18:42.882 INFO:tasks.workunit.client.0.smithi081.stdout:8/742: mknod d2/d3/d19/d1e/d13/d22/db8/cfb 0 2024-03-22T07:18:42.883 INFO:tasks.workunit.client.0.smithi081.stdout:8/743: creat d2/d3/d19/d1e/ffc x:0 0 0 2024-03-22T07:18:42.903 INFO:tasks.workunit.client.0.smithi081.stdout:8/744: dwrite d2/d3/d19/d1e/d63/da0/db7/dc8/f92 [0,4194304] 0 2024-03-22T07:18:42.911 INFO:tasks.workunit.client.0.smithi081.stdout:9/771: dwrite d2/d5d/d58/dfc/ff6 [0,4194304] 0 2024-03-22T07:18:42.912 INFO:tasks.workunit.client.0.smithi081.stdout:9/772: fsync d2/d10/ff8 0 2024-03-22T07:18:42.914 INFO:tasks.workunit.client.0.smithi081.stdout:9/773: link d2/d10/d30/d57/dad/fbe d2/f100 0 2024-03-22T07:18:42.919 INFO:tasks.workunit.client.0.smithi081.stdout:0/785: write d3/dc/d40/fe3 [513619,55123] 0 2024-03-22T07:18:42.920 INFO:tasks.workunit.client.0.smithi081.stdout:0/786: readlink d3/dc/d2b/d4d/l107 0 2024-03-22T07:18:42.920 INFO:tasks.workunit.client.0.smithi081.stdout:0/787: mknod d3/dc/d40/d69/d92/c115 0 2024-03-22T07:18:42.921 INFO:tasks.workunit.client.0.smithi081.stdout:0/788: link d3/dc/d40/d69/db3/c108 d3/d6a/de0/c116 0 2024-03-22T07:18:42.922 INFO:tasks.workunit.client.0.smithi081.stdout:0/789: chown d3/dc/d40/d69/d92/dc1/l67 2125825 1 2024-03-22T07:18:42.922 INFO:tasks.workunit.client.0.smithi081.stdout:0/790: dread - d3/dc/d40/d69/d92/dc1/fcd zero size 2024-03-22T07:18:42.923 INFO:tasks.workunit.client.0.smithi081.stdout:0/791: unlink d3/f1d 0 2024-03-22T07:18:42.923 INFO:tasks.workunit.client.0.smithi081.stdout:0/792: read - d3/d4e/f88 zero size 2024-03-22T07:18:42.923 INFO:tasks.workunit.client.0.smithi081.stdout:0/793: symlink d3/dc/d40/d69/d92/dc1/d91/l117 0 2024-03-22T07:18:42.924 INFO:tasks.workunit.client.0.smithi081.stdout:0/794: truncate d3/dc/d40/d69/d92/dc1/d91/fc5 681033 0 2024-03-22T07:18:42.925 INFO:tasks.workunit.client.0.smithi081.stdout:0/795: mkdir d3/dc/d40/d118 0 2024-03-22T07:18:42.926 INFO:tasks.workunit.client.0.smithi081.stdout:0/796: creat d3/dc/d33/f119 x:0 0 0 2024-03-22T07:18:42.927 INFO:tasks.workunit.client.0.smithi081.stdout:0/797: write d3/dc/fd3 [5170099,131014] 0 2024-03-22T07:18:42.927 INFO:tasks.workunit.client.0.smithi081.stdout:3/607: dwrite db/de/d4b/db0/fd5 [0,4194304] 0 2024-03-22T07:18:42.927 INFO:tasks.workunit.client.0.smithi081.stdout:8/745: dwrite d2/d3/d19/d1e/d13/d22/db8/fbc [0,4194304] 0 2024-03-22T07:18:42.927 INFO:tasks.workunit.client.0.smithi081.stdout:4/899: dwrite da/d18/d1b/d34/f64 [0,4194304] 0 2024-03-22T07:18:42.927 INFO:tasks.workunit.client.0.smithi081.stdout:4/900: chown da/df/d38/dd8/c9c 169910048 1 2024-03-22T07:18:42.927 INFO:tasks.workunit.client.0.smithi081.stdout:2/624: dwrite da/db/d39/d47/d3b/d64/f8f [0,4194304] 0 2024-03-22T07:18:42.928 INFO:tasks.workunit.client.0.smithi081.stdout:7/853: dwrite d9/d10/f2e [0,4194304] 0 2024-03-22T07:18:42.933 INFO:tasks.workunit.client.0.smithi081.stdout:3/608: getdents db/de/db8 0 2024-03-22T07:18:42.933 INFO:tasks.workunit.client.0.smithi081.stdout:3/609: read db/f13 [2390390,121574] 0 2024-03-22T07:18:42.934 INFO:tasks.workunit.client.0.smithi081.stdout:3/610: dread - db/de/d4b/d6a/d70/d3a/f6f zero size 2024-03-22T07:18:42.936 INFO:tasks.workunit.client.0.smithi081.stdout:8/746: creat d2/d3/d19/d1e/d63/ffd x:0 0 0 2024-03-22T07:18:42.936 INFO:tasks.workunit.client.0.smithi081.stdout:8/747: write d2/d3/d28/d2b/f58 [4130879,46949] 0 2024-03-22T07:18:42.937 INFO:tasks.workunit.client.0.smithi081.stdout:4/901: getdents da/d3d/def 0 2024-03-22T07:18:42.938 INFO:tasks.workunit.client.0.smithi081.stdout:4/902: write da/d18/d1b/d1e/d37/f20 [3248771,12663] 0 2024-03-22T07:18:42.940 INFO:tasks.workunit.client.0.smithi081.stdout:2/625: dread da/db/d39/d47/d3b/d41/f54 [0,4194304] 0 2024-03-22T07:18:42.942 INFO:tasks.workunit.client.0.smithi081.stdout:7/854: dread d9/d10/d16/f6b [0,4194304] 0 2024-03-22T07:18:42.947 INFO:tasks.workunit.client.0.smithi081.stdout:2/626: dread da/db/d39/d46/f44 [0,4194304] 0 2024-03-22T07:18:42.950 INFO:tasks.workunit.client.0.smithi081.stdout:8/748: creat d2/d3/d19/d64/ffe x:0 0 0 2024-03-22T07:18:42.951 INFO:tasks.workunit.client.0.smithi081.stdout:2/627: link da/db/d39/d47/d6e/l8a da/db/d6d/lc8 0 2024-03-22T07:18:42.953 INFO:tasks.workunit.client.0.smithi081.stdout:2/628: mkdir da/db/d39/d46/d42/dc9 0 2024-03-22T07:18:42.957 INFO:tasks.workunit.client.0.smithi081.stdout:7/855: write d9/d10/d53/d81/d59/d89/dd4/de0/d6d/fa5 [457476,12686] 0 2024-03-22T07:18:42.957 INFO:tasks.workunit.client.0.smithi081.stdout:7/856: write d9/d10/d2a/f110 [511791,122983] 0 2024-03-22T07:18:42.958 INFO:tasks.workunit.client.0.smithi081.stdout:7/857: write d9/d10/d53/d81/d59/d89/dd4/de0/d70/f8d [1603854,87926] 0 2024-03-22T07:18:42.959 INFO:tasks.workunit.client.0.smithi081.stdout:7/858: mknod d9/d3f/d52/d83/c12f 0 2024-03-22T07:18:42.978 INFO:tasks.workunit.client.0.smithi081.stdout:4/903: dwrite da/d3d/f11a [0,4194304] 0 2024-03-22T07:18:42.979 INFO:tasks.workunit.client.0.smithi081.stdout:4/904: chown da/df/d38/dd8/d4f/d7c 2809348 1 2024-03-22T07:18:42.979 INFO:tasks.workunit.client.0.smithi081.stdout:4/905: read da/f79 [1140630,79270] 0 2024-03-22T07:18:42.979 INFO:tasks.workunit.client.0.smithi081.stdout:4/906: write da/d18/d1b/d1e/d37/f20 [4227047,9361] 0 2024-03-22T07:18:42.979 INFO:tasks.workunit.client.0.smithi081.stdout:4/907: stat da/d18/d1b/d26/d32/lcf 0 2024-03-22T07:18:42.980 INFO:tasks.workunit.client.0.smithi081.stdout:3/611: dwrite db/de/d4b/d6a/d70/d3a/d5d/fbc [0,4194304] 0 2024-03-22T07:18:42.980 INFO:tasks.workunit.client.0.smithi081.stdout:2/629: dwrite da/d11/d21/f33 [0,4194304] 0 2024-03-22T07:18:42.980 INFO:tasks.workunit.client.0.smithi081.stdout:5/849: sync 2024-03-22T07:18:42.980 INFO:tasks.workunit.client.0.smithi081.stdout:5/850: write d4/f89 [4588480,49283] 0 2024-03-22T07:18:42.980 INFO:tasks.workunit.client.0.smithi081.stdout:5/851: write d4/f33 [1551683,112679] 0 2024-03-22T07:18:42.982 INFO:tasks.workunit.client.0.smithi081.stdout:4/908: mknod da/d18/d1b/d26/dce/dfd/d74/c12c 0 2024-03-22T07:18:42.984 INFO:tasks.workunit.client.0.smithi081.stdout:5/852: mknod d4/df/de6/c116 0 2024-03-22T07:18:42.985 INFO:tasks.workunit.client.0.smithi081.stdout:5/853: write d4/d10/d20/d25/f6e [558451,109906] 0 2024-03-22T07:18:42.985 INFO:tasks.workunit.client.0.smithi081.stdout:5/854: chown d4/d9/f15 9813 1 2024-03-22T07:18:42.985 INFO:tasks.workunit.client.0.smithi081.stdout:4/909: creat da/df/d38/dd8/d4f/d7c/d90/f12d x:0 0 0 2024-03-22T07:18:42.986 INFO:tasks.workunit.client.0.smithi081.stdout:5/855: link d4/d75/da5/fcb d4/d75/d98/d103/f117 0 2024-03-22T07:18:42.986 INFO:tasks.workunit.client.0.smithi081.stdout:5/856: truncate d4/d75/f113 628980 0 2024-03-22T07:18:42.988 INFO:tasks.workunit.client.0.smithi081.stdout:4/910: mkdir da/d3d/def/d12e 0 2024-03-22T07:18:42.988 INFO:tasks.workunit.client.0.smithi081.stdout:4/911: write da/d18/d1b/d26/d3e/f113 [830976,38783] 0 2024-03-22T07:18:42.996 INFO:tasks.workunit.client.0.smithi081.stdout:3/612: dread db/de/d4b/f88 [0,4194304] 0 2024-03-22T07:18:42.996 INFO:tasks.workunit.client.0.smithi081.stdout:3/613: chown db/de/l1c 834 1 2024-03-22T07:18:42.996 INFO:tasks.workunit.client.0.smithi081.stdout:3/614: write db/de/f29 [2034372,81133] 0 2024-03-22T07:18:42.997 INFO:tasks.workunit.client.0.smithi081.stdout:3/615: creat db/de/fdc x:0 0 0 2024-03-22T07:18:43.000 INFO:tasks.workunit.client.0.smithi081.stdout:4/912: dread da/d18/d1b/d1e/d5f/fc0 [0,4194304] 0 2024-03-22T07:18:43.001 INFO:tasks.workunit.client.0.smithi081.stdout:3/616: dread db/de/d4b/d6a/f7c [0,4194304] 0 2024-03-22T07:18:43.002 INFO:tasks.workunit.client.0.smithi081.stdout:4/913: link da/d18/d1b/d26/d3e/fa1 da/df/d38/dd8/d4f/daf/ded/f12f 0 2024-03-22T07:18:43.011 INFO:tasks.workunit.client.0.smithi081.stdout:3/617: dread db/f1e [0,4194304] 0 2024-03-22T07:18:43.011 INFO:tasks.workunit.client.0.smithi081.stdout:3/618: stat db/de/d4b/f88 0 2024-03-22T07:18:43.012 INFO:tasks.workunit.client.0.smithi081.stdout:3/619: mkdir db/d38/ddd 0 2024-03-22T07:18:43.012 INFO:tasks.workunit.client.0.smithi081.stdout:0/798: dwrite d3/d32/fbe [0,4194304] 0 2024-03-22T07:18:43.013 INFO:tasks.workunit.client.0.smithi081.stdout:2/630: dwrite da/d11/d21/f4e [0,4194304] 0 2024-03-22T07:18:43.013 INFO:tasks.workunit.client.0.smithi081.stdout:5/857: dwrite d4/df/d73/d8c/dc0/dc9/ff1 [0,4194304] 0 2024-03-22T07:18:43.013 INFO:tasks.workunit.client.0.smithi081.stdout:3/620: creat db/d38/fde x:0 0 0 2024-03-22T07:18:43.016 INFO:tasks.workunit.client.0.smithi081.stdout:2/631: symlink da/db/d39/d47/d3b/d41/dab/dac/dc2/lca 0 2024-03-22T07:18:43.019 INFO:tasks.workunit.client.0.smithi081.stdout:5/858: mknod d4/d10/d20/db2/d2f/c118 0 2024-03-22T07:18:43.023 INFO:tasks.workunit.client.0.smithi081.stdout:5/859: truncate d4/df/d73/d8c/f4c 6694706 0 2024-03-22T07:18:43.024 INFO:tasks.workunit.client.0.smithi081.stdout:2/632: mkdir da/db/d39/d46/d42/dcb 0 2024-03-22T07:18:43.026 INFO:tasks.workunit.client.0.smithi081.stdout:5/860: rmdir d4/d10/d20/d25/da3 39 2024-03-22T07:18:43.026 INFO:tasks.workunit.client.0.smithi081.stdout:5/861: fdatasync d4/d10/f111 0 2024-03-22T07:18:43.027 INFO:tasks.workunit.client.0.smithi081.stdout:3/621: dwrite db/de/d4b/d6a/d70/d3a/d5d/fd3 [0,4194304] 0 2024-03-22T07:18:43.027 INFO:tasks.workunit.client.0.smithi081.stdout:3/622: write db/de/f7d [931459,971] 0 2024-03-22T07:18:43.027 INFO:tasks.workunit.client.0.smithi081.stdout:3/623: write db/de/d4b/d6a/d70/d3a/d5d/fb5 [1880530,126852] 0 2024-03-22T07:18:43.028 INFO:tasks.workunit.client.0.smithi081.stdout:5/862: rename d4/df3/df4 to d4/df/d22/d119 0 2024-03-22T07:18:43.028 INFO:tasks.workunit.client.0.smithi081.stdout:5/863: stat d4/d9/f1b 0 2024-03-22T07:18:43.029 INFO:tasks.workunit.client.0.smithi081.stdout:2/633: mkdir da/db/d39/d45/dcc 0 2024-03-22T07:18:43.031 INFO:tasks.workunit.client.0.smithi081.stdout:3/624: mkdir db/ddf 0 2024-03-22T07:18:43.038 INFO:tasks.workunit.client.0.smithi081.stdout:5/864: write d4/fe [4856200,77966] 0 2024-03-22T07:18:43.038 INFO:tasks.workunit.client.0.smithi081.stdout:5/865: dread d4/d10/d20/dd2/ddf/fed [0,4194304] 0 2024-03-22T07:18:43.039 INFO:tasks.workunit.client.0.smithi081.stdout:5/866: creat d4/d75/da5/f11a x:0 0 0 2024-03-22T07:18:43.039 INFO:tasks.workunit.client.0.smithi081.stdout:3/625: link db/d38/f6c db/de/d4b/fe0 0 2024-03-22T07:18:43.041 INFO:tasks.workunit.client.0.smithi081.stdout:0/799: truncate d3/d32/fbe 1898643 0 2024-03-22T07:18:43.043 INFO:tasks.workunit.client.0.smithi081.stdout:3/626: symlink db/de/d4b/d6a/d70/d3a/le1 0 2024-03-22T07:18:43.045 INFO:tasks.workunit.client.0.smithi081.stdout:3/627: dread db/de/d4b/d6a/f89 [0,4194304] 0 2024-03-22T07:18:43.045 INFO:tasks.workunit.client.0.smithi081.stdout:2/634: rmdir da/db/d39/d47/d3b/d41/d50/d6a/db2 0 2024-03-22T07:18:43.048 INFO:tasks.workunit.client.0.smithi081.stdout:2/635: symlink da/db/d39/d45/dcc/lcd 0 2024-03-22T07:18:43.049 INFO:tasks.workunit.client.0.smithi081.stdout:3/628: rename db/de/d4b/d6a/d70/d3a/l96 to db/de/le2 0 2024-03-22T07:18:43.050 INFO:tasks.workunit.client.0.smithi081.stdout:3/629: rmdir db/de/db8/dd2 39 2024-03-22T07:18:43.051 INFO:tasks.workunit.client.0.smithi081.stdout:3/630: creat db/d38/da1/db7/fe3 x:0 0 0 2024-03-22T07:18:43.051 INFO:tasks.workunit.client.0.smithi081.stdout:2/636: rename da/db/d39/lbf to da/db/d39/d46/lce 0 2024-03-22T07:18:43.053 INFO:tasks.workunit.client.0.smithi081.stdout:3/631: rename db/de/d4b/d6a/d70/d3a/la2 to db/d38/da1/db7/le4 0 2024-03-22T07:18:43.054 INFO:tasks.workunit.client.0.smithi081.stdout:2/637: rmdir da/db/d39/d47/d3b/d41/d50/d6a/d70 39 2024-03-22T07:18:43.056 INFO:tasks.workunit.client.0.smithi081.stdout:2/638: creat da/db/d39/d47/d3b/d41/d50/fcf x:0 0 0 2024-03-22T07:18:43.056 INFO:tasks.workunit.client.0.smithi081.stdout:2/639: readlink da/l1d 0 2024-03-22T07:18:43.057 INFO:tasks.workunit.client.0.smithi081.stdout:2/640: creat da/db/d6d/fd0 x:0 0 0 2024-03-22T07:18:43.058 INFO:tasks.workunit.client.0.smithi081.stdout:2/641: write da/db/d39/fae [40670,36337] 0 2024-03-22T07:18:43.058 INFO:tasks.workunit.client.0.smithi081.stdout:2/642: read da/db/d39/fae [389,57664] 0 2024-03-22T07:18:43.060 INFO:tasks.workunit.client.0.smithi081.stdout:0/800: dread d3/dc/d40/fb6 [0,4194304] 0 2024-03-22T07:18:43.061 INFO:tasks.workunit.client.0.smithi081.stdout:2/643: symlink da/db/d39/d46/d42/dcb/ld1 0 2024-03-22T07:18:43.061 INFO:tasks.workunit.client.0.smithi081.stdout:5/867: dwrite d4/df/d22/d4d/d82/fff [0,4194304] 0 2024-03-22T07:18:43.062 INFO:tasks.workunit.client.0.smithi081.stdout:0/801: rename d3/dc/d40/d69/d92/dcf/dd6 to d3/dc/d40/d118/d11a 0 2024-03-22T07:18:43.063 INFO:tasks.workunit.client.0.smithi081.stdout:0/802: write d3/d6a/f8e [896565,9055] 0 2024-03-22T07:18:43.063 INFO:tasks.workunit.client.0.smithi081.stdout:0/803: write d3/dc/d40/d69/f113 [569225,57366] 0 2024-03-22T07:18:43.064 INFO:tasks.workunit.client.0.smithi081.stdout:1/770: sync 2024-03-22T07:18:43.066 INFO:tasks.workunit.client.0.smithi081.stdout:5/868: creat d4/d10/d20/f11b x:0 0 0 2024-03-22T07:18:43.067 INFO:tasks.workunit.client.0.smithi081.stdout:1/771: creat d6/d6a/d85/df6/d4c/d70/f102 x:0 0 0 2024-03-22T07:18:43.074 INFO:tasks.workunit.client.0.smithi081.stdout:5/869: dread d4/d9/d90/ffa [0,4194304] 0 2024-03-22T07:18:43.084 INFO:tasks.workunit.client.0.smithi081.stdout:5/870: write d4/d9/f5b [3274434,52196] 0 2024-03-22T07:18:43.084 INFO:tasks.workunit.client.0.smithi081.stdout:0/804: dwrite d3/dc/d33/f5b [4194304,4194304] 0 2024-03-22T07:18:43.084 INFO:tasks.workunit.client.0.smithi081.stdout:0/805: readlink d3/dc/d40/d69/d92/dcf/l10e 0 2024-03-22T07:18:43.084 INFO:tasks.workunit.client.0.smithi081.stdout:2/644: dwrite da/d11/d21/fa2 [0,4194304] 0 2024-03-22T07:18:43.085 INFO:tasks.workunit.client.0.smithi081.stdout:2/645: fsync da/d11/f1f 0 2024-03-22T07:18:43.085 INFO:tasks.workunit.client.0.smithi081.stdout:1/772: write f1 [1631091,67753] 0 2024-03-22T07:18:43.085 INFO:tasks.workunit.client.0.smithi081.stdout:1/773: fdatasync d6/d18/f27 0 2024-03-22T07:18:43.085 INFO:tasks.workunit.client.0.smithi081.stdout:1/774: chown d6/d6a/d85/df6/d32/dbe/dc4/fc5 61 1 2024-03-22T07:18:43.085 INFO:tasks.workunit.client.0.smithi081.stdout:1/775: creat d6/d6a/d85/df6/d24/d48/d6e/de7/f103 x:0 0 0 2024-03-22T07:18:43.086 INFO:tasks.workunit.client.0.smithi081.stdout:5/871: creat d4/d75/d98/d103/f11c x:0 0 0 2024-03-22T07:18:43.086 INFO:tasks.workunit.client.0.smithi081.stdout:5/872: creat d4/d10/d20/db2/d2a/f11d x:0 0 0 2024-03-22T07:18:43.087 INFO:tasks.workunit.client.0.smithi081.stdout:5/873: dread d4/d10/d20/d25/da3/fd6 [0,4194304] 0 2024-03-22T07:18:43.087 INFO:tasks.workunit.client.0.smithi081.stdout:5/874: chown d4/d10/d20/db2/d2a/l4b 619 1 2024-03-22T07:18:43.088 INFO:tasks.workunit.client.0.smithi081.stdout:0/806: rmdir d3/dc/d40/d69/d111 0 2024-03-22T07:18:43.089 INFO:tasks.workunit.client.0.smithi081.stdout:0/807: chown d3/d4e 2055378553 1 2024-03-22T07:18:43.089 INFO:tasks.workunit.client.0.smithi081.stdout:0/808: creat d3/dc/d40/d69/dd8/f11b x:0 0 0 2024-03-22T07:18:43.089 INFO:tasks.workunit.client.0.smithi081.stdout:0/809: readlink d3/dc/d2b/d4d/lc3 0 2024-03-22T07:18:43.089 INFO:tasks.workunit.client.0.smithi081.stdout:0/810: chown d3/d32/f5f 1529 1 2024-03-22T07:18:43.090 INFO:tasks.workunit.client.0.smithi081.stdout:2/646: truncate da/db/f3f 1707577 0 2024-03-22T07:18:43.092 INFO:tasks.workunit.client.0.smithi081.stdout:1/776: chown d6/d18/c99 3619225 1 2024-03-22T07:18:43.096 INFO:tasks.workunit.client.0.smithi081.stdout:0/811: write d3/dc/d40/d69/d92/dc1/df6/f10a [3743407,35051] 0 2024-03-22T07:18:43.106 INFO:tasks.workunit.client.0.smithi081.stdout:2/647: dread da/db/d39/d47/d3b/d64/f8f [0,4194304] 0 2024-03-22T07:18:43.106 INFO:tasks.workunit.client.0.smithi081.stdout:2/648: dread - da/db/d6d/fd0 zero size 2024-03-22T07:18:43.106 INFO:tasks.workunit.client.0.smithi081.stdout:2/649: symlink da/db/d39/d47/d3b/d41/d50/ld2 0 2024-03-22T07:18:43.117 INFO:tasks.workunit.client.0.smithi081.stdout:5/875: dwrite d4/d75/d98/fad [0,4194304] 0 2024-03-22T07:18:43.117 INFO:tasks.workunit.client.0.smithi081.stdout:5/876: readlink d4/d10/d20/l107 0 2024-03-22T07:18:43.120 INFO:tasks.workunit.client.0.smithi081.stdout:0/812: dwrite d3/dc/d77/f104 [0,4194304] 0 2024-03-22T07:18:43.120 INFO:tasks.workunit.client.0.smithi081.stdout:0/813: chown d3/dc/d33/f97 13717 1 2024-03-22T07:18:43.127 INFO:tasks.workunit.client.0.smithi081.stdout:5/877: read d4/df/d22/d4d/f99 [1056583,86348] 0 2024-03-22T07:18:43.128 INFO:tasks.workunit.client.0.smithi081.stdout:0/814: dread d3/dc/dc6/fd9 [0,4194304] 0 2024-03-22T07:18:43.128 INFO:tasks.workunit.client.0.smithi081.stdout:0/815: write d3/d6a/fef [158797,8534] 0 2024-03-22T07:18:43.128 INFO:tasks.workunit.client.0.smithi081.stdout:5/878: symlink d4/df/d73/d8c/dc0/dc9/l11e 0 2024-03-22T07:18:43.131 INFO:tasks.workunit.client.0.smithi081.stdout:5/879: creat d4/df/d73/d8c/dc0/dc9/f11f x:0 0 0 2024-03-22T07:18:43.131 INFO:tasks.workunit.client.0.smithi081.stdout:5/880: chown d4/d75/da5 352 1 2024-03-22T07:18:43.132 INFO:tasks.workunit.client.0.smithi081.stdout:0/816: rename d3/dc/d40/d69/d92/dc1/l67 to d3/dc/d2b/l11c 0 2024-03-22T07:18:43.132 INFO:tasks.workunit.client.0.smithi081.stdout:0/817: fsync d3/dc/d33/f7d 0 2024-03-22T07:18:43.133 INFO:tasks.workunit.client.0.smithi081.stdout:0/818: truncate d3/d6a/dd5/fda 1085478 0 2024-03-22T07:18:43.134 INFO:tasks.workunit.client.0.smithi081.stdout:0/819: unlink d3/dc/c89 0 2024-03-22T07:18:43.135 INFO:tasks.workunit.client.0.smithi081.stdout:5/881: mkdir d4/d120 0 2024-03-22T07:18:43.136 INFO:tasks.workunit.client.0.smithi081.stdout:0/820: dread d3/dc/d40/d69/d92/dc1/d91/fc5 [0,4194304] 0 2024-03-22T07:18:43.136 INFO:tasks.workunit.client.0.smithi081.stdout:5/882: creat d4/d9/df7/f121 x:0 0 0 2024-03-22T07:18:43.139 INFO:tasks.workunit.client.0.smithi081.stdout:0/821: dread d3/f24 [0,4194304] 0 2024-03-22T07:18:43.139 INFO:tasks.workunit.client.0.smithi081.stdout:0/822: read d3/dc/d2b/d4d/f5e [839549,37540] 0 2024-03-22T07:18:43.140 INFO:tasks.workunit.client.0.smithi081.stdout:0/823: creat d3/dc/d40/d69/d92/dc1/df6/f11d x:0 0 0 2024-03-22T07:18:43.150 INFO:tasks.workunit.client.0.smithi081.stdout:0/824: write d3/dc/f3f [311032,44219] 0 2024-03-22T07:18:43.151 INFO:tasks.workunit.client.0.smithi081.stdout:0/825: creat d3/dc/d33/dd1/f11e x:0 0 0 2024-03-22T07:18:43.152 INFO:tasks.workunit.client.0.smithi081.stdout:0/826: creat d3/dc/f11f x:0 0 0 2024-03-22T07:18:43.160 INFO:tasks.workunit.client.0.smithi081.stdout:5/883: dwrite d4/d9/f1b [0,4194304] 0 2024-03-22T07:18:43.162 INFO:tasks.workunit.client.0.smithi081.stdout:5/884: creat d4/d10/d20/db2/d2f/d8e/f122 x:0 0 0 2024-03-22T07:18:43.163 INFO:tasks.workunit.client.0.smithi081.stdout:5/885: creat d4/d10/daf/f123 x:0 0 0 2024-03-22T07:18:43.163 INFO:tasks.workunit.client.0.smithi081.stdout:5/886: readlink d4/d9/d90/la6 0 2024-03-22T07:18:43.163 INFO:tasks.workunit.client.0.smithi081.stdout:5/887: creat d4/d10/d20/dd2/ddf/f124 x:0 0 0 2024-03-22T07:18:43.168 INFO:tasks.workunit.client.0.smithi081.stdout:0/827: dwrite d3/d6a/fa3 [0,4194304] 0 2024-03-22T07:18:43.175 INFO:tasks.workunit.client.0.smithi081.stdout:5/888: write d4/d10/d20/d4f/fb6 [1585925,36041] 0 2024-03-22T07:18:43.177 INFO:tasks.workunit.client.0.smithi081.stdout:5/889: link d4/d10/daf/dfe/f109 d4/d9/d90/f125 0 2024-03-22T07:18:43.182 INFO:tasks.workunit.client.0.smithi081.stdout:0/828: dwrite d3/d9b/faa [0,4194304] 0 2024-03-22T07:18:43.182 INFO:tasks.workunit.client.0.smithi081.stdout:0/829: fdatasync d3/dc/dc6/fd9 0 2024-03-22T07:18:43.183 INFO:tasks.workunit.client.0.smithi081.stdout:0/830: dread - d3/dc/d2b/d4d/f103 zero size 2024-03-22T07:18:43.185 INFO:tasks.workunit.client.0.smithi081.stdout:0/831: mkdir d3/dc/d33/d62/df9/d120 0 2024-03-22T07:18:43.185 INFO:tasks.workunit.client.0.smithi081.stdout:0/832: fsync d3/d6a/dd5/fda 0 2024-03-22T07:18:43.187 INFO:tasks.workunit.client.0.smithi081.stdout:0/833: symlink d3/dc/d40/d118/d11a/l121 0 2024-03-22T07:18:43.188 INFO:tasks.workunit.client.0.smithi081.stdout:0/834: rename d3/dc/d40/d69/d92/dc1/f76 to d3/dc/d40/df7/f122 0 2024-03-22T07:18:43.188 INFO:tasks.workunit.client.0.smithi081.stdout:0/835: chown d3/dc/f19 949 1 2024-03-22T07:18:43.188 INFO:tasks.workunit.client.0.smithi081.stdout:0/836: chown d3/d6a 5486 1 2024-03-22T07:18:43.197 INFO:tasks.workunit.client.0.smithi081.stdout:5/890: dwrite d4/d10/daf/fb4 [0,4194304] 0 2024-03-22T07:18:43.200 INFO:tasks.workunit.client.0.smithi081.stdout:5/891: creat d4/df/d22/d49/d115/f126 x:0 0 0 2024-03-22T07:18:43.200 INFO:tasks.workunit.client.0.smithi081.stdout:0/837: write d3/dc/d2b/d4d/fb2 [111269,109106] 0 2024-03-22T07:18:43.211 INFO:tasks.workunit.client.0.smithi081.stdout:5/892: mknod d4/df/d73/d8c/dc0/de1/c127 0 2024-03-22T07:18:43.213 INFO:tasks.workunit.client.0.smithi081.stdout:5/893: creat d4/df/d22/f128 x:0 0 0 2024-03-22T07:18:43.213 INFO:tasks.workunit.client.0.smithi081.stdout:5/894: fdatasync d4/d75/da5/fcb 0 2024-03-22T07:18:43.214 INFO:tasks.workunit.client.0.smithi081.stdout:5/895: creat d4/d120/f129 x:0 0 0 2024-03-22T07:18:43.215 INFO:tasks.workunit.client.0.smithi081.stdout:5/896: mknod d4/d10/d20/db2/d2a/c12a 0 2024-03-22T07:18:43.216 INFO:tasks.workunit.client.0.smithi081.stdout:5/897: getdents d4/df/d73/d8c/dc0/dc9 0 2024-03-22T07:18:43.216 INFO:tasks.workunit.client.0.smithi081.stdout:5/898: chown d4/d10/la4 113 1 2024-03-22T07:18:43.216 INFO:tasks.workunit.client.0.smithi081.stdout:6/894: sync 2024-03-22T07:18:43.217 INFO:tasks.workunit.client.0.smithi081.stdout:5/899: creat d4/d10/d20/d25/db8/de0/f12b x:0 0 0 2024-03-22T07:18:43.217 INFO:tasks.workunit.client.0.smithi081.stdout:5/900: stat d4/d10/d20/ffd 0 2024-03-22T07:18:43.217 INFO:tasks.workunit.client.0.smithi081.stdout:5/901: chown d4/d120 1341 1 2024-03-22T07:18:43.218 INFO:tasks.workunit.client.0.smithi081.stdout:6/895: truncate d3/da/de/f1f 102423 0 2024-03-22T07:18:43.218 INFO:tasks.workunit.client.0.smithi081.stdout:6/896: chown d3/d67/db2/d40/db9/d63 6953253 1 2024-03-22T07:18:43.219 INFO:tasks.workunit.client.0.smithi081.stdout:6/897: write d3/d67/db2/d40/db9/d63/d26/d3f/d4c/fe8 [4397471,13462] 0 2024-03-22T07:18:43.219 INFO:tasks.workunit.client.0.smithi081.stdout:5/902: mkdir d4/d10/d112/d12c 0 2024-03-22T07:18:43.221 INFO:tasks.workunit.client.0.smithi081.stdout:5/903: creat d4/d10/d20/d25/db8/dcf/f12d x:0 0 0 2024-03-22T07:18:43.251 INFO:tasks.workunit.client.0.smithi081.stdout:6/898: dwrite d3/dcc/fed [0,4194304] 0 2024-03-22T07:18:43.251 INFO:tasks.workunit.client.0.smithi081.stdout:5/904: dwrite d4/df3/ffc [0,4194304] 0 2024-03-22T07:18:43.253 INFO:tasks.workunit.client.0.smithi081.stdout:5/905: unlink d4/d10/d20/f11b 0 2024-03-22T07:18:43.253 INFO:tasks.workunit.client.0.smithi081.stdout:5/906: chown d4/df/d73/d8c/dc0 15918 1 2024-03-22T07:18:43.264 INFO:tasks.workunit.client.0.smithi081.stdout:6/899: write d3/d67/db2/d40/f99 [2427159,124889] 0 2024-03-22T07:18:43.271 INFO:tasks.workunit.client.0.smithi081.stdout:6/900: dread d3/da/de/f27 [0,4194304] 0 2024-03-22T07:18:43.271 INFO:tasks.workunit.client.0.smithi081.stdout:6/901: stat d3/dcc 0 2024-03-22T07:18:43.271 INFO:tasks.workunit.client.0.smithi081.stdout:6/902: getdents d3/d67/db2/d40/db9/d63/d26/d3f/d4c/de0 0 2024-03-22T07:18:43.271 INFO:tasks.workunit.client.0.smithi081.stdout:6/903: write d3/d67/db2/d40/db9/d63/f2c [697144,102307] 0 2024-03-22T07:18:43.272 INFO:tasks.workunit.client.0.smithi081.stdout:6/904: creat d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f120 x:0 0 0 2024-03-22T07:18:43.272 INFO:tasks.workunit.client.0.smithi081.stdout:6/905: rename d3/d67/db2/d40/db9/d63/d26 to d3/d67/db2/d40/db9/d63/d26/dfa/d108/d121 22 2024-03-22T07:18:43.272 INFO:tasks.workunit.client.0.smithi081.stdout:6/906: write d3/d67/db2/d40/db9/d63/db4/dc5/f10c [1748894,82931] 0 2024-03-22T07:18:43.272 INFO:tasks.workunit.client.0.smithi081.stdout:5/907: dwrite d4/d75/fbd [0,4194304] 0 2024-03-22T07:18:43.273 INFO:tasks.workunit.client.0.smithi081.stdout:5/908: stat d4/d75/fef 0 2024-03-22T07:18:43.273 INFO:tasks.workunit.client.0.smithi081.stdout:5/909: chown d4/d10/d20/d25/db8/f54 9 1 2024-03-22T07:18:43.273 INFO:tasks.workunit.client.0.smithi081.stdout:6/907: mkdir d3/d67/db2/d40/db9/d63/d26/d3f/d4c/de0/d122 0 2024-03-22T07:18:43.277 INFO:tasks.workunit.client.0.smithi081.stdout:6/908: rename d3/d67/db2/d40/db9/d63/d26/f71 to d3/d67/db2/d40/db9/d63/dd2/f123 0 2024-03-22T07:18:43.289 INFO:tasks.workunit.client.0.smithi081.stdout:6/909: write d3/d67/db2/d40/db9/d63/d26/d3f/d5a/f6b [10217272,2115] 0 2024-03-22T07:18:43.292 INFO:tasks.workunit.client.0.smithi081.stdout:5/910: dwrite d4/d75/da5/f11a [0,4194304] 0 2024-03-22T07:18:43.293 INFO:tasks.workunit.client.0.smithi081.stdout:5/911: write d4/df/d73/f9d [46748,35517] 0 2024-03-22T07:18:43.294 INFO:tasks.workunit.client.0.smithi081.stdout:5/912: mknod d4/d10/d20/d25/da3/c12e 0 2024-03-22T07:18:43.298 INFO:tasks.workunit.client.0.smithi081.stdout:6/910: dread d3/d67/db2/d40/db9/d63/d26/f84 [0,4194304] 0 2024-03-22T07:18:43.300 INFO:tasks.workunit.client.0.smithi081.stdout:6/911: link d3/da/de/db5/dfe/d107/f11f d3/d67/db2/d40/db9/d63/d26/f124 0 2024-03-22T07:18:43.302 INFO:tasks.workunit.client.0.smithi081.stdout:6/912: link d3/d67/db2/d40/db9/d63/d5e/d9d/da6/lb1 d3/da/l125 0 2024-03-22T07:18:43.303 INFO:tasks.workunit.client.0.smithi081.stdout:6/913: dread d3/d67/db2/d40/db9/d63/d5e/f8c [0,4194304] 0 2024-03-22T07:18:43.304 INFO:tasks.workunit.client.0.smithi081.stdout:6/914: rename d3/da/de/db5/fcf to d3/d67/db2/d40/db9/d63/d5e/dc0/f126 0 2024-03-22T07:18:43.305 INFO:tasks.workunit.client.0.smithi081.stdout:6/915: dread d3/d67/dd3/fd1 [0,4194304] 0 2024-03-22T07:18:43.305 INFO:tasks.workunit.client.0.smithi081.stdout:6/916: dread - d3/d67/db2/d40/da8/ffc zero size 2024-03-22T07:18:43.306 INFO:tasks.workunit.client.0.smithi081.stdout:6/917: creat d3/da/f127 x:0 0 0 2024-03-22T07:18:43.312 INFO:tasks.workunit.client.0.smithi081.stdout:5/913: dwrite d4/f89 [4194304,4194304] 0 2024-03-22T07:18:43.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:43 smithi081 ceph-mon[113965]: pgmap v217: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 93 MiB/s rd, 120 MiB/s wr, 517 op/s 2024-03-22T07:18:43.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:43 smithi149 ceph-mon[83447]: pgmap v217: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 93 MiB/s rd, 120 MiB/s wr, 517 op/s 2024-03-22T07:18:43.365 INFO:tasks.workunit.client.0.smithi081.stdout:5/914: unlink d4/d75/faa 0 2024-03-22T07:18:43.387 INFO:tasks.workunit.client.0.smithi081.stdout:6/918: dwrite d3/da/f53 [0,4194304] 0 2024-03-22T07:18:43.387 INFO:tasks.workunit.client.0.smithi081.stdout:6/919: dread - d3/d67/db2/d40/da8/fe9 zero size 2024-03-22T07:18:43.405 INFO:tasks.workunit.client.0.smithi081.stdout:6/920: dwrite d3/f9 [8388608,4194304] 0 2024-03-22T07:18:43.405 INFO:tasks.workunit.client.0.smithi081.stdout:6/921: write d3/da/f8f [116277,68081] 0 2024-03-22T07:18:43.406 INFO:tasks.workunit.client.0.smithi081.stdout:6/922: unlink d3/d67/dd3/cda 0 2024-03-22T07:18:43.406 INFO:tasks.workunit.client.0.smithi081.stdout:6/923: readlink d3/d67/db2/d40/db9/d63/d26/l74 0 2024-03-22T07:18:43.421 INFO:tasks.workunit.client.0.smithi081.stdout:6/924: dwrite d3/d67/db2/d40/db9/d63/db4/dc5/f10c [0,4194304] 0 2024-03-22T07:18:43.422 INFO:tasks.workunit.client.0.smithi081.stdout:6/925: mkdir d3/d67/db2/d128 0 2024-03-22T07:18:43.423 INFO:tasks.workunit.client.0.smithi081.stdout:6/926: creat d3/d67/db2/d40/db9/d63/d7d/dc6/d11e/f129 x:0 0 0 2024-03-22T07:18:43.423 INFO:tasks.workunit.client.0.smithi081.stdout:6/927: chown d3/d67/db2/d40/l51 1093602 1 2024-03-22T07:18:43.424 INFO:tasks.workunit.client.0.smithi081.stdout:6/928: rename d3/da/de/f7f to d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f12a 0 2024-03-22T07:18:43.641 INFO:tasks.workunit.client.0.smithi081.stdout:9/774: sync 2024-03-22T07:18:43.641 INFO:tasks.workunit.client.0.smithi081.stdout:8/749: sync 2024-03-22T07:18:43.642 INFO:tasks.workunit.client.0.smithi081.stdout:8/750: creat d2/d3/d28/d2b/d61/ddb/fff x:0 0 0 2024-03-22T07:18:43.643 INFO:tasks.workunit.client.0.smithi081.stdout:9/775: creat d2/d5d/d58/dfc/f101 x:0 0 0 2024-03-22T07:18:43.645 INFO:tasks.workunit.client.0.smithi081.stdout:9/776: mkdir d2/d10/d30/d102 0 2024-03-22T07:18:43.646 INFO:tasks.workunit.client.0.smithi081.stdout:9/777: creat d2/d73/d85/d7a/f103 x:0 0 0 2024-03-22T07:18:43.650 INFO:tasks.workunit.client.0.smithi081.stdout:8/751: dread d2/d3/d19/d1e/d63/da0/db7/dc8/f92 [0,4194304] 0 2024-03-22T07:18:43.650 INFO:tasks.workunit.client.0.smithi081.stdout:8/752: write d2/d3/d19/d1e/ffc [953849,87594] 0 2024-03-22T07:18:43.651 INFO:tasks.workunit.client.0.smithi081.stdout:8/753: creat d2/d3/d28/f100 x:0 0 0 2024-03-22T07:18:43.664 INFO:tasks.workunit.client.0.smithi081.stdout:8/754: dread d2/d3/d19/d1e/d13/d22/f51 [0,4194304] 0 2024-03-22T07:18:43.664 INFO:tasks.workunit.client.0.smithi081.stdout:8/755: chown d2/d3/d19/d1e/d13/d22/fbe 3037151 1 2024-03-22T07:18:43.665 INFO:tasks.workunit.client.0.smithi081.stdout:8/756: rename d2/d3/d19/d1e/d13/da5/fe7 to d2/d3/d19/d1e/d13/da5/f101 0 2024-03-22T07:18:43.665 INFO:tasks.workunit.client.0.smithi081.stdout:8/757: truncate d2/f5e 4633865 0 2024-03-22T07:18:43.666 INFO:tasks.workunit.client.0.smithi081.stdout:8/758: mknod d2/d3/d19/d1e/d13/c102 0 2024-03-22T07:18:43.666 INFO:tasks.workunit.client.0.smithi081.stdout:8/759: chown d2/d3/d19/d64/l96 7 1 2024-03-22T07:18:43.667 INFO:tasks.workunit.client.0.smithi081.stdout:8/760: creat d2/d3/d19/d1e/d63/da0/f103 x:0 0 0 2024-03-22T07:18:43.668 INFO:tasks.workunit.client.0.smithi081.stdout:8/761: mknod d2/d3/d28/d2b/d61/c104 0 2024-03-22T07:18:43.669 INFO:tasks.workunit.client.0.smithi081.stdout:8/762: mknod d2/d3/d19/d1e/d63/da0/db7/c105 0 2024-03-22T07:18:43.670 INFO:tasks.workunit.client.0.smithi081.stdout:8/763: truncate d2/f24 3760220 0 2024-03-22T07:18:43.670 INFO:tasks.workunit.client.0.smithi081.stdout:8/764: read - d2/d3/d19/d1e/f9d zero size 2024-03-22T07:18:43.670 INFO:tasks.workunit.client.0.smithi081.stdout:8/765: chown d2/d3/d28/d2b/d61 380 1 2024-03-22T07:18:43.671 INFO:tasks.workunit.client.0.smithi081.stdout:9/778: dwrite d2/d10/d30/d57/f9e [4194304,4194304] 0 2024-03-22T07:18:43.671 INFO:tasks.workunit.client.0.smithi081.stdout:8/766: rmdir d2/d3/d28/d2b/d61/dd7 39 2024-03-22T07:18:43.673 INFO:tasks.workunit.client.0.smithi081.stdout:9/779: creat d2/d10/d30/f104 x:0 0 0 2024-03-22T07:18:43.674 INFO:tasks.workunit.client.0.smithi081.stdout:9/780: getdents d2/d5d/d58/dfc/dc1 0 2024-03-22T07:18:43.676 INFO:tasks.workunit.client.0.smithi081.stdout:9/781: rename d2/d10/d30/c96 to d2/d73/d85/ddd/c105 0 2024-03-22T07:18:43.761 INFO:tasks.workunit.client.0.smithi081.stdout:4/914: sync 2024-03-22T07:18:43.761 INFO:tasks.workunit.client.0.smithi081.stdout:7/859: sync 2024-03-22T07:18:43.761 INFO:tasks.workunit.client.0.smithi081.stdout:4/915: chown f0 0 1 2024-03-22T07:18:43.761 INFO:tasks.workunit.client.0.smithi081.stdout:7/860: write d9/d10/d53/f123 [687956,126620] 0 2024-03-22T07:18:43.762 INFO:tasks.workunit.client.0.smithi081.stdout:4/916: dread da/d18/d1b/d1e/f62 [0,4194304] 0 2024-03-22T07:18:43.762 INFO:tasks.workunit.client.0.smithi081.stdout:4/917: chown da/d18/d1b/d1e/d5f/lb7 4 1 2024-03-22T07:18:43.764 INFO:tasks.workunit.client.0.smithi081.stdout:7/861: rmdir d9/d10/d53/d81/d59/d89/dd4 39 2024-03-22T07:18:43.765 INFO:tasks.workunit.client.0.smithi081.stdout:7/862: chown d9/d14/l5e 18984313 1 2024-03-22T07:18:43.766 INFO:tasks.workunit.client.0.smithi081.stdout:7/863: symlink d9/d3f/d12d/l130 0 2024-03-22T07:18:43.766 INFO:tasks.workunit.client.0.smithi081.stdout:4/918: dread da/f1d [0,4194304] 0 2024-03-22T07:18:43.768 INFO:tasks.workunit.client.0.smithi081.stdout:4/919: truncate da/d3d/faa 2571219 0 2024-03-22T07:18:43.769 INFO:tasks.workunit.client.0.smithi081.stdout:7/864: symlink d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/l131 0 2024-03-22T07:18:43.771 INFO:tasks.workunit.client.0.smithi081.stdout:7/865: rmdir d9/d10/d53 39 2024-03-22T07:18:43.771 INFO:tasks.workunit.client.0.smithi081.stdout:7/866: fdatasync d9/d3f/d52/d83/f8e 0 2024-03-22T07:18:43.774 INFO:tasks.workunit.client.0.smithi081.stdout:4/920: rename da/d18/d1b/d1e/d5f/f122 to da/df/d38/dd8/d4a/f130 0 2024-03-22T07:18:43.775 INFO:tasks.workunit.client.0.smithi081.stdout:4/921: dread - da/de6/f118 zero size 2024-03-22T07:18:43.775 INFO:tasks.workunit.client.0.smithi081.stdout:7/867: truncate d9/d3f/d52/d83/f8e 105955 0 2024-03-22T07:18:43.777 INFO:tasks.workunit.client.0.smithi081.stdout:4/922: link da/d18/d1b/d1e/d5f/ceb da/d18/ddc/c131 0 2024-03-22T07:18:43.778 INFO:tasks.workunit.client.0.smithi081.stdout:7/868: symlink d9/d3f/d12d/l132 0 2024-03-22T07:18:43.783 INFO:tasks.workunit.client.0.smithi081.stdout:7/869: unlink d9/d10/d53/d81/d59/d89/dd4/de0/d6d/fc2 0 2024-03-22T07:18:43.787 INFO:tasks.workunit.client.0.smithi081.stdout:7/870: mknod d9/d10/d2a/d73/c133 0 2024-03-22T07:18:43.791 INFO:tasks.workunit.client.0.smithi081.stdout:7/871: symlink d9/d10/d53/d81/d59/d89/dd4/l134 0 2024-03-22T07:18:43.795 INFO:tasks.workunit.client.0.smithi081.stdout:4/923: dwrite da/df/d38/dd8/d4a/fda [0,4194304] 0 2024-03-22T07:18:43.797 INFO:tasks.workunit.client.0.smithi081.stdout:4/924: creat da/d18/d1b/d26/dce/dfd/d74/f132 x:0 0 0 2024-03-22T07:18:43.798 INFO:tasks.workunit.client.0.smithi081.stdout:4/925: link da/d18/f31 da/d18/d1b/d26/dce/dfd/f133 0 2024-03-22T07:18:43.798 INFO:tasks.workunit.client.0.smithi081.stdout:4/926: dread - da/df/d38/dd8/d4f/daf/f128 zero size 2024-03-22T07:18:43.799 INFO:tasks.workunit.client.0.smithi081.stdout:4/927: symlink da/df/d38/l134 0 2024-03-22T07:18:43.799 INFO:tasks.workunit.client.0.smithi081.stdout:4/928: dread - da/df/d38/dd8/fe8 zero size 2024-03-22T07:18:43.800 INFO:tasks.workunit.client.0.smithi081.stdout:4/929: mknod da/df/c135 0 2024-03-22T07:18:43.800 INFO:tasks.workunit.client.0.smithi081.stdout:4/930: stat da/d18/d6e/fa8 0 2024-03-22T07:18:43.800 INFO:tasks.workunit.client.0.smithi081.stdout:4/931: creat da/df/d38/f136 x:0 0 0 2024-03-22T07:18:43.801 INFO:tasks.workunit.client.0.smithi081.stdout:4/932: truncate da/df/d38/dd8/d4f/daf/ff9 4110056 0 2024-03-22T07:18:43.804 INFO:tasks.workunit.client.0.smithi081.stdout:4/933: rename da/d18/d1b/d34/f4c to da/df/d38/dd8/d4f/d7c/d90/d10d/f137 0 2024-03-22T07:18:43.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/934: mkdir da/d57/d138 0 2024-03-22T07:18:43.807 INFO:tasks.workunit.client.0.smithi081.stdout:4/935: stat da/d57/d138 0 2024-03-22T07:18:43.808 INFO:tasks.workunit.client.0.smithi081.stdout:4/936: mknod da/df/d38/dd8/c139 0 2024-03-22T07:18:43.809 INFO:tasks.workunit.client.0.smithi081.stdout:4/937: link da/df/l8f da/df/d38/dd8/d4f/d12b/l13a 0 2024-03-22T07:18:43.811 INFO:tasks.workunit.client.0.smithi081.stdout:7/872: dwrite d9/d10/d2a/f47 [4194304,4194304] 0 2024-03-22T07:18:43.837 INFO:tasks.workunit.client.0.smithi081.stdout:4/938: dwrite da/d18/f21 [0,4194304] 0 2024-03-22T07:18:43.838 INFO:tasks.workunit.client.0.smithi081.stdout:7/873: dwrite d9/d10/d53/d81/d59/fcf [0,4194304] 0 2024-03-22T07:18:43.838 INFO:tasks.workunit.client.0.smithi081.stdout:7/874: chown d9/d10/d53/fb5 34184768 1 2024-03-22T07:18:43.838 INFO:tasks.workunit.client.0.smithi081.stdout:7/875: fdatasync d9/d10/d9e/fc3 0 2024-03-22T07:18:43.839 INFO:tasks.workunit.client.0.smithi081.stdout:4/939: unlink da/l96 0 2024-03-22T07:18:43.841 INFO:tasks.workunit.client.0.smithi081.stdout:4/940: dread da/d18/d1b/d26/d3e/fd1 [0,4194304] 0 2024-03-22T07:18:43.850 INFO:tasks.workunit.client.0.smithi081.stdout:5/915: sync 2024-03-22T07:18:43.850 INFO:tasks.workunit.client.0.smithi081.stdout:0/838: sync 2024-03-22T07:18:43.850 INFO:tasks.workunit.client.0.smithi081.stdout:1/777: sync 2024-03-22T07:18:43.850 INFO:tasks.workunit.client.0.smithi081.stdout:2/650: sync 2024-03-22T07:18:43.850 INFO:tasks.workunit.client.0.smithi081.stdout:3/632: sync 2024-03-22T07:18:43.850 INFO:tasks.workunit.client.0.smithi081.stdout:3/633: chown db/de/d4b/d6a/f7c 3994 1 2024-03-22T07:18:43.850 INFO:tasks.workunit.client.0.smithi081.stdout:0/839: truncate d3/d32/f70 1024452 0 2024-03-22T07:18:43.850 INFO:tasks.workunit.client.0.smithi081.stdout:3/634: write db/d38/f80 [177446,117733] 0 2024-03-22T07:18:43.850 INFO:tasks.workunit.client.0.smithi081.stdout:3/635: write db/d38/f69 [382059,39762] 0 2024-03-22T07:18:43.851 INFO:tasks.workunit.client.0.smithi081.stdout:5/916: chown d4/df/c16 11294 1 2024-03-22T07:18:43.851 INFO:tasks.workunit.client.0.smithi081.stdout:2/651: creat da/d11/d21/fd3 x:0 0 0 2024-03-22T07:18:43.851 INFO:tasks.workunit.client.0.smithi081.stdout:8/767: sync 2024-03-22T07:18:43.852 INFO:tasks.workunit.client.0.smithi081.stdout:6/929: sync 2024-03-22T07:18:43.852 INFO:tasks.workunit.client.0.smithi081.stdout:6/930: truncate d3/d67/db2/d40/db9/d63/d26/d3f/d5a/fce 1043277 0 2024-03-22T07:18:43.852 INFO:tasks.workunit.client.0.smithi081.stdout:6/931: write d3/da/fd [1252194,4866] 0 2024-03-22T07:18:43.854 INFO:tasks.workunit.client.0.smithi081.stdout:4/941: write da/d3d/def/f127 [788989,56410] 0 2024-03-22T07:18:43.854 INFO:tasks.workunit.client.0.smithi081.stdout:5/917: mkdir d4/df/d73/d8c/dc0/d12f 0 2024-03-22T07:18:43.855 INFO:tasks.workunit.client.0.smithi081.stdout:8/768: creat d2/d3/d28/f106 x:0 0 0 2024-03-22T07:18:43.856 INFO:tasks.workunit.client.0.smithi081.stdout:2/652: link da/db/d39/d46/d42/d67/fa4 da/d11/d21/d71/dc3/fd4 0 2024-03-22T07:18:43.857 INFO:tasks.workunit.client.0.smithi081.stdout:2/653: dread - da/db/d39/d46/d42/d67/fa4 zero size 2024-03-22T07:18:43.860 INFO:tasks.workunit.client.0.smithi081.stdout:4/942: link da/d18/f21 da/df/d38/dd8/d4f/d7c/f13b 0 2024-03-22T07:18:43.860 INFO:tasks.workunit.client.0.smithi081.stdout:0/840: dread d3/f22 [0,4194304] 0 2024-03-22T07:18:43.860 INFO:tasks.workunit.client.0.smithi081.stdout:7/876: dwrite d9/d3f/d52/d83/f8e [0,4194304] 0 2024-03-22T07:18:43.862 INFO:tasks.workunit.client.0.smithi081.stdout:8/769: mkdir d2/d3/d19/d1e/d13/d66/d70/d86/d107 0 2024-03-22T07:18:43.864 INFO:tasks.workunit.client.0.smithi081.stdout:2/654: creat da/db/d39/d47/fd5 x:0 0 0 2024-03-22T07:18:43.864 INFO:tasks.workunit.client.0.smithi081.stdout:2/655: chown da/db/d39/d47/l61 15 1 2024-03-22T07:18:43.866 INFO:tasks.workunit.client.0.smithi081.stdout:4/943: unlink da/d18/cdb 0 2024-03-22T07:18:43.866 INFO:tasks.workunit.client.0.smithi081.stdout:4/944: chown da/df/d38/dd8/d4a/ce5 1975 1 2024-03-22T07:18:43.867 INFO:tasks.workunit.client.0.smithi081.stdout:5/918: truncate d4/df/d22/f57 896213 0 2024-03-22T07:18:43.872 INFO:tasks.workunit.client.0.smithi081.stdout:2/656: dread da/d11/d21/f90 [0,4194304] 0 2024-03-22T07:18:43.872 INFO:tasks.workunit.client.0.smithi081.stdout:1/778: dread d6/d6a/f9d [0,4194304] 0 2024-03-22T07:18:43.874 INFO:tasks.workunit.client.0.smithi081.stdout:8/770: symlink d2/d3/d19/d1e/d63/da0/l108 0 2024-03-22T07:18:43.875 INFO:tasks.workunit.client.0.smithi081.stdout:4/945: getdents da/d57 0 2024-03-22T07:18:43.876 INFO:tasks.workunit.client.0.smithi081.stdout:4/946: readlink l2 0 2024-03-22T07:18:43.876 INFO:tasks.workunit.client.0.smithi081.stdout:4/947: fdatasync da/de6/db8/f114 0 2024-03-22T07:18:43.876 INFO:tasks.workunit.client.0.smithi081.stdout:6/932: dwrite d3/d67/db2/d40/db9/d63/d26/f44 [0,4194304] 0 2024-03-22T07:18:43.876 INFO:tasks.workunit.client.0.smithi081.stdout:0/841: dread d3/d6a/f8e [0,4194304] 0 2024-03-22T07:18:43.876 INFO:tasks.workunit.client.0.smithi081.stdout:6/933: getdents d3/d67/dd3/df4 0 2024-03-22T07:18:43.876 INFO:tasks.workunit.client.0.smithi081.stdout:0/842: write d3/dc/d40/d69/dd8/f11b [441808,75218] 0 2024-03-22T07:18:43.877 INFO:tasks.workunit.client.0.smithi081.stdout:2/657: readlink da/db/l10 0 2024-03-22T07:18:43.878 INFO:tasks.workunit.client.0.smithi081.stdout:7/877: symlink d9/d10/d16/d1e/d80/da1/l135 0 2024-03-22T07:18:43.882 INFO:tasks.workunit.client.0.smithi081.stdout:4/948: rmdir da/d18/d1b/d26 39 2024-03-22T07:18:43.885 INFO:tasks.workunit.client.0.smithi081.stdout:6/934: creat d3/d67/db2/d40/db9/d63/d7d/df3/f12b x:0 0 0 2024-03-22T07:18:43.887 INFO:tasks.workunit.client.0.smithi081.stdout:0/843: mknod d3/d6a/de0/d114/c123 0 2024-03-22T07:18:43.889 INFO:tasks.workunit.client.0.smithi081.stdout:9/782: sync 2024-03-22T07:18:43.890 INFO:tasks.workunit.client.0.smithi081.stdout:8/771: symlink d2/d3/d19/d64/l109 0 2024-03-22T07:18:43.894 INFO:tasks.workunit.client.0.smithi081.stdout:0/844: readlink d3/dc/l7f 0 2024-03-22T07:18:43.894 INFO:tasks.workunit.client.0.smithi081.stdout:0/845: stat d3/dc/cac 0 2024-03-22T07:18:43.897 INFO:tasks.workunit.client.0.smithi081.stdout:4/949: getdents da/df/d38/dd8 0 2024-03-22T07:18:43.897 INFO:tasks.workunit.client.0.smithi081.stdout:4/950: truncate da/f1d 2892548 0 2024-03-22T07:18:43.900 INFO:tasks.workunit.client.0.smithi081.stdout:0/846: mknod d3/dc/d33/d62/c124 0 2024-03-22T07:18:43.903 INFO:tasks.workunit.client.0.smithi081.stdout:9/783: rmdir d2/d73/d85/d7a/d97/dda 39 2024-03-22T07:18:43.903 INFO:tasks.workunit.client.0.smithi081.stdout:2/658: rename da/db/c16 to da/cd6 0 2024-03-22T07:18:43.905 INFO:tasks.workunit.client.0.smithi081.stdout:9/784: creat d2/d5/d65/f106 x:0 0 0 2024-03-22T07:18:43.905 INFO:tasks.workunit.client.0.smithi081.stdout:1/779: write d6/d6a/d85/df6/d32/dac/f4a [1508909,58765] 0 2024-03-22T07:18:43.906 INFO:tasks.workunit.client.0.smithi081.stdout:2/659: getdents da/db/d39/d47/d3b/d64 0 2024-03-22T07:18:43.906 INFO:tasks.workunit.client.0.smithi081.stdout:2/660: write da/d56/f6b [608159,96525] 0 2024-03-22T07:18:43.906 INFO:tasks.workunit.client.0.smithi081.stdout:2/661: readlink da/db/d39/d47/d3b/d41/la3 0 2024-03-22T07:18:43.907 INFO:tasks.workunit.client.0.smithi081.stdout:1/780: creat d6/d6a/d85/df6/d4c/d70/dd2/f104 x:0 0 0 2024-03-22T07:18:43.908 INFO:tasks.workunit.client.0.smithi081.stdout:1/781: chown d6/d18/c99 7147511 1 2024-03-22T07:18:43.908 INFO:tasks.workunit.client.0.smithi081.stdout:1/782: creat d6/d6a/d85/df6/d4c/d70/dd2/f105 x:0 0 0 2024-03-22T07:18:43.913 INFO:tasks.workunit.client.0.smithi081.stdout:8/772: dwrite d2/d3/d19/d1e/f5a [4194304,4194304] 0 2024-03-22T07:18:43.913 INFO:tasks.workunit.client.0.smithi081.stdout:9/785: rename d2/f40 to d2/d73/d85/f107 0 2024-03-22T07:18:43.913 INFO:tasks.workunit.client.0.smithi081.stdout:9/786: fdatasync d2/d10/d30/d57/dad/fbe 0 2024-03-22T07:18:43.916 INFO:tasks.workunit.client.0.smithi081.stdout:8/773: creat d2/d3/d19/d1e/d13/d66/dc1/f10a x:0 0 0 2024-03-22T07:18:43.918 INFO:tasks.workunit.client.0.smithi081.stdout:8/774: creat d2/d3/d28/d2b/f10b x:0 0 0 2024-03-22T07:18:43.918 INFO:tasks.workunit.client.0.smithi081.stdout:9/787: mkdir d2/d10/d108 0 2024-03-22T07:18:43.919 INFO:tasks.workunit.client.0.smithi081.stdout:8/775: symlink d2/d3/d19/d1e/d63/da0/db7/dc8/d7c/l10c 0 2024-03-22T07:18:43.920 INFO:tasks.workunit.client.0.smithi081.stdout:9/788: mknod d2/d73/d85/d7a/c109 0 2024-03-22T07:18:43.921 INFO:tasks.workunit.client.0.smithi081.stdout:8/776: rename d2/d3/d23/f3b to d2/d3/d9e/f10d 0 2024-03-22T07:18:43.923 INFO:tasks.workunit.client.0.smithi081.stdout:8/777: unlink d2/d3/d19/f6e 0 2024-03-22T07:18:43.924 INFO:tasks.workunit.client.0.smithi081.stdout:8/778: creat d2/d3/d19/d1e/d13/d22/d38/dbb/f10e x:0 0 0 2024-03-22T07:18:43.925 INFO:tasks.workunit.client.0.smithi081.stdout:8/779: mknod d2/d3/d9e/dc9/dca/c10f 0 2024-03-22T07:18:43.926 INFO:tasks.workunit.client.0.smithi081.stdout:8/780: read d2/f5e [4255395,33328] 0 2024-03-22T07:18:43.930 INFO:tasks.workunit.client.0.smithi081.stdout:1/783: dread d6/fe5 [4194304,4194304] 0 2024-03-22T07:18:43.930 INFO:tasks.workunit.client.0.smithi081.stdout:1/784: chown d6/d6a/f8f 28 1 2024-03-22T07:18:43.930 INFO:tasks.workunit.client.0.smithi081.stdout:1/785: chown d6/d6a/d85/df6/d4c/d70/dd2/ld9 14 1 2024-03-22T07:18:43.931 INFO:tasks.workunit.client.0.smithi081.stdout:1/786: mkdir d6/d6a/d85/df6/d24/d106 0 2024-03-22T07:18:43.932 INFO:tasks.workunit.client.0.smithi081.stdout:1/787: symlink d6/d6a/d85/df6/d24/d48/d6e/de7/l107 0 2024-03-22T07:18:43.933 INFO:tasks.workunit.client.0.smithi081.stdout:1/788: rename d6/d6a/d85/df6/d24/daa/ldb to d6/d18/d60/l108 0 2024-03-22T07:18:43.934 INFO:tasks.workunit.client.0.smithi081.stdout:1/789: creat d6/d6a/d85/df6/d32/dac/d30/d93/dbc/f109 x:0 0 0 2024-03-22T07:18:43.934 INFO:tasks.workunit.client.0.smithi081.stdout:1/790: fdatasync d6/d6a/d85/df6/d32/dac/d30/f7f 0 2024-03-22T07:18:43.935 INFO:tasks.workunit.client.0.smithi081.stdout:1/791: mknod d6/d18/d60/c10a 0 2024-03-22T07:18:43.937 INFO:tasks.workunit.client.0.smithi081.stdout:1/792: symlink d6/d6a/d85/df6/d32/dac/d30/d93/dd7/l10b 0 2024-03-22T07:18:43.937 INFO:tasks.workunit.client.0.smithi081.stdout:1/793: fdatasync d6/d6a/d85/df6/d32/dac/d30/ffc 0 2024-03-22T07:18:43.939 INFO:tasks.workunit.client.0.smithi081.stdout:8/781: dread d2/d3/d23/f53 [0,4194304] 0 2024-03-22T07:18:43.940 INFO:tasks.workunit.client.0.smithi081.stdout:8/782: mkdir d2/d3/d23/d110 0 2024-03-22T07:18:43.940 INFO:tasks.workunit.client.0.smithi081.stdout:8/783: fdatasync d2/d3/d19/d1e/d13/f47 0 2024-03-22T07:18:43.949 INFO:tasks.workunit.client.0.smithi081.stdout:5/919: dwrite d4/d10/d20/d4f/f5f [0,4194304] 0 2024-03-22T07:18:43.949 INFO:tasks.workunit.client.0.smithi081.stdout:5/920: write d4/d10/d20/d25/f8b [8396167,43159] 0 2024-03-22T07:18:43.949 INFO:tasks.workunit.client.0.smithi081.stdout:6/935: dwrite d3/d67/db2/df7/f101 [0,4194304] 0 2024-03-22T07:18:43.950 INFO:tasks.workunit.client.0.smithi081.stdout:8/784: dread d2/d3/d28/d2b/f7a [0,4194304] 0 2024-03-22T07:18:43.951 INFO:tasks.workunit.client.0.smithi081.stdout:8/785: stat d2/d3/d28/f82 0 2024-03-22T07:18:43.951 INFO:tasks.workunit.client.0.smithi081.stdout:8/786: dread - d2/d3/d19/d64/ffe zero size 2024-03-22T07:18:43.954 INFO:tasks.workunit.client.0.smithi081.stdout:8/787: mkdir d2/d3/d19/d1e/d13/d111 0 2024-03-22T07:18:43.961 INFO:tasks.workunit.client.0.smithi081.stdout:4/951: dwrite da/df/d38/dd8/d4f/daf/ded/f12f [0,4194304] 0 2024-03-22T07:18:43.963 INFO:tasks.workunit.client.0.smithi081.stdout:8/788: dread d2/d3/d28/d2b/f46 [0,4194304] 0 2024-03-22T07:18:43.963 INFO:tasks.workunit.client.0.smithi081.stdout:8/789: write d2/d3/f95 [411785,91652] 0 2024-03-22T07:18:43.963 INFO:tasks.workunit.client.0.smithi081.stdout:7/878: dwrite d9/d14/f54 [0,4194304] 0 2024-03-22T07:18:43.964 INFO:tasks.workunit.client.0.smithi081.stdout:4/952: creat da/d57/f13c x:0 0 0 2024-03-22T07:18:43.964 INFO:tasks.workunit.client.0.smithi081.stdout:4/953: creat da/df/d38/f13d x:0 0 0 2024-03-22T07:18:43.964 INFO:tasks.workunit.client.0.smithi081.stdout:4/954: readlink da/d18/d1b/d1e/d37/l2f 0 2024-03-22T07:18:43.964 INFO:tasks.workunit.client.0.smithi081.stdout:4/955: creat da/df/d38/dd8/d4f/daf/d101/f13e x:0 0 0 2024-03-22T07:18:43.969 INFO:tasks.workunit.client.0.smithi081.stdout:2/662: dwrite da/db/d39/d47/d3b/d41/d50/d6a/d70/f95 [4194304,4194304] 0 2024-03-22T07:18:43.970 INFO:tasks.workunit.client.0.smithi081.stdout:1/794: dwrite d6/d6a/d85/df6/d32/dac/d30/da8/fba [0,4194304] 0 2024-03-22T07:18:43.970 INFO:tasks.workunit.client.0.smithi081.stdout:1/795: getdents d6/d6a/d85/df6/d32/df9 0 2024-03-22T07:18:43.970 INFO:tasks.workunit.client.0.smithi081.stdout:7/879: rename d9/d10/fb7 to d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/f136 0 2024-03-22T07:18:43.971 INFO:tasks.workunit.client.0.smithi081.stdout:1/796: write d6/d6a/d85/df6/d32/dac/d30/f61 [3792,125923] 0 2024-03-22T07:18:43.973 INFO:tasks.workunit.client.0.smithi081.stdout:1/797: creat d6/d18/d41/f10c x:0 0 0 2024-03-22T07:18:43.974 INFO:tasks.workunit.client.0.smithi081.stdout:1/798: dread - d6/d6a/d85/df6/d32/dac/d30/ffc zero size 2024-03-22T07:18:43.974 INFO:tasks.workunit.client.0.smithi081.stdout:9/789: dwrite d2/d10/d30/f45 [0,4194304] 0 2024-03-22T07:18:43.974 INFO:tasks.workunit.client.0.smithi081.stdout:9/790: chown d2/d5/d36/f6c 58917370 1 2024-03-22T07:18:44.000 INFO:tasks.workunit.client.0.smithi081.stdout:9/791: dwrite d2/f5f [4194304,4194304] 0 2024-03-22T07:18:44.001 INFO:tasks.workunit.client.0.smithi081.stdout:8/790: dwrite d2/d3/d19/fe9 [0,4194304] 0 2024-03-22T07:18:44.001 INFO:tasks.workunit.client.0.smithi081.stdout:8/791: chown d2/d3/d19/d1e/d13/d22/d38/d78 57 1 2024-03-22T07:18:44.001 INFO:tasks.workunit.client.0.smithi081.stdout:9/792: dread - d2/d5d/d5a/fcd zero size 2024-03-22T07:18:44.001 INFO:tasks.workunit.client.0.smithi081.stdout:8/792: write d2/d3/d19/d1e/d13/d22/db8/fe8 [814593,88529] 0 2024-03-22T07:18:44.001 INFO:tasks.workunit.client.0.smithi081.stdout:8/793: chown d2/d3/d19 137435 1 2024-03-22T07:18:44.001 INFO:tasks.workunit.client.0.smithi081.stdout:8/794: write d2/d3/d19/d1e/d13/d66/d70/fd0 [1219549,99155] 0 2024-03-22T07:18:44.001 INFO:tasks.workunit.client.0.smithi081.stdout:8/795: fdatasync d2/d3/d19/d1e/d13/da5/f101 0 2024-03-22T07:18:44.001 INFO:tasks.workunit.client.0.smithi081.stdout:8/796: dread - d2/d3/d23/ff5 zero size 2024-03-22T07:18:44.001 INFO:tasks.workunit.client.0.smithi081.stdout:7/880: dwrite d9/d10/d16/d1e/d6f/f11e [0,4194304] 0 2024-03-22T07:18:44.018 INFO:tasks.workunit.client.0.smithi081.stdout:9/793: dwrite d2/d2b/f2d [4194304,4194304] 0 2024-03-22T07:18:44.078 INFO:tasks.workunit.client.0.smithi081.stdout:2/663: unlink da/db/c82 0 2024-03-22T07:18:44.080 INFO:tasks.workunit.client.0.smithi081.stdout:4/956: fsync da/df/d38/dd8/d4f/daf/d101/f13e 0 2024-03-22T07:18:44.084 INFO:tasks.workunit.client.0.smithi081.stdout:8/797: symlink d2/d3/d28/d2b/d61/dc6/l112 0 2024-03-22T07:18:44.088 INFO:tasks.workunit.client.0.smithi081.stdout:2/664: read da/d11/d21/f4e [509150,12435] 0 2024-03-22T07:18:44.090 INFO:tasks.workunit.client.0.smithi081.stdout:9/794: creat d2/d10/d30/f10a x:0 0 0 2024-03-22T07:18:44.097 INFO:tasks.workunit.client.0.smithi081.stdout:4/957: truncate da/de6/db8/fbb 121107 0 2024-03-22T07:18:44.097 INFO:tasks.workunit.client.0.smithi081.stdout:4/958: write da/df/df4/f102 [482342,100826] 0 2024-03-22T07:18:44.097 INFO:tasks.workunit.client.0.smithi081.stdout:1/799: mknod d6/d6a/d85/df6/c10d 0 2024-03-22T07:18:44.098 INFO:tasks.workunit.client.0.smithi081.stdout:8/798: symlink d2/d3/d23/d110/l113 0 2024-03-22T07:18:44.098 INFO:tasks.workunit.client.0.smithi081.stdout:2/665: creat da/db/d39/d46/d7e/fd7 x:0 0 0 2024-03-22T07:18:44.103 INFO:tasks.workunit.client.0.smithi081.stdout:1/800: creat d6/d6a/d85/dec/f10e x:0 0 0 2024-03-22T07:18:44.104 INFO:tasks.workunit.client.0.smithi081.stdout:1/801: symlink d6/d18/d60/l10f 0 2024-03-22T07:18:44.104 INFO:tasks.workunit.client.0.smithi081.stdout:1/802: chown d6/d18/da5 207960605 1 2024-03-22T07:18:44.104 INFO:tasks.workunit.client.0.smithi081.stdout:1/803: fdatasync d6/d6a/d85/df6/d24/d48/d6e/de7/f103 0 2024-03-22T07:18:44.104 INFO:tasks.workunit.client.0.smithi081.stdout:2/666: creat da/db/d39/d47/d3b/d41/d50/d6a/d70/fd8 x:0 0 0 2024-03-22T07:18:44.105 INFO:tasks.workunit.client.0.smithi081.stdout:1/804: creat d6/d6a/d85/df6/f110 x:0 0 0 2024-03-22T07:18:44.107 INFO:tasks.workunit.client.0.smithi081.stdout:2/667: getdents da/db/d39/d46/d42 0 2024-03-22T07:18:44.108 INFO:tasks.workunit.client.0.smithi081.stdout:2/668: mknod da/db/d39/d46/d42/cd9 0 2024-03-22T07:18:44.114 INFO:tasks.workunit.client.0.smithi081.stdout:4/959: dwrite da/d18/d1b/d26/f73 [0,4194304] 0 2024-03-22T07:18:44.115 INFO:tasks.workunit.client.0.smithi081.stdout:4/960: rename da/df/d38/dd8/cca to da/df/d38/dc9/c13f 0 2024-03-22T07:18:44.115 INFO:tasks.workunit.client.0.smithi081.stdout:4/961: fsync da/df/d38/dd8/d4f/d7c/d90/dd7/f11f 0 2024-03-22T07:18:44.115 INFO:tasks.workunit.client.0.smithi081.stdout:4/962: fsync da/df/d38/dd8/d4f/daf/f11e 0 2024-03-22T07:18:44.122 INFO:tasks.workunit.client.0.smithi081.stdout:4/963: write da/df/d38/dd8/d4f/daf/ff9 [393243,105518] 0 2024-03-22T07:18:44.124 INFO:tasks.workunit.client.0.smithi081.stdout:4/964: mknod da/d18/ddc/c140 0 2024-03-22T07:18:44.145 INFO:tasks.workunit.client.0.smithi081.stdout:8/799: write d2/d3/d9e/f10d [1216631,127717] 0 2024-03-22T07:18:44.147 INFO:tasks.workunit.client.0.smithi081.stdout:2/669: getdents da/db/d39/d46/d7e 0 2024-03-22T07:18:44.148 INFO:tasks.workunit.client.0.smithi081.stdout:8/800: mknod d2/d3/d19/d1e/d13/d66/d70/d86/d107/c114 0 2024-03-22T07:18:44.163 INFO:tasks.workunit.client.0.smithi081.stdout:4/965: dwrite da/df/d38/dd8/d4f/d7c/d90/dd7/f121 [0,4194304] 0 2024-03-22T07:18:44.165 INFO:tasks.workunit.client.0.smithi081.stdout:9/795: dwrite d2/d73/d85/fb1 [0,4194304] 0 2024-03-22T07:18:44.165 INFO:tasks.workunit.client.0.smithi081.stdout:1/805: dwrite d6/d6a/d85/df6/d32/ffe [0,4194304] 0 2024-03-22T07:18:44.166 INFO:tasks.workunit.client.0.smithi081.stdout:4/966: link da/d18/d1b/d26/d3e/c75 da/df/d38/dd8/d4f/d7c/d90/d10d/c141 0 2024-03-22T07:18:44.166 INFO:tasks.workunit.client.0.smithi081.stdout:4/967: dread - da/d3d/f9b zero size 2024-03-22T07:18:44.167 INFO:tasks.workunit.client.0.smithi081.stdout:4/968: creat da/d18/ddc/f142 x:0 0 0 2024-03-22T07:18:44.168 INFO:tasks.workunit.client.0.smithi081.stdout:4/969: read - da/d18/ddc/f142 zero size 2024-03-22T07:18:44.168 INFO:tasks.workunit.client.0.smithi081.stdout:4/970: stat da/df/d38/f76 0 2024-03-22T07:18:44.169 INFO:tasks.workunit.client.0.smithi081.stdout:4/971: creat da/df/d38/dd8/d4a/f143 x:0 0 0 2024-03-22T07:18:44.169 INFO:tasks.workunit.client.0.smithi081.stdout:4/972: dread - da/de6/db8/ff0 zero size 2024-03-22T07:18:44.171 INFO:tasks.workunit.client.0.smithi081.stdout:4/973: mkdir da/d18/d1b/d34/d144 0 2024-03-22T07:18:44.172 INFO:tasks.workunit.client.0.smithi081.stdout:2/670: dwrite da/db/d39/d47/fd5 [0,4194304] 0 2024-03-22T07:18:44.172 INFO:tasks.workunit.client.0.smithi081.stdout:8/801: dwrite d2/d3/d19/d1e/d13/d22/f6f [0,4194304] 0 2024-03-22T07:18:44.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/671: write da/db/d39/d46/d42/fc7 [637509,10923] 0 2024-03-22T07:18:44.177 INFO:tasks.workunit.client.0.smithi081.stdout:2/672: mkdir da/db/d39/d47/dda 0 2024-03-22T07:18:44.177 INFO:tasks.workunit.client.0.smithi081.stdout:2/673: write da/d11/d21/f90 [4785276,98183] 0 2024-03-22T07:18:44.178 INFO:tasks.workunit.client.0.smithi081.stdout:2/674: truncate da/db/d39/d47/d3b/d41/d50/f66 624657 0 2024-03-22T07:18:44.178 INFO:tasks.workunit.client.0.smithi081.stdout:4/974: write da/d18/d1b/d26/dce/dfd/d74/fbf [1117753,70913] 0 2024-03-22T07:18:44.178 INFO:tasks.workunit.client.0.smithi081.stdout:4/975: read - da/d3d/f108 zero size 2024-03-22T07:18:44.178 INFO:tasks.workunit.client.0.smithi081.stdout:1/806: dwrite d6/d6a/d85/df6/d32/dac/fcd [0,4194304] 0 2024-03-22T07:18:44.179 INFO:tasks.workunit.client.0.smithi081.stdout:8/802: rename d2/d3/d19/d1e/d63/da0/db7/c105 to d2/d3/d19/d1e/d13/d22/c115 0 2024-03-22T07:18:44.181 INFO:tasks.workunit.client.0.smithi081.stdout:1/807: mknod d6/d6a/d85/df6/d32/dac/d30/df2/c111 0 2024-03-22T07:18:44.182 INFO:tasks.workunit.client.0.smithi081.stdout:1/808: truncate d6/d6a/fb9 1255666 0 2024-03-22T07:18:44.182 INFO:tasks.workunit.client.0.smithi081.stdout:1/809: chown d6/d6a/d85/df6/d32/df9 240856 1 2024-03-22T07:18:44.189 INFO:tasks.workunit.client.0.smithi081.stdout:8/803: write d2/d3/d19/fe9 [1206710,116865] 0 2024-03-22T07:18:44.190 INFO:tasks.workunit.client.0.smithi081.stdout:3/636: sync 2024-03-22T07:18:44.190 INFO:tasks.workunit.client.0.smithi081.stdout:3/637: write db/de/d4b/fe0 [5162958,42961] 0 2024-03-22T07:18:44.191 INFO:tasks.workunit.client.0.smithi081.stdout:2/675: dread da/fa9 [0,4194304] 0 2024-03-22T07:18:44.202 INFO:tasks.workunit.client.0.smithi081.stdout:2/676: dread da/d11/d21/f33 [4194304,4194304] 0 2024-03-22T07:18:44.202 INFO:tasks.workunit.client.0.smithi081.stdout:2/677: readlink l1 0 2024-03-22T07:18:44.203 INFO:tasks.workunit.client.0.smithi081.stdout:4/976: dwrite da/d18/d1b/d26/dce/dfd/f51 [4194304,4194304] 0 2024-03-22T07:18:44.207 INFO:tasks.workunit.client.0.smithi081.stdout:1/810: rename d6/d6a/d85/df6/d32/ldd to d6/d6a/d85/df6/d32/dac/l112 0 2024-03-22T07:18:44.207 INFO:tasks.workunit.client.0.smithi081.stdout:1/811: fdatasync d6/d18/f27 0 2024-03-22T07:18:44.208 INFO:tasks.workunit.client.0.smithi081.stdout:8/804: dwrite d2/d3/d19/d64/fee [0,4194304] 0 2024-03-22T07:18:44.209 INFO:tasks.workunit.client.0.smithi081.stdout:3/638: rename db/d38/f6e to db/de/db8/fe5 0 2024-03-22T07:18:44.210 INFO:tasks.workunit.client.0.smithi081.stdout:1/812: rmdir d6/d6a/d85/df6/d32 39 2024-03-22T07:18:44.210 INFO:tasks.workunit.client.0.smithi081.stdout:1/813: chown d6/d6a/d85/df6/d24/d48/d6e/de7/fee 3542836 1 2024-03-22T07:18:44.210 INFO:tasks.workunit.client.0.smithi081.stdout:1/814: write d6/d18/d41/f46 [5141878,16900] 0 2024-03-22T07:18:44.210 INFO:tasks.workunit.client.0.smithi081.stdout:1/815: mknod d6/d6a/d85/df6/d24/daa/c113 0 2024-03-22T07:18:44.211 INFO:tasks.workunit.client.0.smithi081.stdout:1/816: unlink d6/d6a/d85/df6/d24/daa/c113 0 2024-03-22T07:18:44.214 INFO:tasks.workunit.client.0.smithi081.stdout:3/639: symlink db/le6 0 2024-03-22T07:18:44.215 INFO:tasks.workunit.client.0.smithi081.stdout:4/977: dread da/d18/d1b/d26/f43 [0,4194304] 0 2024-03-22T07:18:44.215 INFO:tasks.workunit.client.0.smithi081.stdout:4/978: creat da/d57/f145 x:0 0 0 2024-03-22T07:18:44.215 INFO:tasks.workunit.client.0.smithi081.stdout:4/979: write da/df/d38/dd8/d4a/fda [4668286,49430] 0 2024-03-22T07:18:44.216 INFO:tasks.workunit.client.0.smithi081.stdout:4/980: creat da/de6/db8/f146 x:0 0 0 2024-03-22T07:18:44.218 INFO:tasks.workunit.client.0.smithi081.stdout:3/640: dread db/d38/f47 [0,4194304] 0 2024-03-22T07:18:44.218 INFO:tasks.workunit.client.0.smithi081.stdout:4/981: creat da/d18/d1b/d26/d3e/f147 x:0 0 0 2024-03-22T07:18:44.220 INFO:tasks.workunit.client.0.smithi081.stdout:3/641: link db/de/d4b/fe0 db/de/db8/dcb/fe7 0 2024-03-22T07:18:44.220 INFO:tasks.workunit.client.0.smithi081.stdout:3/642: creat db/de/d4b/fe8 x:0 0 0 2024-03-22T07:18:44.221 INFO:tasks.workunit.client.0.smithi081.stdout:4/982: unlink da/d18/d1b/d26/d32/la9 0 2024-03-22T07:18:44.224 INFO:tasks.workunit.client.0.smithi081.stdout:3/643: dread db/de/d4b/d6a/d70/d3a/f46 [0,4194304] 0 2024-03-22T07:18:44.243 INFO:tasks.workunit.client.0.smithi081.stdout:4/983: dwrite da/d18/d1b/d34/f64 [4194304,4194304] 0 2024-03-22T07:18:44.243 INFO:tasks.workunit.client.0.smithi081.stdout:4/984: write da/df/d38/f136 [491287,87346] 0 2024-03-22T07:18:44.243 INFO:tasks.workunit.client.0.smithi081.stdout:4/985: fdatasync da/f87 0 2024-03-22T07:18:44.243 INFO:tasks.workunit.client.0.smithi081.stdout:3/644: dwrite db/de/d4b/d6a/d70/d3a/d5d/fd3 [0,4194304] 0 2024-03-22T07:18:44.243 INFO:tasks.workunit.client.0.smithi081.stdout:3/645: truncate db/de/fdc 535978 0 2024-03-22T07:18:44.245 INFO:tasks.workunit.client.0.smithi081.stdout:1/817: unlink d6/d6a/d85/df6/d4c/d70/f9c 0 2024-03-22T07:18:44.245 INFO:tasks.workunit.client.0.smithi081.stdout:1/818: readlink d6/d6a/d85/df6/d24/l65 0 2024-03-22T07:18:44.252 INFO:tasks.workunit.client.0.smithi081.stdout:3/646: dread db/de/d4b/d6a/d70/f92 [0,4194304] 0 2024-03-22T07:18:44.253 INFO:tasks.workunit.client.0.smithi081.stdout:3/647: symlink db/ddf/le9 0 2024-03-22T07:18:44.253 INFO:tasks.workunit.client.0.smithi081.stdout:3/648: chown db/de/cc6 0 1 2024-03-22T07:18:44.253 INFO:tasks.workunit.client.0.smithi081.stdout:3/649: fdatasync db/de/d4b/d6a/d70/d3a/f8f 0 2024-03-22T07:18:44.253 INFO:tasks.workunit.client.0.smithi081.stdout:3/650: fdatasync db/d38/da1/fcf 0 2024-03-22T07:18:44.253 INFO:tasks.workunit.client.0.smithi081.stdout:3/651: readlink db/de/l2c 0 2024-03-22T07:18:44.253 INFO:tasks.workunit.client.0.smithi081.stdout:3/652: mknod db/de/d4b/d9a/cea 0 2024-03-22T07:18:44.253 INFO:tasks.workunit.client.0.smithi081.stdout:3/653: creat db/d38/da1/db7/feb x:0 0 0 2024-03-22T07:18:44.254 INFO:tasks.workunit.client.0.smithi081.stdout:1/819: read d6/d6a/d85/df6/d32/dac/fcd [2701265,71598] 0 2024-03-22T07:18:44.254 INFO:tasks.workunit.client.0.smithi081.stdout:1/820: write d6/d6a/d85/fc0 [32355,101201] 0 2024-03-22T07:18:44.256 INFO:tasks.workunit.client.0.smithi081.stdout:1/821: rename d6/d6a/d85/df6/d32/dbe/dc4/ccb to d6/d6a/d85/df6/d24/d48/d6e/c114 0 2024-03-22T07:18:44.256 INFO:tasks.workunit.client.0.smithi081.stdout:3/654: dread db/d38/f80 [0,4194304] 0 2024-03-22T07:18:44.256 INFO:tasks.workunit.client.0.smithi081.stdout:3/655: fsync db/de/d4b/d6a/d70/f40 0 2024-03-22T07:18:44.256 INFO:tasks.workunit.client.0.smithi081.stdout:1/822: mkdir d6/d18/d41/d49/d115 0 2024-03-22T07:18:44.256 INFO:tasks.workunit.client.0.smithi081.stdout:1/823: chown d6/d18/d41/fd1 116583190 1 2024-03-22T07:18:44.256 INFO:tasks.workunit.client.0.smithi081.stdout:3/656: creat db/de/db8/fec x:0 0 0 2024-03-22T07:18:44.257 INFO:tasks.workunit.client.0.smithi081.stdout:3/657: getdents db/de/ddb 0 2024-03-22T07:18:44.257 INFO:tasks.workunit.client.0.smithi081.stdout:3/658: mkdir db/ddf/ded 0 2024-03-22T07:18:44.257 INFO:tasks.workunit.client.0.smithi081.stdout:3/659: write db/de/d4b/d6a/d70/f32 [23287,60442] 0 2024-03-22T07:18:44.258 INFO:tasks.workunit.client.0.smithi081.stdout:1/824: rename d6/d6a/d85/df6/d4c/c8c to d6/d18/d60/c116 0 2024-03-22T07:18:44.258 INFO:tasks.workunit.client.0.smithi081.stdout:1/825: fsync d6/d6a/d85/df6/d4c/f9f 0 2024-03-22T07:18:44.258 INFO:tasks.workunit.client.0.smithi081.stdout:1/826: rename d6/d6a/d85/df6/d24/d48/d5c/fb7 to d6/d18/d60/f117 0 2024-03-22T07:18:44.264 INFO:tasks.workunit.client.0.smithi081.stdout:3/660: write db/de/d4b/d6a/d70/d3a/d5d/fd3 [3651477,32696] 0 2024-03-22T07:18:44.265 INFO:tasks.workunit.client.0.smithi081.stdout:3/661: dread - db/d38/da1/db7/fe3 zero size 2024-03-22T07:18:44.267 INFO:tasks.workunit.client.0.smithi081.stdout:3/662: getdents db/de/d4b 0 2024-03-22T07:18:44.267 INFO:tasks.workunit.client.0.smithi081.stdout:3/663: write db/fa9 [429327,18436] 0 2024-03-22T07:18:44.267 INFO:tasks.workunit.client.0.smithi081.stdout:3/664: truncate db/de/d4b/db0/fd5 4803279 0 2024-03-22T07:18:44.267 INFO:tasks.workunit.client.0.smithi081.stdout:1/827: dread d6/d6a/d85/df6/d32/dac/d30/f34 [0,4194304] 0 2024-03-22T07:18:44.272 INFO:tasks.workunit.client.0.smithi081.stdout:3/665: read db/d38/f51 [1547990,19655] 0 2024-03-22T07:18:44.279 INFO:tasks.workunit.client.0.smithi081.stdout:3/666: dread db/de/d4b/db0/fd5 [0,4194304] 0 2024-03-22T07:18:44.279 INFO:tasks.workunit.client.0.smithi081.stdout:3/667: truncate db/d38/f6c 28923 0 2024-03-22T07:18:44.279 INFO:tasks.workunit.client.0.smithi081.stdout:3/668: write db/fa9 [1274865,54280] 0 2024-03-22T07:18:44.279 INFO:tasks.workunit.client.0.smithi081.stdout:1/828: link d6/d18/d60/ld0 d6/d6a/d85/df6/d76/l118 0 2024-03-22T07:18:44.279 INFO:tasks.workunit.client.0.smithi081.stdout:1/829: chown d6/d6a/d85/df6/d32/dac/d30/da8 26125 1 2024-03-22T07:18:44.280 INFO:tasks.workunit.client.0.smithi081.stdout:1/830: mknod d6/d18/d41/d49/d101/c119 0 2024-03-22T07:18:44.289 INFO:tasks.workunit.client.0.smithi081.stdout:3/669: dread db/de/d4b/d6a/d70/d3a/d5d/fbc [0,4194304] 0 2024-03-22T07:18:44.290 INFO:tasks.workunit.client.0.smithi081.stdout:3/670: mknod db/de/ddb/cee 0 2024-03-22T07:18:44.301 INFO:tasks.workunit.client.0.smithi081.stdout:1/831: dwrite d6/d18/f4f [4194304,4194304] 0 2024-03-22T07:18:44.309 INFO:tasks.workunit.client.0.smithi081.stdout:3/671: dwrite db/de/db8/dcb/fe7 [0,4194304] 0 2024-03-22T07:18:44.323 INFO:tasks.workunit.client.0.smithi081.stdout:1/832: dwrite d6/d6a/fb9 [0,4194304] 0 2024-03-22T07:18:44.324 INFO:tasks.workunit.client.0.smithi081.stdout:1/833: creat d6/d6a/d85/df6/d24/d48/d6e/f11a x:0 0 0 2024-03-22T07:18:44.324 INFO:tasks.workunit.client.0.smithi081.stdout:1/834: dread - d6/d6a/d85/df6/d24/d48/d6e/de7/f103 zero size 2024-03-22T07:18:44.324 INFO:tasks.workunit.client.0.smithi081.stdout:3/672: dwrite db/f77 [0,4194304] 0 2024-03-22T07:18:44.337 INFO:tasks.workunit.client.0.smithi081.stdout:3/673: dread db/f54 [0,4194304] 0 2024-03-22T07:18:44.338 INFO:tasks.workunit.client.0.smithi081.stdout:3/674: creat db/d38/ddd/fef x:0 0 0 2024-03-22T07:18:44.338 INFO:tasks.workunit.client.0.smithi081.stdout:3/675: mknod db/de/db8/cf0 0 2024-03-22T07:18:44.338 INFO:tasks.workunit.client.0.smithi081.stdout:1/835: dread d6/d6a/d85/df6/d32/dac/d30/f83 [0,4194304] 0 2024-03-22T07:18:44.341 INFO:tasks.workunit.client.0.smithi081.stdout:3/676: dread db/f5c [0,4194304] 0 2024-03-22T07:18:44.342 INFO:tasks.workunit.client.0.smithi081.stdout:3/677: symlink db/ddf/lf1 0 2024-03-22T07:18:44.342 INFO:tasks.workunit.client.0.smithi081.stdout:3/678: write db/de/d4b/d6a/f7c [3867915,60973] 0 2024-03-22T07:18:44.346 INFO:tasks.workunit.client.0.smithi081.stdout:1/836: dread d6/d6a/d85/df6/f25 [0,4194304] 0 2024-03-22T07:18:44.358 INFO:tasks.workunit.client.0.smithi081.stdout:3/679: dwrite db/de/f7d [0,4194304] 0 2024-03-22T07:18:44.366 INFO:tasks.workunit.client.0.smithi081.stdout:1/837: dwrite d6/d6a/d85/df6/d32/fef [0,4194304] 0 2024-03-22T07:18:44.367 INFO:tasks.workunit.client.0.smithi081.stdout:3/680: dread db/de/d4b/d6a/d70/f44 [0,4194304] 0 2024-03-22T07:18:44.368 INFO:tasks.workunit.client.0.smithi081.stdout:3/681: write db/de/d4b/d6a/d70/d3a/d5d/fb5 [328885,126587] 0 2024-03-22T07:18:44.369 INFO:tasks.workunit.client.0.smithi081.stdout:3/682: mknod db/de/db8/dd2/cf2 0 2024-03-22T07:18:44.369 INFO:tasks.workunit.client.0.smithi081.stdout:3/683: truncate db/d38/d8d/fc8 20395 0 2024-03-22T07:18:44.380 INFO:tasks.workunit.client.0.smithi081.stdout:3/684: dread db/de/d4b/d6a/fbd [0,4194304] 0 2024-03-22T07:18:44.380 INFO:tasks.workunit.client.0.smithi081.stdout:1/838: dwrite d6/d6a/d85/df6/d24/d48/d6e/fae [0,4194304] 0 2024-03-22T07:18:44.405 INFO:tasks.workunit.client.0.smithi081.stdout:1/839: dwrite d6/d6a/d85/df6/d32/dac/d30/d93/dbc/fa3 [4194304,4194304] 0 2024-03-22T07:18:44.412 INFO:tasks.workunit.client.0.smithi081.stdout:1/840: mknod d6/d6a/d85/dec/c11b 0 2024-03-22T07:18:44.413 INFO:tasks.workunit.client.0.smithi081.stdout:1/841: read d6/d18/d41/fd1 [104167,52404] 0 2024-03-22T07:18:44.521 INFO:tasks.workunit.client.0.smithi081.stdout:0/847: sync 2024-03-22T07:18:44.522 INFO:tasks.workunit.client.0.smithi081.stdout:0/848: mkdir d3/dc/d40/d69/dbf/d125 0 2024-03-22T07:18:44.522 INFO:tasks.workunit.client.0.smithi081.stdout:0/849: truncate d3/dc/d33/f96 4815338 0 2024-03-22T07:18:44.523 INFO:tasks.workunit.client.0.smithi081.stdout:0/850: unlink d3/dc/d33/f97 0 2024-03-22T07:18:44.537 INFO:tasks.workunit.client.0.smithi081.stdout:0/851: dwrite d3/f2e [0,4194304] 0 2024-03-22T07:18:44.538 INFO:tasks.workunit.client.0.smithi081.stdout:0/852: fsync d3/dc/f73 0 2024-03-22T07:18:44.539 INFO:tasks.workunit.client.0.smithi081.stdout:0/853: mkdir d3/d126 0 2024-03-22T07:18:44.540 INFO:tasks.workunit.client.0.smithi081.stdout:0/854: read d3/d6a/fef [13453,28510] 0 2024-03-22T07:18:44.540 INFO:tasks.workunit.client.0.smithi081.stdout:0/855: getdents d3/d4e 0 2024-03-22T07:18:44.542 INFO:tasks.workunit.client.0.smithi081.stdout:0/856: mknod d3/d32/df3/c127 0 2024-03-22T07:18:44.545 INFO:tasks.workunit.client.0.smithi081.stdout:0/857: rename d3/dc/le8 to d3/dc/d40/d118/d11a/l128 0 2024-03-22T07:18:44.545 INFO:tasks.workunit.client.0.smithi081.stdout:0/858: creat d3/dc/d40/d69/dd8/f129 x:0 0 0 2024-03-22T07:18:44.545 INFO:tasks.workunit.client.0.smithi081.stdout:0/859: creat d3/dc/d40/d69/d92/dc1/df6/f12a x:0 0 0 2024-03-22T07:18:44.546 INFO:tasks.workunit.client.0.smithi081.stdout:0/860: rmdir d3/dc/dc6 39 2024-03-22T07:18:44.547 INFO:tasks.workunit.client.0.smithi081.stdout:0/861: creat d3/dc/d40/d69/d92/dc1/f12b x:0 0 0 2024-03-22T07:18:44.547 INFO:tasks.workunit.client.0.smithi081.stdout:0/862: read d3/dc/d40/d69/d92/dc1/d74/fb7 [57141,8370] 0 2024-03-22T07:18:44.547 INFO:tasks.workunit.client.0.smithi081.stdout:0/863: chown d3/dc/d2b/lc4 475475842 1 2024-03-22T07:18:44.548 INFO:tasks.workunit.client.0.smithi081.stdout:0/864: chown d3/d6a/fc2 3492 1 2024-03-22T07:18:44.555 INFO:tasks.workunit.client.0.smithi081.stdout:0/865: rmdir d3 39 2024-03-22T07:18:44.555 INFO:tasks.workunit.client.0.smithi081.stdout:0/866: write d3/dc/d40/d69/dbf/fc0 [272553,12284] 0 2024-03-22T07:18:44.555 INFO:tasks.workunit.client.0.smithi081.stdout:0/867: chown d3/dc/d40/d69 76189781 1 2024-03-22T07:18:44.556 INFO:tasks.workunit.client.0.smithi081.stdout:0/868: creat d3/dc/dc6/f12c x:0 0 0 2024-03-22T07:18:44.575 INFO:tasks.workunit.client.0.smithi081.stdout:0/869: dwrite d3/d6a/dd5/fff [0,4194304] 0 2024-03-22T07:18:44.575 INFO:tasks.workunit.client.0.smithi081.stdout:0/870: creat d3/dc/d40/d118/f12d x:0 0 0 2024-03-22T07:18:44.576 INFO:tasks.workunit.client.0.smithi081.stdout:0/871: rmdir d3/d4e/dee/d112 0 2024-03-22T07:18:44.577 INFO:tasks.workunit.client.0.smithi081.stdout:0/872: rmdir d3/dc/d40/d69/d92/df4 39 2024-03-22T07:18:44.577 INFO:tasks.workunit.client.0.smithi081.stdout:0/873: readlink d3/dc/d2b/l90 0 2024-03-22T07:18:44.578 INFO:tasks.workunit.client.0.smithi081.stdout:0/874: creat d3/d9b/da4/f12e x:0 0 0 2024-03-22T07:18:44.578 INFO:tasks.workunit.client.0.smithi081.stdout:0/875: fsync d3/dc/d2b/fa0 0 2024-03-22T07:18:44.579 INFO:tasks.workunit.client.0.smithi081.stdout:0/876: mknod d3/dc/d2b/d4d/c12f 0 2024-03-22T07:18:44.588 INFO:tasks.workunit.client.0.smithi081.stdout:0/877: write d3/dc/d40/d69/d92/dc1/f38 [407064,106459] 0 2024-03-22T07:18:44.588 INFO:tasks.workunit.client.0.smithi081.stdout:0/878: chown d3/dc/d2b/d4d/c71 7 1 2024-03-22T07:18:44.588 INFO:tasks.workunit.client.0.smithi081.stdout:0/879: truncate d3/dc/d40/d69/dd8/f129 361454 0 2024-03-22T07:18:44.589 INFO:tasks.workunit.client.0.smithi081.stdout:0/880: truncate d3/dc/d40/d69/d92/dc1/fc8 3282486 0 2024-03-22T07:18:44.589 INFO:tasks.workunit.client.0.smithi081.stdout:0/881: getdents d3/dc/d40/d69/d92/dc1/df6 0 2024-03-22T07:18:44.591 INFO:tasks.workunit.client.0.smithi081.stdout:0/882: link d3/dc/d2b/d39/l55 d3/d9b/d10b/l130 0 2024-03-22T07:18:44.592 INFO:tasks.workunit.client.0.smithi081.stdout:0/883: write d3/d9b/fe4 [133627,42301] 0 2024-03-22T07:18:44.592 INFO:tasks.workunit.client.0.smithi081.stdout:0/884: fdatasync d3/dc/d2b/f2f 0 2024-03-22T07:18:44.592 INFO:tasks.workunit.client.0.smithi081.stdout:0/885: mkdir d3/dc/d2b/d39/d131 0 2024-03-22T07:18:44.593 INFO:tasks.workunit.client.0.smithi081.stdout:0/886: fsync d3/d6a/fa3 0 2024-03-22T07:18:44.602 INFO:tasks.workunit.client.0.smithi081.stdout:0/887: dread d3/dc/d33/f5a [0,4194304] 0 2024-03-22T07:18:44.603 INFO:tasks.workunit.client.0.smithi081.stdout:0/888: mknod d3/dc/d2b/d39/c132 0 2024-03-22T07:18:44.603 INFO:tasks.workunit.client.0.smithi081.stdout:0/889: dread - d3/dc/d40/d69/d92/dc1/df6/f11d zero size 2024-03-22T07:18:44.625 INFO:tasks.workunit.client.0.smithi081.stdout:5/921: sync 2024-03-22T07:18:44.625 INFO:tasks.workunit.client.0.smithi081.stdout:5/922: fdatasync d4/df/d22/d49/d115/f126 0 2024-03-22T07:18:44.625 INFO:tasks.workunit.client.0.smithi081.stdout:5/923: write d4/fe [7301925,101864] 0 2024-03-22T07:18:44.625 INFO:tasks.workunit.client.0.smithi081.stdout:5/924: stat d4/df/d73/d8c/dc0/de1/ff8 0 2024-03-22T07:18:44.625 INFO:tasks.workunit.client.0.smithi081.stdout:5/925: write d4/d10/d20/db2/d2a/f88 [1300902,82499] 0 2024-03-22T07:18:44.626 INFO:tasks.workunit.client.0.smithi081.stdout:5/926: write d4/d10/f1d [3630195,63169] 0 2024-03-22T07:18:44.760 INFO:tasks.workunit.client.0.smithi081.stdout:6/936: sync 2024-03-22T07:18:44.760 INFO:tasks.workunit.client.0.smithi081.stdout:7/881: sync 2024-03-22T07:18:44.761 INFO:tasks.workunit.client.0.smithi081.stdout:6/937: dread - d3/d67/db2/d40/db9/d63/d26/d52/fc1 zero size 2024-03-22T07:18:44.761 INFO:tasks.workunit.client.0.smithi081.stdout:7/882: creat d9/d10/d53/d81/d59/f137 x:0 0 0 2024-03-22T07:18:44.762 INFO:tasks.workunit.client.0.smithi081.stdout:6/938: unlink d3/d67/db2/d40/db9/d63/d26/d52/lee 0 2024-03-22T07:18:44.762 INFO:tasks.workunit.client.0.smithi081.stdout:6/939: write d3/da/f127 [366702,25218] 0 2024-03-22T07:18:44.762 INFO:tasks.workunit.client.0.smithi081.stdout:6/940: readlink d3/d67/la2 0 2024-03-22T07:18:44.767 INFO:tasks.workunit.client.0.smithi081.stdout:6/941: rename d3/da/de/c18 to d3/d67/db2/d40/db9/d63/db4/dc5/c12c 0 2024-03-22T07:18:44.768 INFO:tasks.workunit.client.0.smithi081.stdout:7/883: rmdir d9/d10/d53/d81/d59 39 2024-03-22T07:18:44.769 INFO:tasks.workunit.client.0.smithi081.stdout:6/942: rmdir d3/d67/db2/d40/db9/d63/d7d/df3 39 2024-03-22T07:18:44.770 INFO:tasks.workunit.client.0.smithi081.stdout:6/943: rename d3/d67/db2/d40/db9/d63/d26/d3f/l58 to d3/d67/db2/d40/l12d 0 2024-03-22T07:18:44.790 INFO:tasks.workunit.client.0.smithi081.stdout:6/944: mknod d3/da/d10e/c12e 0 2024-03-22T07:18:44.791 INFO:tasks.workunit.client.0.smithi081.stdout:6/945: mkdir d3/d67/db2/d40/db9/d12f 0 2024-03-22T07:18:44.791 INFO:tasks.workunit.client.0.smithi081.stdout:6/946: write d3/d67/db2/fa9 [826660,17700] 0 2024-03-22T07:18:44.792 INFO:tasks.workunit.client.0.smithi081.stdout:6/947: symlink d3/d67/db2/d40/db9/d63/d26/dfa/d108/l130 0 2024-03-22T07:18:44.792 INFO:tasks.workunit.client.0.smithi081.stdout:6/948: write d3/da/d48/fe4 [767884,54014] 0 2024-03-22T07:18:44.796 INFO:tasks.workunit.client.0.smithi081.stdout:6/949: dread d3/d67/db2/d40/db9/d63/f68 [0,4194304] 0 2024-03-22T07:18:44.797 INFO:tasks.workunit.client.0.smithi081.stdout:6/950: symlink d3/d67/db2/d40/db9/d10f/l131 0 2024-03-22T07:18:44.798 INFO:tasks.workunit.client.0.smithi081.stdout:6/951: unlink d3/d67/db2/d40/db9/d63/d7d/l88 0 2024-03-22T07:18:44.799 INFO:tasks.workunit.client.0.smithi081.stdout:6/952: creat d3/d67/db2/d40/db9/d63/d26/dfa/d108/f132 x:0 0 0 2024-03-22T07:18:44.799 INFO:tasks.workunit.client.0.smithi081.stdout:6/953: dread - d3/d67/db2/d40/db9/d63/d26/d3f/fa1 zero size 2024-03-22T07:18:44.800 INFO:tasks.workunit.client.0.smithi081.stdout:6/954: truncate d3/d67/db2/d40/db9/d63/d5e/d9d/fcb 59326 0 2024-03-22T07:18:44.802 INFO:tasks.workunit.client.0.smithi081.stdout:6/955: rename d3/d67/db2/d40/db9/d63/d26/d3f/d5a/c91 to d3/d67/db2/d128/c133 0 2024-03-22T07:18:44.802 INFO:tasks.workunit.client.0.smithi081.stdout:6/956: dread d3/d67/dd3/fd1 [0,4194304] 0 2024-03-22T07:18:44.804 INFO:tasks.workunit.client.0.smithi081.stdout:6/957: creat d3/da/d10e/f134 x:0 0 0 2024-03-22T07:18:44.804 INFO:tasks.workunit.client.0.smithi081.stdout:6/958: stat d3/d67/db2/fb8 0 2024-03-22T07:18:44.804 INFO:tasks.workunit.client.0.smithi081.stdout:6/959: dread - d3/d67/db2/d40/da8/fe9 zero size 2024-03-22T07:18:44.811 INFO:tasks.workunit.client.0.smithi081.stdout:6/960: rmdir d3/da 39 2024-03-22T07:18:44.811 INFO:tasks.workunit.client.0.smithi081.stdout:6/961: getdents d3/d67/db2 0 2024-03-22T07:18:44.812 INFO:tasks.workunit.client.0.smithi081.stdout:6/962: truncate d3/d67/db2/d40/db9/d63/d26/f44 2955672 0 2024-03-22T07:18:44.813 INFO:tasks.workunit.client.0.smithi081.stdout:6/963: mkdir d3/d67/db2/d40/db9/d10f/d135 0 2024-03-22T07:18:44.816 INFO:tasks.workunit.client.0.smithi081.stdout:6/964: write d3/d67/db2/d40/db9/d63/f68 [281672,92038] 0 2024-03-22T07:18:44.818 INFO:tasks.workunit.client.0.smithi081.stdout:6/965: unlink d3/da/de/db5/ccd 0 2024-03-22T07:18:44.818 INFO:tasks.workunit.client.0.smithi081.stdout:6/966: chown d3/da/f59 8540597 1 2024-03-22T07:18:44.824 INFO:tasks.workunit.client.0.smithi081.stdout:6/967: rename d3/da/de/f61 to d3/da/de/db5/dfe/d11a/f136 0 2024-03-22T07:18:44.826 INFO:tasks.workunit.client.0.smithi081.stdout:6/968: unlink d3/da/de/db5/dfe/d11a/ff8 0 2024-03-22T07:18:44.827 INFO:tasks.workunit.client.0.smithi081.stdout:6/969: truncate d3/d67/db2/d40/db9/d63/d26/d3f/d4c/f12a 140209 0 2024-03-22T07:18:44.827 INFO:tasks.workunit.client.0.smithi081.stdout:6/970: dread - d3/d67/db2/d40/db9/d63/d26/f124 zero size 2024-03-22T07:18:44.838 INFO:tasks.workunit.client.0.smithi081.stdout:3/685: sync 2024-03-22T07:18:44.838 INFO:tasks.workunit.client.0.smithi081.stdout:1/842: sync 2024-03-22T07:18:44.838 INFO:tasks.workunit.client.0.smithi081.stdout:4/986: sync 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:8/805: sync 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:9/796: sync 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:3/686: chown db/de/d4b/d6a/d70/d3a/d5d/f9f 3 1 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:2/678: sync 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:0/890: sync 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:3/687: creat db/de/d4b/d6a/d70/d3a/d5d/ff3 x:0 0 0 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:5/927: sync 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:0/891: fsync d3/dc/f36 0 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:0/892: chown d3/d6a/fd2 125897393 1 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:0/893: fdatasync d3/d4e/f88 0 2024-03-22T07:18:44.839 INFO:tasks.workunit.client.0.smithi081.stdout:5/928: write d4/d75/da5/fd7 [798952,107660] 0 2024-03-22T07:18:44.840 INFO:tasks.workunit.client.0.smithi081.stdout:8/806: creat d2/d3/d9e/dc9/dca/f116 x:0 0 0 2024-03-22T07:18:44.840 INFO:tasks.workunit.client.0.smithi081.stdout:8/807: chown d2/d3/d28/d2b/d61/c85 703710 1 2024-03-22T07:18:44.840 INFO:tasks.workunit.client.0.smithi081.stdout:3/688: readlink db/d23/l2b 0 2024-03-22T07:18:44.841 INFO:tasks.workunit.client.0.smithi081.stdout:2/679: mknod da/db/d6d/cdb 0 2024-03-22T07:18:44.843 INFO:tasks.workunit.client.0.smithi081.stdout:5/929: mknod d4/d9/d90/c130 0 2024-03-22T07:18:44.844 INFO:tasks.workunit.client.0.smithi081.stdout:9/797: mkdir d2/d73/d85/d10b 0 2024-03-22T07:18:44.845 INFO:tasks.workunit.client.0.smithi081.stdout:8/808: mknod d2/d3/d9e/c117 0 2024-03-22T07:18:44.846 INFO:tasks.workunit.client.0.smithi081.stdout:2/680: creat da/db/d39/d47/dda/fdc x:0 0 0 2024-03-22T07:18:44.847 INFO:tasks.workunit.client.0.smithi081.stdout:9/798: rmdir d2/d10/d30/d57/d66/d75/d94 39 2024-03-22T07:18:44.847 INFO:tasks.workunit.client.0.smithi081.stdout:9/799: chown d2/d73/d85/f4a 822805761 1 2024-03-22T07:18:44.849 INFO:tasks.workunit.client.0.smithi081.stdout:2/681: truncate da/d11/f1f 1989004 0 2024-03-22T07:18:44.851 INFO:tasks.workunit.client.0.smithi081.stdout:9/800: mknod d2/d73/d85/d7a/c10c 0 2024-03-22T07:18:44.853 INFO:tasks.workunit.client.0.smithi081.stdout:8/809: mknod d2/d3/d19/d1e/d13/d22/c118 0 2024-03-22T07:18:44.853 INFO:tasks.workunit.client.0.smithi081.stdout:8/810: dread - d2/d3/d19/d64/ffe zero size 2024-03-22T07:18:44.853 INFO:tasks.workunit.client.0.smithi081.stdout:8/811: write d2/d3/d19/d1e/d13/d22/d2f/f40 [2926207,2315] 0 2024-03-22T07:18:44.854 INFO:tasks.workunit.client.0.smithi081.stdout:3/689: dread db/de/f7d [0,4194304] 0 2024-03-22T07:18:44.854 INFO:tasks.workunit.client.0.smithi081.stdout:3/690: dread - db/de/d4b/db0/fd1 zero size 2024-03-22T07:18:44.854 INFO:tasks.workunit.client.0.smithi081.stdout:3/691: stat db/d38/d8d 0 2024-03-22T07:18:44.854 INFO:tasks.workunit.client.0.smithi081.stdout:2/682: symlink da/db/d39/d46/ldd 0 2024-03-22T07:18:44.854 INFO:tasks.workunit.client.0.smithi081.stdout:2/683: write da/db/d39/d47/d3b/d41/d50/d6a/d70/f95 [1467346,45508] 0 2024-03-22T07:18:44.855 INFO:tasks.workunit.client.0.smithi081.stdout:2/684: write da/f28 [768030,117298] 0 2024-03-22T07:18:44.855 INFO:tasks.workunit.client.0.smithi081.stdout:2/685: dread - da/db/d39/d46/f2e zero size 2024-03-22T07:18:44.855 INFO:tasks.workunit.client.0.smithi081.stdout:2/686: chown da/d11/d21/f4e 179 1 2024-03-22T07:18:44.856 INFO:tasks.workunit.client.0.smithi081.stdout:8/812: truncate d2/d3/d28/f49 586459 0 2024-03-22T07:18:44.856 INFO:tasks.workunit.client.0.smithi081.stdout:4/987: write da/df/d38/dd8/d4f/d7c/d90/dd7/f121 [2980938,68615] 0 2024-03-22T07:18:44.857 INFO:tasks.workunit.client.0.smithi081.stdout:2/687: creat da/db/d39/fde x:0 0 0 2024-03-22T07:18:44.858 INFO:tasks.workunit.client.0.smithi081.stdout:0/894: dwrite d3/dc/f1c [4194304,4194304] 0 2024-03-22T07:18:44.861 INFO:tasks.workunit.client.0.smithi081.stdout:8/813: rename d2/d3/d28/d2b/d61/dc6/fcd to d2/d3/d19/d1e/d63/da0/d9c/f119 0 2024-03-22T07:18:44.863 INFO:tasks.workunit.client.0.smithi081.stdout:2/688: unlink l1 0 2024-03-22T07:18:44.863 INFO:tasks.workunit.client.0.smithi081.stdout:2/689: chown da/db/d39/fae 21583010 1 2024-03-22T07:18:44.863 INFO:tasks.workunit.client.0.smithi081.stdout:2/690: readlink da/db/d39/d45/l85 0 2024-03-22T07:18:44.866 INFO:tasks.workunit.client.0.smithi081.stdout:0/895: symlink d3/dc/d2b/d39/dc9/l133 0 2024-03-22T07:18:44.866 INFO:tasks.workunit.client.0.smithi081.stdout:0/896: chown d3/dc/d2b/d39/dc9/ff0 6028571 1 2024-03-22T07:18:44.868 INFO:tasks.workunit.client.0.smithi081.stdout:4/988: mkdir da/d148 0 2024-03-22T07:18:44.871 INFO:tasks.workunit.client.0.smithi081.stdout:4/989: chown da/d3d/c69 102646646 1 2024-03-22T07:18:44.872 INFO:tasks.workunit.client.0.smithi081.stdout:8/814: symlink d2/d3/d19/d1e/d13/d22/l11a 0 2024-03-22T07:18:44.875 INFO:tasks.workunit.client.0.smithi081.stdout:2/691: rename da/f25 to da/db/d39/d46/d42/db4/dbb/fdf 0 2024-03-22T07:18:44.881 INFO:tasks.workunit.client.0.smithi081.stdout:0/897: read d3/f3e [1417932,6895] 0 2024-03-22T07:18:44.881 INFO:tasks.workunit.client.0.smithi081.stdout:5/930: dwrite d4/d10/d20/d25/f8b [0,4194304] 0 2024-03-22T07:18:44.882 INFO:tasks.workunit.client.0.smithi081.stdout:0/898: mknod d3/dc/d40/df7/c134 0 2024-03-22T07:18:44.884 INFO:tasks.workunit.client.0.smithi081.stdout:3/692: dwrite db/de/d4b/d6a/d70/d3a/f8f [0,4194304] 0 2024-03-22T07:18:44.884 INFO:tasks.workunit.client.0.smithi081.stdout:3/693: fdatasync db/de/fdc 0 2024-03-22T07:18:44.884 INFO:tasks.workunit.client.0.smithi081.stdout:0/899: mkdir d3/dc/d40/d69/d92/dc1/df6/d135 0 2024-03-22T07:18:44.886 INFO:tasks.workunit.client.0.smithi081.stdout:9/801: dwrite d2/d73/d85/fa0 [0,4194304] 0 2024-03-22T07:18:44.887 INFO:tasks.workunit.client.0.smithi081.stdout:1/843: link d6/d18/d60/ld0 d6/d6a/d85/df6/db0/l11c 0 2024-03-22T07:18:44.887 INFO:tasks.workunit.client.0.smithi081.stdout:5/931: rename d4/d10/d20/db2/d2f/c53 to d4/c131 0 2024-03-22T07:18:44.891 INFO:tasks.workunit.client.0.smithi081.stdout:0/900: creat d3/d6a/de0/f136 x:0 0 0 2024-03-22T07:18:44.896 INFO:tasks.workunit.client.0.smithi081.stdout:4/990: dwrite da/d18/d1b/d26/d3e/fd1 [0,4194304] 0 2024-03-22T07:18:44.896 INFO:tasks.workunit.client.0.smithi081.stdout:5/932: link d4/d10/d20/d25/db8/f3b d4/d10/d20/db2/f132 0 2024-03-22T07:18:44.896 INFO:tasks.workunit.client.0.smithi081.stdout:5/933: creat d4/d10/d20/d25/db8/de0/f133 x:0 0 0 2024-03-22T07:18:44.900 INFO:tasks.workunit.client.0.smithi081.stdout:9/802: dread d2/d73/d74/dd9/fdb [0,4194304] 0 2024-03-22T07:18:44.901 INFO:tasks.workunit.client.0.smithi081.stdout:8/815: dwrite d2/d3/d23/f53 [0,4194304] 0 2024-03-22T07:18:44.901 INFO:tasks.workunit.client.0.smithi081.stdout:4/991: unlink da/d57/ca2 0 2024-03-22T07:18:44.906 INFO:tasks.workunit.client.0.smithi081.stdout:4/992: link da/d3d/f108 da/d18/d1b/d34/d144/f149 0 2024-03-22T07:18:44.907 INFO:tasks.workunit.client.0.smithi081.stdout:5/934: getdents d4/d10/daf 0 2024-03-22T07:18:44.908 INFO:tasks.workunit.client.0.smithi081.stdout:9/803: rmdir d2/d5d/d5a 39 2024-03-22T07:18:44.914 INFO:tasks.workunit.client.0.smithi081.stdout:5/935: mkdir d4/df/d22/d49/d134 0 2024-03-22T07:18:44.914 INFO:tasks.workunit.client.0.smithi081.stdout:5/936: chown d4/df/d22/f58 162 1 2024-03-22T07:18:44.915 INFO:tasks.workunit.client.0.smithi081.stdout:9/804: dread d2/d5/d9d/f7b [0,4194304] 0 2024-03-22T07:18:44.919 INFO:tasks.workunit.client.0.smithi081.stdout:1/844: dwrite d6/f7b [4194304,4194304] 0 2024-03-22T07:18:44.919 INFO:tasks.workunit.client.0.smithi081.stdout:0/901: dwrite d3/f44 [0,4194304] 0 2024-03-22T07:18:44.920 INFO:tasks.workunit.client.0.smithi081.stdout:5/937: rename d4/d10/d20/c41 to d4/d10/c135 0 2024-03-22T07:18:44.923 INFO:tasks.workunit.client.0.smithi081.stdout:1/845: truncate d6/d6a/d85/df6/d32/dac/fcd 1212211 0 2024-03-22T07:18:44.924 INFO:tasks.workunit.client.0.smithi081.stdout:1/846: mknod d6/d6a/d85/df6/db0/c11d 0 2024-03-22T07:18:44.928 INFO:tasks.workunit.client.0.smithi081.stdout:0/902: creat d3/d4e/f137 x:0 0 0 2024-03-22T07:18:44.928 INFO:tasks.workunit.client.0.smithi081.stdout:0/903: truncate d3/dc/f73 866669 0 2024-03-22T07:18:44.930 INFO:tasks.workunit.client.0.smithi081.stdout:5/938: mknod d4/df/d73/d8c/dc0/d12f/c136 0 2024-03-22T07:18:44.932 INFO:tasks.workunit.client.0.smithi081.stdout:0/904: getdents d3/d6a 0 2024-03-22T07:18:44.932 INFO:tasks.workunit.client.0.smithi081.stdout:0/905: creat d3/dc/d40/d118/f138 x:0 0 0 2024-03-22T07:18:44.932 INFO:tasks.workunit.client.0.smithi081.stdout:9/805: dwrite d2/d78/d93/f9f [0,4194304] 0 2024-03-22T07:18:44.932 INFO:tasks.workunit.client.0.smithi081.stdout:9/806: chown d2/d10/d30/d57/f7d 1895824 1 2024-03-22T07:18:44.933 INFO:tasks.workunit.client.0.smithi081.stdout:5/939: creat d4/df/de6/dec/f137 x:0 0 0 2024-03-22T07:18:44.933 INFO:tasks.workunit.client.0.smithi081.stdout:5/940: write d4/d10/f8d [188214,96764] 0 2024-03-22T07:18:44.933 INFO:tasks.workunit.client.0.smithi081.stdout:5/941: dread - d4/d10/d20/db2/f104 zero size 2024-03-22T07:18:44.933 INFO:tasks.workunit.client.0.smithi081.stdout:5/942: chown d4/d10/d20/d25/db8/de0/f133 10853 1 2024-03-22T07:18:44.933 INFO:tasks.workunit.client.0.smithi081.stdout:5/943: fdatasync d4/df/f46 0 2024-03-22T07:18:44.935 INFO:tasks.workunit.client.0.smithi081.stdout:0/906: creat d3/d4e/dee/f139 x:0 0 0 2024-03-22T07:18:44.946 INFO:tasks.workunit.client.0.smithi081.stdout:9/807: rename d2/d5/le5 to d2/d73/d85/ddd/l10d 0 2024-03-22T07:18:44.947 INFO:tasks.workunit.client.0.smithi081.stdout:9/808: dread d2/d10/d30/d57/fb8 [0,4194304] 0 2024-03-22T07:18:44.948 INFO:tasks.workunit.client.0.smithi081.stdout:5/944: link d4/d9/f91 d4/d10/d20/db2/d2f/f138 0 2024-03-22T07:18:44.948 INFO:tasks.workunit.client.0.smithi081.stdout:5/945: write d4/df/d73/fab [4222496,7896] 0 2024-03-22T07:18:44.954 INFO:tasks.workunit.client.0.smithi081.stdout:0/907: unlink d3/dc/d2b/d39/dc9/l133 0 2024-03-22T07:18:44.956 INFO:tasks.workunit.client.0.smithi081.stdout:5/946: unlink d4/df/d22/d4d/cce 0 2024-03-22T07:18:44.956 INFO:tasks.workunit.client.0.smithi081.stdout:5/947: write f1 [429853,103440] 0 2024-03-22T07:18:44.958 INFO:tasks.workunit.client.0.smithi081.stdout:0/908: unlink d3/dc/d2b/c8b 0 2024-03-22T07:18:44.959 INFO:tasks.workunit.client.0.smithi081.stdout:0/909: creat d3/dc/d40/d69/d92/dc1/df6/d135/f13a x:0 0 0 2024-03-22T07:18:44.962 INFO:tasks.workunit.client.0.smithi081.stdout:5/948: read d4/d10/d20/db2/d2a/f9b [873724,96627] 0 2024-03-22T07:18:44.968 INFO:tasks.workunit.client.0.smithi081.stdout:1/847: link d6/d6a/d85/df6/d32/fef d6/d18/d60/f11e 0 2024-03-22T07:18:44.971 INFO:tasks.workunit.client.0.smithi081.stdout:1/848: mknod d6/d6a/d85/df6/d4c/d70/dd2/c11f 0 2024-03-22T07:18:44.971 INFO:tasks.workunit.client.0.smithi081.stdout:1/849: read d6/fc [2199024,73956] 0 2024-03-22T07:18:44.971 INFO:tasks.workunit.client.0.smithi081.stdout:5/949: write d4/d10/d20/db2/d2f/fc4 [1658861,33484] 0 2024-03-22T07:18:44.971 INFO:tasks.workunit.client.0.smithi081.stdout:5/950: creat d4/d75/da5/f139 x:0 0 0 2024-03-22T07:18:44.976 INFO:tasks.workunit.client.0.smithi081.stdout:5/951: write d4/d10/d20/d25/f9e [4718229,109582] 0 2024-03-22T07:18:44.978 INFO:tasks.workunit.client.0.smithi081.stdout:1/850: dread d6/d6a/d85/df6/db0/fb4 [0,4194304] 0 2024-03-22T07:18:44.979 INFO:tasks.workunit.client.0.smithi081.stdout:1/851: creat d6/dc6/f120 x:0 0 0 2024-03-22T07:18:44.985 INFO:tasks.workunit.client.0.smithi081.stdout:5/952: dread d4/df/f6d [0,4194304] 0 2024-03-22T07:18:44.987 INFO:tasks.workunit.client.0.smithi081.stdout:5/953: link d4/df/d73/fab d4/d9/d90/f13a 0 2024-03-22T07:18:45.001 INFO:tasks.workunit.client.0.smithi081.stdout:1/852: dwrite d6/d18/f4f [4194304,4194304] 0 2024-03-22T07:18:45.144 INFO:tasks.workunit.client.0.smithi081.stdout:7/884: sync 2024-03-22T07:18:45.164 INFO:tasks.workunit.client.0.smithi081.stdout:7/885: dwrite d9/d10/d16/d1e/d6f/f94 [0,4194304] 0 2024-03-22T07:18:45.182 INFO:tasks.workunit.client.0.smithi081.stdout:7/886: dwrite d9/d10/d53/d81/d59/fcf [0,4194304] 0 2024-03-22T07:18:45.183 INFO:tasks.workunit.client.0.smithi081.stdout:7/887: mkdir d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d10c/d138 0 2024-03-22T07:18:45.202 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:45 smithi081 ceph-mon[113965]: pgmap v218: 65 pgs: 65 active+clean; 1.9 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 129 MiB/s rd, 167 MiB/s wr, 359 op/s 2024-03-22T07:18:45.204 INFO:tasks.workunit.client.0.smithi081.stdout:7/888: dwrite d9/d10/d53/f82 [0,4194304] 0 2024-03-22T07:18:45.205 INFO:tasks.workunit.client.0.smithi081.stdout:7/889: readlink d9/d10/d4b/l118 0 2024-03-22T07:18:45.205 INFO:tasks.workunit.client.0.smithi081.stdout:7/890: write d9/f11 [9167635,31869] 0 2024-03-22T07:18:45.208 INFO:tasks.workunit.client.0.smithi081.stdout:7/891: symlink d9/d10/d53/d81/d59/d89/dd4/de0/d6d/l139 0 2024-03-22T07:18:45.210 INFO:tasks.workunit.client.0.smithi081.stdout:7/892: dread d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/fe2 [0,4194304] 0 2024-03-22T07:18:45.212 INFO:tasks.workunit.client.0.smithi081.stdout:7/893: mknod d9/dc/df6/c13a 0 2024-03-22T07:18:45.214 INFO:tasks.workunit.client.0.smithi081.stdout:7/894: symlink d9/dc/dbc/l13b 0 2024-03-22T07:18:45.214 INFO:tasks.workunit.client.0.smithi081.stdout:7/895: chown d9/d3f/d105 103528 1 2024-03-22T07:18:45.214 INFO:tasks.workunit.client.0.smithi081.stdout:7/896: readlink d9/d10/d16/d1e/le3 0 2024-03-22T07:18:45.215 INFO:tasks.workunit.client.0.smithi081.stdout:7/897: creat d9/d10/d53/d81/d59/dbe/f13c x:0 0 0 2024-03-22T07:18:45.215 INFO:tasks.workunit.client.0.smithi081.stdout:7/898: read - d9/d10/d53/d81/d59/dbe/fff zero size 2024-03-22T07:18:45.216 INFO:tasks.workunit.client.0.smithi081.stdout:7/899: write d9/de4/f115 [216497,52405] 0 2024-03-22T07:18:45.216 INFO:tasks.workunit.client.0.smithi081.stdout:7/900: rmdir d9/d10/d16/d1e/d80/da1 39 2024-03-22T07:18:45.217 INFO:tasks.workunit.client.0.smithi081.stdout:7/901: symlink d9/d10/d16/d1e/d80/da1/l13d 0 2024-03-22T07:18:45.218 INFO:tasks.workunit.client.0.smithi081.stdout:7/902: creat d9/d3f/d12d/f13e x:0 0 0 2024-03-22T07:18:45.218 INFO:tasks.workunit.client.0.smithi081.stdout:7/903: creat d9/de4/f13f x:0 0 0 2024-03-22T07:18:45.220 INFO:tasks.workunit.client.0.smithi081.stdout:7/904: link f4 d9/f140 0 2024-03-22T07:18:45.221 INFO:tasks.workunit.client.0.smithi081.stdout:7/905: symlink d9/d3f/d12d/l141 0 2024-03-22T07:18:45.221 INFO:tasks.workunit.client.0.smithi081.stdout:7/906: chown d9/d10/d119/c12b 18 1 2024-03-22T07:18:45.222 INFO:tasks.workunit.client.0.smithi081.stdout:7/907: mkdir d9/d142 0 2024-03-22T07:18:45.223 INFO:tasks.workunit.client.0.smithi081.stdout:7/908: mkdir d9/d3f/d52/d83/d143 0 2024-03-22T07:18:45.223 INFO:tasks.workunit.client.0.smithi081.stdout:7/909: truncate d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/fdf 258416 0 2024-03-22T07:18:45.224 INFO:tasks.workunit.client.0.smithi081.stdout:7/910: symlink d9/d3f/d52/d83/d143/l144 0 2024-03-22T07:18:45.233 INFO:tasks.workunit.client.0.smithi081.stdout:7/911: dread d9/d3f/db3/fc6 [0,4194304] 0 2024-03-22T07:18:45.233 INFO:tasks.workunit.client.0.smithi081.stdout:7/912: write d9/d10/d53/d81/d59/d89/dd4/f7f [35955,26621] 0 2024-03-22T07:18:45.233 INFO:tasks.workunit.client.0.smithi081.stdout:7/913: chown d9/d10/d16/db6 518032 1 2024-03-22T07:18:45.234 INFO:tasks.workunit.client.0.smithi081.stdout:7/914: dread d9/d3f/f79 [0,4194304] 0 2024-03-22T07:18:45.235 INFO:tasks.workunit.client.0.smithi081.stdout:7/915: mkdir d9/d3f/d52/d99/d145 0 2024-03-22T07:18:45.235 INFO:tasks.workunit.client.0.smithi081.stdout:7/916: truncate d9/d14/f24 5223253 0 2024-03-22T07:18:45.236 INFO:tasks.workunit.client.0.smithi081.stdout:7/917: mkdir d9/d10/d16/d1e/d80/da1/d146 0 2024-03-22T07:18:45.244 INFO:tasks.workunit.client.0.smithi081.stdout:7/918: dread d9/d10/d53/d81/d59/f9f [0,4194304] 0 2024-03-22T07:18:45.262 INFO:tasks.workunit.client.0.smithi081.stdout:7/919: dwrite d9/dc/f4a [0,4194304] 0 2024-03-22T07:18:45.263 INFO:tasks.workunit.client.0.smithi081.stdout:7/920: creat d9/d3f/d105/f147 x:0 0 0 2024-03-22T07:18:45.263 INFO:tasks.workunit.client.0.smithi081.stdout:7/921: chown d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/l131 10 1 2024-03-22T07:18:45.265 INFO:tasks.workunit.client.0.smithi081.stdout:7/922: dread d9/d10/d16/db6/dd9/fe8 [0,4194304] 0 2024-03-22T07:18:45.267 INFO:tasks.workunit.client.0.smithi081.stdout:7/923: unlink d9/c10f 0 2024-03-22T07:18:45.273 INFO:tasks.workunit.client.0.smithi081.stdout:7/924: dread d9/d3f/d52/d83/f8e [0,4194304] 0 2024-03-22T07:18:45.274 INFO:tasks.workunit.client.0.smithi081.stdout:7/925: symlink d9/d10/d9e/l148 0 2024-03-22T07:18:45.274 INFO:tasks.workunit.client.0.smithi081.stdout:6/971: sync 2024-03-22T07:18:45.275 INFO:tasks.workunit.client.0.smithi081.stdout:7/926: stat d9/d10/d53/d81/l92 0 2024-03-22T07:18:45.276 INFO:tasks.workunit.client.0.smithi081.stdout:6/972: creat d3/d67/db2/d40/db9/f137 x:0 0 0 2024-03-22T07:18:45.276 INFO:tasks.workunit.client.0.smithi081.stdout:7/927: dread d9/d10/d16/f6b [0,4194304] 0 2024-03-22T07:18:45.278 INFO:tasks.workunit.client.0.smithi081.stdout:6/973: symlink d3/d67/db2/d40/db9/d63/d26/d3f/d5a/l138 0 2024-03-22T07:18:45.279 INFO:tasks.workunit.client.0.smithi081.stdout:7/928: link d9/d10/d53/f123 d9/d10/d2a/f149 0 2024-03-22T07:18:45.279 INFO:tasks.workunit.client.0.smithi081.stdout:7/929: readlink d9/d10/d2a/l36 0 2024-03-22T07:18:45.283 INFO:tasks.workunit.client.0.smithi081.stdout:7/930: mknod d9/d10/d16/d1e/d80/da1/d146/c14a 0 2024-03-22T07:18:45.288 INFO:tasks.workunit.client.0.smithi081.stdout:7/931: dread d9/d3f/db3/fc6 [0,4194304] 0 2024-03-22T07:18:45.296 INFO:tasks.workunit.client.0.smithi081.stdout:6/974: dwrite d3/d67/f93 [0,4194304] 0 2024-03-22T07:18:45.299 INFO:tasks.workunit.client.0.smithi081.stdout:6/975: mknod d3/d67/db2/d40/db9/d63/d7d/df3/c139 0 2024-03-22T07:18:45.300 INFO:tasks.workunit.client.0.smithi081.stdout:6/976: mkdir d3/d67/db2/d40/db9/d63/d7d/dc6/d11e/d13a 0 2024-03-22T07:18:45.300 INFO:tasks.workunit.client.0.smithi081.stdout:6/977: read - d3/d67/db2/d40/db9/d63/d5e/f9e zero size 2024-03-22T07:18:45.301 INFO:tasks.workunit.client.0.smithi081.stdout:6/978: stat d3/da/d48 0 2024-03-22T07:18:45.301 INFO:tasks.workunit.client.0.smithi081.stdout:6/979: fdatasync d3/d67/db2/d40/db9/d63/d5e/f116 0 2024-03-22T07:18:45.302 INFO:tasks.workunit.client.0.smithi081.stdout:6/980: mkdir d3/d67/db2/d40/db9/d63/d26/d52/d13b 0 2024-03-22T07:18:45.302 INFO:tasks.workunit.client.0.smithi081.stdout:6/981: write d3/d67/db2/d40/db9/d63/d26/dfa/f113 [818492,41032] 0 2024-03-22T07:18:45.302 INFO:tasks.workunit.client.0.smithi081.stdout:6/982: read - d3/d67/db2/d40/db9/d63/d26/d3f/d5a/fa7 zero size 2024-03-22T07:18:45.302 INFO:tasks.workunit.client.0.smithi081.stdout:6/983: chown d3/da/d48/fde 28 1 2024-03-22T07:18:45.302 INFO:tasks.workunit.client.0.smithi081.stdout:6/984: mknod d3/da/de/db5/c13c 0 2024-03-22T07:18:45.303 INFO:tasks.workunit.client.0.smithi081.stdout:6/985: link d3/d67/db2/d40/db9/d10f/l131 d3/d67/dd3/df4/l13d 0 2024-03-22T07:18:45.305 INFO:tasks.workunit.client.0.smithi081.stdout:6/986: mkdir d3/d67/db2/d40/db9/d63/d26/d3f/d5a/d13e 0 2024-03-22T07:18:45.305 INFO:tasks.workunit.client.0.smithi081.stdout:7/932: dwrite d9/d10/d16/d1e/d80/ffe [0,4194304] 0 2024-03-22T07:18:45.306 INFO:tasks.workunit.client.0.smithi081.stdout:7/933: write d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/f124 [71465,13807] 0 2024-03-22T07:18:45.306 INFO:tasks.workunit.client.0.smithi081.stdout:7/934: readlink d9/dc/le5 0 2024-03-22T07:18:45.329 INFO:tasks.workunit.client.0.smithi081.stdout:6/987: dwrite d3/da/f53 [0,4194304] 0 2024-03-22T07:18:45.330 INFO:tasks.workunit.client.0.smithi081.stdout:7/935: dwrite d9/d3f/d52/ff0 [0,4194304] 0 2024-03-22T07:18:45.330 INFO:tasks.workunit.client.0.smithi081.stdout:7/936: truncate d9/d10/d53/d81/d59/d89/dd4/de0/f108 702241 0 2024-03-22T07:18:45.333 INFO:tasks.workunit.client.0.smithi081.stdout:6/988: rename d3/d67/db2/d40/db9/d63/d26/d3f/c2f to d3/da/de/c13f 0 2024-03-22T07:18:45.333 INFO:tasks.workunit.client.0.smithi081.stdout:6/989: chown d3/d67/db2/d40/db9/c11c 9226 1 2024-03-22T07:18:45.334 INFO:tasks.workunit.client.0.smithi081.stdout:7/937: link d9/d3f/d52/ldb d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/l14b 0 2024-03-22T07:18:45.334 INFO:tasks.workunit.client.0.smithi081.stdout:7/938: chown d9/d3f/d52/fc8 93441533 1 2024-03-22T07:18:45.335 INFO:tasks.workunit.client.0.smithi081.stdout:7/939: symlink d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/dc1/l14c 0 2024-03-22T07:18:45.335 INFO:tasks.workunit.client.0.smithi081.stdout:7/940: chown d9/d10/d16/fda 6356531 1 2024-03-22T07:18:45.336 INFO:tasks.workunit.client.0.smithi081.stdout:7/941: truncate d9/dc/f4a 4980650 0 2024-03-22T07:18:45.336 INFO:tasks.workunit.client.0.smithi081.stdout:7/942: getdents d9/d10/d2a/d73 0 2024-03-22T07:18:45.340 INFO:tasks.workunit.client.0.smithi081.stdout:6/990: write d3/da/f45 [830397,68833] 0 2024-03-22T07:18:45.342 INFO:tasks.workunit.client.0.smithi081.stdout:6/991: mknod d3/da/d48/c140 0 2024-03-22T07:18:45.344 INFO:tasks.workunit.client.0.smithi081.stdout:6/992: truncate d3/dcc/fed 3740034 0 2024-03-22T07:18:45.344 INFO:tasks.workunit.client.0.smithi081.stdout:6/993: chown d3/d67/db2/d40/db9/d63/ld6 24925586 1 2024-03-22T07:18:45.345 INFO:tasks.workunit.client.0.smithi081.stdout:7/943: write d9/d10/d2a/f47 [5595609,6744] 0 2024-03-22T07:18:45.346 INFO:tasks.workunit.client.0.smithi081.stdout:6/994: rmdir d3 39 2024-03-22T07:18:45.346 INFO:tasks.workunit.client.0.smithi081.stdout:6/995: fdatasync d3/d67/db2/d40/db9/d63/db4/fff 0 2024-03-22T07:18:45.351 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:45 smithi149 ceph-mon[83447]: pgmap v218: 65 pgs: 65 active+clean; 1.9 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 129 MiB/s rd, 167 MiB/s wr, 359 op/s 2024-03-22T07:18:45.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.971+0000 7fe8c28a9700 1 -- 172.21.15.81:0/109253396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc101d10 msgr2=0x7fe8bc1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:45.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.971+0000 7fe8c28a9700 1 --2- 172.21.15.81:0/109253396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc101d10 0x7fe8bc1020e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac004940 tx=0x7fe8ac009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:45.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.971+0000 7fe8c28a9700 1 -- 172.21.15.81:0/109253396 shutdown_connections 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.971+0000 7fe8c28a9700 1 --2- 172.21.15.81:0/109253396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8bc1026b0 0x7fe8bc0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.971+0000 7fe8c28a9700 1 --2- 172.21.15.81:0/109253396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc101d10 0x7fe8bc1020e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.971+0000 7fe8c28a9700 1 -- 172.21.15.81:0/109253396 >> 172.21.15.81:0/109253396 conn(0x7fe8bc0f7880 msgr2=0x7fe8bc0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.971+0000 7fe8c28a9700 1 -- 172.21.15.81:0/109253396 shutdown_connections 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.971+0000 7fe8c28a9700 1 -- 172.21.15.81:0/109253396 wait complete. 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.972+0000 7fe8c28a9700 1 Processor -- start 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.972+0000 7fe8c28a9700 1 -- start start 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8c28a9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc101d10 0x7fe8bc113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8c28a9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8bc1026b0 0x7fe8bc114410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8c28a9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc114af0 con 0x7fe8bc101d10 2024-03-22T07:18:45.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8c28a9700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc10de30 con 0x7fe8bc1026b0 2024-03-22T07:18:45.381 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:18:45.381 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (7m) 63s ago 17m 24.4M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:18:45.381 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (5m) 63s ago 17m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:18:45.381 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (4m) 34s ago 16m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:18:45.381 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (7m) 63s ago 17m 87.4M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:18:45.381 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (86s) 63s ago 12m 12.3M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5405937045b2 2024-03-22T07:18:45.381 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (68s) 63s ago 12m 13.7M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5080d1e80a87 2024-03-22T07:18:45.381 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (37s) 34s ago 12m 14.6M - 19.0.0-1667-gdb0330b1 4f325f8440e9 6e2fdd449b68 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (52s) 34s ago 12m 14.9M - 19.0.0-1667-gdb0330b1 4f325f8440e9 90fe0ec77272 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (8m) 63s ago 19m 510M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (8m) 34s ago 16m 451M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (6m) 63s ago 19m 49.2M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (5m) 34s ago 16m 40.3M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (7m) 63s ago 17m 17.1M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (7m) 34s ago 16m 16.5M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (4m) 63s ago 15m 98.0M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 033c080308b6 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (4m) 63s ago 15m 108M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 8341fe32f3fa 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (3m) 63s ago 15m 84.7M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 0b8c514dced1 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (2m) 34s ago 14m 107M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c8234f7ed003 2024-03-22T07:18:45.382 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (2m) 34s ago 14m 103M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 437455ba97c0 2024-03-22T07:18:45.383 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (102s) 34s ago 13m 89.6M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0d8d31a867d9 2024-03-22T07:18:45.383 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (7m) 63s ago 17m 63.4M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:18:45.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8bbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc101d10 0x7fe8bc113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:45.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8bbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc101d10 0x7fe8bc113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47352/0 (socket says 172.21.15.81:47352) 2024-03-22T07:18:45.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8bbfff700 1 -- 172.21.15.81:0/1567397284 learned_addr learned my addr 172.21.15.81:0/1567397284 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:18:45.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8bb7fe700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8bc1026b0 0x7fe8bc114410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:45.383 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8bb7fe700 1 -- 172.21.15.81:0/1567397284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc101d10 msgr2=0x7fe8bc113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8bb7fe700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc101d10 0x7fe8bc113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.973+0000 7fe8bb7fe700 1 -- 172.21.15.81:0/1567397284 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac005040 con 0x7fe8bc1026b0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.974+0000 7fe8bb7fe700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8bc1026b0 0x7fe8bc114410 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000b550 tx=0x7fe8b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.974+0000 7fe8b8ff9700 1 -- 172.21.15.81:0/1567397284 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8b000f6f0 con 0x7fe8bc1026b0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.974+0000 7fe8b8ff9700 1 -- 172.21.15.81:0/1567397284 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8b000fd20 con 0x7fe8bc1026b0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.974+0000 7fe8b8ff9700 1 -- 172.21.15.81:0/1567397284 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8b00103a0 con 0x7fe8bc1026b0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.974+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc10e0b0 con 0x7fe8bc1026b0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.974+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc10e590 con 0x7fe8bc1026b0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.976+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc0fd460 con 0x7fe8bc1026b0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.977+0000 7fe8b8ff9700 1 -- 172.21.15.81:0/1567397284 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fe8b0011400 con 0x7fe8bc1026b0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.977+0000 7fe8b8ff9700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe8a406d450 0x7fe8a406f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.977+0000 7fe8b8ff9700 1 -- 172.21.15.81:0/1567397284 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe8b008c630 con 0x7fe8bc1026b0 2024-03-22T07:18:45.384 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.977+0000 7fe8bbfff700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe8a406d450 0x7fe8a406f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.978+0000 7fe8bbfff700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe8a406d450 0x7fe8a406f900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac004f90 tx=0x7fe8ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:44.983+0000 7fe8b8ff9700 1 -- 172.21.15.81:0/1567397284 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8b0059bf0 con 0x7fe8bc1026b0 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.201+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8bc066220 con 0x7fe8a406d450 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.216+0000 7fe8b8ff9700 1 -- 172.21.15.81:0/1567397284 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fe8bc066220 con 0x7fe8a406d450 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe8a406d450 msgr2=0x7fe8a406f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe8a406d450 0x7fe8a406f900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac004f90 tx=0x7fe8ac009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8bc1026b0 msgr2=0x7fe8bc114410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8bc1026b0 0x7fe8bc114410 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000b550 tx=0x7fe8b00091a0 comp rx=0 tx=0).stop 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 shutdown_connections 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe8a406d450 0x7fe8a406f900 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc101d10 0x7fe8bc113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 --2- 172.21.15.81:0/1567397284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8bc1026b0 0x7fe8bc114410 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 >> 172.21.15.81:0/1567397284 conn(0x7fe8bc0f7880 msgr2=0x7fe8bc104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:45.385 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 shutdown_connections 2024-03-22T07:18:45.386 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:45.219+0000 7fe8c28a9700 1 -- 172.21.15.81:0/1567397284 wait complete. 2024-03-22T07:18:45.386 INFO:tasks.workunit.client.0.smithi081.stdout:6/996: dwrite d3/d67/db2/d40/db9/d63/db4/fff [0,4194304] 0 2024-03-22T07:18:45.386 INFO:tasks.workunit.client.0.smithi081.stdout:6/997: fdatasync d3/d67/db2/d40/db9/d63/d26/d52/ff6 0 2024-03-22T07:18:45.386 INFO:tasks.workunit.client.0.smithi081.stdout:7/944: dwrite f4 [0,4194304] 0 2024-03-22T07:18:45.386 INFO:tasks.workunit.client.0.smithi081.stdout:7/945: mkdir d9/d3f/d52/d14d 0 2024-03-22T07:18:45.386 INFO:tasks.workunit.client.0.smithi081.stdout:7/946: stat d9/dc 0 2024-03-22T07:18:45.387 INFO:tasks.workunit.client.0.smithi081.stdout:6/998: write d3/d67/db2/d40/db9/d63/d26/d3f/d5a/fa3 [2822459,52881] 0 2024-03-22T07:18:45.387 INFO:tasks.workunit.client.0.smithi081.stdout:6/999: fsync d3/d67/db2/d40/db9/d63/db4/dc5/f10c 0 2024-03-22T07:18:45.392 INFO:tasks.workunit.client.0.smithi081.stdout:7/947: dread d9/f11 [4194304,4194304] 0 2024-03-22T07:18:45.394 INFO:tasks.workunit.client.0.smithi081.stdout:7/948: creat d9/dc/df6/f14e x:0 0 0 2024-03-22T07:18:45.394 INFO:tasks.workunit.client.0.smithi081.stdout:7/949: read - d9/d3f/d12d/f13e zero size 2024-03-22T07:18:45.402 INFO:tasks.workunit.client.0.smithi081.stdout:7/950: write d9/d10/d16/d1e/d6f/f11e [2120637,117021] 0 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:4/993: sync 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:2/692: sync 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:5/954: sync 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:8/816: sync 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:9/809: sync 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:5/955: fsync d4/d9/f91 0 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:3/694: sync 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:9/810: chown d2/d5d/d58/dfc/de6/cf1 36 1 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:1/853: sync 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:0/910: sync 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:1/854: write d6/d6a/d85/df6/d4c/d70/d7d/f88 [3294669,54767] 0 2024-03-22T07:18:45.404 INFO:tasks.workunit.client.0.smithi081.stdout:1/855: chown d6/d6a/d85/df6/d24/d48/d6e/c114 0 1 2024-03-22T07:18:45.405 INFO:tasks.workunit.client.0.smithi081.stdout:2/693: write da/db/d39/d46/f36 [1464670,129950] 0 2024-03-22T07:18:45.405 INFO:tasks.workunit.client.0.smithi081.stdout:7/951: dread d9/d10/d2a/f149 [0,4194304] 0 2024-03-22T07:18:45.407 INFO:tasks.workunit.client.0.smithi081.stdout:8/817: creat d2/d3/d19/d1e/d13/d22/d38/f11b x:0 0 0 2024-03-22T07:18:45.407 INFO:tasks.workunit.client.0.smithi081.stdout:8/818: truncate d2/d3/d19/d1e/d13/d22/fbe 491577 0 2024-03-22T07:18:45.407 INFO:tasks.workunit.client.0.smithi081.stdout:1/856: rename d6/d18/d60/cca to d6/d18/d41/d49/d115/c121 0 2024-03-22T07:18:45.407 INFO:tasks.workunit.client.0.smithi081.stdout:1/857: write d6/f100 [30972,74447] 0 2024-03-22T07:18:45.409 INFO:tasks.workunit.client.0.smithi081.stdout:9/811: symlink d2/d73/d85/d10b/l10e 0 2024-03-22T07:18:45.409 INFO:tasks.workunit.client.0.smithi081.stdout:9/812: stat d2/d5/d36/f6c 0 2024-03-22T07:18:45.409 INFO:tasks.workunit.client.0.smithi081.stdout:9/813: chown d2/d10/d30/d57/d66/d75/l92 3 1 2024-03-22T07:18:45.409 INFO:tasks.workunit.client.0.smithi081.stdout:4/994: mknod da/df/c14a 0 2024-03-22T07:18:45.410 INFO:tasks.workunit.client.0.smithi081.stdout:7/952: creat d9/d10/d16/f14f x:0 0 0 2024-03-22T07:18:45.413 INFO:tasks.workunit.client.0.smithi081.stdout:9/814: symlink d2/d78/l10f 0 2024-03-22T07:18:45.413 INFO:tasks.workunit.client.0.smithi081.stdout:0/911: dread d3/dc/d40/f9a [0,4194304] 0 2024-03-22T07:18:45.413 INFO:tasks.workunit.client.0.smithi081.stdout:0/912: dread - d3/dc/d33/d62/ff5 zero size 2024-03-22T07:18:45.414 INFO:tasks.workunit.client.0.smithi081.stdout:4/995: creat da/d18/d6e/f14b x:0 0 0 2024-03-22T07:18:45.415 INFO:tasks.workunit.client.0.smithi081.stdout:8/819: mkdir d2/d3/d19/d11c 0 2024-03-22T07:18:45.415 INFO:tasks.workunit.client.0.smithi081.stdout:8/820: chown d2/d3/d19/d1e/d13/d22/cad 7549555 1 2024-03-22T07:18:45.416 INFO:tasks.workunit.client.0.smithi081.stdout:8/821: write d2/d3/d19/d1e/d13/d22/d38/dbb/f10e [265050,108356] 0 2024-03-22T07:18:45.417 INFO:tasks.workunit.client.0.smithi081.stdout:9/815: mknod d2/d5d/d5a/c110 0 2024-03-22T07:18:45.417 INFO:tasks.workunit.client.0.smithi081.stdout:2/694: mknod da/d11/ce0 0 2024-03-22T07:18:45.418 INFO:tasks.workunit.client.0.smithi081.stdout:4/996: mknod da/d148/c14c 0 2024-03-22T07:18:45.421 INFO:tasks.workunit.client.0.smithi081.stdout:2/695: creat da/db/d39/d46/d7e/fe1 x:0 0 0 2024-03-22T07:18:45.422 INFO:tasks.workunit.client.0.smithi081.stdout:4/997: symlink da/d18/ddc/l14d 0 2024-03-22T07:18:45.422 INFO:tasks.workunit.client.0.smithi081.stdout:4/998: creat da/d18/ddc/f14e x:0 0 0 2024-03-22T07:18:45.424 INFO:tasks.workunit.client.0.smithi081.stdout:8/822: rename d2/d3/d19/d1e/d63/c89 to d2/d3/d28/c11d 0 2024-03-22T07:18:45.425 INFO:tasks.workunit.client.0.smithi081.stdout:0/913: dread d3/f22 [0,4194304] 0 2024-03-22T07:18:45.425 INFO:tasks.workunit.client.0.smithi081.stdout:9/816: mknod d2/d5/d36/c111 0 2024-03-22T07:18:45.426 INFO:tasks.workunit.client.0.smithi081.stdout:9/817: chown d2/d10/d30/d57/fb8 24594 1 2024-03-22T07:18:45.426 INFO:tasks.workunit.client.0.smithi081.stdout:9/818: creat d2/d5d/d58/dfc/f112 x:0 0 0 2024-03-22T07:18:45.429 INFO:tasks.workunit.client.0.smithi081.stdout:0/914: mkdir d3/d6a/de0/d13b 0 2024-03-22T07:18:45.430 INFO:tasks.workunit.client.0.smithi081.stdout:0/915: read - d3/dc/d40/d118/f12d zero size 2024-03-22T07:18:45.431 INFO:tasks.workunit.client.0.smithi081.stdout:0/916: stat d3/dc/d40/d69/d92/df4/c102 0 2024-03-22T07:18:45.431 INFO:tasks.workunit.client.0.smithi081.stdout:9/819: symlink d2/d5/d9d/l113 0 2024-03-22T07:18:45.432 INFO:tasks.workunit.client.0.smithi081.stdout:0/917: fsync d3/f3d 0 2024-03-22T07:18:45.432 INFO:tasks.workunit.client.0.smithi081.stdout:0/918: chown d3/dc/d40/f9a 56273935 1 2024-03-22T07:18:45.432 INFO:tasks.workunit.client.0.smithi081.stdout:0/919: creat d3/d4e/dee/f13c x:0 0 0 2024-03-22T07:18:45.433 INFO:tasks.workunit.client.0.smithi081.stdout:9/820: mknod d2/d73/d74/dd9/c114 0 2024-03-22T07:18:45.434 INFO:tasks.workunit.client.0.smithi081.stdout:2/696: dread da/d11/d21/f53 [0,4194304] 0 2024-03-22T07:18:45.434 INFO:tasks.workunit.client.0.smithi081.stdout:2/697: readlink da/db/d39/d47/l61 0 2024-03-22T07:18:45.434 INFO:tasks.workunit.client.0.smithi081.stdout:2/698: stat da/d11/d21/fa2 0 2024-03-22T07:18:45.435 INFO:tasks.workunit.client.0.smithi081.stdout:0/920: creat d3/d32/df3/f13d x:0 0 0 2024-03-22T07:18:45.435 INFO:tasks.workunit.client.0.smithi081.stdout:9/821: rmdir d2/d73/d85/d7a/d97 39 2024-03-22T07:18:45.436 INFO:tasks.workunit.client.0.smithi081.stdout:0/921: mknod d3/d9b/da4/c13e 0 2024-03-22T07:18:45.438 INFO:tasks.workunit.client.0.smithi081.stdout:0/922: rename d3/dc/d40/d118/d11a/fe7 to d3/dc/d40/d69/d92/dc1/d91/d9d/f13f 0 2024-03-22T07:18:45.440 INFO:tasks.workunit.client.0.smithi081.stdout:9/822: symlink d2/d5/d36/l115 0 2024-03-22T07:18:45.440 INFO:tasks.workunit.client.0.smithi081.stdout:9/823: getdents d2/d5d/d58/dfc/de6 0 2024-03-22T07:18:45.441 INFO:tasks.workunit.client.0.smithi081.stdout:5/956: dwrite d4/d10/d20/db2/d2a/f88 [0,4194304] 0 2024-03-22T07:18:45.441 INFO:tasks.workunit.client.0.smithi081.stdout:9/824: creat d2/d2b/f116 x:0 0 0 2024-03-22T07:18:45.442 INFO:tasks.workunit.client.0.smithi081.stdout:5/957: symlink d4/d9/d90/l13b 0 2024-03-22T07:18:45.443 INFO:tasks.workunit.client.0.smithi081.stdout:1/858: write d6/d6a/d85/df6/d32/fef [3488017,88811] 0 2024-03-22T07:18:45.443 INFO:tasks.workunit.client.0.smithi081.stdout:2/699: dread da/db/d39/d46/f44 [0,4194304] 0 2024-03-22T07:18:45.443 INFO:tasks.workunit.client.0.smithi081.stdout:3/695: dwrite db/d23/f61 [0,4194304] 0 2024-03-22T07:18:45.443 INFO:tasks.workunit.client.0.smithi081.stdout:9/825: mknod d2/d5d/c117 0 2024-03-22T07:18:45.444 INFO:tasks.workunit.client.0.smithi081.stdout:9/826: fsync d2/d5d/d5a/fa5 0 2024-03-22T07:18:45.444 INFO:tasks.workunit.client.0.smithi081.stdout:1/859: symlink d6/d6a/d85/df6/d24/daa/l122 0 2024-03-22T07:18:45.445 INFO:tasks.workunit.client.0.smithi081.stdout:5/958: unlink d4/d75/da5/fd7 0 2024-03-22T07:18:45.445 INFO:tasks.workunit.client.0.smithi081.stdout:5/959: dread - d4/d10/d20/d4f/fd8 zero size 2024-03-22T07:18:45.453 INFO:tasks.workunit.client.0.smithi081.stdout:8/823: dwrite d2/d3/d23/fb3 [0,4194304] 0 2024-03-22T07:18:45.453 INFO:tasks.workunit.client.0.smithi081.stdout:3/696: rmdir db/d23 39 2024-03-22T07:18:45.453 INFO:tasks.workunit.client.0.smithi081.stdout:3/697: creat db/de/d4b/d6a/d70/ff4 x:0 0 0 2024-03-22T07:18:45.457 INFO:tasks.workunit.client.0.smithi081.stdout:2/700: rename da/db/d39/d47/d3b/d41/d50/d6a/d70/fd8 to da/db/d39/d46/d42/fe2 0 2024-03-22T07:18:45.457 INFO:tasks.workunit.client.0.smithi081.stdout:2/701: chown da/db/d39/d47/d3b/d63/c8d 1599451 1 2024-03-22T07:18:45.460 INFO:tasks.workunit.client.0.smithi081.stdout:2/702: creat da/db/d39/d46/d42/db4/dbb/fe3 x:0 0 0 2024-03-22T07:18:45.461 INFO:tasks.workunit.client.0.smithi081.stdout:2/703: rename da/db/d39/d46/f44 to da/db/d39/d47/d3b/d41/d50/fe4 0 2024-03-22T07:18:45.461 INFO:tasks.workunit.client.0.smithi081.stdout:2/704: truncate da/db/d39/d47/d3b/d41/dab/fad 103935 0 2024-03-22T07:18:45.464 INFO:tasks.workunit.client.0.smithi081.stdout:3/698: write db/d38/f6c [2404961,51578] 0 2024-03-22T07:18:45.464 INFO:tasks.workunit.client.0.smithi081.stdout:3/699: write db/de/d4b/d6a/d70/ff4 [711554,59155] 0 2024-03-22T07:18:45.464 INFO:tasks.workunit.client.0.smithi081.stdout:3/700: chown db/de/d4b/d6a/d70/f92 4 1 2024-03-22T07:18:45.465 INFO:tasks.workunit.client.0.smithi081.stdout:3/701: symlink db/d38/d8d/lf5 0 2024-03-22T07:18:45.467 INFO:tasks.workunit.client.0.smithi081.stdout:8/824: dread d2/d3/d19/d1e/f5c [0,4194304] 0 2024-03-22T07:18:45.467 INFO:tasks.workunit.client.0.smithi081.stdout:8/825: write d2/d3/d28/f106 [542452,109594] 0 2024-03-22T07:18:45.470 INFO:tasks.workunit.client.0.smithi081.stdout:8/826: mkdir d2/d3/d19/d64/d11e 0 2024-03-22T07:18:45.471 INFO:tasks.workunit.client.0.smithi081.stdout:8/827: symlink d2/d3/d28/d2b/d61/ddb/l11f 0 2024-03-22T07:18:45.471 INFO:tasks.workunit.client.0.smithi081.stdout:4/999: dwrite da/df/d38/dd8/d4f/fad [0,4194304] 0 2024-03-22T07:18:45.473 INFO:tasks.workunit.client.0.smithi081.stdout:5/960: dwrite d4/d75/fbd [0,4194304] 0 2024-03-22T07:18:45.473 INFO:tasks.workunit.client.0.smithi081.stdout:8/828: unlink d2/d3/d23/c12 0 2024-03-22T07:18:45.475 INFO:tasks.workunit.client.0.smithi081.stdout:0/923: dwrite d3/dc/f19 [0,4194304] 0 2024-03-22T07:18:45.475 INFO:tasks.workunit.client.0.smithi081.stdout:3/702: dread db/d38/f51 [0,4194304] 0 2024-03-22T07:18:45.475 INFO:tasks.workunit.client.0.smithi081.stdout:3/703: chown db/de/d4b/d6a/d70/cad 125904220 1 2024-03-22T07:18:45.475 INFO:tasks.workunit.client.0.smithi081.stdout:3/704: fdatasync db/de/db8/fe5 0 2024-03-22T07:18:45.479 INFO:tasks.workunit.client.0.smithi081.stdout:2/705: dwrite da/db/d39/d47/d3b/d41/f51 [0,4194304] 0 2024-03-22T07:18:45.480 INFO:tasks.workunit.client.0.smithi081.stdout:5/961: creat d4/d10/d20/d25/f13c x:0 0 0 2024-03-22T07:18:45.480 INFO:tasks.workunit.client.0.smithi081.stdout:5/962: dread - d4/d9/d90/f125 zero size 2024-03-22T07:18:45.481 INFO:tasks.workunit.client.0.smithi081.stdout:0/924: rename d3/f2e to d3/d9b/d10b/f140 0 2024-03-22T07:18:45.481 INFO:tasks.workunit.client.0.smithi081.stdout:8/829: dread d2/d3/d28/f106 [0,4194304] 0 2024-03-22T07:18:45.481 INFO:tasks.workunit.client.0.smithi081.stdout:8/830: fsync d2/d3/d19/d1e/f5f 0 2024-03-22T07:18:45.482 INFO:tasks.workunit.client.0.smithi081.stdout:0/925: dread d3/dc/d2b/d39/f75 [0,4194304] 0 2024-03-22T07:18:45.486 INFO:tasks.workunit.client.0.smithi081.stdout:5/963: mkdir d4/df/d73/d13d 0 2024-03-22T07:18:45.488 INFO:tasks.workunit.client.0.smithi081.stdout:0/926: unlink d3/dc/d40/f94 0 2024-03-22T07:18:45.489 INFO:tasks.workunit.client.0.smithi081.stdout:0/927: stat d3/dc/d40/d118/f12d 0 2024-03-22T07:18:45.489 INFO:tasks.workunit.client.0.smithi081.stdout:0/928: write d3/dc/f3f [289376,99696] 0 2024-03-22T07:18:45.490 INFO:tasks.workunit.client.0.smithi081.stdout:5/964: truncate d4/df/d73/d8c/f87 2975647 0 2024-03-22T07:18:45.492 INFO:tasks.workunit.client.0.smithi081.stdout:3/705: dwrite db/de/d4b/d6a/d70/d3a/d5d/ff3 [0,4194304] 0 2024-03-22T07:18:45.492 INFO:tasks.workunit.client.0.smithi081.stdout:3/706: write db/de/d4b/db0/fd1 [657897,112623] 0 2024-03-22T07:18:45.493 INFO:tasks.workunit.client.0.smithi081.stdout:2/706: rename da/db/d6d/cdb to da/ce5 0 2024-03-22T07:18:45.496 INFO:tasks.workunit.client.0.smithi081.stdout:0/929: rmdir d3/dc/d33 39 2024-03-22T07:18:45.496 INFO:tasks.workunit.client.0.smithi081.stdout:0/930: fdatasync d3/dc/f73 0 2024-03-22T07:18:45.496 INFO:tasks.workunit.client.0.smithi081.stdout:0/931: read - d3/d6a/de0/f136 zero size 2024-03-22T07:18:45.496 INFO:tasks.workunit.client.0.smithi081.stdout:0/932: chown d3/dc/l26 329 1 2024-03-22T07:18:45.500 INFO:tasks.workunit.client.0.smithi081.stdout:3/707: creat db/de/ddb/ff6 x:0 0 0 2024-03-22T07:18:45.500 INFO:tasks.workunit.client.0.smithi081.stdout:3/708: fsync db/f77 0 2024-03-22T07:18:45.502 INFO:tasks.workunit.client.0.smithi081.stdout:2/707: creat da/db/d39/d47/d3b/d41/d50/d6a/fe6 x:0 0 0 2024-03-22T07:18:45.502 INFO:tasks.workunit.client.0.smithi081.stdout:0/933: write d3/dc/d33/d62/f68 [3817489,50886] 0 2024-03-22T07:18:45.509 INFO:tasks.workunit.client.0.smithi081.stdout:2/708: rename da/db/d39/d46/d7e/fe1 to da/db/d39/d46/fe7 0 2024-03-22T07:18:45.509 INFO:tasks.workunit.client.0.smithi081.stdout:2/709: stat da/db/d39/d47/d3b/d41/d50/f66 0 2024-03-22T07:18:45.510 INFO:tasks.workunit.client.0.smithi081.stdout:0/934: mknod d3/dc/d2b/c141 0 2024-03-22T07:18:45.510 INFO:tasks.workunit.client.0.smithi081.stdout:0/935: dread - d3/dc/d2b/d39/dc9/ff0 zero size 2024-03-22T07:18:45.510 INFO:tasks.workunit.client.0.smithi081.stdout:0/936: write d3/dc/d33/d62/f63 [4266603,64609] 0 2024-03-22T07:18:45.514 INFO:tasks.workunit.client.0.smithi081.stdout:2/710: rename da/db/d39/d47/d3b/d41/d50/d6a/d70 to da/db/d39/d45/dcc/de8 0 2024-03-22T07:18:45.514 INFO:tasks.workunit.client.0.smithi081.stdout:2/711: fdatasync da/d11/d21/f33 0 2024-03-22T07:18:45.514 INFO:tasks.workunit.client.0.smithi081.stdout:2/712: stat da/d11 0 2024-03-22T07:18:45.514 INFO:tasks.workunit.client.0.smithi081.stdout:3/709: dwrite db/de/d4b/db0/fd5 [0,4194304] 0 2024-03-22T07:18:45.516 INFO:tasks.workunit.client.0.smithi081.stdout:2/713: symlink da/db/d6d/le9 0 2024-03-22T07:18:45.517 INFO:tasks.workunit.client.0.smithi081.stdout:2/714: truncate da/db/d39/d45/f4c 2738774 0 2024-03-22T07:18:45.518 INFO:tasks.workunit.client.0.smithi081.stdout:3/710: mkdir db/ddf/ded/df7 0 2024-03-22T07:18:45.518 INFO:tasks.workunit.client.0.smithi081.stdout:3/711: creat db/ff8 x:0 0 0 2024-03-22T07:18:45.520 INFO:tasks.workunit.client.0.smithi081.stdout:0/937: rename d3/dc/d40/d69/d92/dc1/df6 to d3/dc/d40/d69/d92/dc1/d91/d9d/d142 0 2024-03-22T07:18:45.523 INFO:tasks.workunit.client.0.smithi081.stdout:3/712: truncate db/f5c 3968752 0 2024-03-22T07:18:45.523 INFO:tasks.workunit.client.0.smithi081.stdout:3/713: creat db/de/db8/ff9 x:0 0 0 2024-03-22T07:18:45.526 INFO:tasks.workunit.client.0.smithi081.stdout:3/714: truncate db/f77 3208472 0 2024-03-22T07:18:45.528 INFO:tasks.workunit.client.0.smithi081.stdout:2/715: rename da/db/d39/d46/d42/f74 to da/db/d39/d47/d6e/fea 0 2024-03-22T07:18:45.531 INFO:tasks.workunit.client.0.smithi081.stdout:3/715: unlink db/de/cc6 0 2024-03-22T07:18:45.531 INFO:tasks.workunit.client.0.smithi081.stdout:3/716: write db/f18 [4601308,24807] 0 2024-03-22T07:18:45.531 INFO:tasks.workunit.client.0.smithi081.stdout:3/717: chown db/de/ddb/ff6 314132050 1 2024-03-22T07:18:45.534 INFO:tasks.workunit.client.0.smithi081.stdout:3/718: truncate db/de/f4c 962249 0 2024-03-22T07:18:45.535 INFO:tasks.workunit.client.0.smithi081.stdout:3/719: dread db/de/d4b/d6a/f89 [0,4194304] 0 2024-03-22T07:18:45.540 INFO:tasks.workunit.client.0.smithi081.stdout:3/720: dread db/de/d4b/d6a/d70/f32 [0,4194304] 0 2024-03-22T07:18:45.542 INFO:tasks.workunit.client.0.smithi081.stdout:2/716: rmdir da/db/d39 39 2024-03-22T07:18:45.542 INFO:tasks.workunit.client.0.smithi081.stdout:2/717: dread - da/db/d39/d47/dda/fdc zero size 2024-03-22T07:18:45.542 INFO:tasks.workunit.client.0.smithi081.stdout:2/718: readlink da/db/d39/d47/d6e/l8a 0 2024-03-22T07:18:45.544 INFO:tasks.workunit.client.0.smithi081.stdout:3/721: mkdir db/de/d4b/d6a/dfa 0 2024-03-22T07:18:45.544 INFO:tasks.workunit.client.0.smithi081.stdout:3/722: creat db/de/d4b/d6a/d70/d3a/d5d/ffb x:0 0 0 2024-03-22T07:18:45.544 INFO:tasks.workunit.client.0.smithi081.stdout:2/719: getdents da/db/d39/d47/d3b/d41/dab 0 2024-03-22T07:18:45.554 INFO:tasks.workunit.client.0.smithi081.stdout:2/720: mknod da/db/d39/d46/ceb 0 2024-03-22T07:18:45.567 INFO:tasks.workunit.client.0.smithi081.stdout:0/938: dwrite d3/d32/f59 [0,4194304] 0 2024-03-22T07:18:45.569 INFO:tasks.workunit.client.0.smithi081.stdout:3/723: dwrite db/f2f [0,4194304] 0 2024-03-22T07:18:45.571 INFO:tasks.workunit.client.0.smithi081.stdout:2/721: dwrite da/db/d39/d46/d42/d67/fa4 [0,4194304] 0 2024-03-22T07:18:45.572 INFO:tasks.workunit.client.0.smithi081.stdout:0/939: mknod d3/dc/d2b/d39/c143 0 2024-03-22T07:18:45.573 INFO:tasks.workunit.client.0.smithi081.stdout:0/940: creat d3/dc/d40/d69/d92/dc1/d91/d9d/d142/d135/f144 x:0 0 0 2024-03-22T07:18:45.574 INFO:tasks.workunit.client.0.smithi081.stdout:0/941: mkdir d3/dc/d2b/d39/d131/d145 0 2024-03-22T07:18:45.574 INFO:tasks.workunit.client.0.smithi081.stdout:0/942: write d3/d9b/fe4 [572529,112690] 0 2024-03-22T07:18:45.574 INFO:tasks.workunit.client.0.smithi081.stdout:0/943: write d3/d4e/dee/f13c [401663,101413] 0 2024-03-22T07:18:45.575 INFO:tasks.workunit.client.0.smithi081.stdout:0/944: creat d3/dc/d77/f146 x:0 0 0 2024-03-22T07:18:45.575 INFO:tasks.workunit.client.0.smithi081.stdout:0/945: read - d3/dc/d33/dd1/f11e zero size 2024-03-22T07:18:45.576 INFO:tasks.workunit.client.0.smithi081.stdout:0/946: symlink d3/d6a/de0/d13b/l147 0 2024-03-22T07:18:45.576 INFO:tasks.workunit.client.0.smithi081.stdout:0/947: fdatasync d3/d9b/faa 0 2024-03-22T07:18:45.576 INFO:tasks.workunit.client.0.smithi081.stdout:0/948: symlink d3/dc/d33/d62/df9/d120/l148 0 2024-03-22T07:18:45.576 INFO:tasks.workunit.client.0.smithi081.stdout:0/949: chown d3/d4e/c8a 23424888 1 2024-03-22T07:18:45.577 INFO:tasks.workunit.client.0.smithi081.stdout:0/950: rmdir d3/d6a 39 2024-03-22T07:18:45.577 INFO:tasks.workunit.client.0.smithi081.stdout:0/951: getdents d3/dc/d40/d69/dbf/d125 0 2024-03-22T07:18:45.577 INFO:tasks.workunit.client.0.smithi081.stdout:0/952: chown d3/dc/c80 307160 1 2024-03-22T07:18:45.581 INFO:tasks.workunit.client.0.smithi081.stdout:0/953: rename d3/d4e to d3/dc/d33/dd1/dec/d149 0 2024-03-22T07:18:45.581 INFO:tasks.workunit.client.0.smithi081.stdout:0/954: chown d3 92617 1 2024-03-22T07:18:45.581 INFO:tasks.workunit.client.0.smithi081.stdout:0/955: write d3/dc/d40/d69/d92/dc1/d91/d9d/d142/f12a [734019,75639] 0 2024-03-22T07:18:45.581 INFO:tasks.workunit.client.0.smithi081.stdout:0/956: fsync d3/dc/d33/dd1/dec/d149/f137 0 2024-03-22T07:18:45.581 INFO:tasks.workunit.client.0.smithi081.stdout:0/957: write d3/dc/d40/d118/d11a/f101 [492244,125464] 0 2024-03-22T07:18:45.582 INFO:tasks.workunit.client.0.smithi081.stdout:0/958: readlink d3/d6a/dd5/l109 0 2024-03-22T07:18:45.582 INFO:tasks.workunit.client.0.smithi081.stdout:0/959: write d3/dc/d40/d69/d92/dc1/d74/fb7 [1749850,75699] 0 2024-03-22T07:18:45.582 INFO:tasks.workunit.client.0.smithi081.stdout:0/960: rename d3/dc/d2b/d4d/lc3 to d3/dc/d40/d69/d92/dc1/d91/l14a 0 2024-03-22T07:18:45.582 INFO:tasks.workunit.client.0.smithi081.stdout:0/961: write d3/dc/d40/d69/d92/dc1/d91/d9d/f13f [17852,21789] 0 2024-03-22T07:18:45.583 INFO:tasks.workunit.client.0.smithi081.stdout:2/722: write da/d11/d21/f33 [2787532,26224] 0 2024-03-22T07:18:45.584 INFO:tasks.workunit.client.0.smithi081.stdout:0/962: unlink d3/f22 0 2024-03-22T07:18:45.591 INFO:tasks.workunit.client.0.smithi081.stdout:3/724: dwrite db/de/f4c [0,4194304] 0 2024-03-22T07:18:45.591 INFO:tasks.workunit.client.0.smithi081.stdout:3/725: chown db/de/db8/cf0 26542 1 2024-03-22T07:18:45.594 INFO:tasks.workunit.client.0.smithi081.stdout:3/726: creat db/de/d4b/d6a/dfa/ffc x:0 0 0 2024-03-22T07:18:45.594 INFO:tasks.workunit.client.0.smithi081.stdout:3/727: creat db/de/d4b/d6a/ffd x:0 0 0 2024-03-22T07:18:45.606 INFO:tasks.workunit.client.0.smithi081.stdout:3/728: dread db/f2f [0,4194304] 0 2024-03-22T07:18:45.607 INFO:tasks.workunit.client.0.smithi081.stdout:3/729: unlink db/de/l58 0 2024-03-22T07:18:45.609 INFO:tasks.workunit.client.0.smithi081.stdout:2/723: dwrite da/f13 [0,4194304] 0 2024-03-22T07:18:45.609 INFO:tasks.workunit.client.0.smithi081.stdout:0/963: dwrite d3/d32/f5f [0,4194304] 0 2024-03-22T07:18:45.609 INFO:tasks.workunit.client.0.smithi081.stdout:0/964: readlink d3/dc/d40/lde 0 2024-03-22T07:18:45.611 INFO:tasks.workunit.client.0.smithi081.stdout:2/724: creat da/db/d39/d47/d3b/d41/dab/dac/dc2/fec x:0 0 0 2024-03-22T07:18:45.612 INFO:tasks.workunit.client.0.smithi081.stdout:0/965: creat d3/d126/f14b x:0 0 0 2024-03-22T07:18:45.614 INFO:tasks.workunit.client.0.smithi081.stdout:0/966: getdents d3/d32/df3 0 2024-03-22T07:18:45.616 INFO:tasks.workunit.client.0.smithi081.stdout:0/967: creat d3/d6a/dd5/f14c x:0 0 0 2024-03-22T07:18:45.625 INFO:tasks.workunit.client.0.smithi081.stdout:0/968: rmdir d3/dc/d2b 39 2024-03-22T07:18:45.626 INFO:tasks.workunit.client.0.smithi081.stdout:0/969: creat d3/dc/d33/d62/f14d x:0 0 0 2024-03-22T07:18:45.627 INFO:tasks.workunit.client.0.smithi081.stdout:0/970: rename d3/d6a/cdb to d3/d6a/de0/d13b/c14e 0 2024-03-22T07:18:45.627 INFO:tasks.workunit.client.0.smithi081.stdout:0/971: stat d3/dc/d40/d69 0 2024-03-22T07:18:45.628 INFO:tasks.workunit.client.0.smithi081.stdout:3/730: dwrite db/d23/fa7 [0,4194304] 0 2024-03-22T07:18:45.638 INFO:tasks.workunit.client.0.smithi081.stdout:0/972: write d3/dc/f8d [2529925,128312] 0 2024-03-22T07:18:45.638 INFO:tasks.workunit.client.0.smithi081.stdout:0/973: chown d3/dc/d33/f5a 9 1 2024-03-22T07:18:45.638 INFO:tasks.workunit.client.0.smithi081.stdout:0/974: truncate d3/d6a/de0/f136 586058 0 2024-03-22T07:18:45.645 INFO:tasks.workunit.client.0.smithi081.stdout:3/731: dwrite db/d38/f69 [0,4194304] 0 2024-03-22T07:18:45.645 INFO:tasks.workunit.client.0.smithi081.stdout:3/732: dread - db/de/d4b/fd9 zero size 2024-03-22T07:18:45.654 INFO:tasks.workunit.client.0.smithi081.stdout:3/733: read db/d23/f61 [3319065,92378] 0 2024-03-22T07:18:45.660 INFO:tasks.workunit.client.0.smithi081.stdout:0/975: dwrite d3/f8 [4194304,4194304] 0 2024-03-22T07:18:45.660 INFO:tasks.workunit.client.0.smithi081.stdout:0/976: readlink d3/d6a/dd5/l109 0 2024-03-22T07:18:45.662 INFO:tasks.workunit.client.0.smithi081.stdout:0/977: creat d3/dc/d40/d69/f14f x:0 0 0 2024-03-22T07:18:45.662 INFO:tasks.workunit.client.0.smithi081.stdout:0/978: readlink d3/dc/l1e 0 2024-03-22T07:18:45.667 INFO:tasks.workunit.client.0.smithi081.stdout:3/734: dwrite db/de/d4b/d6a/ffd [0,4194304] 0 2024-03-22T07:18:45.671 INFO:tasks.workunit.client.0.smithi081.stdout:3/735: getdents db/d38/da1/db7 0 2024-03-22T07:18:45.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/979: dwrite d3/dc/d2b/d39/f65 [0,4194304] 0 2024-03-22T07:18:45.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/980: dread - d3/dc/d40/d69/d92/dc1/f12b zero size 2024-03-22T07:18:45.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/981: fsync d3/dc/dc6/f12c 0 2024-03-22T07:18:45.683 INFO:tasks.workunit.client.0.smithi081.stdout:0/982: write d3/d6a/fc2 [504041,9817] 0 2024-03-22T07:18:45.684 INFO:tasks.workunit.client.0.smithi081.stdout:0/983: chown d3/dc/d33 7787743 1 2024-03-22T07:18:45.684 INFO:tasks.workunit.client.0.smithi081.stdout:0/984: symlink d3/dc/d2b/l150 0 2024-03-22T07:18:45.697 INFO:tasks.workunit.client.0.smithi081.stdout:0/985: dwrite d3/d32/fbe [0,4194304] 0 2024-03-22T07:18:45.718 INFO:tasks.workunit.client.0.smithi081.stdout:0/986: dwrite d3/d6a/f8e [4194304,4194304] 0 2024-03-22T07:18:45.740 INFO:tasks.workunit.client.0.smithi081.stdout:0/987: dwrite d3/f24 [0,4194304] 0 2024-03-22T07:18:45.740 INFO:tasks.workunit.client.0.smithi081.stdout:0/988: chown d3/d6a/de0/c116 15539 1 2024-03-22T07:18:45.741 INFO:tasks.workunit.client.0.smithi081.stdout:0/989: rename d3/dc/d40/d69/f14f to d3/dc/d40/d69/d92/dc1/d91/d9d/d142/d135/f151 0 2024-03-22T07:18:45.742 INFO:tasks.workunit.client.0.smithi081.stdout:0/990: rename d3/dc/d40/f9a to d3/dc/d33/d62/f152 0 2024-03-22T07:18:45.743 INFO:tasks.workunit.client.0.smithi081.stdout:0/991: write d3/dc/d33/dd1/dec/ff1 [622813,13515] 0 2024-03-22T07:18:45.743 INFO:tasks.workunit.client.0.smithi081.stdout:0/992: mkdir d3/d9b/da4/d153 0 2024-03-22T07:18:45.743 INFO:tasks.workunit.client.0.smithi081.stdout:0/993: truncate d3/dc/d77/f146 816408 0 2024-03-22T07:18:45.745 INFO:tasks.workunit.client.0.smithi081.stdout:0/994: rmdir d3/dc/d40/d69/dbf/d125 0 2024-03-22T07:18:45.746 INFO:tasks.workunit.client.0.smithi081.stdout:0/995: symlink d3/d6a/de0/d114/l154 0 2024-03-22T07:18:45.746 INFO:tasks.workunit.client.0.smithi081.stdout:0/996: symlink d3/dc/d40/df7/l155 0 2024-03-22T07:18:45.755 INFO:tasks.workunit.client.0.smithi081.stdout:0/997: dread d3/dc/f15 [0,4194304] 0 2024-03-22T07:18:45.755 INFO:tasks.workunit.client.0.smithi081.stdout:0/998: readlink d3/le9 0 2024-03-22T07:18:45.756 INFO:tasks.workunit.client.0.smithi081.stdout:0/999: rmdir d3/d6a/de0/d13b 39 2024-03-22T07:18:45.924 INFO:tasks.workunit.client.0.smithi081.stdout:7/953: sync 2024-03-22T07:18:45.925 INFO:tasks.workunit.client.0.smithi081.stdout:7/954: mknod d9/d3f/c150 0 2024-03-22T07:18:45.926 INFO:tasks.workunit.client.0.smithi081.stdout:7/955: creat d9/d10/d53/d81/d59/d89/f151 x:0 0 0 2024-03-22T07:18:45.932 INFO:tasks.workunit.client.0.smithi081.stdout:7/956: write d9/d10/d53/d81/d59/f9f [2123992,41074] 0 2024-03-22T07:18:45.951 INFO:tasks.workunit.client.0.smithi081.stdout:7/957: dwrite d9/d10/d53/fb5 [0,4194304] 0 2024-03-22T07:18:45.959 INFO:tasks.workunit.client.0.smithi081.stdout:7/958: dread d9/d10/d16/d1e/f35 [0,4194304] 0 2024-03-22T07:18:45.961 INFO:tasks.workunit.client.0.smithi081.stdout:7/959: creat d9/d10/d16/db6/f152 x:0 0 0 2024-03-22T07:18:45.961 INFO:tasks.workunit.client.0.smithi081.stdout:7/960: write d9/dc/f90 [5134911,69607] 0 2024-03-22T07:18:45.961 INFO:tasks.workunit.client.0.smithi081.stdout:7/961: fsync d9/dc/ff7 0 2024-03-22T07:18:45.962 INFO:tasks.workunit.client.0.smithi081.stdout:7/962: rmdir d9/dc 39 2024-03-22T07:18:45.963 INFO:tasks.workunit.client.0.smithi081.stdout:7/963: link d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d10c/c122 d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/dc1/c153 0 2024-03-22T07:18:45.983 INFO:tasks.workunit.client.0.smithi081.stdout:7/964: dwrite d9/d10/d16/d1e/d6f/ffb [0,4194304] 0 2024-03-22T07:18:45.995 INFO:tasks.workunit.client.0.smithi081.stdout:7/965: dread d9/d3f/d52/ff0 [0,4194304] 0 2024-03-22T07:18:45.996 INFO:tasks.workunit.client.0.smithi081.stdout:7/966: mknod d9/d3f/d52/d83/d143/c154 0 2024-03-22T07:18:45.996 INFO:tasks.workunit.client.0.smithi081.stdout:7/967: stat d9/l44 0 2024-03-22T07:18:45.996 INFO:tasks.workunit.client.0.smithi081.stdout:7/968: symlink d9/de4/l155 0 2024-03-22T07:18:45.997 INFO:tasks.workunit.client.0.smithi081.stdout:7/969: mknod d9/d10/d16/d84/c156 0 2024-03-22T07:18:45.997 INFO:tasks.workunit.client.0.smithi081.stdout:7/970: write d9/d3f/d12d/f13e [8515,43774] 0 2024-03-22T07:18:45.998 INFO:tasks.workunit.client.0.smithi081.stdout:7/971: creat d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/f157 x:0 0 0 2024-03-22T07:18:45.998 INFO:tasks.workunit.client.0.smithi081.stdout:7/972: stat d9/cee 0 2024-03-22T07:18:45.998 INFO:tasks.workunit.client.0.smithi081.stdout:7/973: chown d9/d3f/d52/d83/f8e 1 1 2024-03-22T07:18:45.999 INFO:tasks.workunit.client.0.smithi081.stdout:7/974: link d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/f157 d9/d3f/db3/f158 0 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:5/965: sync 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:8/831: sync 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:9/827: sync 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:5/966: fdatasync d4/d10/d20/d4f/fd8 0 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:2/725: sync 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:5/967: stat d4/df 0 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:3/736: sync 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:1/860: sync 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:9/828: creat d2/d5/d65/f118 x:0 0 0 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:3/737: creat db/d38/ffe x:0 0 0 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:3/738: chown db/de/f7d 50529368 1 2024-03-22T07:18:46.005 INFO:tasks.workunit.client.0.smithi081.stdout:3/739: write db/d38/ddd/fef [517386,8958] 0 2024-03-22T07:18:46.006 INFO:tasks.workunit.client.0.smithi081.stdout:3/740: creat db/d38/d8d/db4/fff x:0 0 0 2024-03-22T07:18:46.006 INFO:tasks.workunit.client.0.smithi081.stdout:3/741: write db/de/d4b/d6a/d70/fcd [437526,35881] 0 2024-03-22T07:18:46.006 INFO:tasks.workunit.client.0.smithi081.stdout:3/742: creat db/de/d4b/d6a/f100 x:0 0 0 2024-03-22T07:18:46.006 INFO:tasks.workunit.client.0.smithi081.stdout:3/743: fsync db/d38/f51 0 2024-03-22T07:18:46.006 INFO:tasks.workunit.client.0.smithi081.stdout:3/744: fdatasync db/de/f24 0 2024-03-22T07:18:46.006 INFO:tasks.workunit.client.0.smithi081.stdout:3/745: creat db/de/ddb/f101 x:0 0 0 2024-03-22T07:18:46.007 INFO:tasks.workunit.client.0.smithi081.stdout:2/726: write da/d11/d21/f90 [4214972,27608] 0 2024-03-22T07:18:46.007 INFO:tasks.workunit.client.0.smithi081.stdout:2/727: write da/d11/d21/f33 [971827,60990] 0 2024-03-22T07:18:46.007 INFO:tasks.workunit.client.0.smithi081.stdout:2/728: fsync da/db/d39/d47/fd5 0 2024-03-22T07:18:46.007 INFO:tasks.workunit.client.0.smithi081.stdout:8/832: dread d2/d3/d19/d1e/f8a [0,4194304] 0 2024-03-22T07:18:46.007 INFO:tasks.workunit.client.0.smithi081.stdout:5/968: mknod d4/df/d22/d49/d115/c13e 0 2024-03-22T07:18:46.007 INFO:tasks.workunit.client.0.smithi081.stdout:9/829: mknod d2/d5/d65/c119 0 2024-03-22T07:18:46.009 INFO:tasks.workunit.client.0.smithi081.stdout:2/729: symlink da/db/d39/d46/d42/db4/dbb/led 0 2024-03-22T07:18:46.010 INFO:tasks.workunit.client.0.smithi081.stdout:5/969: mkdir d4/df/d22/d49/d115/d13f 0 2024-03-22T07:18:46.010 INFO:tasks.workunit.client.0.smithi081.stdout:5/970: creat d4/d9/df7/f140 x:0 0 0 2024-03-22T07:18:46.010 INFO:tasks.workunit.client.0.smithi081.stdout:5/971: dread - d4/d10/f111 zero size 2024-03-22T07:18:46.011 INFO:tasks.workunit.client.0.smithi081.stdout:9/830: mkdir d2/d5/d65/d11a 0 2024-03-22T07:18:46.016 INFO:tasks.workunit.client.0.smithi081.stdout:2/730: creat da/db/d39/d46/d42/dcb/fee x:0 0 0 2024-03-22T07:18:46.017 INFO:tasks.workunit.client.0.smithi081.stdout:5/972: unlink d4/d10/d20/d25/db8/c3d 0 2024-03-22T07:18:46.019 INFO:tasks.workunit.client.0.smithi081.stdout:9/831: dread d2/f11 [0,4194304] 0 2024-03-22T07:18:46.019 INFO:tasks.workunit.client.0.smithi081.stdout:2/731: unlink da/db/d39/cb3 0 2024-03-22T07:18:46.019 INFO:tasks.workunit.client.0.smithi081.stdout:2/732: chown da/db/f35 7236 1 2024-03-22T07:18:46.020 INFO:tasks.workunit.client.0.smithi081.stdout:9/832: rmdir d2/d73/d85/d7a/d97/dda 39 2024-03-22T07:18:46.020 INFO:tasks.workunit.client.0.smithi081.stdout:9/833: readlink d2/d5d/lcb 0 2024-03-22T07:18:46.021 INFO:tasks.workunit.client.0.smithi081.stdout:2/733: creat da/d11/d21/d71/dc3/fef x:0 0 0 2024-03-22T07:18:46.024 INFO:tasks.workunit.client.0.smithi081.stdout:8/833: dwrite d2/d3/d19/d1e/ffc [0,4194304] 0 2024-03-22T07:18:46.031 INFO:tasks.workunit.client.0.smithi081.stdout:9/834: write d2/f3 [3723529,26394] 0 2024-03-22T07:18:46.034 INFO:tasks.workunit.client.0.smithi081.stdout:9/835: mknod d2/d10/d30/d57/d66/d75/d94/c11b 0 2024-03-22T07:18:46.035 INFO:tasks.workunit.client.0.smithi081.stdout:9/836: creat d2/d10/d30/d57/de2/f11c x:0 0 0 2024-03-22T07:18:46.044 INFO:tasks.workunit.client.0.smithi081.stdout:8/834: dwrite d2/d3/d19/d1e/d13/d22/d38/dbb/f10e [0,4194304] 0 2024-03-22T07:18:46.045 INFO:tasks.workunit.client.0.smithi081.stdout:9/837: write d2/d10/f35 [179517,3508] 0 2024-03-22T07:18:46.045 INFO:tasks.workunit.client.0.smithi081.stdout:2/734: dwrite da/db/d39/d46/d7e/fd7 [0,4194304] 0 2024-03-22T07:18:46.046 INFO:tasks.workunit.client.0.smithi081.stdout:1/861: rename d6/d6a/d85/df6/d32/dbe/ca1 to d6/d6a/d85/df6/d32/dac/d30/c123 0 2024-03-22T07:18:46.049 INFO:tasks.workunit.client.0.smithi081.stdout:9/838: mkdir d2/d5d/d58/d11d 0 2024-03-22T07:18:46.049 INFO:tasks.workunit.client.0.smithi081.stdout:9/839: truncate d2/d5d/d58/dfc/fee 604813 0 2024-03-22T07:18:46.049 INFO:tasks.workunit.client.0.smithi081.stdout:9/840: chown d2/d2b/l8d 343946 1 2024-03-22T07:18:46.050 INFO:tasks.workunit.client.0.smithi081.stdout:1/862: unlink d6/d6a/d85/df6/d24/d48/d6e/de7/fee 0 2024-03-22T07:18:46.052 INFO:tasks.workunit.client.0.smithi081.stdout:5/973: rename d4/d75/le7 to d4/d9/d90/d10d/l141 0 2024-03-22T07:18:46.053 INFO:tasks.workunit.client.0.smithi081.stdout:9/841: dread d2/d2b/f81 [0,4194304] 0 2024-03-22T07:18:46.053 INFO:tasks.workunit.client.0.smithi081.stdout:9/842: fsync d2/d73/fd1 0 2024-03-22T07:18:46.054 INFO:tasks.workunit.client.0.smithi081.stdout:9/843: rmdir d2/d73/d74/dd9 39 2024-03-22T07:18:46.056 INFO:tasks.workunit.client.0.smithi081.stdout:9/844: symlink d2/d10/d30/d57/l11e 0 2024-03-22T07:18:46.058 INFO:tasks.workunit.client.0.smithi081.stdout:2/735: read da/db/d39/d46/d7e/fd7 [1056623,28277] 0 2024-03-22T07:18:46.082 INFO:tasks.workunit.client.0.smithi081.stdout:8/835: dwrite d2/f24 [0,4194304] 0 2024-03-22T07:18:46.084 INFO:tasks.workunit.client.0.smithi081.stdout:8/836: mknod d2/d3/d28/d2b/d61/d7e/dda/c120 0 2024-03-22T07:18:46.084 INFO:tasks.workunit.client.0.smithi081.stdout:8/837: chown d2/d3/d19/d1e/d63/da0/d9c 1571020 1 2024-03-22T07:18:46.105 INFO:tasks.workunit.client.0.smithi081.stdout:5/974: dwrite d4/df/d22/f58 [4194304,4194304] 0 2024-03-22T07:18:46.105 INFO:tasks.workunit.client.0.smithi081.stdout:2/736: dwrite da/db/d39/d46/d42/db4/dbb/fdf [0,4194304] 0 2024-03-22T07:18:46.105 INFO:tasks.workunit.client.0.smithi081.stdout:9/845: dwrite d2/f3 [4194304,4194304] 0 2024-03-22T07:18:46.105 INFO:tasks.workunit.client.0.smithi081.stdout:8/838: dwrite d2/d3/d28/f100 [0,4194304] 0 2024-03-22T07:18:46.106 INFO:tasks.workunit.client.0.smithi081.stdout:2/737: getdents da/db/d39/d46/d42 0 2024-03-22T07:18:46.106 INFO:tasks.workunit.client.0.smithi081.stdout:9/846: creat d2/d2b/d49/f11f x:0 0 0 2024-03-22T07:18:46.109 INFO:tasks.workunit.client.0.smithi081.stdout:8/839: creat d2/d3/d23/d110/f121 x:0 0 0 2024-03-22T07:18:46.113 INFO:tasks.workunit.client.0.smithi081.stdout:8/840: symlink d2/d3/d19/d1e/d13/d66/d70/d86/l122 0 2024-03-22T07:18:46.115 INFO:tasks.workunit.client.0.smithi081.stdout:9/847: dread d2/d2b/f3b [0,4194304] 0 2024-03-22T07:18:46.115 INFO:tasks.workunit.client.0.smithi081.stdout:8/841: symlink d2/d3/d19/d1e/d63/da0/db7/l123 0 2024-03-22T07:18:46.115 INFO:tasks.workunit.client.0.smithi081.stdout:8/842: truncate d2/d3/d19/d1e/f5f 879895 0 2024-03-22T07:18:46.116 INFO:tasks.workunit.client.0.smithi081.stdout:9/848: rmdir d2/d10/d30/deb 0 2024-03-22T07:18:46.117 INFO:tasks.workunit.client.0.smithi081.stdout:9/849: truncate d2/d5/d65/f118 932235 0 2024-03-22T07:18:46.117 INFO:tasks.workunit.client.0.smithi081.stdout:8/843: getdents d2 0 2024-03-22T07:18:46.119 INFO:tasks.workunit.client.0.smithi081.stdout:9/850: mknod d2/d73/d85/ddd/c120 0 2024-03-22T07:18:46.119 INFO:tasks.workunit.client.0.smithi081.stdout:9/851: chown d2/d10/d30/d57/f76 128424 1 2024-03-22T07:18:46.119 INFO:tasks.workunit.client.0.smithi081.stdout:8/844: rmdir d2/d3/d19/d1e/d13/d66 39 2024-03-22T07:18:46.120 INFO:tasks.workunit.client.0.smithi081.stdout:9/852: link d2/d78/la1 d2/d2b/d49/l121 0 2024-03-22T07:18:46.122 INFO:tasks.workunit.client.0.smithi081.stdout:8/845: unlink d2/d3/d28/d2b/d61/d7e/dda/ldc 0 2024-03-22T07:18:46.123 INFO:tasks.workunit.client.0.smithi081.stdout:9/853: rename d2/d78/d93/fdf to d2/d5d/d58/dc2/dfa/f122 0 2024-03-22T07:18:46.125 INFO:tasks.workunit.client.0.smithi081.stdout:2/738: write da/db/f27 [1297088,14442] 0 2024-03-22T07:18:46.131 INFO:tasks.workunit.client.0.smithi081.stdout:8/846: write d2/f5e [2767149,102385] 0 2024-03-22T07:18:46.132 INFO:tasks.workunit.client.0.smithi081.stdout:8/847: link d2/d3/d19/d1e/d13/d22/d38/d78/la4 d2/d3/d23/d110/l124 0 2024-03-22T07:18:46.133 INFO:tasks.workunit.client.0.smithi081.stdout:8/848: mkdir d2/d3/d19/d1e/d63/da0/d125 0 2024-03-22T07:18:46.134 INFO:tasks.workunit.client.0.smithi081.stdout:8/849: mknod d2/d3/d28/d2b/d61/dc6/c126 0 2024-03-22T07:18:46.134 INFO:tasks.workunit.client.0.smithi081.stdout:8/850: stat d2/d3/d19/d1e/f5a 0 2024-03-22T07:18:46.135 INFO:tasks.workunit.client.0.smithi081.stdout:8/851: creat d2/d3/d23/d110/f127 x:0 0 0 2024-03-22T07:18:46.153 INFO:tasks.workunit.client.0.smithi081.stdout:5/975: dwrite d4/d10/d20/dd2/dea/f10a [0,4194304] 0 2024-03-22T07:18:46.159 INFO:tasks.workunit.client.0.smithi081.stdout:2/739: dwrite da/db/d39/d47/fd5 [4194304,4194304] 0 2024-03-22T07:18:46.159 INFO:tasks.workunit.client.0.smithi081.stdout:2/740: chown da/db/d39/d46/d7e 425 1 2024-03-22T07:18:46.159 INFO:tasks.workunit.client.0.smithi081.stdout:9/854: dwrite d2/d2b/d49/f5c [0,4194304] 0 2024-03-22T07:18:46.161 INFO:tasks.workunit.client.0.smithi081.stdout:2/741: chown da/d11/l17 0 1 2024-03-22T07:18:46.161 INFO:tasks.workunit.client.0.smithi081.stdout:2/742: chown da/db/d39/d46/d42/dcb/fee 625890029 1 2024-03-22T07:18:46.165 INFO:tasks.workunit.client.0.smithi081.stdout:9/855: creat d2/d73/d85/d7a/d97/f123 x:0 0 0 2024-03-22T07:18:46.165 INFO:tasks.workunit.client.0.smithi081.stdout:9/856: dread - d2/d73/d85/d7a/f103 zero size 2024-03-22T07:18:46.166 INFO:tasks.workunit.client.0.smithi081.stdout:9/857: creat d2/d10/d30/d57/d66/d75/d94/f124 x:0 0 0 2024-03-22T07:18:46.169 INFO:tasks.workunit.client.0.smithi081.stdout:9/858: rename d2/d5d/lbb to d2/d10/d30/d57/d66/d75/l125 0 2024-03-22T07:18:46.169 INFO:tasks.workunit.client.0.smithi081.stdout:9/859: write d2/d10/d30/d57/dad/fbe [764256,56828] 0 2024-03-22T07:18:46.172 INFO:tasks.workunit.client.0.smithi081.stdout:2/743: write da/d11/d21/fa2 [2185620,2358] 0 2024-03-22T07:18:46.172 INFO:tasks.workunit.client.0.smithi081.stdout:5/976: dwrite d4/df3/ffc [0,4194304] 0 2024-03-22T07:18:46.172 INFO:tasks.workunit.client.0.smithi081.stdout:5/977: write d4/df/d22/d49/d115/f126 [624585,95141] 0 2024-03-22T07:18:46.174 INFO:tasks.workunit.client.0.smithi081.stdout:2/744: rmdir da/db/d39/d47/d3b/d41/d50/d6a 39 2024-03-22T07:18:46.174 INFO:tasks.workunit.client.0.smithi081.stdout:2/745: read da/db/d39/d47/d3b/d41/d50/f9d [37567,34107] 0 2024-03-22T07:18:46.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/746: mknod da/db/d39/d47/d3b/d41/d50/d6a/cf0 0 2024-03-22T07:18:46.175 INFO:tasks.workunit.client.0.smithi081.stdout:2/747: rmdir da/db/d39/d47/dda 39 2024-03-22T07:18:46.186 INFO:tasks.workunit.client.0.smithi081.stdout:9/860: dwrite d2/fe [0,4194304] 0 2024-03-22T07:18:46.191 INFO:tasks.workunit.client.0.smithi081.stdout:5/978: dread d4/d10/d20/d25/f9e [0,4194304] 0 2024-03-22T07:18:46.191 INFO:tasks.workunit.client.0.smithi081.stdout:5/979: fdatasync d4/d10/d20/db2/fa2 0 2024-03-22T07:18:46.191 INFO:tasks.workunit.client.0.smithi081.stdout:2/748: read da/d11/d21/f90 [2372007,3228] 0 2024-03-22T07:18:46.192 INFO:tasks.workunit.client.0.smithi081.stdout:2/749: read da/db/d39/d47/d6e/fea [238790,1200] 0 2024-03-22T07:18:46.192 INFO:tasks.workunit.client.0.smithi081.stdout:2/750: chown da/db/d39/d47/d3b/d41/dab 210 1 2024-03-22T07:18:46.232 INFO:tasks.workunit.client.0.smithi081.stdout:9/861: dwrite d2/d5d/d5a/fa5 [0,4194304] 0 2024-03-22T07:18:46.233 INFO:tasks.workunit.client.0.smithi081.stdout:5/980: dwrite d4/d10/d20/d25/f6e [0,4194304] 0 2024-03-22T07:18:46.234 INFO:tasks.workunit.client.0.smithi081.stdout:2/751: dwrite da/fa9 [4194304,4194304] 0 2024-03-22T07:18:46.234 INFO:tasks.workunit.client.0.smithi081.stdout:2/752: write da/db/d39/d46/fe7 [456650,38552] 0 2024-03-22T07:18:46.234 INFO:tasks.workunit.client.0.smithi081.stdout:9/862: symlink d2/d5d/d58/l126 0 2024-03-22T07:18:46.239 INFO:tasks.workunit.client.0.smithi081.stdout:2/753: link da/db/d39/d47/d3b/d41/f86 da/d56/ff1 0 2024-03-22T07:18:46.242 INFO:tasks.workunit.client.0.smithi081.stdout:2/754: chown da/l23 25 1 2024-03-22T07:18:46.247 INFO:tasks.workunit.client.0.smithi081.stdout:9/863: rename d2/d2b to d2/d73/d85/d127 0 2024-03-22T07:18:46.247 INFO:tasks.workunit.client.0.smithi081.stdout:9/864: mkdir d2/d5/d65/d128 0 2024-03-22T07:18:46.249 INFO:tasks.workunit.client.0.smithi081.stdout:5/981: dwrite d4/d10/daf/dfe/f105 [0,4194304] 0 2024-03-22T07:18:46.249 INFO:tasks.workunit.client.0.smithi081.stdout:5/982: stat d4/df/d22/d49/d115/d13f 0 2024-03-22T07:18:46.249 INFO:tasks.workunit.client.0.smithi081.stdout:5/983: read - d4/d10/d20/ffd zero size 2024-03-22T07:18:46.249 INFO:tasks.workunit.client.0.smithi081.stdout:5/984: readlink d4/d9/d90/d10d/l141 0 2024-03-22T07:18:46.253 INFO:tasks.workunit.client.0.smithi081.stdout:5/985: stat d4/d9/d90/d10d/l141 0 2024-03-22T07:18:46.255 INFO:tasks.workunit.client.0.smithi081.stdout:5/986: symlink d4/df/d22/d49/l142 0 2024-03-22T07:18:46.256 INFO:tasks.workunit.client.0.smithi081.stdout:5/987: mkdir d4/d75/d98/d143 0 2024-03-22T07:18:46.257 INFO:tasks.workunit.client.0.smithi081.stdout:5/988: write d4/d75/f113 [237369,15637] 0 2024-03-22T07:18:46.257 INFO:tasks.workunit.client.0.smithi081.stdout:5/989: write d4/df/d73/d8c/f87 [2058601,80742] 0 2024-03-22T07:18:46.257 INFO:tasks.workunit.client.0.smithi081.stdout:5/990: chown d4/d10/d20/d25/f8b 2042849601 1 2024-03-22T07:18:46.257 INFO:tasks.workunit.client.0.smithi081.stdout:5/991: getdents d4/d120 0 2024-03-22T07:18:46.264 INFO:tasks.workunit.client.0.smithi081.stdout:2/755: dwrite da/db/d39/d47/d3b/d41/f54 [0,4194304] 0 2024-03-22T07:18:46.266 INFO:tasks.workunit.client.0.smithi081.stdout:2/756: dread da/db/d39/d46/d42/fc7 [0,4194304] 0 2024-03-22T07:18:46.266 INFO:tasks.workunit.client.0.smithi081.stdout:2/757: fdatasync da/db/d39/d46/d42/dcb/fee 0 2024-03-22T07:18:46.270 INFO:tasks.workunit.client.0.smithi081.stdout:9/865: dwrite d2/d5d/d58/f90 [0,4194304] 0 2024-03-22T07:18:46.270 INFO:tasks.workunit.client.0.smithi081.stdout:9/866: fdatasync d2/f3 0 2024-03-22T07:18:46.270 INFO:tasks.workunit.client.0.smithi081.stdout:9/867: write d2/d5d/d58/dc2/dfa/f122 [444687,25586] 0 2024-03-22T07:18:46.270 INFO:tasks.workunit.client.0.smithi081.stdout:9/868: write d2/d10/d30/f9c [447383,91696] 0 2024-03-22T07:18:46.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:46.341 INFO:tasks.workunit.client.0.smithi081.stdout:9/869: creat d2/f129 x:0 0 0 2024-03-22T07:18:46.349 INFO:tasks.workunit.client.0.smithi081.stdout:9/870: getdents d2/d73/d85/ddd 0 2024-03-22T07:18:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:46.359 INFO:tasks.workunit.client.0.smithi081.stdout:9/871: dread d2/d5/d36/f6c [0,4194304] 0 2024-03-22T07:18:46.360 INFO:tasks.workunit.client.0.smithi081.stdout:9/872: write d2/d5/d65/f118 [510165,53081] 0 2024-03-22T07:18:46.377 INFO:tasks.workunit.client.0.smithi081.stdout:9/873: dwrite d2/d78/d93/fc6 [0,4194304] 0 2024-03-22T07:18:46.379 INFO:tasks.workunit.client.0.smithi081.stdout:9/874: creat d2/d5/d36/f12a x:0 0 0 2024-03-22T07:18:46.380 INFO:tasks.workunit.client.0.smithi081.stdout:9/875: symlink d2/d5/d36/l12b 0 2024-03-22T07:18:46.386 INFO:tasks.workunit.client.0.smithi081.stdout:9/876: dread d2/d5/f3f [4194304,4194304] 0 2024-03-22T07:18:46.386 INFO:tasks.workunit.client.0.smithi081.stdout:9/877: chown d2/d73/d74 644725545 1 2024-03-22T07:18:46.387 INFO:tasks.workunit.client.0.smithi081.stdout:9/878: rmdir d2/d5/d65/d11a 0 2024-03-22T07:18:46.388 INFO:tasks.workunit.client.0.smithi081.stdout:9/879: rmdir d2/d73/d85/d127 39 2024-03-22T07:18:46.388 INFO:tasks.workunit.client.0.smithi081.stdout:9/880: readlink d2/d5/d36/l115 0 2024-03-22T07:18:46.389 INFO:tasks.workunit.client.0.smithi081.stdout:9/881: symlink d2/d73/d85/d7a/d97/dda/l12c 0 2024-03-22T07:18:46.389 INFO:tasks.workunit.client.0.smithi081.stdout:9/882: symlink d2/d5/d65/l12d 0 2024-03-22T07:18:46.390 INFO:tasks.workunit.client.0.smithi081.stdout:9/883: truncate d2/f89 844478 0 2024-03-22T07:18:46.390 INFO:tasks.workunit.client.0.smithi081.stdout:9/884: dread - d2/d78/fc9 zero size 2024-03-22T07:18:46.394 INFO:tasks.workunit.client.0.smithi081.stdout:9/885: symlink d2/d10/d108/l12e 0 2024-03-22T07:18:46.394 INFO:tasks.workunit.client.0.smithi081.stdout:9/886: fdatasync d2/d5/fd 0 2024-03-22T07:18:46.400 INFO:tasks.workunit.client.0.smithi081.stdout:9/887: dread d2/d78/fa6 [0,4194304] 0 2024-03-22T07:18:46.401 INFO:tasks.workunit.client.0.smithi081.stdout:9/888: link d2/d73/d85/d127/ff4 d2/d10/d30/d57/d66/f12f 0 2024-03-22T07:18:46.402 INFO:tasks.workunit.client.0.smithi081.stdout:9/889: creat d2/d73/d74/dd9/f130 x:0 0 0 2024-03-22T07:18:46.404 INFO:tasks.workunit.client.0.smithi081.stdout:9/890: unlink d2/d73/d74/fa4 0 2024-03-22T07:18:46.406 INFO:tasks.workunit.client.0.smithi081.stdout:9/891: unlink d2/d5/d36/l12b 0 2024-03-22T07:18:46.407 INFO:tasks.workunit.client.0.smithi081.stdout:9/892: rmdir d2/d78 39 2024-03-22T07:18:46.432 INFO:tasks.workunit.client.0.smithi081.stdout:9/893: dwrite d2/f12 [0,4194304] 0 2024-03-22T07:18:46.433 INFO:tasks.workunit.client.0.smithi081.stdout:9/894: truncate d2/d10/fb0 50195 0 2024-03-22T07:18:46.433 INFO:tasks.workunit.client.0.smithi081.stdout:9/895: dread - d2/d73/d85/d127/f116 zero size 2024-03-22T07:18:46.433 INFO:tasks.workunit.client.0.smithi081.stdout:9/896: mknod d2/d73/d85/d7a/c131 0 2024-03-22T07:18:46.437 INFO:tasks.workunit.client.0.smithi081.stdout:9/897: rename d2/d5d/d58/dfc/f101 to d2/f132 0 2024-03-22T07:18:46.438 INFO:tasks.workunit.client.0.smithi081.stdout:9/898: readlink d2/d10/d30/l33 0 2024-03-22T07:18:46.440 INFO:tasks.workunit.client.0.smithi081.stdout:9/899: dread d2/d73/d85/d127/f60 [0,4194304] 0 2024-03-22T07:18:46.442 INFO:tasks.workunit.client.0.smithi081.stdout:9/900: mkdir d2/d5d/d58/d11d/d133 0 2024-03-22T07:18:46.443 INFO:tasks.workunit.client.0.smithi081.stdout:9/901: creat d2/d73/d85/d127/f134 x:0 0 0 2024-03-22T07:18:46.445 INFO:tasks.workunit.client.0.smithi081.stdout:9/902: rename d2/d5/d36/f12a to d2/d5/d65/f135 0 2024-03-22T07:18:46.446 INFO:tasks.workunit.client.0.smithi081.stdout:9/903: mkdir d2/d5d/d58/d11d/d133/d136 0 2024-03-22T07:18:46.446 INFO:tasks.workunit.client.0.smithi081.stdout:9/904: chown d2/f1e 0 1 2024-03-22T07:18:46.446 INFO:tasks.workunit.client.0.smithi081.stdout:9/905: fsync d2/d5/d9d/f8e 0 2024-03-22T07:18:46.446 INFO:tasks.workunit.client.0.smithi081.stdout:9/906: chown d2/d73/d85/d7a/c10c 80 1 2024-03-22T07:18:46.447 INFO:tasks.workunit.client.0.smithi081.stdout:9/907: mkdir d2/d73/d85/d7a/d97/d137 0 2024-03-22T07:18:46.447 INFO:tasks.workunit.client.0.smithi081.stdout:9/908: write d2/d73/fd3 [1039355,92197] 0 2024-03-22T07:18:46.448 INFO:tasks.workunit.client.0.smithi081.stdout:9/909: rename d2/d10/f29 to d2/d73/d85/f138 0 2024-03-22T07:18:46.449 INFO:tasks.workunit.client.0.smithi081.stdout:9/910: creat d2/d10/d30/d57/d66/f139 x:0 0 0 2024-03-22T07:18:46.449 INFO:tasks.workunit.client.0.smithi081.stdout:9/911: chown d2/d73/d74/fe0 15272 1 2024-03-22T07:18:46.466 INFO:tasks.workunit.client.0.smithi081.stdout:9/912: dwrite d2/d73/d85/d7a/d97/fa8 [0,4194304] 0 2024-03-22T07:18:46.467 INFO:tasks.workunit.client.0.smithi081.stdout:9/913: truncate d2/d73/d85/d127/f116 711340 0 2024-03-22T07:18:46.467 INFO:tasks.workunit.client.0.smithi081.stdout:9/914: truncate d2/d78/fc5 285997 0 2024-03-22T07:18:46.480 INFO:tasks.workunit.client.0.smithi081.stdout:9/915: dwrite d2/d10/d30/f9c [0,4194304] 0 2024-03-22T07:18:46.495 INFO:tasks.workunit.client.0.smithi081.stdout:9/916: dwrite d2/d5/d9d/f8e [0,4194304] 0 2024-03-22T07:18:46.499 INFO:tasks.workunit.client.0.smithi081.stdout:3/746: sync 2024-03-22T07:18:46.499 INFO:tasks.workunit.client.0.smithi081.stdout:1/863: sync 2024-03-22T07:18:46.499 INFO:tasks.workunit.client.0.smithi081.stdout:7/975: sync 2024-03-22T07:18:46.499 INFO:tasks.workunit.client.0.smithi081.stdout:7/976: read - d9/d10/d53/ff9 zero size 2024-03-22T07:18:46.499 INFO:tasks.workunit.client.0.smithi081.stdout:3/747: creat db/de/db8/f102 x:0 0 0 2024-03-22T07:18:46.500 INFO:tasks.workunit.client.0.smithi081.stdout:8/852: sync 2024-03-22T07:18:46.500 INFO:tasks.workunit.client.0.smithi081.stdout:8/853: chown d2/c5 262299086 1 2024-03-22T07:18:46.500 INFO:tasks.workunit.client.0.smithi081.stdout:8/854: readlink d2/d3/d19/d1e/d13/d22/l42 0 2024-03-22T07:18:46.500 INFO:tasks.workunit.client.0.smithi081.stdout:8/855: chown d2/d3/d28/d2b/f7a 1 1 2024-03-22T07:18:46.500 INFO:tasks.workunit.client.0.smithi081.stdout:8/856: chown d2/f24 313098233 1 2024-03-22T07:18:46.500 INFO:tasks.workunit.client.0.smithi081.stdout:1/864: symlink d6/d6a/d85/df6/d4c/d70/dd2/l124 0 2024-03-22T07:18:46.500 INFO:tasks.workunit.client.0.smithi081.stdout:9/917: link d2/d10/d30/cd5 d2/d5d/d58/d11d/d133/d136/c13a 0 2024-03-22T07:18:46.501 INFO:tasks.workunit.client.0.smithi081.stdout:5/992: sync 2024-03-22T07:18:46.501 INFO:tasks.workunit.client.0.smithi081.stdout:2/758: sync 2024-03-22T07:18:46.501 INFO:tasks.workunit.client.0.smithi081.stdout:3/748: mknod db/de/db8/dd2/c103 0 2024-03-22T07:18:46.501 INFO:tasks.workunit.client.0.smithi081.stdout:3/749: readlink db/d38/l94 0 2024-03-22T07:18:46.501 INFO:tasks.workunit.client.0.smithi081.stdout:3/750: write db/d38/f80 [745507,114076] 0 2024-03-22T07:18:46.501 INFO:tasks.workunit.client.0.smithi081.stdout:3/751: chown db/de/d4b/f88 1040536 1 2024-03-22T07:18:46.502 INFO:tasks.workunit.client.0.smithi081.stdout:9/918: rename d2/d5d/d58/dfc/de6/cf1 to d2/d5/d65/c13b 0 2024-03-22T07:18:46.502 INFO:tasks.workunit.client.0.smithi081.stdout:5/993: mkdir d4/df/d22/d49/d134/d144 0 2024-03-22T07:18:46.503 INFO:tasks.workunit.client.0.smithi081.stdout:3/752: mkdir db/de/d4b/d6a/d70/d3a/d5d/d104 0 2024-03-22T07:18:46.503 INFO:tasks.workunit.client.0.smithi081.stdout:3/753: write db/d23/fa7 [4774460,120557] 0 2024-03-22T07:18:46.505 INFO:tasks.workunit.client.0.smithi081.stdout:7/977: link d9/ca3 d9/d10/d16/d1e/c159 0 2024-03-22T07:18:46.506 INFO:tasks.workunit.client.0.smithi081.stdout:3/754: dread db/de/d4b/d6a/d70/ff4 [0,4194304] 0 2024-03-22T07:18:46.507 INFO:tasks.workunit.client.0.smithi081.stdout:5/994: mkdir d4/d9/d90/d145 0 2024-03-22T07:18:46.507 INFO:tasks.workunit.client.0.smithi081.stdout:2/759: dread da/db/d39/d47/fa1 [0,4194304] 0 2024-03-22T07:18:46.510 INFO:tasks.workunit.client.0.smithi081.stdout:7/978: creat d9/d10/d119/f15a x:0 0 0 2024-03-22T07:18:46.510 INFO:tasks.workunit.client.0.smithi081.stdout:5/995: rmdir d4/d10/d20/d25/da3 39 2024-03-22T07:18:46.511 INFO:tasks.workunit.client.0.smithi081.stdout:2/760: mkdir da/db/d39/d46/d7e/df2 0 2024-03-22T07:18:46.514 INFO:tasks.workunit.client.0.smithi081.stdout:9/919: write d2/d73/d85/d127/f77 [836651,130362] 0 2024-03-22T07:18:46.514 INFO:tasks.workunit.client.0.smithi081.stdout:9/920: write d2/d10/d30/d57/f7d [1514633,16638] 0 2024-03-22T07:18:46.515 INFO:tasks.workunit.client.0.smithi081.stdout:3/755: dread db/de/d4b/d6a/ffd [0,4194304] 0 2024-03-22T07:18:46.515 INFO:tasks.workunit.client.0.smithi081.stdout:3/756: creat db/d38/d8d/f105 x:0 0 0 2024-03-22T07:18:46.515 INFO:tasks.workunit.client.0.smithi081.stdout:2/761: rename da/db/d39/fde to da/db/d39/d47/d3b/d63/ff3 0 2024-03-22T07:18:46.518 INFO:tasks.workunit.client.0.smithi081.stdout:7/979: getdents d9/d10/d16/d1e 0 2024-03-22T07:18:46.521 INFO:tasks.workunit.client.0.smithi081.stdout:5/996: link d4/df/d73/f9d d4/df/d22/d49/f146 0 2024-03-22T07:18:46.521 INFO:tasks.workunit.client.0.smithi081.stdout:5/997: truncate d4/d10/d20/d25/db8/de0/f12b 489486 0 2024-03-22T07:18:46.523 INFO:tasks.workunit.client.0.smithi081.stdout:3/757: truncate db/d38/f6c 699755 0 2024-03-22T07:18:46.524 INFO:tasks.workunit.client.0.smithi081.stdout:2/762: creat da/db/d39/d47/dda/ff4 x:0 0 0 2024-03-22T07:18:46.525 INFO:tasks.workunit.client.0.smithi081.stdout:7/980: creat d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/d9d/f15b x:0 0 0 2024-03-22T07:18:46.525 INFO:tasks.workunit.client.0.smithi081.stdout:3/758: mknod db/de/d4b/d6a/dfa/c106 0 2024-03-22T07:18:46.526 INFO:tasks.workunit.client.0.smithi081.stdout:2/763: mkdir da/db/d39/d47/dda/df5 0 2024-03-22T07:18:46.527 INFO:tasks.workunit.client.0.smithi081.stdout:7/981: creat d9/d14/f15c x:0 0 0 2024-03-22T07:18:46.527 INFO:tasks.workunit.client.0.smithi081.stdout:7/982: dread - d9/d14/f15c zero size 2024-03-22T07:18:46.527 INFO:tasks.workunit.client.0.smithi081.stdout:3/759: chown c3 4661395 1 2024-03-22T07:18:46.527 INFO:tasks.workunit.client.0.smithi081.stdout:3/760: truncate db/f18 5574394 0 2024-03-22T07:18:46.527 INFO:tasks.workunit.client.0.smithi081.stdout:3/761: chown db/de/d4b 3589283 1 2024-03-22T07:18:46.527 INFO:tasks.workunit.client.0.smithi081.stdout:3/762: fdatasync db/de/d4b/d6a/f89 0 2024-03-22T07:18:46.528 INFO:tasks.workunit.client.0.smithi081.stdout:7/983: mknod d9/d10/d53/d81/d59/dbe/c15d 0 2024-03-22T07:18:46.531 INFO:tasks.workunit.client.0.smithi081.stdout:8/857: dwrite d2/d3/d23/fb3 [4194304,4194304] 0 2024-03-22T07:18:46.531 INFO:tasks.workunit.client.0.smithi081.stdout:1/865: dwrite d6/d6a/d85/df6/d4c/f6c [0,4194304] 0 2024-03-22T07:18:46.534 INFO:tasks.workunit.client.0.smithi081.stdout:5/998: dread d4/d10/d20/d25/da3/fa7 [0,4194304] 0 2024-03-22T07:18:46.535 INFO:tasks.workunit.client.0.smithi081.stdout:2/764: dread da/db/f3f [0,4194304] 0 2024-03-22T07:18:46.536 INFO:tasks.workunit.client.0.smithi081.stdout:5/999: mknod d4/d10/d20/d25/da3/c147 0 2024-03-22T07:18:46.555 INFO:tasks.workunit.client.0.smithi081.stdout:8/858: dwrite d2/d3/d28/d2b/d61/fea [0,4194304] 0 2024-03-22T07:18:46.556 INFO:tasks.workunit.client.0.smithi081.stdout:2/765: dwrite da/fa9 [4194304,4194304] 0 2024-03-22T07:18:46.556 INFO:tasks.workunit.client.0.smithi081.stdout:2/766: fdatasync da/db/d39/d47/d3b/d41/d50/f66 0 2024-03-22T07:18:46.556 INFO:tasks.workunit.client.0.smithi081.stdout:2/767: creat da/db/d39/d47/d3b/d41/dab/dac/dc2/ff6 x:0 0 0 2024-03-22T07:18:46.559 INFO:tasks.workunit.client.0.smithi081.stdout:8/859: unlink d2/d3/d19/d1e/d13/d22/d2f/d73/feb 0 2024-03-22T07:18:46.559 INFO:tasks.workunit.client.0.smithi081.stdout:8/860: fdatasync d2/d3/d28/d2b/f67 0 2024-03-22T07:18:46.569 INFO:tasks.workunit.client.0.smithi081.stdout:2/768: dread da/db/d39/d46/d42/db4/dbb/fdf [0,4194304] 0 2024-03-22T07:18:46.569 INFO:tasks.workunit.client.0.smithi081.stdout:2/769: read - da/db/d39/d46/f2e zero size 2024-03-22T07:18:46.570 INFO:tasks.workunit.client.0.smithi081.stdout:2/770: mkdir da/db/d39/d45/df7 0 2024-03-22T07:18:46.570 INFO:tasks.workunit.client.0.smithi081.stdout:8/861: write d2/d3/d19/d1e/d13/d22/f6f [1059997,113096] 0 2024-03-22T07:18:46.570 INFO:tasks.workunit.client.0.smithi081.stdout:8/862: chown d2/d3/d19/d1e/c33 466791 1 2024-03-22T07:18:46.571 INFO:tasks.workunit.client.0.smithi081.stdout:8/863: chown d2/d3/d19/d1e/d13/ff7 976134 1 2024-03-22T07:18:46.571 INFO:tasks.workunit.client.0.smithi081.stdout:2/771: dread da/db/d39/d46/d42/fc7 [0,4194304] 0 2024-03-22T07:18:46.571 INFO:tasks.workunit.client.0.smithi081.stdout:2/772: getdents da/db/d39/d45/dcc/de8/dba 0 2024-03-22T07:18:46.577 INFO:tasks.workunit.client.0.smithi081.stdout:2/773: dread da/d11/f1f [0,4194304] 0 2024-03-22T07:18:46.577 INFO:tasks.workunit.client.0.smithi081.stdout:2/774: fsync da/db/d39/d45/f4c 0 2024-03-22T07:18:46.579 INFO:tasks.workunit.client.0.smithi081.stdout:2/775: rename da/db/d39/d46/d42/d67/fa4 to da/db/d6d/d72/ff8 0 2024-03-22T07:18:46.580 INFO:tasks.workunit.client.0.smithi081.stdout:2/776: link da/db/l10 da/db/d39/d46/d42/db4/lf9 0 2024-03-22T07:18:46.581 INFO:tasks.workunit.client.0.smithi081.stdout:2/777: mknod da/db/d39/d46/d42/db4/dbb/cfa 0 2024-03-22T07:18:46.581 INFO:tasks.workunit.client.0.smithi081.stdout:2/778: read da/d11/d21/f90 [3708133,70605] 0 2024-03-22T07:18:46.591 INFO:tasks.workunit.client.0.smithi081.stdout:8/864: dwrite d2/d3/d19/d1e/d13/d22/d38/fc5 [0,4194304] 0 2024-03-22T07:18:46.593 INFO:tasks.workunit.client.0.smithi081.stdout:8/865: symlink d2/d3/d19/d1e/l128 0 2024-03-22T07:18:46.594 INFO:tasks.workunit.client.0.smithi081.stdout:8/866: fdatasync d2/d3/d28/f49 0 2024-03-22T07:18:46.598 INFO:tasks.workunit.client.0.smithi081.stdout:2/779: dwrite da/db/d39/d47/d3b/d41/d50/d6a/fe6 [0,4194304] 0 2024-03-22T07:18:46.601 INFO:tasks.workunit.client.0.smithi081.stdout:2/780: mknod da/db/d39/d45/cfb 0 2024-03-22T07:18:46.603 INFO:tasks.workunit.client.0.smithi081.stdout:2/781: mkdir da/db/d39/d45/dcc/de8/dfc 0 2024-03-22T07:18:46.604 INFO:tasks.workunit.client.0.smithi081.stdout:2/782: mknod da/db/d39/d46/d7e/df2/cfd 0 2024-03-22T07:18:46.604 INFO:tasks.workunit.client.0.smithi081.stdout:2/783: write da/d11/d21/d71/dc3/fef [999945,125934] 0 2024-03-22T07:18:46.604 INFO:tasks.workunit.client.0.smithi081.stdout:8/867: truncate d2/d3/d19/d1e/d13/d22/f6f 4047840 0 2024-03-22T07:18:46.606 INFO:tasks.workunit.client.0.smithi081.stdout:8/868: truncate d2/d3/d28/f106 426966 0 2024-03-22T07:18:46.606 INFO:tasks.workunit.client.0.smithi081.stdout:8/869: chown d2/d3/d28/d2b/d61/dc6 1 1 2024-03-22T07:18:46.607 INFO:tasks.workunit.client.0.smithi081.stdout:2/784: mknod da/db/d39/cfe 0 2024-03-22T07:18:46.608 INFO:tasks.workunit.client.0.smithi081.stdout:8/870: mknod d2/d3/d9e/c129 0 2024-03-22T07:18:46.609 INFO:tasks.workunit.client.0.smithi081.stdout:8/871: rmdir d2/d3/d9e/dc9 39 2024-03-22T07:18:46.610 INFO:tasks.workunit.client.0.smithi081.stdout:8/872: mkdir d2/d3/d19/d1e/d13/d66/d70/d12a 0 2024-03-22T07:18:46.619 INFO:tasks.workunit.client.0.smithi081.stdout:8/873: dread d2/d3/d19/d1e/d13/d22/d38/fc5 [0,4194304] 0 2024-03-22T07:18:46.623 INFO:tasks.workunit.client.0.smithi081.stdout:8/874: getdents d2/d3/d19/d64 0 2024-03-22T07:18:46.624 INFO:tasks.workunit.client.0.smithi081.stdout:8/875: write d2/d3/f1b [4596960,102240] 0 2024-03-22T07:18:46.624 INFO:tasks.workunit.client.0.smithi081.stdout:8/876: stat d2/d3/d28/d2b/f10b 0 2024-03-22T07:18:46.625 INFO:tasks.workunit.client.0.smithi081.stdout:2/785: dwrite da/db/d39/d47/d3b/d41/d50/fcf [0,4194304] 0 2024-03-22T07:18:46.627 INFO:tasks.workunit.client.0.smithi081.stdout:2/786: readlink da/l1d 0 2024-03-22T07:18:46.629 INFO:tasks.workunit.client.0.smithi081.stdout:2/787: link da/db/d39/d47/f5b da/db/d39/d47/d3b/d64/fff 0 2024-03-22T07:18:46.632 INFO:tasks.workunit.client.0.smithi081.stdout:8/877: dread d2/d3/d19/d1e/d13/f60 [0,4194304] 0 2024-03-22T07:18:46.632 INFO:tasks.workunit.client.0.smithi081.stdout:8/878: truncate d2/d3/d28/d2b/d61/f74 396412 0 2024-03-22T07:18:46.632 INFO:tasks.workunit.client.0.smithi081.stdout:8/879: symlink d2/d3/d23/l12b 0 2024-03-22T07:18:46.633 INFO:tasks.workunit.client.0.smithi081.stdout:8/880: chown d2/d3/d9e/dc9 784081 1 2024-03-22T07:18:46.634 INFO:tasks.workunit.client.0.smithi081.stdout:2/788: write da/db/d39/d46/f2e [111219,70833] 0 2024-03-22T07:18:46.645 INFO:tasks.workunit.client.0.smithi081.stdout:2/789: dread da/db/d6d/f73 [0,4194304] 0 2024-03-22T07:18:46.645 INFO:tasks.workunit.client.0.smithi081.stdout:2/790: write da/db/d39/d47/f3d [1059494,73182] 0 2024-03-22T07:18:46.646 INFO:tasks.workunit.client.0.smithi081.stdout:2/791: rename da/d56/f6b to da/db/d39/d47/d3b/d41/dab/dac/dc2/f100 0 2024-03-22T07:18:46.647 INFO:tasks.workunit.client.0.smithi081.stdout:2/792: creat da/db/d39/d47/d3b/d63/f101 x:0 0 0 2024-03-22T07:18:46.648 INFO:tasks.workunit.client.0.smithi081.stdout:2/793: readlink da/d11/d21/l2a 0 2024-03-22T07:18:46.651 INFO:tasks.workunit.client.0.smithi081.stdout:8/881: dwrite d2/d3/d19/d1e/d63/da0/d9c/f119 [4194304,4194304] 0 2024-03-22T07:18:46.659 INFO:tasks.workunit.client.0.smithi081.stdout:8/882: dread d2/d3/d28/d2b/d61/fe0 [0,4194304] 0 2024-03-22T07:18:46.659 INFO:tasks.workunit.client.0.smithi081.stdout:8/883: truncate d2/d3/d28/d2b/f10b 864749 0 2024-03-22T07:18:46.675 INFO:tasks.workunit.client.0.smithi081.stdout:8/884: dwrite d2/d3/d19/d1e/d13/f60 [4194304,4194304] 0 2024-03-22T07:18:46.675 INFO:tasks.workunit.client.0.smithi081.stdout:8/885: write d2/d3/d19/d1e/d13/d22/d2f/fd1 [1436362,126746] 0 2024-03-22T07:18:46.676 INFO:tasks.workunit.client.0.smithi081.stdout:8/886: mkdir d2/d3/d19/d1e/d13/d66/d70/d12a/d12c 0 2024-03-22T07:18:46.685 INFO:tasks.workunit.client.0.smithi081.stdout:8/887: dread d2/d3/d28/d2b/d61/fea [0,4194304] 0 2024-03-22T07:18:46.686 INFO:tasks.workunit.client.0.smithi081.stdout:8/888: creat d2/d3/d28/d2b/d61/d7e/dda/f12d x:0 0 0 2024-03-22T07:18:46.686 INFO:tasks.workunit.client.0.smithi081.stdout:8/889: write d2/d3/d28/d2b/f67 [459834,114202] 0 2024-03-22T07:18:46.691 INFO:tasks.workunit.client.0.smithi081.stdout:8/890: dread d2/d3/d19/d1e/d13/d22/d38/fc5 [0,4194304] 0 2024-03-22T07:18:46.691 INFO:tasks.workunit.client.0.smithi081.stdout:8/891: truncate d2/d3/d19/d1e/d13/d66/dc1/f10a 410240 0 2024-03-22T07:18:46.692 INFO:tasks.workunit.client.0.smithi081.stdout:8/892: dread d2/d3/d19/d1e/d13/d22/d2f/f77 [0,4194304] 0 2024-03-22T07:18:46.693 INFO:tasks.workunit.client.0.smithi081.stdout:8/893: link d2/d3/d19/d1e/d13/d22/d38/d78/la4 d2/d3/d19/d1e/d13/d66/d70/l12e 0 2024-03-22T07:18:46.694 INFO:tasks.workunit.client.0.smithi081.stdout:8/894: rmdir d2/d3/d19/d1e/d13/d22/db8 39 2024-03-22T07:18:46.695 INFO:tasks.workunit.client.0.smithi081.stdout:8/895: mkdir d2/d3/d19/d1e/d13/d66/d70/d12a/d12c/d12f 0 2024-03-22T07:18:46.695 INFO:tasks.workunit.client.0.smithi081.stdout:8/896: dread - d2/d3/d23/ff5 zero size 2024-03-22T07:18:46.695 INFO:tasks.workunit.client.0.smithi081.stdout:8/897: stat d2/d3/d19/d1e/d13/d66/d70/d12a/d12c/d12f 0 2024-03-22T07:18:46.697 INFO:tasks.workunit.client.0.smithi081.stdout:8/898: unlink d2/d3/d19/d1e/d63/da0/db7/dc8/cb9 0 2024-03-22T07:18:46.699 INFO:tasks.workunit.client.0.smithi081.stdout:8/899: rename d2/d3/d19/d1e/f3c to d2/d3/d19/d1e/d13/d66/d70/d86/f130 0 2024-03-22T07:18:46.699 INFO:tasks.workunit.client.0.smithi081.stdout:8/900: truncate d2/d3/d28/f82 9163963 0 2024-03-22T07:18:46.703 INFO:tasks.workunit.client.0.smithi081.stdout:8/901: truncate d2/d3/d19/d1e/d13/d22/f6f 1658639 0 2024-03-22T07:18:46.710 INFO:tasks.workunit.client.0.smithi081.stdout:8/902: write d2/d3/d19/d1e/d13/f60 [2071934,19616] 0 2024-03-22T07:18:46.711 INFO:tasks.workunit.client.0.smithi081.stdout:8/903: mkdir d2/d3/d19/d1e/d13/d22/d38/d131 0 2024-03-22T07:18:46.712 INFO:tasks.workunit.client.0.smithi081.stdout:8/904: mkdir d2/d3/d28/d2b/d61/d7e/dda/d132 0 2024-03-22T07:18:46.712 INFO:tasks.workunit.client.0.smithi081.stdout:8/905: chown d2/d3/d19/d1e/d13/d66/d70/le2 7670 1 2024-03-22T07:18:46.717 INFO:tasks.workunit.client.0.smithi081.stdout:8/906: dread d2/f5e [0,4194304] 0 2024-03-22T07:18:46.718 INFO:tasks.workunit.client.0.smithi081.stdout:8/907: creat d2/d3/d19/d1e/d13/d66/d70/f133 x:0 0 0 2024-03-22T07:18:46.719 INFO:tasks.workunit.client.0.smithi081.stdout:8/908: mkdir d2/d3/d19/d1e/d63/da0/d134 0 2024-03-22T07:18:46.720 INFO:tasks.workunit.client.0.smithi081.stdout:8/909: getdents d2/d3/d19/d1e 0 2024-03-22T07:18:46.722 INFO:tasks.workunit.client.0.smithi081.stdout:8/910: rmdir d2/d3/d19/d1e/d13/d66/d70 39 2024-03-22T07:18:46.722 INFO:tasks.workunit.client.0.smithi081.stdout:8/911: readlink d2/d3/d19/d1e/d63/da0/db7/dc8/l87 0 2024-03-22T07:18:46.723 INFO:tasks.workunit.client.0.smithi081.stdout:8/912: link d2/d3/d28/d2b/l4f d2/d3/d28/d2b/d61/d7e/ded/l135 0 2024-03-22T07:18:46.724 INFO:tasks.workunit.client.0.smithi081.stdout:8/913: getdents d2/d3/d19/d1e/d13/da5 0 2024-03-22T07:18:46.725 INFO:tasks.workunit.client.0.smithi081.stdout:8/914: write d2/d3/d19/d1e/d13/ff6 [575432,61035] 0 2024-03-22T07:18:46.726 INFO:tasks.workunit.client.0.smithi081.stdout:8/915: creat d2/d3/d19/d1e/d63/da0/db7/f136 x:0 0 0 2024-03-22T07:18:46.727 INFO:tasks.workunit.client.0.smithi081.stdout:8/916: creat d2/d3/f137 x:0 0 0 2024-03-22T07:18:46.743 INFO:tasks.workunit.client.0.smithi081.stdout:8/917: dwrite d2/d3/d28/d2b/d61/fea [0,4194304] 0 2024-03-22T07:18:46.744 INFO:tasks.workunit.client.0.smithi081.stdout:8/918: dread d2/d3/d28/f106 [0,4194304] 0 2024-03-22T07:18:46.745 INFO:tasks.workunit.client.0.smithi081.stdout:8/919: symlink d2/d3/d19/d1e/d13/d22/d2f/d73/l138 0 2024-03-22T07:18:46.746 INFO:tasks.workunit.client.0.smithi081.stdout:8/920: link d2/d3/d19/d1e/f5a d2/d3/d28/d2b/f139 0 2024-03-22T07:18:46.746 INFO:tasks.workunit.client.0.smithi081.stdout:8/921: stat d2/d3/d19/d64/ff3 0 2024-03-22T07:18:46.747 INFO:tasks.workunit.client.0.smithi081.stdout:8/922: chown d2/d3/d19/d1e/d63/l7d 13540 1 2024-03-22T07:18:46.747 INFO:tasks.workunit.client.0.smithi081.stdout:8/923: symlink d2/d3/d19/d1e/d13/d22/l13a 0 2024-03-22T07:18:46.748 INFO:tasks.workunit.client.0.smithi081.stdout:8/924: write d2/d3/d19/d1e/d63/da0/db7/f136 [494067,29722] 0 2024-03-22T07:18:46.748 INFO:tasks.workunit.client.0.smithi081.stdout:8/925: dread d2/d3/d19/d1e/d63/f65 [0,4194304] 0 2024-03-22T07:18:46.751 INFO:tasks.workunit.client.0.smithi081.stdout:8/926: getdents d2/d3/d19/d1e/d13/d22/d2f/d73 0 2024-03-22T07:18:46.759 INFO:tasks.workunit.client.0.smithi081.stdout:8/927: dread d2/d3/d19/d1e/d13/d22/db8/fbc [0,4194304] 0 2024-03-22T07:18:46.768 INFO:tasks.workunit.client.0.smithi081.stdout:8/928: write d2/d3/d19/d1e/d13/d22/f51 [3590919,107695] 0 2024-03-22T07:18:46.768 INFO:tasks.workunit.client.0.smithi081.stdout:8/929: write d2/d3/d19/d64/fee [4730526,1597] 0 2024-03-22T07:18:46.769 INFO:tasks.workunit.client.0.smithi081.stdout:8/930: unlink d2/d3/d19/d1e/d13/d22/d2f/d73/l138 0 2024-03-22T07:18:46.769 INFO:tasks.workunit.client.0.smithi081.stdout:8/931: chown d2/d3/d19/d1e/d63/da0/db7/dc8/d7c/c88 23651 1 2024-03-22T07:18:46.769 INFO:tasks.workunit.client.0.smithi081.stdout:8/932: fsync d2/d3/d28/f4a 0 2024-03-22T07:18:46.769 INFO:tasks.workunit.client.0.smithi081.stdout:8/933: fsync d2/d3/f4 0 2024-03-22T07:18:46.775 INFO:tasks.workunit.client.0.smithi081.stdout:8/934: dread d2/d3/d19/d1e/d63/da0/db7/dc8/f92 [0,4194304] 0 2024-03-22T07:18:46.776 INFO:tasks.workunit.client.0.smithi081.stdout:8/935: mknod d2/d3/d28/d2b/d61/dd7/c13b 0 2024-03-22T07:18:46.786 INFO:tasks.workunit.client.0.smithi081.stdout:8/936: read d2/d3/d19/d1e/d63/da0/db7/dc8/d7c/fc4 [2155079,104631] 0 2024-03-22T07:18:46.787 INFO:tasks.workunit.client.0.smithi081.stdout:8/937: mkdir d2/d3/d19/d1e/d13/d66/dc1/d13c 0 2024-03-22T07:18:46.788 INFO:tasks.workunit.client.0.smithi081.stdout:8/938: mknod d2/d3/d19/d1e/d13/d22/d2f/c13d 0 2024-03-22T07:18:46.791 INFO:tasks.workunit.client.0.smithi081.stdout:8/939: dread d2/d3/d19/f25 [0,4194304] 0 2024-03-22T07:18:46.791 INFO:tasks.workunit.client.0.smithi081.stdout:8/940: mkdir d2/d3/d9e/dc9/dca/d13e 0 2024-03-22T07:18:46.792 INFO:tasks.workunit.client.0.smithi081.stdout:8/941: mkdir d2/d3/d19/d1e/d13/d66/d70/d86/d13f 0 2024-03-22T07:18:46.793 INFO:tasks.workunit.client.0.smithi081.stdout:8/942: mknod d2/d3/d19/d1e/d13/d22/db8/c140 0 2024-03-22T07:18:46.802 INFO:tasks.workunit.client.0.smithi081.stdout:8/943: write d2/f15 [8194814,79762] 0 2024-03-22T07:18:46.829 INFO:tasks.workunit.client.0.smithi081.stdout:8/944: dwrite d2/d3/d19/d1e/d13/d22/fbe [0,4194304] 0 2024-03-22T07:18:46.833 INFO:tasks.workunit.client.0.smithi081.stdout:8/945: dread d2/d3/d28/f6c [0,4194304] 0 2024-03-22T07:18:46.834 INFO:tasks.workunit.client.0.smithi081.stdout:8/946: rename d2/d3/d19/d1e/d13/f39 to d2/f141 0 2024-03-22T07:18:46.840 INFO:tasks.workunit.client.0.smithi081.stdout:8/947: dread d2/d3/d19/d1e/d13/d22/f51 [0,4194304] 0 2024-03-22T07:18:46.840 INFO:tasks.workunit.client.0.smithi081.stdout:8/948: fsync d2/d3/d28/d2b/fa8 0 2024-03-22T07:18:46.841 INFO:tasks.workunit.client.0.smithi081.stdout:8/949: rename d2/d3/d28/f100 to d2/f142 0 2024-03-22T07:18:46.842 INFO:tasks.workunit.client.0.smithi081.stdout:8/950: creat d2/d3/d19/d1e/d63/da0/db7/dc8/d7c/f143 x:0 0 0 2024-03-22T07:18:46.842 INFO:tasks.workunit.client.0.smithi081.stdout:8/951: creat d2/d3/d19/d1e/d63/da0/d125/f144 x:0 0 0 2024-03-22T07:18:46.842 INFO:tasks.workunit.client.0.smithi081.stdout:8/952: fsync d2/d3/d28/f82 0 2024-03-22T07:18:46.847 INFO:tasks.workunit.client.0.smithi081.stdout:8/953: write d2/d3/d19/d1e/d63/f65 [135780,129427] 0 2024-03-22T07:18:46.848 INFO:tasks.workunit.client.0.smithi081.stdout:8/954: creat d2/d3/d19/d1e/d13/d22/db8/f145 x:0 0 0 2024-03-22T07:18:46.849 INFO:tasks.workunit.client.0.smithi081.stdout:8/955: rename d2/d3/d19/d1e/d13/d22/d38/d78/la4 to d2/d3/d19/d1e/d13/d111/l146 0 2024-03-22T07:18:46.849 INFO:tasks.workunit.client.0.smithi081.stdout:8/956: fdatasync d2/d3/d19/d1e/d63/da0/d9c/f119 0 2024-03-22T07:18:46.850 INFO:tasks.workunit.client.0.smithi081.stdout:8/957: creat d2/d3/d19/d1e/d13/d66/d70/d86/d107/f147 x:0 0 0 2024-03-22T07:18:46.851 INFO:tasks.workunit.client.0.smithi081.stdout:8/958: unlink d2/c4e 0 2024-03-22T07:18:46.852 INFO:tasks.workunit.client.0.smithi081.stdout:8/959: link f0 d2/d3/d19/d1e/d13/da5/f148 0 2024-03-22T07:18:46.852 INFO:tasks.workunit.client.0.smithi081.stdout:8/960: read - d2/d3/d28/d2b/d61/ddb/fff zero size 2024-03-22T07:18:46.853 INFO:tasks.workunit.client.0.smithi081.stdout:8/961: symlink d2/d3/d28/d2b/d61/dc6/l149 0 2024-03-22T07:18:46.853 INFO:tasks.workunit.client.0.smithi081.stdout:9/921: sync 2024-03-22T07:18:46.854 INFO:tasks.workunit.client.0.smithi081.stdout:9/922: chown d2/l3d 38572 1 2024-03-22T07:18:46.854 INFO:tasks.workunit.client.0.smithi081.stdout:9/923: dread - d2/d73/d74/fe0 zero size 2024-03-22T07:18:46.855 INFO:tasks.workunit.client.0.smithi081.stdout:9/924: creat d2/d10/f13c x:0 0 0 2024-03-22T07:18:46.855 INFO:tasks.workunit.client.0.smithi081.stdout:9/925: chown d2/d5d/d58/d11d/d133/d136 13 1 2024-03-22T07:18:46.858 INFO:tasks.workunit.client.0.smithi081.stdout:8/962: read d2/d3/d19/d1e/f5c [3845747,20164] 0 2024-03-22T07:18:46.859 INFO:tasks.workunit.client.0.smithi081.stdout:8/963: mknod d2/d3/d19/c14a 0 2024-03-22T07:18:46.860 INFO:tasks.workunit.client.0.smithi081.stdout:8/964: mkdir d2/d3/d23/d14b 0 2024-03-22T07:18:46.862 INFO:tasks.workunit.client.0.smithi081.stdout:8/965: rmdir d2/d3/d28/d2b/d61/d7e 39 2024-03-22T07:18:46.862 INFO:tasks.workunit.client.0.smithi081.stdout:8/966: dread - d2/d3/d19/d1e/d13/d22/d38/f11b zero size 2024-03-22T07:18:46.862 INFO:tasks.workunit.client.0.smithi081.stdout:8/967: chown d2/d3/d9e 55 1 2024-03-22T07:18:46.864 INFO:tasks.workunit.client.0.smithi081.stdout:9/926: dread d2/d73/d85/d127/d49/d86/fb7 [0,4194304] 0 2024-03-22T07:18:46.864 INFO:tasks.workunit.client.0.smithi081.stdout:9/927: fdatasync d2/d10/d30/d57/f9e 0 2024-03-22T07:18:46.865 INFO:tasks.workunit.client.0.smithi081.stdout:8/968: write d2/f141 [790996,21171] 0 2024-03-22T07:18:46.866 INFO:tasks.workunit.client.0.smithi081.stdout:8/969: stat d2/d3/d19 0 2024-03-22T07:18:46.867 INFO:tasks.workunit.client.0.smithi081.stdout:8/970: rename d2/d3/d19/d1e/d13/l8e to d2/d3/d28/d2b/d61/d7e/dda/d132/l14c 0 2024-03-22T07:18:46.867 INFO:tasks.workunit.client.0.smithi081.stdout:8/971: fsync d2/d3/d28/f4a 0 2024-03-22T07:18:46.874 INFO:tasks.workunit.client.0.smithi081.stdout:9/928: write d2/d10/d30/d57/f9e [4627310,51928] 0 2024-03-22T07:18:46.876 INFO:tasks.workunit.client.0.smithi081.stdout:9/929: mkdir d2/d10/d30/d57/d66/d75/dca/d13d 0 2024-03-22T07:18:46.879 INFO:tasks.workunit.client.0.smithi081.stdout:9/930: mkdir d2/d10/d30/d102/d13e 0 2024-03-22T07:18:46.879 INFO:tasks.workunit.client.0.smithi081.stdout:9/931: write d2/d10/d30/d57/d66/d75/fc4 [361719,130794] 0 2024-03-22T07:18:46.879 INFO:tasks.workunit.client.0.smithi081.stdout:9/932: creat d2/d10/d30/d57/d66/d75/d94/f13f x:0 0 0 2024-03-22T07:18:46.879 INFO:tasks.workunit.client.0.smithi081.stdout:9/933: write d2/d73/d74/dd9/f130 [288404,62708] 0 2024-03-22T07:18:46.884 INFO:tasks.workunit.client.0.smithi081.stdout:9/934: dread d2/d73/d85/d127/f41 [0,4194304] 0 2024-03-22T07:18:46.884 INFO:tasks.workunit.client.0.smithi081.stdout:8/972: dwrite d2/d3/d19/d1e/d63/da0/f103 [0,4194304] 0 2024-03-22T07:18:46.885 INFO:tasks.workunit.client.0.smithi081.stdout:9/935: creat d2/d5d/d5a/f140 x:0 0 0 2024-03-22T07:18:46.887 INFO:tasks.workunit.client.0.smithi081.stdout:9/936: rmdir d2/d5d 39 2024-03-22T07:18:46.888 INFO:tasks.workunit.client.0.smithi081.stdout:9/937: creat d2/d10/d30/d57/d66/f141 x:0 0 0 2024-03-22T07:18:46.890 INFO:tasks.workunit.client.0.smithi081.stdout:9/938: rename d2/f5f to d2/d73/d74/f142 0 2024-03-22T07:18:46.892 INFO:tasks.workunit.client.0.smithi081.stdout:9/939: creat d2/d10/f143 x:0 0 0 2024-03-22T07:18:46.892 INFO:tasks.workunit.client.0.smithi081.stdout:9/940: write d2/d5/d65/f106 [855015,27941] 0 2024-03-22T07:18:46.893 INFO:tasks.workunit.client.0.smithi081.stdout:9/941: truncate d2/d78/ffd 3751253 0 2024-03-22T07:18:46.900 INFO:tasks.workunit.client.0.smithi081.stdout:8/973: dwrite d2/d3/d28/d2b/fa8 [0,4194304] 0 2024-03-22T07:18:46.900 INFO:tasks.workunit.client.0.smithi081.stdout:8/974: getdents d2/d3/d28/d2b/d61/dc6/de6 0 2024-03-22T07:18:46.921 INFO:tasks.workunit.client.0.smithi081.stdout:8/975: dwrite d2/d3/d19/d1e/d63/da0/db7/dc8/d7c/fc4 [0,4194304] 0 2024-03-22T07:18:46.921 INFO:tasks.workunit.client.0.smithi081.stdout:9/942: dwrite d2/d73/d85/d127/f2d [0,4194304] 0 2024-03-22T07:18:46.922 INFO:tasks.workunit.client.0.smithi081.stdout:3/763: sync 2024-03-22T07:18:46.925 INFO:tasks.workunit.client.0.smithi081.stdout:3/764: getdents db/de/d4b/d6a/d70/d3a/d5d 0 2024-03-22T07:18:46.926 INFO:tasks.workunit.client.0.smithi081.stdout:2/794: sync 2024-03-22T07:18:46.926 INFO:tasks.workunit.client.0.smithi081.stdout:7/984: sync 2024-03-22T07:18:46.926 INFO:tasks.workunit.client.0.smithi081.stdout:1/866: sync 2024-03-22T07:18:46.926 INFO:tasks.workunit.client.0.smithi081.stdout:1/867: fsync d6/d18/de3/ff1 0 2024-03-22T07:18:46.927 INFO:tasks.workunit.client.0.smithi081.stdout:1/868: stat d6/d18/d41/le0 0 2024-03-22T07:18:46.929 INFO:tasks.workunit.client.0.smithi081.stdout:2/795: symlink da/db/d39/l102 0 2024-03-22T07:18:46.929 INFO:tasks.workunit.client.0.smithi081.stdout:1/869: creat d6/d6a/d85/df6/d76/f125 x:0 0 0 2024-03-22T07:18:46.931 INFO:tasks.workunit.client.0.smithi081.stdout:8/976: write d2/d3/d28/d2b/d61/f74 [61989,35151] 0 2024-03-22T07:18:46.932 INFO:tasks.workunit.client.0.smithi081.stdout:8/977: link d2/d3/d23/d110/f127 d2/d3/d19/d1e/d13/d111/f14d 0 2024-03-22T07:18:46.935 INFO:tasks.workunit.client.0.smithi081.stdout:8/978: dread d2/d3/f4 [0,4194304] 0 2024-03-22T07:18:46.936 INFO:tasks.workunit.client.0.smithi081.stdout:7/985: dread d9/d3f/db3/fc6 [0,4194304] 0 2024-03-22T07:18:46.936 INFO:tasks.workunit.client.0.smithi081.stdout:7/986: fdatasync d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/f136 0 2024-03-22T07:18:46.936 INFO:tasks.workunit.client.0.smithi081.stdout:7/987: fdatasync d9/d10/d53/d81/d59/d89/dd4/de0/d6d/d8c/fdf 0 2024-03-22T07:18:46.938 INFO:tasks.workunit.client.0.smithi081.stdout:7/988: getdents d9/dc/dbc 0 2024-03-22T07:18:46.940 INFO:tasks.workunit.client.0.smithi081.stdout:2/796: read da/db/d39/d46/fb6 [3343373,63281] 0 2024-03-22T07:18:46.941 INFO:tasks.workunit.client.0.smithi081.stdout:2/797: getdents da/db/d39/d46 0 2024-03-22T07:18:46.942 INFO:tasks.workunit.client.0.smithi081.stdout:2/798: rename da/db/d39/d46/d42/db4/dbb to da/db/d39/d47/d3b/dc5/d103 0 2024-03-22T07:18:46.946 INFO:tasks.workunit.client.0.smithi081.stdout:2/799: dread da/db/f3f [0,4194304] 0 2024-03-22T07:18:46.946 INFO:tasks.workunit.client.0.smithi081.stdout:2/800: fdatasync da/d11/d21/d71/dc3/fd4 0 2024-03-22T07:18:46.948 INFO:tasks.workunit.client.0.smithi081.stdout:2/801: symlink da/db/d39/d47/d3b/d41/l104 0 2024-03-22T07:18:46.949 INFO:tasks.workunit.client.0.smithi081.stdout:2/802: mknod da/db/d39/d47/d3b/dc5/c105 0 2024-03-22T07:18:46.949 INFO:tasks.workunit.client.0.smithi081.stdout:2/803: fsync da/db/d39/d47/dda/ff4 0 2024-03-22T07:18:46.951 INFO:tasks.workunit.client.0.smithi081.stdout:8/979: write d2/d3/d28/f4a [2292330,13917] 0 2024-03-22T07:18:46.952 INFO:tasks.workunit.client.0.smithi081.stdout:3/765: dwrite db/de/d4b/d6a/d70/d3a/d5d/ffb [0,4194304] 0 2024-03-22T07:18:46.955 INFO:tasks.workunit.client.0.smithi081.stdout:3/766: creat db/ddf/ded/f107 x:0 0 0 2024-03-22T07:18:46.957 INFO:tasks.workunit.client.0.smithi081.stdout:3/767: creat db/d38/da1/db7/f108 x:0 0 0 2024-03-22T07:18:46.958 INFO:tasks.workunit.client.0.smithi081.stdout:3/768: mkdir db/ddf/ded/d109 0 2024-03-22T07:18:46.960 INFO:tasks.workunit.client.0.smithi081.stdout:3/769: creat db/d38/da1/f10a x:0 0 0 2024-03-22T07:18:46.960 INFO:tasks.workunit.client.0.smithi081.stdout:3/770: fdatasync db/d38/fde 0 2024-03-22T07:18:46.962 INFO:tasks.workunit.client.0.smithi081.stdout:3/771: creat db/de/f10b x:0 0 0 2024-03-22T07:18:46.973 INFO:tasks.workunit.client.0.smithi081.stdout:1/870: dwrite d6/d18/f4f [4194304,4194304] 0 2024-03-22T07:18:46.973 INFO:tasks.workunit.client.0.smithi081.stdout:7/989: dwrite d9/d10/d9e/f120 [0,4194304] 0 2024-03-22T07:18:46.973 INFO:tasks.workunit.client.0.smithi081.stdout:7/990: chown d9/d10/d16/d1e/d6f/f94 252027 1 2024-03-22T07:18:46.974 INFO:tasks.workunit.client.0.smithi081.stdout:2/804: dwrite da/db/f29 [0,4194304] 0 2024-03-22T07:18:46.974 INFO:tasks.workunit.client.0.smithi081.stdout:2/805: write da/db/d39/d47/d6e/fea [732506,123092] 0 2024-03-22T07:18:46.982 INFO:tasks.workunit.client.0.smithi081.stdout:7/991: write d9/f140 [313936,101227] 0 2024-03-22T07:18:46.982 INFO:tasks.workunit.client.0.smithi081.stdout:7/992: write d9/d10/d4b/d9b/fdc [397069,80048] 0 2024-03-22T07:18:46.998 INFO:tasks.workunit.client.0.smithi081.stdout:2/806: dwrite da/db/d39/d47/fa1 [0,4194304] 0 2024-03-22T07:18:46.998 INFO:tasks.workunit.client.0.smithi081.stdout:2/807: chown da/d11/f1f 62780435 1 2024-03-22T07:18:47.000 INFO:tasks.workunit.client.0.smithi081.stdout:1/871: dwrite d6/d6a/d85/df6/d32/dbe/fa0 [0,4194304] 0 2024-03-22T07:18:47.001 INFO:tasks.workunit.client.0.smithi081.stdout:7/993: dwrite d9/d3f/d12d/f13e [0,4194304] 0 2024-03-22T07:18:47.006 INFO:tasks.workunit.client.0.smithi081.stdout:1/872: creat d6/d6a/d85/df6/d32/dbe/dc4/f126 x:0 0 0 2024-03-22T07:18:47.006 INFO:tasks.workunit.client.0.smithi081.stdout:1/873: chown d6/d18/d41/lb6 0 1 2024-03-22T07:18:47.007 INFO:tasks.workunit.client.0.smithi081.stdout:1/874: symlink d6/d18/d41/d49/d101/l127 0 2024-03-22T07:18:47.018 INFO:tasks.workunit.client.0.smithi081.stdout:2/808: dwrite da/db/d39/d46/fb6 [4194304,4194304] 0 2024-03-22T07:18:47.018 INFO:tasks.workunit.client.0.smithi081.stdout:2/809: chown da/db/d39/d45/dcc/de8/dfc 10 1 2024-03-22T07:18:47.024 INFO:tasks.workunit.client.0.smithi081.stdout:7/994: dwrite d9/d14/f15c [0,4194304] 0 2024-03-22T07:18:47.024 INFO:tasks.workunit.client.0.smithi081.stdout:7/995: chown d9/d3f/d52/d99 38221034 1 2024-03-22T07:18:47.025 INFO:tasks.workunit.client.0.smithi081.stdout:1/875: dwrite d6/d6a/d85/df6/f25 [0,4194304] 0 2024-03-22T07:18:47.029 INFO:tasks.workunit.client.0.smithi081.stdout:7/996: rmdir d9/d10/d16 39 2024-03-22T07:18:47.034 INFO:tasks.workunit.client.0.smithi081.stdout:1/876: write d6/d6a/d85/df6/d4c/f6c [829980,129518] 0 2024-03-22T07:18:47.034 INFO:tasks.workunit.client.0.smithi081.stdout:1/877: fsync d6/d6a/d85/df6/d24/d48/d6e/f11a 0 2024-03-22T07:18:47.035 INFO:tasks.workunit.client.0.smithi081.stdout:1/878: truncate d6/d18/d41/f46 1900400 0 2024-03-22T07:18:47.035 INFO:tasks.workunit.client.0.smithi081.stdout:1/879: chown d6/d6a/d85/df6/d32/dac/fcd 211409 1 2024-03-22T07:18:47.036 INFO:tasks.workunit.client.0.smithi081.stdout:2/810: dwrite da/db/d39/d47/d3b/d64/fff [4194304,4194304] 0 2024-03-22T07:18:47.037 INFO:tasks.workunit.client.0.smithi081.stdout:7/997: write d9/d10/d16/d1e/f35 [1708310,9229] 0 2024-03-22T07:18:47.039 INFO:tasks.workunit.client.0.smithi081.stdout:7/998: creat d9/d10/d4b/de6/f15e x:0 0 0 2024-03-22T07:18:47.039 INFO:tasks.workunit.client.0.smithi081.stdout:7/999: write d9/d10/d119/f15a [221177,11774] 0 2024-03-22T07:18:47.065 INFO:tasks.workunit.client.0.smithi081.stdout:2/811: dwrite da/db/d39/d47/d3b/d64/fff [4194304,4194304] 0 2024-03-22T07:18:47.065 INFO:tasks.workunit.client.0.smithi081.stdout:2/812: chown da/db/d39/d47/d3b/d41/dab/dac/dc2/ff6 0 1 2024-03-22T07:18:47.065 INFO:tasks.workunit.client.0.smithi081.stdout:2/813: fsync da/d56/f58 0 2024-03-22T07:18:47.066 INFO:tasks.workunit.client.0.smithi081.stdout:2/814: readlink da/db/d39/d45/l85 0 2024-03-22T07:18:47.067 INFO:tasks.workunit.client.0.smithi081.stdout:2/815: chown da/c91 228 1 2024-03-22T07:18:47.072 INFO:tasks.workunit.client.0.smithi081.stdout:2/816: symlink da/db/d39/d45/dcc/de8/l106 0 2024-03-22T07:18:47.072 INFO:tasks.workunit.client.0.smithi081.stdout:2/817: dread - da/db/d39/d47/dda/ff4 zero size 2024-03-22T07:18:47.096 INFO:tasks.workunit.client.0.smithi081.stdout:2/818: dwrite da/db/f27 [0,4194304] 0 2024-03-22T07:18:47.096 INFO:tasks.workunit.client.0.smithi081.stdout:2/819: fdatasync da/db/d39/d46/d42/db4/fbd 0 2024-03-22T07:18:47.099 INFO:tasks.workunit.client.0.smithi081.stdout:2/820: link da/db/d39/d47/d3b/d41/d50/ld2 da/db/d39/d47/d3b/d41/dab/dac/dc2/l107 0 2024-03-22T07:18:47.099 INFO:tasks.workunit.client.0.smithi081.stdout:2/821: creat da/db/d39/d47/dda/f108 x:0 0 0 2024-03-22T07:18:47.099 INFO:tasks.workunit.client.0.smithi081.stdout:2/822: chown da/d11/l17 280 1 2024-03-22T07:18:47.100 INFO:tasks.workunit.client.0.smithi081.stdout:2/823: creat da/db/d39/d46/d42/db4/f109 x:0 0 0 2024-03-22T07:18:47.101 INFO:tasks.workunit.client.0.smithi081.stdout:2/824: rmdir da/db/d39/d45/df7 0 2024-03-22T07:18:47.110 INFO:tasks.workunit.client.0.smithi081.stdout:2/825: write da/db/d39/d47/d3b/d41/d50/d6a/fe6 [1402420,95454] 0 2024-03-22T07:18:47.111 INFO:tasks.workunit.client.0.smithi081.stdout:2/826: unlink da/db/d39/d46/f2e 0 2024-03-22T07:18:47.112 INFO:tasks.workunit.client.0.smithi081.stdout:2/827: creat da/db/d39/d46/d42/dc9/f10a x:0 0 0 2024-03-22T07:18:47.115 INFO:tasks.workunit.client.0.smithi081.stdout:2/828: read da/db/f35 [961199,126088] 0 2024-03-22T07:18:47.296 INFO:tasks.workunit.client.0.smithi081.stdout:9/943: sync 2024-03-22T07:18:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:47 smithi081 ceph-mon[113965]: from='client.44297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:47 smithi081 ceph-mon[113965]: pgmap v219: 65 pgs: 65 active+clean; 1.9 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 117 MiB/s rd, 153 MiB/s wr, 281 op/s 2024-03-22T07:18:47.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:47 smithi149 ceph-mon[83447]: from='client.44297 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:47.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:47 smithi149 ceph-mon[83447]: pgmap v219: 65 pgs: 65 active+clean; 1.9 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 117 MiB/s rd, 153 MiB/s wr, 281 op/s 2024-03-22T07:18:47.431 INFO:tasks.workunit.client.0.smithi081.stdout:9/944: truncate d2/d5/d9d/f44 1836850 0 2024-03-22T07:18:47.431 INFO:tasks.workunit.client.0.smithi081.stdout:9/945: dread d2/d10/d30/d57/fb8 [0,4194304] 0 2024-03-22T07:18:47.439 INFO:tasks.workunit.client.0.smithi081.stdout:9/946: unlink d2/d73/d85/d7a/c109 0 2024-03-22T07:18:47.439 INFO:tasks.workunit.client.0.smithi081.stdout:9/947: creat d2/d5d/d58/dc2/f144 x:0 0 0 2024-03-22T07:18:47.442 INFO:tasks.workunit.client.0.smithi081.stdout:9/948: unlink d2/d10/d30/d57/d66/d75/l92 0 2024-03-22T07:18:47.442 INFO:tasks.workunit.client.0.smithi081.stdout:9/949: dread - d2/d5/d65/f135 zero size 2024-03-22T07:18:47.442 INFO:tasks.workunit.client.0.smithi081.stdout:9/950: fsync d2/f132 0 2024-03-22T07:18:47.442 INFO:tasks.workunit.client.0.smithi081.stdout:9/951: read - d2/d73/d85/d127/ff4 zero size 2024-03-22T07:18:47.442 INFO:tasks.workunit.client.0.smithi081.stdout:9/952: creat d2/d10/d30/d102/f145 x:0 0 0 2024-03-22T07:18:47.472 INFO:tasks.workunit.client.0.smithi081.stdout:9/953: rmdir d2/d5d/d58/dfc 39 2024-03-22T07:18:47.472 INFO:tasks.workunit.client.0.smithi081.stdout:9/954: write d2/d73/d85/d127/f77 [1666824,33932] 0 2024-03-22T07:18:47.472 INFO:tasks.workunit.client.0.smithi081.stdout:9/955: write d2/d73/d85/d127/f41 [9103225,119155] 0 2024-03-22T07:18:47.473 INFO:tasks.workunit.client.0.smithi081.stdout:9/956: getdents d2/d10/d108 0 2024-03-22T07:18:47.475 INFO:tasks.workunit.client.0.smithi081.stdout:8/980: sync 2024-03-22T07:18:47.475 INFO:tasks.workunit.client.0.smithi081.stdout:3/772: sync 2024-03-22T07:18:47.475 INFO:tasks.workunit.client.0.smithi081.stdout:3/773: dread - db/de/d4b/d6a/d70/d3a/d5d/f9f zero size 2024-03-22T07:18:47.475 INFO:tasks.workunit.client.0.smithi081.stdout:2/829: sync 2024-03-22T07:18:47.475 INFO:tasks.workunit.client.0.smithi081.stdout:1/880: sync 2024-03-22T07:18:47.475 INFO:tasks.workunit.client.0.smithi081.stdout:1/881: dread - d6/d6a/d85/df6/d32/dac/d30/d93/fd5 zero size 2024-03-22T07:18:47.476 INFO:tasks.workunit.client.0.smithi081.stdout:1/882: creat d6/dc6/f128 x:0 0 0 2024-03-22T07:18:47.476 INFO:tasks.workunit.client.0.smithi081.stdout:9/957: rename d2/d5d/lcb to d2/d5d/d58/dfc/dc1/l146 0 2024-03-22T07:18:47.477 INFO:tasks.workunit.client.0.smithi081.stdout:3/774: creat db/de/d4b/f10c x:0 0 0 2024-03-22T07:18:47.477 INFO:tasks.workunit.client.0.smithi081.stdout:2/830: symlink da/d56/l10b 0 2024-03-22T07:18:47.480 INFO:tasks.workunit.client.0.smithi081.stdout:3/775: rename db/d23/fb2 to db/de/d4b/d9a/f10d 0 2024-03-22T07:18:47.480 INFO:tasks.workunit.client.0.smithi081.stdout:3/776: truncate db/d23/f52 704535 0 2024-03-22T07:18:47.481 INFO:tasks.workunit.client.0.smithi081.stdout:2/831: rmdir da/db/d39/d47/d3b/d41/d50/d6a 39 2024-03-22T07:18:47.483 INFO:tasks.workunit.client.0.smithi081.stdout:1/883: dread d6/d6a/d85/df6/d32/ffe [0,4194304] 0 2024-03-22T07:18:47.483 INFO:tasks.workunit.client.0.smithi081.stdout:3/777: link db/d23/c63 db/ddf/ded/df7/c10e 0 2024-03-22T07:18:47.483 INFO:tasks.workunit.client.0.smithi081.stdout:3/778: fsync db/f2f 0 2024-03-22T07:18:47.484 INFO:tasks.workunit.client.0.smithi081.stdout:3/779: mknod db/ddf/ded/df7/c10f 0 2024-03-22T07:18:47.485 INFO:tasks.workunit.client.0.smithi081.stdout:1/884: rmdir d6/d6a/d85/df6/d24/d106 0 2024-03-22T07:18:47.486 INFO:tasks.workunit.client.0.smithi081.stdout:1/885: symlink d6/d6a/d85/df6/d4c/d70/l129 0 2024-03-22T07:18:47.487 INFO:tasks.workunit.client.0.smithi081.stdout:9/958: dread d2/f3 [4194304,4194304] 0 2024-03-22T07:18:47.489 INFO:tasks.workunit.client.0.smithi081.stdout:9/959: mknod d2/d10/d30/d57/d66/d75/d94/c147 0 2024-03-22T07:18:47.489 INFO:tasks.workunit.client.0.smithi081.stdout:9/960: fdatasync d2/f3 0 2024-03-22T07:18:47.489 INFO:tasks.workunit.client.0.smithi081.stdout:9/961: fsync d2/d10/d30/d102/f145 0 2024-03-22T07:18:47.490 INFO:tasks.workunit.client.0.smithi081.stdout:9/962: rename d2/f132 to d2/d73/d85/d127/d49/f148 0 2024-03-22T07:18:47.490 INFO:tasks.workunit.client.0.smithi081.stdout:9/963: creat d2/d5d/d58/dc2/f149 x:0 0 0 2024-03-22T07:18:47.491 INFO:tasks.workunit.client.0.smithi081.stdout:2/832: dread da/d11/d21/f4e [0,4194304] 0 2024-03-22T07:18:47.492 INFO:tasks.workunit.client.0.smithi081.stdout:9/964: dread d2/d73/fd3 [0,4194304] 0 2024-03-22T07:18:47.492 INFO:tasks.workunit.client.0.smithi081.stdout:9/965: creat d2/d10/d30/d57/d66/d75/dca/f14a x:0 0 0 2024-03-22T07:18:47.496 INFO:tasks.workunit.client.0.smithi081.stdout:8/981: dwrite d2/f142 [0,4194304] 0 2024-03-22T07:18:47.496 INFO:tasks.workunit.client.0.smithi081.stdout:8/982: write d2/d3/d9e/fbd [113716,23830] 0 2024-03-22T07:18:47.496 INFO:tasks.workunit.client.0.smithi081.stdout:9/966: dread d2/d5d/d58/f84 [0,4194304] 0 2024-03-22T07:18:47.651 INFO:tasks.workunit.client.0.smithi081.stdout:2/833: mkdir da/d11/d21/d71/dc3/d10c 0 2024-03-22T07:18:47.651 INFO:tasks.workunit.client.0.smithi081.stdout:2/834: readlink da/db/d39/d47/d3b/d41/dab/dac/lb1 0 2024-03-22T07:18:47.653 INFO:tasks.workunit.client.0.smithi081.stdout:8/983: getdents d2/d3/d28/d2b/d61/dc6 0 2024-03-22T07:18:47.723 INFO:tasks.workunit.client.0.smithi081.stdout:2/835: rename da/db/d39/d46/fb6 to da/db/d39/d45/dcc/de8/f10d 0 2024-03-22T07:18:47.727 INFO:tasks.workunit.client.0.smithi081.stdout:2/836: mknod da/db/c10e 0 2024-03-22T07:18:47.727 INFO:tasks.workunit.client.0.smithi081.stdout:2/837: fsync da/db/d39/d47/d3b/d41/dab/dac/dc2/fec 0 2024-03-22T07:18:47.727 INFO:tasks.workunit.client.0.smithi081.stdout:2/838: fdatasync da/db/d39/d47/d3b/d41/f51 0 2024-03-22T07:18:47.727 INFO:tasks.workunit.client.0.smithi081.stdout:2/839: chown da/db/d39/d45/dcc 0 1 2024-03-22T07:18:47.728 INFO:tasks.workunit.client.0.smithi081.stdout:8/984: rmdir d2/d3/d28/d2b/d61/dc6/de6 0 2024-03-22T07:18:47.729 INFO:tasks.workunit.client.0.smithi081.stdout:2/840: mknod da/db/d39/d47/d3b/d63/c10f 0 2024-03-22T07:18:47.731 INFO:tasks.workunit.client.0.smithi081.stdout:8/985: creat d2/f14e x:0 0 0 2024-03-22T07:18:47.731 INFO:tasks.workunit.client.0.smithi081.stdout:2/841: rmdir da/db/d39/d47/d3b/d41/d50 39 2024-03-22T07:18:47.732 INFO:tasks.workunit.client.0.smithi081.stdout:8/986: symlink d2/d3/d19/d1e/d13/d22/d38/l14f 0 2024-03-22T07:18:47.733 INFO:tasks.workunit.client.0.smithi081.stdout:1/886: rmdir d6/d6a/d85/df6/d32/dac/d30/d93/dbc 39 2024-03-22T07:18:47.734 INFO:tasks.workunit.client.0.smithi081.stdout:2/842: rename da/d11/d21/l2a to da/d11/d21/d71/dc3/d10c/l110 0 2024-03-22T07:18:47.734 INFO:tasks.workunit.client.0.smithi081.stdout:2/843: chown da/d56/c79 0 1 2024-03-22T07:18:47.735 INFO:tasks.workunit.client.0.smithi081.stdout:8/987: symlink d2/d3/d28/d2b/l150 0 2024-03-22T07:18:47.736 INFO:tasks.workunit.client.0.smithi081.stdout:8/988: creat d2/d3/d19/d1e/d13/d66/dc1/d13c/f151 x:0 0 0 2024-03-22T07:18:47.736 INFO:tasks.workunit.client.0.smithi081.stdout:8/989: stat d2/d3/d19/d1e 0 2024-03-22T07:18:47.739 INFO:tasks.workunit.client.0.smithi081.stdout:1/887: dread d6/d18/f74 [0,4194304] 0 2024-03-22T07:18:47.740 INFO:tasks.workunit.client.0.smithi081.stdout:1/888: unlink d6/d6a/d85/df6/d32/dbe/dc4/fc5 0 2024-03-22T07:18:47.740 INFO:tasks.workunit.client.0.smithi081.stdout:1/889: chown d6/f7b 32705 1 2024-03-22T07:18:47.741 INFO:tasks.workunit.client.0.smithi081.stdout:1/890: fsync d6/d6a/d85/df6/d4c/d70/d7d/f88 0 2024-03-22T07:18:47.741 INFO:tasks.workunit.client.0.smithi081.stdout:1/891: write d6/fc [4238549,25087] 0 2024-03-22T07:18:47.742 INFO:tasks.workunit.client.0.smithi081.stdout:1/892: creat d6/d6a/d85/df6/d32/df9/f12a x:0 0 0 2024-03-22T07:18:47.742 INFO:tasks.workunit.client.0.smithi081.stdout:1/893: write d6/d6a/d85/df6/d32/dac/d30/f61 [486433,26621] 0 2024-03-22T07:18:47.742 INFO:tasks.workunit.client.0.smithi081.stdout:1/894: chown d6/d6a/d85/df6/d24/d48/d6e/de7 92712562 1 2024-03-22T07:18:47.743 INFO:tasks.workunit.client.0.smithi081.stdout:1/895: mknod d6/d6a/d85/df6/d4c/c12b 0 2024-03-22T07:18:47.743 INFO:tasks.workunit.client.0.smithi081.stdout:1/896: chown d6/d18/d41/d49/d115 14 1 2024-03-22T07:18:47.744 INFO:tasks.workunit.client.0.smithi081.stdout:2/844: dread da/f28 [0,4194304] 0 2024-03-22T07:18:47.744 INFO:tasks.workunit.client.0.smithi081.stdout:2/845: fdatasync da/db/d39/d47/d3b/d41/d50/f66 0 2024-03-22T07:18:47.744 INFO:tasks.workunit.client.0.smithi081.stdout:1/897: mknod d6/dc6/c12c 0 2024-03-22T07:18:47.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/846: symlink da/db/d39/d46/d42/dc9/l111 0 2024-03-22T07:18:47.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/847: chown da/d56/c7c 1034 1 2024-03-22T07:18:47.746 INFO:tasks.workunit.client.0.smithi081.stdout:1/898: symlink d6/d6a/d85/df6/d24/da6/l12d 0 2024-03-22T07:18:47.746 INFO:tasks.workunit.client.0.smithi081.stdout:1/899: truncate d6/d6a/d85/df6/d32/dac/f95 95048 0 2024-03-22T07:18:47.746 INFO:tasks.workunit.client.0.smithi081.stdout:2/848: truncate f5 3904617 0 2024-03-22T07:18:47.747 INFO:tasks.workunit.client.0.smithi081.stdout:2/849: chown da/db/d39/d47/l61 2692452 1 2024-03-22T07:18:47.753 INFO:tasks.workunit.client.0.smithi081.stdout:8/990: dwrite d2/d3/d19/d1e/d13/dba/fd4 [0,4194304] 0 2024-03-22T07:18:47.754 INFO:tasks.workunit.client.0.smithi081.stdout:8/991: dread - d2/d3/d28/d2b/d61/d7e/dda/f12d zero size 2024-03-22T07:18:47.754 INFO:tasks.workunit.client.0.smithi081.stdout:1/900: dread d6/d18/f27 [0,4194304] 0 2024-03-22T07:18:47.755 INFO:tasks.workunit.client.0.smithi081.stdout:2/850: dread da/db/d39/d47/f3d [0,4194304] 0 2024-03-22T07:18:47.756 INFO:tasks.workunit.client.0.smithi081.stdout:1/901: mkdir d6/d6a/d85/df6/d24/d48/d6e/de7/d12e 0 2024-03-22T07:18:47.761 INFO:tasks.workunit.client.0.smithi081.stdout:1/902: truncate d6/d6a/d85/df6/d32/fef 2679151 0 2024-03-22T07:18:47.769 INFO:tasks.workunit.client.0.smithi081.stdout:8/992: dread d2/d3/d19/d1e/d63/da0/f103 [0,4194304] 0 2024-03-22T07:18:47.769 INFO:tasks.workunit.client.0.smithi081.stdout:8/993: creat d2/f152 x:0 0 0 2024-03-22T07:18:47.772 INFO:tasks.workunit.client.0.smithi081.stdout:8/994: truncate d2/d3/d19/d1e/d63/da0/db7/dc8/f92 3322978 0 2024-03-22T07:18:47.774 INFO:tasks.workunit.client.0.smithi081.stdout:8/995: mknod d2/d3/d28/d2b/d61/d7e/dda/d132/c153 0 2024-03-22T07:18:47.774 INFO:tasks.workunit.client.0.smithi081.stdout:8/996: dread - d2/d3/d19/d1e/d13/ff7 zero size 2024-03-22T07:18:47.774 INFO:tasks.workunit.client.0.smithi081.stdout:8/997: truncate d2/d3/d19/d1e/d13/d66/dc1/d13c/f151 118109 0 2024-03-22T07:18:47.774 INFO:tasks.workunit.client.0.smithi081.stdout:8/998: read d2/d3/f95 [81553,125049] 0 2024-03-22T07:18:47.774 INFO:tasks.workunit.client.0.smithi081.stdout:8/999: write d2/d3/d19/d1e/d13/ff6 [645113,126283] 0 2024-03-22T07:18:47.776 INFO:tasks.workunit.client.0.smithi081.stdout:1/903: dread d6/d6a/d85/df6/d32/dac/f3c [0,4194304] 0 2024-03-22T07:18:47.776 INFO:tasks.workunit.client.0.smithi081.stdout:1/904: stat d6/d6a/d85/df6/d24/d48/d6e/de7/l107 0 2024-03-22T07:18:47.776 INFO:tasks.workunit.client.0.smithi081.stdout:2/851: dwrite da/db/f27 [4194304,4194304] 0 2024-03-22T07:18:47.779 INFO:tasks.workunit.client.0.smithi081.stdout:2/852: link da/db/d39/d47/d3b/dc5/d103/cfa da/db/d39/d46/d42/db4/c112 0 2024-03-22T07:18:47.780 INFO:tasks.workunit.client.0.smithi081.stdout:1/905: dread d6/d18/f27 [0,4194304] 0 2024-03-22T07:18:47.780 INFO:tasks.workunit.client.0.smithi081.stdout:1/906: dread - d6/d6a/d85/df6/d4c/d70/f102 zero size 2024-03-22T07:18:47.780 INFO:tasks.workunit.client.0.smithi081.stdout:1/907: dread - d6/d6a/d85/df6/d4c/d70/dd2/f104 zero size 2024-03-22T07:18:47.780 INFO:tasks.workunit.client.0.smithi081.stdout:1/908: write d6/d6a/d85/df6/d32/dac/d30/d93/fe9 [1088211,79182] 0 2024-03-22T07:18:47.786 INFO:tasks.workunit.client.0.smithi081.stdout:2/853: mknod da/db/d39/d47/d3b/d41/dab/dac/c113 0 2024-03-22T07:18:47.787 INFO:tasks.workunit.client.0.smithi081.stdout:1/909: unlink d6/f2a 0 2024-03-22T07:18:47.788 INFO:tasks.workunit.client.0.smithi081.stdout:2/854: mkdir da/db/d39/d46/d42/dc9/d114 0 2024-03-22T07:18:47.788 INFO:tasks.workunit.client.0.smithi081.stdout:2/855: write da/db/d39/d46/d42/fe2 [701788,100814] 0 2024-03-22T07:18:47.791 INFO:tasks.workunit.client.0.smithi081.stdout:2/856: dread da/db/d39/d47/d3b/d41/d50/f62 [0,4194304] 0 2024-03-22T07:18:47.796 INFO:tasks.workunit.client.0.smithi081.stdout:1/910: dread d6/d6a/d85/df6/f87 [0,4194304] 0 2024-03-22T07:18:47.810 INFO:tasks.workunit.client.0.smithi081.stdout:2/857: dwrite da/db/d39/d47/d3b/d41/dab/fad [0,4194304] 0 2024-03-22T07:18:47.823 INFO:tasks.workunit.client.0.smithi081.stdout:1/911: link d6/d6a/fb9 d6/d6a/d85/df6/d32/f12f 0 2024-03-22T07:18:47.823 INFO:tasks.workunit.client.0.smithi081.stdout:2/858: write da/db/d39/d45/dcc/de8/f95 [4844410,18467] 0 2024-03-22T07:18:47.823 INFO:tasks.workunit.client.0.smithi081.stdout:2/859: creat da/db/d39/d47/dda/df5/f115 x:0 0 0 2024-03-22T07:18:47.823 INFO:tasks.workunit.client.0.smithi081.stdout:1/912: rename d6/d6a/d85/df6/d32/lce to d6/d18/de3/l130 0 2024-03-22T07:18:47.823 INFO:tasks.workunit.client.0.smithi081.stdout:1/913: fsync d6/d18/f4f 0 2024-03-22T07:18:47.824 INFO:tasks.workunit.client.0.smithi081.stdout:3/780: sync 2024-03-22T07:18:47.825 INFO:tasks.workunit.client.0.smithi081.stdout:9/967: sync 2024-03-22T07:18:47.825 INFO:tasks.workunit.client.0.smithi081.stdout:9/968: fdatasync d2/d5d/d58/dfc/ff6 0 2024-03-22T07:18:47.826 INFO:tasks.workunit.client.0.smithi081.stdout:3/781: truncate db/de/f20 3389465 0 2024-03-22T07:18:47.826 INFO:tasks.workunit.client.0.smithi081.stdout:3/782: write db/ff8 [381665,9689] 0 2024-03-22T07:18:47.827 INFO:tasks.workunit.client.0.smithi081.stdout:3/783: chown db/de/d4b/d9a/cea 97346880 1 2024-03-22T07:18:47.827 INFO:tasks.workunit.client.0.smithi081.stdout:9/969: getdents d2/d5d/d58/d11d/d133 0 2024-03-22T07:18:47.829 INFO:tasks.workunit.client.0.smithi081.stdout:9/970: creat d2/d73/d85/d127/f14b x:0 0 0 2024-03-22T07:18:47.830 INFO:tasks.workunit.client.0.smithi081.stdout:9/971: mkdir d2/d73/d74/d14c 0 2024-03-22T07:18:47.830 INFO:tasks.workunit.client.0.smithi081.stdout:9/972: chown d2/d73/d85/d127/f2d 7520 1 2024-03-22T07:18:47.831 INFO:tasks.workunit.client.0.smithi081.stdout:9/973: mknod d2/d5d/d58/c14d 0 2024-03-22T07:18:47.831 INFO:tasks.workunit.client.0.smithi081.stdout:9/974: stat d2/d73/d85/d127/d49/d86/la7 0 2024-03-22T07:18:47.833 INFO:tasks.workunit.client.0.smithi081.stdout:1/914: write d6/d18/d60/fc9 [581020,55521] 0 2024-03-22T07:18:47.833 INFO:tasks.workunit.client.0.smithi081.stdout:9/975: creat d2/d5d/d58/d11d/d133/f14e x:0 0 0 2024-03-22T07:18:47.834 INFO:tasks.workunit.client.0.smithi081.stdout:9/976: creat d2/d10/d30/f14f x:0 0 0 2024-03-22T07:18:47.835 INFO:tasks.workunit.client.0.smithi081.stdout:3/784: dread db/de/f64 [0,4194304] 0 2024-03-22T07:18:47.836 INFO:tasks.workunit.client.0.smithi081.stdout:3/785: write db/de/d4b/db0/fd5 [4477049,105915] 0 2024-03-22T07:18:47.836 INFO:tasks.workunit.client.0.smithi081.stdout:3/786: write db/d38/da1/db7/f108 [395647,46532] 0 2024-03-22T07:18:47.836 INFO:tasks.workunit.client.0.smithi081.stdout:3/787: write db/de/d4b/d6a/d70/d3a/d5d/fa8 [144521,23853] 0 2024-03-22T07:18:47.837 INFO:tasks.workunit.client.0.smithi081.stdout:3/788: symlink db/d38/l110 0 2024-03-22T07:18:47.842 INFO:tasks.workunit.client.0.smithi081.stdout:3/789: truncate db/de/d4b/db0/fd5 4226918 0 2024-03-22T07:18:47.844 INFO:tasks.workunit.client.0.smithi081.stdout:3/790: creat db/de/d4b/d6a/d70/f111 x:0 0 0 2024-03-22T07:18:47.846 INFO:tasks.workunit.client.0.smithi081.stdout:3/791: mkdir db/ddf/d112 0 2024-03-22T07:18:47.846 INFO:tasks.workunit.client.0.smithi081.stdout:3/792: truncate db/de/db8/f102 749255 0 2024-03-22T07:18:47.847 INFO:tasks.workunit.client.0.smithi081.stdout:3/793: creat db/ddf/f113 x:0 0 0 2024-03-22T07:18:47.853 INFO:tasks.workunit.client.0.smithi081.stdout:1/915: dwrite d6/d18/d41/fd1 [0,4194304] 0 2024-03-22T07:18:47.853 INFO:tasks.workunit.client.0.smithi081.stdout:1/916: fdatasync d6/d18/da5/fe1 0 2024-03-22T07:18:47.853 INFO:tasks.workunit.client.0.smithi081.stdout:1/917: chown d6/d18/c80 1 1 2024-03-22T07:18:47.857 INFO:tasks.workunit.client.0.smithi081.stdout:9/977: dwrite d2/d10/d30/d57/d66/d75/fc4 [0,4194304] 0 2024-03-22T07:18:47.857 INFO:tasks.workunit.client.0.smithi081.stdout:9/978: readlink d2/d5d/d5a/le1 0 2024-03-22T07:18:47.860 INFO:tasks.workunit.client.0.smithi081.stdout:9/979: mkdir d2/d73/d74/dd9/d150 0 2024-03-22T07:18:47.860 INFO:tasks.workunit.client.0.smithi081.stdout:9/980: readlink d2/d5d/d58/dfc/lcc 0 2024-03-22T07:18:47.861 INFO:tasks.workunit.client.0.smithi081.stdout:9/981: stat d2/d10/d30/d57/d66/d75/l125 0 2024-03-22T07:18:47.862 INFO:tasks.workunit.client.0.smithi081.stdout:9/982: mknod d2/d5d/d58/dc2/dfa/c151 0 2024-03-22T07:18:47.864 INFO:tasks.workunit.client.0.smithi081.stdout:9/983: link d2/d73/d85/d127/d49/d86/la7 d2/d5/d36/l152 0 2024-03-22T07:18:47.865 INFO:tasks.workunit.client.0.smithi081.stdout:9/984: truncate d2/f8 4131622 0 2024-03-22T07:18:47.867 INFO:tasks.workunit.client.0.smithi081.stdout:9/985: creat d2/d73/d85/f153 x:0 0 0 2024-03-22T07:18:47.870 INFO:tasks.workunit.client.0.smithi081.stdout:1/918: dwrite d6/d6a/d85/df6/d32/dac/d30/ffc [0,4194304] 0 2024-03-22T07:18:47.874 INFO:tasks.workunit.client.0.smithi081.stdout:1/919: creat d6/d6a/d85/df6/d32/dac/d30/df2/f131 x:0 0 0 2024-03-22T07:18:47.885 INFO:tasks.workunit.client.0.smithi081.stdout:9/986: dwrite d2/d73/d85/d127/ff4 [0,4194304] 0 2024-03-22T07:18:47.888 INFO:tasks.workunit.client.0.smithi081.stdout:1/920: read d6/d6a/d85/df6/d32/f12f [3669024,48798] 0 2024-03-22T07:18:47.888 INFO:tasks.workunit.client.0.smithi081.stdout:1/921: write d6/d6a/d85/df6/d4c/f6c [4166775,50645] 0 2024-03-22T07:18:47.889 INFO:tasks.workunit.client.0.smithi081.stdout:1/922: mkdir d6/d6a/d85/df6/d24/d48/d5c/d132 0 2024-03-22T07:18:47.910 INFO:tasks.workunit.client.0.smithi081.stdout:1/923: dwrite d6/d6a/d85/df6/d32/dac/d30/d93/fe9 [0,4194304] 0 2024-03-22T07:18:47.911 INFO:tasks.workunit.client.0.smithi081.stdout:1/924: truncate d6/d6a/d85/df6/d4c/d70/d7d/f88 3592739 0 2024-03-22T07:18:47.965 INFO:tasks.workunit.client.0.smithi081.stdout:1/925: dwrite d6/d6a/d85/df6/d32/dbe/dc4/f126 [0,4194304] 0 2024-03-22T07:18:48.019 INFO:tasks.workunit.client.0.smithi081.stdout:1/926: write d6/f5b [303847,68078] 0 2024-03-22T07:18:48.019 INFO:tasks.workunit.client.0.smithi081.stdout:1/927: chown d6/d18/d41 10565 1 2024-03-22T07:18:48.019 INFO:tasks.workunit.client.0.smithi081.stdout:1/928: symlink d6/d6a/d85/df6/d32/dac/d30/d93/dd7/l133 0 2024-03-22T07:18:48.036 INFO:tasks.workunit.client.0.smithi081.stdout:1/929: dwrite d6/d6a/d85/df6/d32/dac/d30/f3d [0,4194304] 0 2024-03-22T07:18:48.056 INFO:tasks.workunit.client.0.smithi081.stdout:1/930: dwrite d6/d6a/d85/df6/d24/d48/f69 [0,4194304] 0 2024-03-22T07:18:48.063 INFO:tasks.workunit.client.0.smithi081.stdout:1/931: dread d6/d6a/d85/df6/d24/d48/d5c/f77 [0,4194304] 0 2024-03-22T07:18:48.070 INFO:tasks.workunit.client.0.smithi081.stdout:1/932: link d6/c19 d6/d18/d41/c134 0 2024-03-22T07:18:48.070 INFO:tasks.workunit.client.0.smithi081.stdout:1/933: chown d6/d6a/d85/df6/d4c/c5a 2406102 1 2024-03-22T07:18:48.070 INFO:tasks.workunit.client.0.smithi081.stdout:1/934: chown d6/d18 6176453 1 2024-03-22T07:18:48.093 INFO:tasks.workunit.client.0.smithi081.stdout:1/935: dwrite d6/d6a/d85/df6/d32/dac/d30/f7f [4194304,4194304] 0 2024-03-22T07:18:48.093 INFO:tasks.workunit.client.0.smithi081.stdout:1/936: fdatasync d6/d6a/d85/df6/f71 0 2024-03-22T07:18:48.104 INFO:tasks.workunit.client.0.smithi081.stdout:1/937: rename d6/d6a/d85/df6/d4c/d70/l129 to d6/d6a/l135 0 2024-03-22T07:18:48.105 INFO:tasks.workunit.client.0.smithi081.stdout:1/938: rmdir d6/d6a/d85/df6/d24/d48/d5c/d132 0 2024-03-22T07:18:48.106 INFO:tasks.workunit.client.0.smithi081.stdout:1/939: mknod d6/d18/d41/d49/c136 0 2024-03-22T07:18:48.107 INFO:tasks.workunit.client.0.smithi081.stdout:1/940: readlink d6/d6a/d85/df6/d32/dac/d30/da8/lbf 0 2024-03-22T07:18:48.107 INFO:tasks.workunit.client.0.smithi081.stdout:1/941: readlink d6/d18/d41/d49/d101/l127 0 2024-03-22T07:18:48.122 INFO:tasks.workunit.client.0.smithi081.stdout:1/942: dwrite d6/d6a/d85/df6/d76/f8d [0,4194304] 0 2024-03-22T07:18:48.132 INFO:tasks.workunit.client.0.smithi081.stdout:2/860: sync 2024-03-22T07:18:48.132 INFO:tasks.workunit.client.0.smithi081.stdout:3/794: sync 2024-03-22T07:18:48.133 INFO:tasks.workunit.client.0.smithi081.stdout:9/987: sync 2024-03-22T07:18:48.133 INFO:tasks.workunit.client.0.smithi081.stdout:3/795: fsync db/de/f7b 0 2024-03-22T07:18:48.133 INFO:tasks.workunit.client.0.smithi081.stdout:2/861: dread - da/db/d39/d47/dda/ff4 zero size 2024-03-22T07:18:48.133 INFO:tasks.workunit.client.0.smithi081.stdout:9/988: mknod d2/d10/d30/d57/d66/d75/dca/d13d/c154 0 2024-03-22T07:18:48.135 INFO:tasks.workunit.client.0.smithi081.stdout:2/862: unlink da/db/d39/d47/d6e/fea 0 2024-03-22T07:18:48.135 INFO:tasks.workunit.client.0.smithi081.stdout:3/796: mknod db/de/db8/dcb/c114 0 2024-03-22T07:18:48.136 INFO:tasks.workunit.client.0.smithi081.stdout:3/797: mknod db/d38/d8d/db4/c115 0 2024-03-22T07:18:48.137 INFO:tasks.workunit.client.0.smithi081.stdout:3/798: write db/de/d4b/db0/fd1 [1151795,124449] 0 2024-03-22T07:18:48.137 INFO:tasks.workunit.client.0.smithi081.stdout:2/863: unlink da/db/lf 0 2024-03-22T07:18:48.138 INFO:tasks.workunit.client.0.smithi081.stdout:3/799: mknod db/c116 0 2024-03-22T07:18:48.141 INFO:tasks.workunit.client.0.smithi081.stdout:3/800: dread db/de/d4b/d6a/d70/ff4 [0,4194304] 0 2024-03-22T07:18:48.141 INFO:tasks.workunit.client.0.smithi081.stdout:3/801: chown db/d38/da1/fcf 83718 1 2024-03-22T07:18:48.141 INFO:tasks.workunit.client.0.smithi081.stdout:3/802: write db/d38/da1/db7/fe3 [427472,93400] 0 2024-03-22T07:18:48.143 INFO:tasks.workunit.client.0.smithi081.stdout:9/989: dread d2/d73/d85/fa0 [0,4194304] 0 2024-03-22T07:18:48.143 INFO:tasks.workunit.client.0.smithi081.stdout:9/990: write d2/d10/d30/d57/f7d [407913,52187] 0 2024-03-22T07:18:48.150 INFO:tasks.workunit.client.0.smithi081.stdout:3/803: write db/d23/fc9 [2025986,59387] 0 2024-03-22T07:18:48.166 INFO:tasks.workunit.client.0.smithi081.stdout:3/804: dwrite db/d38/f6c [0,4194304] 0 2024-03-22T07:18:48.167 INFO:tasks.workunit.client.0.smithi081.stdout:3/805: creat db/d23/f117 x:0 0 0 2024-03-22T07:18:48.167 INFO:tasks.workunit.client.0.smithi081.stdout:3/806: dread - db/de/d4b/d6a/d70/d3a/f6f zero size 2024-03-22T07:18:48.168 INFO:tasks.workunit.client.0.smithi081.stdout:3/807: mknod db/d38/c118 0 2024-03-22T07:18:48.184 INFO:tasks.workunit.client.0.smithi081.stdout:3/808: dwrite db/de/d4b/fd9 [0,4194304] 0 2024-03-22T07:18:48.186 INFO:tasks.workunit.client.0.smithi081.stdout:3/809: creat db/d38/d8d/f119 x:0 0 0 2024-03-22T07:18:48.186 INFO:tasks.workunit.client.0.smithi081.stdout:3/810: truncate db/de/d4b/d6a/d70/d3a/d5d/fd3 4835339 0 2024-03-22T07:18:48.193 INFO:tasks.workunit.client.0.smithi081.stdout:3/811: write db/de/d4b/d6a/d70/f44 [571906,100239] 0 2024-03-22T07:18:48.194 INFO:tasks.workunit.client.0.smithi081.stdout:3/812: symlink db/ddf/ded/d109/l11a 0 2024-03-22T07:18:48.194 INFO:tasks.workunit.client.0.smithi081.stdout:3/813: write db/de/f10b [270782,72372] 0 2024-03-22T07:18:48.293 INFO:tasks.workunit.client.0.smithi081.stdout:1/943: sync 2024-03-22T07:18:48.293 INFO:tasks.workunit.client.0.smithi081.stdout:1/944: chown d6/d6a/d85/df6/d32/dac/d30/df2/f131 3125888 1 2024-03-22T07:18:48.293 INFO:tasks.workunit.client.0.smithi081.stdout:1/945: creat d6/d6a/d85/df6/d24/da6/f137 x:0 0 0 2024-03-22T07:18:48.294 INFO:tasks.workunit.client.0.smithi081.stdout:1/946: truncate d6/d6a/d85/df6/f25 1161606 0 2024-03-22T07:18:48.294 INFO:tasks.workunit.client.0.smithi081.stdout:1/947: fdatasync d6/d6a/d85/df6/d76/f79 0 2024-03-22T07:18:48.295 INFO:tasks.workunit.client.0.smithi081.stdout:1/948: symlink d6/d6a/d85/df6/d76/l138 0 2024-03-22T07:18:48.295 INFO:tasks.workunit.client.0.smithi081.stdout:1/949: read - d6/d6a/d85/df6/f71 zero size 2024-03-22T07:18:48.297 INFO:tasks.workunit.client.0.smithi081.stdout:1/950: rename d6/d6a/d85/df6/d24/d48/d5c/c82 to d6/d6a/d85/df6/c139 0 2024-03-22T07:18:48.307 INFO:tasks.workunit.client.0.smithi081.stdout:1/951: dread d6/d6a/d85/df6/d4c/fe8 [0,4194304] 0 2024-03-22T07:18:48.308 INFO:tasks.workunit.client.0.smithi081.stdout:1/952: creat d6/d6a/d85/df6/d24/dde/f13a x:0 0 0 2024-03-22T07:18:48.308 INFO:tasks.workunit.client.0.smithi081.stdout:1/953: readlink d6/d18/d60/lff 0 2024-03-22T07:18:48.309 INFO:tasks.workunit.client.0.smithi081.stdout:1/954: creat d6/d6a/d85/df6/d32/f13b x:0 0 0 2024-03-22T07:18:48.309 INFO:tasks.workunit.client.0.smithi081.stdout:1/955: write d6/d6a/d85/df6/d4c/d70/dd2/f104 [589450,10428] 0 2024-03-22T07:18:48.317 INFO:tasks.workunit.client.0.smithi081.stdout:1/956: link d6/d18/f27 d6/d6a/d85/df6/d32/dac/d30/d93/dd7/f13c 0 2024-03-22T07:18:48.318 INFO:tasks.workunit.client.0.smithi081.stdout:1/957: mknod d6/d6a/d85/df6/d76/c13d 0 2024-03-22T07:18:48.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:48 smithi081 ceph-mon[113965]: pgmap v220: 65 pgs: 65 active+clean; 2.0 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 139 MiB/s rd, 180 MiB/s wr, 322 op/s 2024-03-22T07:18:48.319 INFO:tasks.workunit.client.0.smithi081.stdout:1/958: stat d6/d6a/d85/df6/c54 0 2024-03-22T07:18:48.319 INFO:tasks.workunit.client.0.smithi081.stdout:1/959: chown d6/d6a/d85/df6/d4c/fe8 1487478445 1 2024-03-22T07:18:48.319 INFO:tasks.workunit.client.0.smithi081.stdout:1/960: fsync d6/d6a/d85/df6/d24/d48/d6e/f75 0 2024-03-22T07:18:48.320 INFO:tasks.workunit.client.0.smithi081.stdout:1/961: getdents d6/d6a/d85/df6/d32/dac/d30/d93 0 2024-03-22T07:18:48.320 INFO:tasks.workunit.client.0.smithi081.stdout:1/962: mknod d6/d6a/d85/df6/d32/dac/d30/df2/c13e 0 2024-03-22T07:18:48.321 INFO:tasks.workunit.client.0.smithi081.stdout:1/963: mkdir d6/d6a/d85/df6/d32/dac/d30/d93/dd7/d13f 0 2024-03-22T07:18:48.329 INFO:tasks.workunit.client.0.smithi081.stdout:1/964: write d6/d6a/d85/df6/d24/f3b [3637293,83500] 0 2024-03-22T07:18:48.330 INFO:tasks.workunit.client.0.smithi081.stdout:1/965: read d6/d6a/d85/df6/d24/f6f [142929,39099] 0 2024-03-22T07:18:48.330 INFO:tasks.workunit.client.0.smithi081.stdout:1/966: write d6/d6a/d85/df6/d24/f6f [718803,57284] 0 2024-03-22T07:18:48.330 INFO:tasks.workunit.client.0.smithi081.stdout:1/967: mkdir d6/d6a/d85/df6/d32/d140 0 2024-03-22T07:18:48.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:48 smithi149 ceph-mon[83447]: pgmap v220: 65 pgs: 65 active+clean; 2.0 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 139 MiB/s rd, 180 MiB/s wr, 322 op/s 2024-03-22T07:18:48.442 INFO:tasks.workunit.client.0.smithi081.stdout:9/991: sync 2024-03-22T07:18:48.442 INFO:tasks.workunit.client.0.smithi081.stdout:3/814: sync 2024-03-22T07:18:48.443 INFO:tasks.workunit.client.0.smithi081.stdout:2/864: sync 2024-03-22T07:18:48.443 INFO:tasks.workunit.client.0.smithi081.stdout:3/815: rename db/de/d4b/d6a/d70/cae to db/ddf/c11b 0 2024-03-22T07:18:48.444 INFO:tasks.workunit.client.0.smithi081.stdout:2/865: creat da/db/d39/d47/d3b/d41/dab/dac/f116 x:0 0 0 2024-03-22T07:18:48.445 INFO:tasks.workunit.client.0.smithi081.stdout:9/992: symlink d2/d10/d30/d57/dad/l155 0 2024-03-22T07:18:48.447 INFO:tasks.workunit.client.0.smithi081.stdout:9/993: rename d2/d5/fd to d2/f156 0 2024-03-22T07:18:48.447 INFO:tasks.workunit.client.0.smithi081.stdout:9/994: creat d2/d10/d30/d57/d66/f157 x:0 0 0 2024-03-22T07:18:48.447 INFO:tasks.workunit.client.0.smithi081.stdout:9/995: stat d2/d73/d85/d7a/d97/d137 0 2024-03-22T07:18:48.448 INFO:tasks.workunit.client.0.smithi081.stdout:9/996: mknod d2/d73/d85/ddd/c158 0 2024-03-22T07:18:48.448 INFO:tasks.workunit.client.0.smithi081.stdout:3/816: dread db/de/d4b/d6a/ffd [0,4194304] 0 2024-03-22T07:18:48.454 INFO:tasks.workunit.client.0.smithi081.stdout:2/866: write da/fa9 [2046975,46104] 0 2024-03-22T07:18:48.460 INFO:tasks.workunit.client.0.smithi081.stdout:9/997: write d2/f20 [1174142,120917] 0 2024-03-22T07:18:48.471 INFO:tasks.workunit.client.0.smithi081.stdout:3/817: dwrite db/d38/ffe [0,4194304] 0 2024-03-22T07:18:48.472 INFO:tasks.workunit.client.0.smithi081.stdout:2/867: read da/db/d39/d47/d3b/d41/d50/fcf [2910708,94368] 0 2024-03-22T07:18:48.475 INFO:tasks.workunit.client.0.smithi081.stdout:3/818: symlink db/d38/d8d/l11c 0 2024-03-22T07:18:48.476 INFO:tasks.workunit.client.0.smithi081.stdout:2/868: unlink da/db/d39/d46/d7e/df2/cfd 0 2024-03-22T07:18:48.478 INFO:tasks.workunit.client.0.smithi081.stdout:3/819: link db/de/db8/fe5 db/d38/d8d/f11d 0 2024-03-22T07:18:48.479 INFO:tasks.workunit.client.0.smithi081.stdout:2/869: mkdir da/db/d39/d47/d3b/d41/dab/dac/dc2/d117 0 2024-03-22T07:18:48.481 INFO:tasks.workunit.client.0.smithi081.stdout:2/870: rename da/d11/d21/d71/dc3/fef to da/db/d6d/f118 0 2024-03-22T07:18:48.483 INFO:tasks.workunit.client.0.smithi081.stdout:3/820: mknod db/de/c11e 0 2024-03-22T07:18:48.485 INFO:tasks.workunit.client.0.smithi081.stdout:2/871: mknod da/db/d39/d47/dda/df5/c119 0 2024-03-22T07:18:48.488 INFO:tasks.workunit.client.0.smithi081.stdout:1/968: sync 2024-03-22T07:18:48.489 INFO:tasks.workunit.client.0.smithi081.stdout:9/998: dwrite d2/f12 [4194304,4194304] 0 2024-03-22T07:18:48.489 INFO:tasks.workunit.client.0.smithi081.stdout:2/872: dread da/d56/f58 [0,4194304] 0 2024-03-22T07:18:48.489 INFO:tasks.workunit.client.0.smithi081.stdout:1/969: creat d6/d6a/d85/df6/d32/dac/d30/d93/dd7/d13f/f141 x:0 0 0 2024-03-22T07:18:48.495 INFO:tasks.workunit.client.0.smithi081.stdout:1/970: dread d6/d18/d41/f46 [0,4194304] 0 2024-03-22T07:18:48.497 INFO:tasks.workunit.client.0.smithi081.stdout:1/971: mkdir d6/d6a/d85/df6/d32/dac/d30/d93/d142 0 2024-03-22T07:18:48.499 INFO:tasks.workunit.client.0.smithi081.stdout:1/972: creat d6/d6a/d85/df6/d4c/d70/f143 x:0 0 0 2024-03-22T07:18:48.499 INFO:tasks.workunit.client.0.smithi081.stdout:1/973: write d6/fc [2982236,71707] 0 2024-03-22T07:18:48.510 INFO:tasks.workunit.client.0.smithi081.stdout:3/821: dwrite db/de/d4b/db0/fd1 [0,4194304] 0 2024-03-22T07:18:48.512 INFO:tasks.workunit.client.0.smithi081.stdout:1/974: write d6/d6a/d85/df6/d24/daa/fcc [2873444,52270] 0 2024-03-22T07:18:48.512 INFO:tasks.workunit.client.0.smithi081.stdout:1/975: write d6/f7b [2092384,22978] 0 2024-03-22T07:18:48.513 INFO:tasks.workunit.client.0.smithi081.stdout:2/873: dwrite da/db/d39/d46/d42/fc7 [0,4194304] 0 2024-03-22T07:18:48.513 INFO:tasks.workunit.client.0.smithi081.stdout:2/874: creat da/db/d39/d47/dda/f11a x:0 0 0 2024-03-22T07:18:48.513 INFO:tasks.workunit.client.0.smithi081.stdout:2/875: chown da/db/d39/d47/d3b/d41/dab/dac/dc2/ff6 6215 1 2024-03-22T07:18:48.513 INFO:tasks.workunit.client.0.smithi081.stdout:1/976: rename d6/d6a/d85/df6/d24/d48/cb3 to d6/d6a/d85/df6/d24/da6/c144 0 2024-03-22T07:18:48.514 INFO:tasks.workunit.client.0.smithi081.stdout:9/999: dwrite d2/d5d/d58/dc2/f149 [0,4194304] 0 2024-03-22T07:18:48.516 INFO:tasks.workunit.client.0.smithi081.stdout:2/876: rmdir da/db/d39/d47/d3b/dc5/d103 39 2024-03-22T07:18:48.517 INFO:tasks.workunit.client.0.smithi081.stdout:1/977: getdents d6/d6a/d85/df6/d32/dac 0 2024-03-22T07:18:48.518 INFO:tasks.workunit.client.0.smithi081.stdout:1/978: getdents d6/d18 0 2024-03-22T07:18:48.520 INFO:tasks.workunit.client.0.smithi081.stdout:2/877: mkdir da/db/d39/d45/dcc/de8/d11b 0 2024-03-22T07:18:48.524 INFO:tasks.workunit.client.0.smithi081.stdout:2/878: rename da/db/d39/d47/d3b/d41/dab/dac/dc2 to da/db/d39/d46/d42/dcb/d11c 0 2024-03-22T07:18:48.524 INFO:tasks.workunit.client.0.smithi081.stdout:2/879: creat da/db/d39/d47/dda/f11d x:0 0 0 2024-03-22T07:18:48.524 INFO:tasks.workunit.client.0.smithi081.stdout:2/880: chown da/db/d39/d47/d3b/dc5/d103 0 1 2024-03-22T07:18:48.524 INFO:tasks.workunit.client.0.smithi081.stdout:2/881: chown da/db/d39/d47/d6e/l94 3733687 1 2024-03-22T07:18:48.525 INFO:tasks.workunit.client.0.smithi081.stdout:2/882: mknod da/db/d39/d47/d3b/dc5/d103/c11e 0 2024-03-22T07:18:48.526 INFO:tasks.workunit.client.0.smithi081.stdout:1/979: write d6/d6a/d85/df6/d24/d48/f69 [1788690,83377] 0 2024-03-22T07:18:48.527 INFO:tasks.workunit.client.0.smithi081.stdout:3/822: dwrite db/de/d4b/d6a/d70/d3a/f46 [0,4194304] 0 2024-03-22T07:18:48.528 INFO:tasks.workunit.client.0.smithi081.stdout:2/883: link da/d11/d21/c24 da/db/d39/d47/c11f 0 2024-03-22T07:18:48.532 INFO:tasks.workunit.client.0.smithi081.stdout:3/823: creat db/de/d4b/d9a/f11f x:0 0 0 2024-03-22T07:18:48.533 INFO:tasks.workunit.client.0.smithi081.stdout:2/884: rmdir da/db/d39/d47/d3b/d41/dab 39 2024-03-22T07:18:48.537 INFO:tasks.workunit.client.0.smithi081.stdout:2/885: truncate da/fa9 3287276 0 2024-03-22T07:18:48.539 INFO:tasks.workunit.client.0.smithi081.stdout:3/824: getdents db/de 0 2024-03-22T07:18:48.539 INFO:tasks.workunit.client.0.smithi081.stdout:3/825: write db/de/f24 [17113696,65938] 0 2024-03-22T07:18:48.540 INFO:tasks.workunit.client.0.smithi081.stdout:2/886: symlink da/db/d39/d46/d42/db4/l120 0 2024-03-22T07:18:48.540 INFO:tasks.workunit.client.0.smithi081.stdout:1/980: dread d6/d6a/d85/df6/d24/d48/d5c/f7e [4194304,4194304] 0 2024-03-22T07:18:48.543 INFO:tasks.workunit.client.0.smithi081.stdout:2/887: creat da/db/d39/d46/d42/dcb/d11c/d117/f121 x:0 0 0 2024-03-22T07:18:48.547 INFO:tasks.workunit.client.0.smithi081.stdout:3/826: rename db/d23 to db/de/d4b/d6a/d70/d3a/d5d/d104/d120 0 2024-03-22T07:18:48.549 INFO:tasks.workunit.client.0.smithi081.stdout:3/827: getdents db/de/db8/dd2 0 2024-03-22T07:18:48.549 INFO:tasks.workunit.client.0.smithi081.stdout:3/828: dread - db/de/d4b/d6a/d70/d3a/d5d/d104/d120/f117 zero size 2024-03-22T07:18:48.549 INFO:tasks.workunit.client.0.smithi081.stdout:3/829: chown db/de/fd4 3806191 1 2024-03-22T07:18:48.549 INFO:tasks.workunit.client.0.smithi081.stdout:3/830: truncate db/ff8 435008 0 2024-03-22T07:18:48.556 INFO:tasks.workunit.client.0.smithi081.stdout:2/888: dread da/db/f29 [0,4194304] 0 2024-03-22T07:18:48.561 INFO:tasks.workunit.client.0.smithi081.stdout:2/889: write da/db/f29 [255509,118795] 0 2024-03-22T07:18:48.561 INFO:tasks.workunit.client.0.smithi081.stdout:2/890: dread - da/db/d39/d47/dda/df5/f115 zero size 2024-03-22T07:18:48.565 INFO:tasks.workunit.client.0.smithi081.stdout:3/831: dwrite db/de/fd4 [0,4194304] 0 2024-03-22T07:18:48.565 INFO:tasks.workunit.client.0.smithi081.stdout:3/832: chown db/de/d4b/d6a/d70/l85 0 1 2024-03-22T07:18:48.567 INFO:tasks.workunit.client.0.smithi081.stdout:1/981: link d6/d6a/d85/df6/c10d d6/d6a/d85/df6/d32/dac/c145 0 2024-03-22T07:18:48.577 INFO:tasks.workunit.client.0.smithi081.stdout:1/982: write d6/d6a/d85/df6/d32/f59 [2650701,35450] 0 2024-03-22T07:18:48.577 INFO:tasks.workunit.client.0.smithi081.stdout:1/983: chown d6/d6a/d85/df6/d32/dac/d30/c123 103 1 2024-03-22T07:18:48.578 INFO:tasks.workunit.client.0.smithi081.stdout:1/984: creat d6/d18/d41/d49/d115/f146 x:0 0 0 2024-03-22T07:18:48.582 INFO:tasks.workunit.client.0.smithi081.stdout:3/833: write db/de/d4b/d6a/d70/d3a/d5d/d104/d120/f3f [3887836,13968] 0 2024-03-22T07:18:48.582 INFO:tasks.workunit.client.0.smithi081.stdout:3/834: truncate db/de/d4b/f84 1196778 0 2024-03-22T07:18:48.582 INFO:tasks.workunit.client.0.smithi081.stdout:3/835: chown db/ddf 1669019 1 2024-03-22T07:18:48.582 INFO:tasks.workunit.client.0.smithi081.stdout:3/836: write db/de/d4b/d6a/d70/fc4 [2154855,110734] 0 2024-03-22T07:18:48.583 INFO:tasks.workunit.client.0.smithi081.stdout:3/837: unlink db/de/d4b/d6a/d70/l9d 0 2024-03-22T07:18:48.584 INFO:tasks.workunit.client.0.smithi081.stdout:3/838: write db/de/d4b/d6a/d70/f3e [2128979,122761] 0 2024-03-22T07:18:48.584 INFO:tasks.workunit.client.0.smithi081.stdout:3/839: write db/de/d4b/d6a/d70/d3a/d5d/d104/d120/fc9 [1063495,82758] 0 2024-03-22T07:18:48.584 INFO:tasks.workunit.client.0.smithi081.stdout:3/840: getdents db/d38/da1 0 2024-03-22T07:18:48.607 INFO:tasks.workunit.client.0.smithi081.stdout:3/841: dwrite db/de/fd4 [0,4194304] 0 2024-03-22T07:18:48.607 INFO:tasks.workunit.client.0.smithi081.stdout:3/842: write db/de/d4b/d6a/dfa/ffc [316701,82851] 0 2024-03-22T07:18:48.607 INFO:tasks.workunit.client.0.smithi081.stdout:1/985: dwrite d6/d18/d41/feb [0,4194304] 0 2024-03-22T07:18:48.608 INFO:tasks.workunit.client.0.smithi081.stdout:3/843: creat db/d38/da1/f121 x:0 0 0 2024-03-22T07:18:48.611 INFO:tasks.workunit.client.0.smithi081.stdout:3/844: symlink db/de/db8/l122 0 2024-03-22T07:18:48.611 INFO:tasks.workunit.client.0.smithi081.stdout:3/845: read db/de/d4b/d6a/f89 [61263,118987] 0 2024-03-22T07:18:48.612 INFO:tasks.workunit.client.0.smithi081.stdout:1/986: stat d6/d6a/d85/df6/d32/fef 0 2024-03-22T07:18:48.612 INFO:tasks.workunit.client.0.smithi081.stdout:3/846: dread db/de/d4b/d6a/f89 [0,4194304] 0 2024-03-22T07:18:48.612 INFO:tasks.workunit.client.0.smithi081.stdout:3/847: chown db 8313661 1 2024-03-22T07:18:48.613 INFO:tasks.workunit.client.0.smithi081.stdout:3/848: write db/de/d4b/d6a/d70/d3a/d5d/d104/d120/f3f [3088351,116307] 0 2024-03-22T07:18:48.618 INFO:tasks.workunit.client.0.smithi081.stdout:1/987: read d6/d6a/d85/df6/d76/fe6 [796567,24367] 0 2024-03-22T07:18:48.622 INFO:tasks.workunit.client.0.smithi081.stdout:3/849: dread db/d38/ffe [0,4194304] 0 2024-03-22T07:18:48.622 INFO:tasks.workunit.client.0.smithi081.stdout:3/850: chown db/fa9 284 1 2024-03-22T07:18:48.622 INFO:tasks.workunit.client.0.smithi081.stdout:3/851: chown db/de/d4b/d6a/d70/f32 802868 1 2024-03-22T07:18:48.623 INFO:tasks.workunit.client.0.smithi081.stdout:3/852: rmdir db/de/db8/dcb 39 2024-03-22T07:18:48.624 INFO:tasks.workunit.client.0.smithi081.stdout:3/853: creat db/ddf/ded/d109/f123 x:0 0 0 2024-03-22T07:18:48.624 INFO:tasks.workunit.client.0.smithi081.stdout:3/854: mknod db/de/ddb/c124 0 2024-03-22T07:18:48.625 INFO:tasks.workunit.client.0.smithi081.stdout:3/855: symlink db/de/d4b/d6a/d70/d3a/d5d/l125 0 2024-03-22T07:18:48.625 INFO:tasks.workunit.client.0.smithi081.stdout:3/856: mkdir db/de/d4b/d126 0 2024-03-22T07:18:48.642 INFO:tasks.workunit.client.0.smithi081.stdout:1/988: dwrite d6/d6a/d85/df6/d32/fef [4194304,4194304] 0 2024-03-22T07:18:48.645 INFO:tasks.workunit.client.0.smithi081.stdout:3/857: dwrite db/de/d4b/d6a/d70/f111 [0,4194304] 0 2024-03-22T07:18:48.657 INFO:tasks.workunit.client.0.smithi081.stdout:1/989: dwrite d6/d6a/d85/df6/d32/ffe [0,4194304] 0 2024-03-22T07:18:48.658 INFO:tasks.workunit.client.0.smithi081.stdout:1/990: link d6/d18/d41/d49/l58 d6/d18/d41/l147 0 2024-03-22T07:18:48.659 INFO:tasks.workunit.client.0.smithi081.stdout:1/991: mkdir d6/d6a/d85/df6/d32/dbe/dc4/d148 0 2024-03-22T07:18:48.659 INFO:tasks.workunit.client.0.smithi081.stdout:1/992: creat d6/d6a/d85/df6/d32/dac/d30/df2/f149 x:0 0 0 2024-03-22T07:18:48.660 INFO:tasks.workunit.client.0.smithi081.stdout:1/993: truncate d6/d6a/d85/df6/d76/f125 468303 0 2024-03-22T07:18:48.660 INFO:tasks.workunit.client.0.smithi081.stdout:1/994: fdatasync d6/d6a/d85/df6/d32/ffe 0 2024-03-22T07:18:48.660 INFO:tasks.workunit.client.0.smithi081.stdout:1/995: write d6/d6a/d85/df6/d32/dac/d30/f34 [5180912,95185] 0 2024-03-22T07:18:48.660 INFO:tasks.workunit.client.0.smithi081.stdout:1/996: truncate d6/d6a/d85/df6/d32/dac/d30/df2/f149 413929 0 2024-03-22T07:18:48.660 INFO:tasks.workunit.client.0.smithi081.stdout:1/997: mknod d6/d6a/d85/df6/d24/dde/c14a 0 2024-03-22T07:18:48.662 INFO:tasks.workunit.client.0.smithi081.stdout:1/998: rename d6/d18/d41/d49/d101 to d6/d6a/d85/df6/d76/d14b 0 2024-03-22T07:18:48.662 INFO:tasks.workunit.client.0.smithi081.stdout:1/999: read - d6/d6a/d85/df6/d24/d48/d6e/f11a zero size 2024-03-22T07:18:48.666 INFO:tasks.workunit.client.0.smithi081.stdout:3/858: dwrite db/d38/f69 [0,4194304] 0 2024-03-22T07:18:48.818 INFO:tasks.workunit.client.0.smithi081.stdout:2/891: sync 2024-03-22T07:18:48.819 INFO:tasks.workunit.client.0.smithi081.stdout:2/892: mkdir da/db/d39/d45/dcc/de8/d11b/d122 0 2024-03-22T07:18:48.819 INFO:tasks.workunit.client.0.smithi081.stdout:2/893: chown da/l23 299 1 2024-03-22T07:18:48.838 INFO:tasks.workunit.client.0.smithi081.stdout:2/894: dwrite da/db/d39/d47/d3b/dc5/d103/fdf [0,4194304] 0 2024-03-22T07:18:48.838 INFO:tasks.workunit.client.0.smithi081.stdout:2/895: write da/db/d39/d46/f6c [12014,71954] 0 2024-03-22T07:18:48.838 INFO:tasks.workunit.client.0.smithi081.stdout:2/896: creat da/d11/d21/f123 x:0 0 0 2024-03-22T07:18:48.843 INFO:tasks.workunit.client.0.smithi081.stdout:2/897: dread da/db/d39/d46/f36 [0,4194304] 0 2024-03-22T07:18:48.844 INFO:tasks.workunit.client.0.smithi081.stdout:2/898: write da/db/d39/d46/d42/fe2 [2444,68685] 0 2024-03-22T07:18:48.845 INFO:tasks.workunit.client.0.smithi081.stdout:2/899: truncate da/db/d6d/fd0 67669 0 2024-03-22T07:18:48.858 INFO:tasks.workunit.client.0.smithi081.stdout:2/900: dwrite da/db/d39/d47/dda/ff4 [0,4194304] 0 2024-03-22T07:18:48.859 INFO:tasks.workunit.client.0.smithi081.stdout:2/901: creat da/db/d39/d46/d42/dcb/f124 x:0 0 0 2024-03-22T07:18:48.859 INFO:tasks.workunit.client.0.smithi081.stdout:2/902: chown da/db/d39/d46/d7e/df2 57213 1 2024-03-22T07:18:48.861 INFO:tasks.workunit.client.0.smithi081.stdout:2/903: symlink da/db/d39/d46/d42/dc9/d114/l125 0 2024-03-22T07:18:48.861 INFO:tasks.workunit.client.0.smithi081.stdout:2/904: chown da/d11/d21 839026 1 2024-03-22T07:18:48.861 INFO:tasks.workunit.client.0.smithi081.stdout:2/905: chown da/db/d39/d47/d3b/d63/ca0 27791 1 2024-03-22T07:18:48.861 INFO:tasks.workunit.client.0.smithi081.stdout:2/906: creat da/db/d6d/f126 x:0 0 0 2024-03-22T07:18:48.862 INFO:tasks.workunit.client.0.smithi081.stdout:2/907: mknod da/db/d39/d45/dcc/de8/d11b/d122/c127 0 2024-03-22T07:18:48.862 INFO:tasks.workunit.client.0.smithi081.stdout:2/908: getdents da/db/d39/d47/d3b/d64 0 2024-03-22T07:18:48.863 INFO:tasks.workunit.client.0.smithi081.stdout:2/909: mkdir da/d11/d128 0 2024-03-22T07:18:48.865 INFO:tasks.workunit.client.0.smithi081.stdout:2/910: link da/db/d39/d47/l5d da/db/d39/d47/d3b/d41/dab/dac/l129 0 2024-03-22T07:18:48.866 INFO:tasks.workunit.client.0.smithi081.stdout:2/911: dread da/db/d39/fae [0,4194304] 0 2024-03-22T07:18:48.866 INFO:tasks.workunit.client.0.smithi081.stdout:2/912: write da/f28 [2751802,127054] 0 2024-03-22T07:18:48.867 INFO:tasks.workunit.client.0.smithi081.stdout:2/913: rename da/db/d39/d47/d3b/dc5/d103/c11e to da/c12a 0 2024-03-22T07:18:48.872 INFO:tasks.workunit.client.0.smithi081.stdout:2/914: link da/db/d39/d47/l88 da/db/d39/d47/d3b/d41/d50/d6a/l12b 0 2024-03-22T07:18:48.881 INFO:tasks.workunit.client.0.smithi081.stdout:2/915: dread da/db/f27 [4194304,4194304] 0 2024-03-22T07:18:48.881 INFO:tasks.workunit.client.0.smithi081.stdout:2/916: fdatasync da/db/d39/d46/d42/db4/fbd 0 2024-03-22T07:18:48.882 INFO:tasks.workunit.client.0.smithi081.stdout:2/917: link da/db/d39/d47/d3b/d63/ff3 da/db/d39/d45/dcc/de8/f12c 0 2024-03-22T07:18:48.882 INFO:tasks.workunit.client.0.smithi081.stdout:2/918: write da/db/d39/d47/d3b/d41/d50/f62 [663098,9573] 0 2024-03-22T07:18:48.882 INFO:tasks.workunit.client.0.smithi081.stdout:2/919: truncate da/db/d6d/f118 1288750 0 2024-03-22T07:18:48.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/859: sync 2024-03-22T07:18:48.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/860: chown db/f13 212500890 1 2024-03-22T07:18:48.905 INFO:tasks.workunit.client.0.smithi081.stdout:2/920: dwrite da/fa9 [0,4194304] 0 2024-03-22T07:18:48.905 INFO:tasks.workunit.client.0.smithi081.stdout:2/921: chown da/db/d39/d45/dcc/de8/l84 2067950 1 2024-03-22T07:18:48.905 INFO:tasks.workunit.client.0.smithi081.stdout:2/922: readlink da/db/d39/d45/dcc/de8/l106 0 2024-03-22T07:18:48.905 INFO:tasks.workunit.client.0.smithi081.stdout:2/923: chown da/c91 89 1 2024-03-22T07:18:48.921 INFO:tasks.workunit.client.0.smithi081.stdout:2/924: dwrite da/db/d39/d47/d3b/dc5/d103/fe3 [0,4194304] 0 2024-03-22T07:18:48.925 INFO:tasks.workunit.client.0.smithi081.stdout:2/925: rename da/d11 to da/db/d39/d47/d3b/d64/d12d 0 2024-03-22T07:18:48.927 INFO:tasks.workunit.client.0.smithi081.stdout:2/926: chown da/ce 8116 1 2024-03-22T07:18:48.928 INFO:tasks.workunit.client.0.smithi081.stdout:2/927: stat da/db/d39/d47/d3b/d64/d12d/f4d 0 2024-03-22T07:18:48.934 INFO:tasks.workunit.client.0.smithi081.stdout:2/928: read da/db/d39/d47/fa1 [1373242,111092] 0 2024-03-22T07:18:48.963 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade status' 2024-03-22T07:18:49.029 INFO:tasks.workunit.client.0.smithi081.stdout:3/861: sync 2024-03-22T07:18:49.029 INFO:tasks.workunit.client.0.smithi081.stdout:3/862: dread - db/de/ddb/ff6 zero size 2024-03-22T07:18:49.036 INFO:tasks.workunit.client.0.smithi081.stdout:3/863: dread db/de/d4b/d6a/d70/d3a/d5d/ffb [0,4194304] 0 2024-03-22T07:18:49.043 INFO:tasks.workunit.client.0.smithi081.stdout:3/864: dread db/de/f4e [0,4194304] 0 2024-03-22T07:18:49.044 INFO:tasks.workunit.client.0.smithi081.stdout:3/865: getdents db/de/ddb 0 2024-03-22T07:18:49.044 INFO:tasks.workunit.client.0.smithi081.stdout:3/866: stat db/d38/da1/db7/le4 0 2024-03-22T07:18:49.045 INFO:tasks.workunit.client.0.smithi081.stdout:3/867: truncate db/d38/f6c 4645223 0 2024-03-22T07:18:49.045 INFO:tasks.workunit.client.0.smithi081.stdout:3/868: truncate db/fc 4158237 0 2024-03-22T07:18:49.046 INFO:tasks.workunit.client.0.smithi081.stdout:3/869: mknod db/d38/da1/c127 0 2024-03-22T07:18:49.046 INFO:tasks.workunit.client.0.smithi081.stdout:3/870: getdents db/d38/ddd 0 2024-03-22T07:18:49.046 INFO:tasks.workunit.client.0.smithi081.stdout:3/871: chown db/de/d4b/d6a/d70/d3a/d5d/c79 4 1 2024-03-22T07:18:49.046 INFO:tasks.workunit.client.0.smithi081.stdout:3/872: truncate db/d38/f51 4339516 0 2024-03-22T07:18:49.048 INFO:tasks.workunit.client.0.smithi081.stdout:3/873: mknod db/d38/da1/db7/c128 0 2024-03-22T07:18:49.049 INFO:tasks.workunit.client.0.smithi081.stdout:3/874: symlink db/ddf/ded/df7/l129 0 2024-03-22T07:18:49.049 INFO:tasks.workunit.client.0.smithi081.stdout:3/875: chown db/d38/da1/c127 10 1 2024-03-22T07:18:49.050 INFO:tasks.workunit.client.0.smithi081.stdout:3/876: symlink db/de/d4b/d6a/l12a 0 2024-03-22T07:18:49.055 INFO:tasks.workunit.client.0.smithi081.stdout:3/877: rename db/f2f to db/ddf/f12b 0 2024-03-22T07:18:49.063 INFO:tasks.workunit.client.0.smithi081.stdout:3/878: dread db/d38/da1/fc3 [0,4194304] 0 2024-03-22T07:18:49.063 INFO:tasks.workunit.client.0.smithi081.stdout:3/879: dread - db/d38/da1/db7/feb zero size 2024-03-22T07:18:49.064 INFO:tasks.workunit.client.0.smithi081.stdout:3/880: mknod db/d38/c12c 0 2024-03-22T07:18:49.066 INFO:tasks.workunit.client.0.smithi081.stdout:3/881: symlink db/de/d4b/d9a/l12d 0 2024-03-22T07:18:49.066 INFO:tasks.workunit.client.0.smithi081.stdout:3/882: read - db/ddf/ded/f107 zero size 2024-03-22T07:18:49.066 INFO:tasks.workunit.client.0.smithi081.stdout:3/883: fsync db/de/d4b/db0/fd5 0 2024-03-22T07:18:49.086 INFO:tasks.workunit.client.0.smithi081.stdout:3/884: dwrite db/de/db8/f102 [0,4194304] 0 2024-03-22T07:18:49.086 INFO:tasks.workunit.client.0.smithi081.stdout:3/885: read - db/d38/da1/f121 zero size 2024-03-22T07:18:49.089 INFO:tasks.workunit.client.0.smithi081.stdout:3/886: link db/de/l1c db/de/d4b/d6a/dfa/l12e 0 2024-03-22T07:18:49.091 INFO:tasks.workunit.client.0.smithi081.stdout:3/887: getdents db/de/d4b/d6a/d70/d3a/d5d 0 2024-03-22T07:18:49.091 INFO:tasks.workunit.client.0.smithi081.stdout:3/888: getdents db/d38/ddd 0 2024-03-22T07:18:49.091 INFO:tasks.workunit.client.0.smithi081.stdout:3/889: creat db/de/d4b/d6a/d70/f12f x:0 0 0 2024-03-22T07:18:49.091 INFO:tasks.workunit.client.0.smithi081.stdout:3/890: chown db/d38/d8d/db4 6 1 2024-03-22T07:18:49.108 INFO:tasks.workunit.client.0.smithi081.stdout:3/891: dwrite db/d38/da1/fc3 [4194304,4194304] 0 2024-03-22T07:18:49.110 INFO:tasks.workunit.client.0.smithi081.stdout:3/892: symlink db/de/d4b/d9a/l130 0 2024-03-22T07:18:49.113 INFO:tasks.workunit.client.0.smithi081.stdout:3/893: read db/d38/f47 [289794,72271] 0 2024-03-22T07:18:49.125 INFO:tasks.workunit.client.0.smithi081.stdout:2/929: sync 2024-03-22T07:18:49.126 INFO:tasks.workunit.client.0.smithi081.stdout:2/930: mknod da/db/d39/d47/d3b/d63/c12e 0 2024-03-22T07:18:49.126 INFO:tasks.workunit.client.0.smithi081.stdout:2/931: creat da/db/d39/d47/dda/df5/f12f x:0 0 0 2024-03-22T07:18:49.126 INFO:tasks.workunit.client.0.smithi081.stdout:2/932: read - da/db/d39/d46/d42/dcb/f124 zero size 2024-03-22T07:18:49.126 INFO:tasks.workunit.client.0.smithi081.stdout:2/933: fdatasync da/db/d39/d47/d3b/d64/d12d/f1b 0 2024-03-22T07:18:49.127 INFO:tasks.workunit.client.0.smithi081.stdout:2/934: write da/d56/f58 [857782,84205] 0 2024-03-22T07:18:49.129 INFO:tasks.workunit.client.0.smithi081.stdout:2/935: mknod da/db/d39/d47/d3b/d64/d12d/d21/d71/dc3/c130 0 2024-03-22T07:18:49.152 INFO:tasks.workunit.client.0.smithi081.stdout:2/936: dwrite da/db/d39/d47/d3b/d64/fff [0,4194304] 0 2024-03-22T07:18:49.335 INFO:tasks.workunit.client.0.smithi081.stdout:2/937: stat da/db/d39/d47/d3b/d63/ff3 0 2024-03-22T07:18:49.339 INFO:tasks.workunit.client.0.smithi081.stdout:2/938: rmdir da/db/d39/d47/d3b/d41/d50/d6a 39 2024-03-22T07:18:49.339 INFO:tasks.workunit.client.0.smithi081.stdout:2/939: getdents da/db/d39/d47/d3b/d64 0 2024-03-22T07:18:49.339 INFO:tasks.workunit.client.0.smithi081.stdout:2/940: fdatasync da/db/d39/d47/dda/f11a 0 2024-03-22T07:18:49.339 INFO:tasks.workunit.client.0.smithi081.stdout:2/941: rename da/db/d39/d47/d3b/dc5/d103/cc6 to da/db/d39/d46/d7e/df2/c131 0 2024-03-22T07:18:49.339 INFO:tasks.workunit.client.0.smithi081.stdout:2/942: dread da/fa9 [0,4194304] 0 2024-03-22T07:18:49.339 INFO:tasks.workunit.client.0.smithi081.stdout:2/943: link da/db/d39/d47/d3b/d64/d12d/d21/d71/dc3/fd4 da/db/d39/d46/d42/dcb/d11c/d117/f132 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:2/944: creat da/db/d39/d47/d3b/d41/d50/d6a/f133 x:0 0 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:2/945: truncate da/db/d39/d45/dcc/de8/f10d 5778318 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/894: sync 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/895: unlink db/de/d4b/fe8 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/896: dwrite db/de/f7d [0,4194304] 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/897: mknod db/de/db8/c131 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/898: dread db/d38/f80 [0,4194304] 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/899: dread - db/de/d4b/d6a/d70/d3a/f6f zero size 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/900: link db/d38/d8d/cac db/de/d4b/d6a/d70/d3a/c132 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/901: getdents db/de/d4b/d6a/dfa 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/902: creat db/d38/f133 x:0 0 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/903: rename db/fa6 to db/de/f134 0 2024-03-22T07:18:49.340 INFO:tasks.workunit.client.0.smithi081.stdout:3/904: readlink db/de/d4b/d6a/d70/l93 0 2024-03-22T07:18:49.341 INFO:tasks.workunit.client.0.smithi081.stdout:3/905: write db/de/f10b [1139267,98914] 0 2024-03-22T07:18:49.341 INFO:tasks.workunit.client.0.smithi081.stdout:3/906: dread - db/de/db8/ff9 zero size 2024-03-22T07:18:49.341 INFO:tasks.workunit.client.0.smithi081.stdout:3/907: chown db/de/d4b/d6a/d70/f32 130111601 1 2024-03-22T07:18:49.341 INFO:tasks.workunit.client.0.smithi081.stdout:3/908: getdents db/de/d4b/d6a/d70/d3a/d5d/d104 0 2024-03-22T07:18:49.341 INFO:tasks.workunit.client.0.smithi081.stdout:3/909: write db/d38/da1/f10a [720888,38953] 0 2024-03-22T07:18:49.341 INFO:tasks.workunit.client.0.smithi081.stdout:2/946: sync 2024-03-22T07:18:49.341 INFO:tasks.workunit.client.0.smithi081.stdout:2/947: dread - da/db/d39/d47/d3b/d63/f101 zero size 2024-03-22T07:18:49.342 INFO:tasks.workunit.client.0.smithi081.stdout:2/948: link da/db/d39/d47/l5d da/db/d39/d47/d3b/l134 0 2024-03-22T07:18:49.342 INFO:tasks.workunit.client.0.smithi081.stdout:2/949: dwrite da/db/d39/d46/d42/db4/f109 [0,4194304] 0 2024-03-22T07:18:49.342 INFO:tasks.workunit.client.0.smithi081.stdout:2/950: mkdir da/db/d39/d47/d3b/d63/d135 0 2024-03-22T07:18:49.342 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:18:49.343 INFO:tasks.workunit.client.0.smithi081.stdout:2/951: creat da/db/d39/d45/dcc/de8/f136 x:0 0 0 2024-03-22T07:18:49.343 INFO:tasks.workunit.client.0.smithi081.stdout:2/952: dwrite da/db/d39/d47/d3b/d63/f101 [0,4194304] 0 2024-03-22T07:18:49.343 INFO:tasks.workunit.client.0.smithi081.stdout:2/953: truncate da/db/d39/d47/dda/f108 158702 0 2024-03-22T07:18:49.343 INFO:tasks.workunit.client.0.smithi081.stdout:2/954: dwrite da/db/d39/d45/dcc/de8/f40 [4194304,4194304] 0 2024-03-22T07:18:49.344 INFO:tasks.workunit.client.0.smithi081.stdout:2/955: write da/db/d6d/f118 [106099,5216] 0 2024-03-22T07:18:49.344 INFO:tasks.workunit.client.0.smithi081.stdout:2/956: fdatasync da/f13 0 2024-03-22T07:18:49.344 INFO:tasks.workunit.client.0.smithi081.stdout:2/957: fdatasync da/db/f29 0 2024-03-22T07:18:49.454 INFO:tasks.workunit.client.0.smithi081.stdout:3/910: sync 2024-03-22T07:18:49.455 INFO:tasks.workunit.client.0.smithi081.stdout:3/911: dread db/de/db8/fe5 [0,4194304] 0 2024-03-22T07:18:49.456 INFO:tasks.workunit.client.0.smithi081.stdout:3/912: getdents db/de/d4b/db0 0 2024-03-22T07:18:49.457 INFO:tasks.workunit.client.0.smithi081.stdout:3/913: mkdir db/de/d4b/d6a/d70/d3a/d135 0 2024-03-22T07:18:49.459 INFO:tasks.workunit.client.0.smithi081.stdout:3/914: getdents db/de/d4b 0 2024-03-22T07:18:49.460 INFO:tasks.workunit.client.0.smithi081.stdout:3/915: mknod db/de/d4b/db0/c136 0 2024-03-22T07:18:49.460 INFO:tasks.workunit.client.0.smithi081.stdout:3/916: write db/de/db8/fe5 [359558,17757] 0 2024-03-22T07:18:49.461 INFO:tasks.workunit.client.0.smithi081.stdout:3/917: mknod db/ddf/ded/c137 0 2024-03-22T07:18:49.462 INFO:tasks.workunit.client.0.smithi081.stdout:3/918: creat db/de/d4b/db0/f138 x:0 0 0 2024-03-22T07:18:49.477 INFO:tasks.workunit.client.0.smithi081.stdout:3/919: dwrite db/ddf/f12b [0,4194304] 0 2024-03-22T07:18:49.477 INFO:tasks.workunit.client.0.smithi081.stdout:3/920: stat db/l3d 0 2024-03-22T07:18:49.477 INFO:tasks.workunit.client.0.smithi081.stdout:3/921: creat db/de/f139 x:0 0 0 2024-03-22T07:18:49.571 INFO:tasks.workunit.client.0.smithi081.stdout:2/958: sync 2024-03-22T07:18:49.572 INFO:tasks.workunit.client.0.smithi081.stdout:2/959: truncate da/db/d39/d47/f3d 596087 0 2024-03-22T07:18:49.586 INFO:tasks.workunit.client.0.smithi081.stdout:2/960: dwrite da/db/d39/d47/d3b/d63/f101 [0,4194304] 0 2024-03-22T07:18:49.588 INFO:tasks.workunit.client.0.smithi081.stdout:2/961: rename da/db/d39/d45/dcc/de8/d11b/d122 to da/db/d39/d47/d3b/d63/d137 0 2024-03-22T07:18:49.589 INFO:tasks.workunit.client.0.smithi081.stdout:2/962: unlink da/c12a 0 2024-03-22T07:18:49.590 INFO:tasks.workunit.client.0.smithi081.stdout:2/963: mkdir da/db/d39/d47/d3b/dc5/d138 0 2024-03-22T07:18:49.591 INFO:tasks.workunit.client.0.smithi081.stdout:2/964: rmdir da/db/d39/d46/d42/db4 39 2024-03-22T07:18:49.591 INFO:tasks.workunit.client.0.smithi081.stdout:2/965: symlink da/db/d39/d47/d3b/d64/d12d/d21/d71/dc3/d10c/l139 0 2024-03-22T07:18:49.609 INFO:tasks.workunit.client.0.smithi081.stdout:2/966: dwrite da/db/d39/d47/d3b/d41/d50/fcf [0,4194304] 0 2024-03-22T07:18:49.610 INFO:tasks.workunit.client.0.smithi081.stdout:2/967: symlink da/db/d39/d47/d3b/d41/d50/d9a/l13a 0 2024-03-22T07:18:49.639 INFO:tasks.workunit.client.0.smithi081.stdout:2/968: dwrite f5 [0,4194304] 0 2024-03-22T07:18:49.640 INFO:tasks.workunit.client.0.smithi081.stdout:2/969: getdents da/db/d39/d47/d3b/d64 0 2024-03-22T07:18:49.689 INFO:tasks.workunit.client.0.smithi081.stdout:3/922: sync 2024-03-22T07:18:49.690 INFO:tasks.workunit.client.0.smithi081.stdout:3/923: creat db/de/d4b/d6a/d70/d3a/f13a x:0 0 0 2024-03-22T07:18:49.707 INFO:tasks.workunit.client.0.smithi081.stdout:3/924: dwrite db/de/d4b/d6a/d70/d3a/d5d/fb5 [0,4194304] 0 2024-03-22T07:18:49.709 INFO:tasks.workunit.client.0.smithi081.stdout:3/925: mkdir db/ddf/d13b 0 2024-03-22T07:18:49.711 INFO:tasks.workunit.client.0.smithi081.stdout:3/926: link db/l30 db/de/db8/dcb/l13c 0 2024-03-22T07:18:49.712 INFO:tasks.workunit.client.0.smithi081.stdout:3/927: mknod db/de/d4b/d126/c13d 0 2024-03-22T07:18:49.713 INFO:tasks.workunit.client.0.smithi081.stdout:3/928: mkdir db/de/db8/dd2/d13e 0 2024-03-22T07:18:49.722 INFO:tasks.workunit.client.0.smithi081.stdout:3/929: write db/de/f4c [7661475,79827] 0 2024-03-22T07:18:49.723 INFO:tasks.workunit.client.0.smithi081.stdout:3/930: dread db/de/d4b/d6a/d70/fcd [0,4194304] 0 2024-03-22T07:18:49.724 INFO:tasks.workunit.client.0.smithi081.stdout:3/931: readlink db/de/d4b/d6a/d70/l3b 0 2024-03-22T07:18:49.730 INFO:tasks.workunit.client.0.smithi081.stdout:3/932: write db/de/f41 [3005260,101853] 0 2024-03-22T07:18:49.731 INFO:tasks.workunit.client.0.smithi081.stdout:3/933: truncate db/de/f24 9438049 0 2024-03-22T07:18:49.731 INFO:tasks.workunit.client.0.smithi081.stdout:3/934: chown db/fd 456116225 1 2024-03-22T07:18:49.733 INFO:tasks.workunit.client.0.smithi081.stdout:3/935: truncate db/de/d4b/d6a/d70/d3a/d5d/fbc 3575778 0 2024-03-22T07:18:49.734 INFO:tasks.workunit.client.0.smithi081.stdout:3/936: symlink db/ddf/d112/l13f 0 2024-03-22T07:18:49.735 INFO:tasks.workunit.client.0.smithi081.stdout:3/937: creat db/ddf/d13b/f140 x:0 0 0 2024-03-22T07:18:49.737 INFO:tasks.workunit.client.0.smithi081.stdout:3/938: mkdir db/de/d4b/d6a/d70/d141 0 2024-03-22T07:18:49.779 INFO:tasks.workunit.client.0.smithi081.stdout:2/970: sync 2024-03-22T07:18:49.795 INFO:tasks.workunit.client.0.smithi081.stdout:2/971: dwrite da/db/d39/d47/dda/f11a [0,4194304] 0 2024-03-22T07:18:49.812 INFO:tasks.workunit.client.0.smithi081.stdout:2/972: dwrite da/db/d39/d47/d3b/d64/d12d/f1f [0,4194304] 0 2024-03-22T07:18:49.813 INFO:tasks.workunit.client.0.smithi081.stdout:2/973: unlink da/db/d39/d46/ldd 0 2024-03-22T07:18:49.815 INFO:tasks.workunit.client.0.smithi081.stdout:2/974: creat da/db/d39/d47/d3b/f13b x:0 0 0 2024-03-22T07:18:49.823 INFO:tasks.workunit.client.0.smithi081.stdout:3/939: sync 2024-03-22T07:18:49.824 INFO:tasks.workunit.client.0.smithi081.stdout:3/940: mknod db/de/d4b/d6a/dfa/c142 0 2024-03-22T07:18:49.826 INFO:tasks.workunit.client.0.smithi081.stdout:3/941: mkdir db/de/d4b/d6a/d70/d3a/d5d/d104/d120/d143 0 2024-03-22T07:18:49.826 INFO:tasks.workunit.client.0.smithi081.stdout:3/942: read db/d38/f51 [2394324,13265] 0 2024-03-22T07:18:49.827 INFO:tasks.workunit.client.0.smithi081.stdout:3/943: creat db/de/db8/dd2/d13e/f144 x:0 0 0 2024-03-22T07:18:49.831 INFO:tasks.workunit.client.0.smithi081.stdout:2/975: dwrite da/db/d39/d47/fd5 [4194304,4194304] 0 2024-03-22T07:18:49.843 INFO:tasks.workunit.client.0.smithi081.stdout:3/944: dwrite db/d38/d8d/fc8 [0,4194304] 0 2024-03-22T07:18:49.844 INFO:tasks.workunit.client.0.smithi081.stdout:2/976: write da/db/d39/d47/f43 [210641,2340] 0 2024-03-22T07:18:49.867 INFO:tasks.workunit.client.0.smithi081.stdout:2/977: dwrite da/db/d39/d45/dcc/de8/f136 [0,4194304] 0 2024-03-22T07:18:49.867 INFO:tasks.workunit.client.0.smithi081.stdout:3/945: dwrite db/de/d4b/db0/f138 [0,4194304] 0 2024-03-22T07:18:49.867 INFO:tasks.workunit.client.0.smithi081.stdout:3/946: chown db/d38/f51 0 1 2024-03-22T07:18:49.867 INFO:tasks.workunit.client.0.smithi081.stdout:3/947: fdatasync db/de/db8/dd2/d13e/f144 0 2024-03-22T07:18:49.868 INFO:tasks.workunit.client.0.smithi081.stdout:3/948: dread db/de/d4b/d6a/d70/ff4 [0,4194304] 0 2024-03-22T07:18:49.870 INFO:tasks.workunit.client.0.smithi081.stdout:2/978: unlink da/db/f29 0 2024-03-22T07:18:49.871 INFO:tasks.workunit.client.0.smithi081.stdout:2/979: stat da/db/d39/d47/f43 0 2024-03-22T07:18:49.871 INFO:tasks.workunit.client.0.smithi081.stdout:2/980: chown da/db/d39/d47/d3b/d64/d12d/d21/d71/dc3 62163 1 2024-03-22T07:18:49.872 INFO:tasks.workunit.client.0.smithi081.stdout:3/949: link db/de/d4b/d6a/f7c db/d38/f145 0 2024-03-22T07:18:49.872 INFO:tasks.workunit.client.0.smithi081.stdout:3/950: chown db/de/d4b/d6a/d70/fc4 9209101 1 2024-03-22T07:18:49.877 INFO:tasks.workunit.client.0.smithi081.stdout:3/951: dread db/de/d4b/d6a/d70/f44 [0,4194304] 0 2024-03-22T07:18:49.878 INFO:tasks.workunit.client.0.smithi081.stdout:3/952: creat db/de/d4b/d6a/d70/d141/f146 x:0 0 0 2024-03-22T07:18:49.878 INFO:tasks.workunit.client.0.smithi081.stdout:3/953: creat db/d38/f147 x:0 0 0 2024-03-22T07:18:49.878 INFO:tasks.workunit.client.0.smithi081.stdout:3/954: write db/de/d4b/d6a/d70/d3a/d5d/f9f [173694,90367] 0 2024-03-22T07:18:49.878 INFO:tasks.workunit.client.0.smithi081.stdout:3/955: write db/d38/da1/fcf [20045,81623] 0 2024-03-22T07:18:49.878 INFO:tasks.workunit.client.0.smithi081.stdout:3/956: fdatasync db/de/f4c 0 2024-03-22T07:18:49.879 INFO:tasks.workunit.client.0.smithi081.stdout:3/957: rmdir db/de/d4b/db0 39 2024-03-22T07:18:49.880 INFO:tasks.workunit.client.0.smithi081.stdout:3/958: rmdir db/de/d4b/db0 39 2024-03-22T07:18:49.881 INFO:tasks.workunit.client.0.smithi081.stdout:3/959: rmdir db/ddf/ded/df7 39 2024-03-22T07:18:49.881 INFO:tasks.workunit.client.0.smithi081.stdout:3/960: write db/de/f10b [1388350,23222] 0 2024-03-22T07:18:49.881 INFO:tasks.workunit.client.0.smithi081.stdout:3/961: dread - db/de/d4b/d6a/f100 zero size 2024-03-22T07:18:49.887 INFO:tasks.workunit.client.0.smithi081.stdout:3/962: dread db/de/d4b/d6a/f7c [0,4194304] 0 2024-03-22T07:18:49.888 INFO:tasks.workunit.client.0.smithi081.stdout:3/963: stat db/de/d4b/d6a/d70/d3a/f6f 0 2024-03-22T07:18:49.888 INFO:tasks.workunit.client.0.smithi081.stdout:3/964: symlink db/de/d4b/d6a/d70/d3a/d5d/d104/d120/d143/l148 0 2024-03-22T07:18:49.889 INFO:tasks.workunit.client.0.smithi081.stdout:3/965: mknod db/de/db8/dd2/d13e/c149 0 2024-03-22T07:18:49.890 INFO:tasks.workunit.client.0.smithi081.stdout:3/966: link db/de/f41 db/de/d4b/d6a/d70/d3a/d135/f14a 0 2024-03-22T07:18:49.890 INFO:tasks.workunit.client.0.smithi081.stdout:2/981: dwrite da/db/d39/d46/d42/dcb/d11c/d117/f121 [0,4194304] 0 2024-03-22T07:18:49.891 INFO:tasks.workunit.client.0.smithi081.stdout:3/967: symlink db/ddf/ded/l14b 0 2024-03-22T07:18:49.894 INFO:tasks.workunit.client.0.smithi081.stdout:2/982: rename da/db/d39/d47/d3b/l134 to da/db/d39/d47/d3b/d41/dab/dac/l13c 0 2024-03-22T07:18:49.894 INFO:tasks.workunit.client.0.smithi081.stdout:2/983: chown da/db/d39/d47/d3b/d41/la3 560795245 1 2024-03-22T07:18:49.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/968: getdents db 0 2024-03-22T07:18:49.896 INFO:tasks.workunit.client.0.smithi081.stdout:3/969: write db/de/d4b/d6a/d70/d3a/d5d/d104/d120/f4d [438857,74578] 0 2024-03-22T07:18:49.898 INFO:tasks.workunit.client.0.smithi081.stdout:2/984: creat da/db/d39/d47/d3b/d64/d12d/d21/d71/dc3/d10c/f13d x:0 0 0 2024-03-22T07:18:49.900 INFO:tasks.workunit.client.0.smithi081.stdout:3/970: unlink db/de/d4b/d6a/d70/d3a/f7a 0 2024-03-22T07:18:49.901 INFO:tasks.workunit.client.0.smithi081.stdout:3/971: creat db/ddf/f14c x:0 0 0 2024-03-22T07:18:49.902 INFO:tasks.workunit.client.0.smithi081.stdout:2/985: rmdir da/db/d39/d47/d3b/d63/d135 0 2024-03-22T07:18:49.905 INFO:tasks.workunit.client.0.smithi081.stdout:3/972: symlink db/d38/d8d/l14d 0 2024-03-22T07:18:49.905 INFO:tasks.workunit.client.0.smithi081.stdout:3/973: read db/d38/f80 [800360,75989] 0 2024-03-22T07:18:49.905 INFO:tasks.workunit.client.0.smithi081.stdout:3/974: write db/d38/fde [219930,14517] 0 2024-03-22T07:18:49.906 INFO:tasks.workunit.client.0.smithi081.stdout:2/986: creat da/db/d39/d47/d3b/dc5/f13e x:0 0 0 2024-03-22T07:18:49.906 INFO:tasks.workunit.client.0.smithi081.stdout:2/987: fdatasync da/db/d39/d47/d3b/d64/d12d/d21/d71/dc3/fd4 0 2024-03-22T07:18:49.908 INFO:tasks.workunit.client.0.smithi081.stdout:3/975: link db/ddf/f113 db/de/d4b/d6a/d70/d3a/d135/f14e 0 2024-03-22T07:18:49.910 INFO:tasks.workunit.client.0.smithi081.stdout:2/988: rmdir da/db/d39/d45/dcc/de8 39 2024-03-22T07:18:49.910 INFO:tasks.workunit.client.0.smithi081.stdout:2/989: write da/db/d39/d46/d7e/fd7 [2566139,8832] 0 2024-03-22T07:18:49.927 INFO:tasks.workunit.client.0.smithi081.stdout:2/990: dwrite da/db/d39/d47/dda/df5/f12f [0,4194304] 0 2024-03-22T07:18:49.930 INFO:tasks.workunit.client.0.smithi081.stdout:2/991: rmdir da/db/d39/d47/d3b/d64/d12d/d128 0 2024-03-22T07:18:49.931 INFO:tasks.workunit.client.0.smithi081.stdout:2/992: symlink da/db/d39/d47/d3b/d64/l13f 0 2024-03-22T07:18:49.949 INFO:tasks.workunit.client.0.smithi081.stdout:2/993: dwrite da/db/d39/d47/d3b/d64/d12d/d21/f53 [0,4194304] 0 2024-03-22T07:18:49.950 INFO:tasks.workunit.client.0.smithi081.stdout:2/994: creat da/db/d39/d47/d3b/d41/dab/dac/f140 x:0 0 0 2024-03-22T07:18:49.950 INFO:tasks.workunit.client.0.smithi081.stdout:2/995: dread - da/db/d39/d47/d3b/d41/dab/dac/f116 zero size 2024-03-22T07:18:49.950 INFO:tasks.workunit.client.0.smithi081.stdout:2/996: fdatasync da/db/d39/d46/d42/fe2 0 2024-03-22T07:18:49.950 INFO:tasks.workunit.client.0.smithi081.stdout:2/997: fsync da/db/d39/d46/d42/fc7 0 2024-03-22T07:18:49.951 INFO:tasks.workunit.client.0.smithi081.stdout:2/998: rename da/db/d39/d47/dda/fdc to da/db/d39/d47/d3b/d64/d12d/f141 0 2024-03-22T07:18:49.963 INFO:tasks.workunit.client.0.smithi081.stdout:2/999: dwrite da/db/d39/d47/dda/f108 [0,4194304] 0 2024-03-22T07:18:50.052 INFO:tasks.workunit.client.0.smithi081.stdout:3/976: sync 2024-03-22T07:18:50.052 INFO:tasks.workunit.client.0.smithi081.stdout:3/977: write db/f54 [4073708,66878] 0 2024-03-22T07:18:50.060 INFO:tasks.workunit.client.0.smithi081.stdout:3/978: dread db/d38/f69 [0,4194304] 0 2024-03-22T07:18:50.061 INFO:tasks.workunit.client.0.smithi081.stdout:3/979: mknod db/de/db8/dd2/d13e/c14f 0 2024-03-22T07:18:50.062 INFO:tasks.workunit.client.0.smithi081.stdout:3/980: rmdir db/de/d4b/d6a/d70/d3a/d5d/d104 39 2024-03-22T07:18:50.243 INFO:tasks.workunit.client.0.smithi081.stdout:3/981: sync 2024-03-22T07:18:50.243 INFO:tasks.workunit.client.0.smithi081.stdout:3/982: read - db/de/d4b/d6a/d70/d3a/f13a zero size 2024-03-22T07:18:50.243 INFO:tasks.workunit.client.0.smithi081.stdout:3/983: chown db/de/db8 0 1 2024-03-22T07:18:50.243 INFO:tasks.workunit.client.0.smithi081.stdout:3/984: write db/de/d4b/d6a/d70/ff4 [1685063,54708] 0 2024-03-22T07:18:50.266 INFO:tasks.workunit.client.0.smithi081.stdout:3/985: dwrite db/de/d4b/d6a/d70/f40 [0,4194304] 0 2024-03-22T07:18:50.266 INFO:tasks.workunit.client.0.smithi081.stdout:3/986: chown db/d38/da1 23 1 2024-03-22T07:18:50.266 INFO:tasks.workunit.client.0.smithi081.stdout:3/987: truncate db/de/d4b/d6a/d70/d3a/f13a 101726 0 2024-03-22T07:18:50.266 INFO:tasks.workunit.client.0.smithi081.stdout:3/988: stat db/de/d4b/d6a/d70/d3a/f6f 0 2024-03-22T07:18:50.266 INFO:tasks.workunit.client.0.smithi081.stdout:3/989: mkdir db/de/d4b/d6a/d70/d3a/d5d/d104/d120/d143/d150 0 2024-03-22T07:18:50.267 INFO:tasks.workunit.client.0.smithi081.stdout:3/990: creat db/de/d4b/d6a/d70/d3a/d5d/d104/f151 x:0 0 0 2024-03-22T07:18:50.285 INFO:tasks.workunit.client.0.smithi081.stdout:3/991: dwrite db/d38/d8d/db4/fba [0,4194304] 0 2024-03-22T07:18:50.286 INFO:tasks.workunit.client.0.smithi081.stdout:3/992: write db/d38/da1/f10a [1016202,57211] 0 2024-03-22T07:18:50.286 INFO:tasks.workunit.client.0.smithi081.stdout:3/993: rename db to db/d38/da1/d152 22 2024-03-22T07:18:50.286 INFO:tasks.workunit.client.0.smithi081.stdout:3/994: chown db/ddf/ded/df7/l129 210132 1 2024-03-22T07:18:50.286 INFO:tasks.workunit.client.0.smithi081.stdout:3/995: dread db/de/d4b/d6a/d70/fcd [0,4194304] 0 2024-03-22T07:18:50.288 INFO:tasks.workunit.client.0.smithi081.stdout:3/996: mknod db/d38/d8d/c153 0 2024-03-22T07:18:50.289 INFO:tasks.workunit.client.0.smithi081.stdout:3/997: chown db/de/laa 1795327 1 2024-03-22T07:18:50.289 INFO:tasks.workunit.client.0.smithi081.stdout:3/998: creat db/de/d4b/d6a/d70/d3a/d135/f154 x:0 0 0 2024-03-22T07:18:50.303 INFO:tasks.workunit.client.0.smithi081.stdout:3/999: dwrite db/de/d4b/d6a/d70/f5b [0,4194304] 0 2024-03-22T07:18:50.307 INFO:tasks.workunit.client.0.smithi081.stderr:+ rm -rf -- ./tmp.TCwELreJ9l 2024-03-22T07:18:51.199 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:50 smithi081 ceph-mon[113965]: pgmap v221: 65 pgs: 65 active+clean; 2.2 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 158 MiB/s rd, 200 MiB/s wr, 287 op/s 2024-03-22T07:18:51.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:50 smithi149 ceph-mon[83447]: pgmap v221: 65 pgs: 65 active+clean; 2.2 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 158 MiB/s rd, 200 MiB/s wr, 287 op/s 2024-03-22T07:18:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.760+0000 7f102dbe4700 1 -- 172.21.15.81:0/425214739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 msgr2=0x7f10280fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.760+0000 7f102dbe4700 1 --2- 172.21.15.81:0/425214739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 0x7f10280fce30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f101c004940 tx=0x7f101c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.760+0000 7f102dbe4700 1 -- 172.21.15.81:0/425214739 shutdown_connections 2024-03-22T07:18:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.760+0000 7f102dbe4700 1 --2- 172.21.15.81:0/425214739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 0x7f10280fce30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.760+0000 7f102dbe4700 1 --2- 172.21.15.81:0/425214739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1028102e20 0x7f10281031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.760+0000 7f102dbe4700 1 -- 172.21.15.81:0/425214739 >> 172.21.15.81:0/425214739 conn(0x7f10280f8970 msgr2=0x7f10280fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.760+0000 7f102dbe4700 1 -- 172.21.15.81:0/425214739 shutdown_connections 2024-03-22T07:18:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.760+0000 7f102dbe4700 1 -- 172.21.15.81:0/425214739 wait complete. 2024-03-22T07:18:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.760+0000 7f102dbe4700 1 Processor -- start 2024-03-22T07:18:51.762 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f102dbe4700 1 -- start start 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f102dbe4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1028102e20 0x7f10280702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f102dbe4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 0x7f1028070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f102dbe4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028070d90 con 0x7f10281037c0 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f102dbe4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028070ed0 con 0x7f1028102e20 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f1026ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 0x7f1028070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f10277fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1028102e20 0x7f10280702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f1026ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 0x7f1028070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47360/0 (socket says 172.21.15.81:47360) 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f1026ffd700 1 -- 172.21.15.81:0/136846390 learned_addr learned my addr 172.21.15.81:0/136846390 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f1026ffd700 1 -- 172.21.15.81:0/136846390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1028102e20 msgr2=0x7f10280702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f1026ffd700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1028102e20 0x7f10280702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f1026ffd700 1 -- 172.21.15.81:0/136846390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c005040 con 0x7f10281037c0 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.761+0000 7f10277fe700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1028102e20 0x7f10280702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:18:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.762+0000 7f1026ffd700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 0x7f1028070800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f101c000c00 tx=0x7f101c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:51.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.762+0000 7f100ffff700 1 -- 172.21.15.81:0/136846390 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f101c021070 con 0x7f10281037c0 2024-03-22T07:18:51.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.762+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1028071120 con 0x7f10281037c0 2024-03-22T07:18:51.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.762+0000 7f100ffff700 1 -- 172.21.15.81:0/136846390 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f101c013c00 con 0x7f10281037c0 2024-03-22T07:18:51.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.762+0000 7f100ffff700 1 -- 172.21.15.81:0/136846390 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f101c00e070 con 0x7f10281037c0 2024-03-22T07:18:51.764 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.762+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102811a1c0 con 0x7f10281037c0 2024-03-22T07:18:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.763+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10281052f0 con 0x7f10281037c0 2024-03-22T07:18:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.763+0000 7f100ffff700 1 -- 172.21.15.81:0/136846390 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f101c01ca60 con 0x7f10281037c0 2024-03-22T07:18:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.764+0000 7f100ffff700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f101006d380 0x7f101006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.764+0000 7f100ffff700 1 -- 172.21.15.81:0/136846390 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f101c08dd60 con 0x7f10281037c0 2024-03-22T07:18:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.764+0000 7f10277fe700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f101006d380 0x7f101006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.764+0000 7f10277fe700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f101006d380 0x7f101006f830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1018004fd0 tx=0x7f101800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:51.769 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:51.769+0000 7f100ffff700 1 -- 172.21.15.81:0/136846390 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f101c057810 con 0x7f10281037c0 2024-03-22T07:18:52.023 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.022+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10280719b0 con 0x7f101006d380 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.023+0000 7f100ffff700 1 -- 172.21.15.81:0/136846390 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f10280719b0 con 0x7f101006d380 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stdout: "target_image": null, 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stdout: "in_progress": false, 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stdout: "which": "", 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stdout: "services_complete": [], 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stdout: "progress": null, 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stdout: "message": "", 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stdout: "is_paused": false 2024-03-22T07:18:52.024 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:18:52.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f101006d380 msgr2=0x7f101006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:52.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f101006d380 0x7f101006f830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1018004fd0 tx=0x7f101800b040 comp rx=0 tx=0).stop 2024-03-22T07:18:52.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 msgr2=0x7f1028070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:52.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 0x7f1028070800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f101c000c00 tx=0x7f101c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:52.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 shutdown_connections 2024-03-22T07:18:52.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f101006d380 0x7f101006f830 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:52.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1028102e20 0x7f10280702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:52.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 --2- 172.21.15.81:0/136846390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10281037c0 0x7f1028070800 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:52.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 >> 172.21.15.81:0/136846390 conn(0x7f10280f8970 msgr2=0x7f10281021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:52.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 shutdown_connections 2024-03-22T07:18:52.027 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:52.026+0000 7f102dbe4700 1 -- 172.21.15.81:0/136846390 wait complete. 2024-03-22T07:18:52.720 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph health detail' 2024-03-22T07:18:53.110 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:18:53.246 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:52 smithi081 ceph-mon[113965]: pgmap v222: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 130 MiB/s rd, 162 MiB/s wr, 298 op/s 2024-03-22T07:18:53.246 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:52 smithi081 ceph-mon[113965]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:52 smithi149 ceph-mon[83447]: pgmap v222: 65 pgs: 65 active+clean; 2.3 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 130 MiB/s rd, 162 MiB/s wr, 298 op/s 2024-03-22T07:18:53.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:52 smithi149 ceph-mon[83447]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:54.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.959+0000 7fd39240a700 1 -- 172.21.15.81:0/1154480500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd38c0fb720 msgr2=0x7fd38c0fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:54.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.959+0000 7fd39240a700 1 --2- 172.21.15.81:0/1154480500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd38c0fb720 0x7fd38c0fbaf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd37c004970 tx=0x7fd37c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:54.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.959+0000 7fd39240a700 1 -- 172.21.15.81:0/1154480500 shutdown_connections 2024-03-22T07:18:54.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.959+0000 7fd39240a700 1 --2- 172.21.15.81:0/1154480500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c0fc030 0x7fd38c10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:54.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.959+0000 7fd39240a700 1 --2- 172.21.15.81:0/1154480500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd38c0fb720 0x7fd38c0fbaf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:54.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.959+0000 7fd39240a700 1 -- 172.21.15.81:0/1154480500 >> 172.21.15.81:0/1154480500 conn(0x7fd38c0714c0 msgr2=0x7fd38c0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:54.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.959+0000 7fd39240a700 1 -- 172.21.15.81:0/1154480500 shutdown_connections 2024-03-22T07:18:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.959+0000 7fd39240a700 1 -- 172.21.15.81:0/1154480500 wait complete. 2024-03-22T07:18:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.960+0000 7fd39240a700 1 Processor -- start 2024-03-22T07:18:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.961+0000 7fd39240a700 1 -- start start 2024-03-22T07:18:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.961+0000 7fd39240a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c0fb720 0x7fd38c10d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.961+0000 7fd39240a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd38c0fc030 0x7fd38c108560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.961+0000 7fd39240a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c108bb0 con 0x7fd38c0fc030 2024-03-22T07:18:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.961+0000 7fd39240a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c108d20 con 0x7fd38c0fb720 2024-03-22T07:18:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.962+0000 7fd38b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd38c0fc030 0x7fd38c108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.962+0000 7fd38b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd38c0fc030 0x7fd38c108560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39428/0 (socket says 172.21.15.81:39428) 2024-03-22T07:18:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.962+0000 7fd38b7fe700 1 -- 172.21.15.81:0/2990062712 learned_addr learned my addr 172.21.15.81:0/2990062712 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:18:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.962+0000 7fd38bfff700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c0fb720 0x7fd38c10d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.962+0000 7fd38bfff700 1 -- 172.21.15.81:0/2990062712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd38c0fc030 msgr2=0x7fd38c108560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.962+0000 7fd38bfff700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd38c0fc030 0x7fd38c108560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:54.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.962+0000 7fd38bfff700 1 -- 172.21.15.81:0/2990062712 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c005040 con 0x7fd38c0fb720 2024-03-22T07:18:54.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.962+0000 7fd38bfff700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c0fb720 0x7fd38c10d3e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b370 tx=0x7fd37c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:54.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.963+0000 7fd388ff9700 1 -- 172.21.15.81:0/2990062712 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd37c00e070 con 0x7fd38c0fb720 2024-03-22T07:18:54.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.963+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c108fa0 con 0x7fd38c0fb720 2024-03-22T07:18:54.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.963+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c126d10 con 0x7fd38c0fb720 2024-03-22T07:18:54.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.963+0000 7fd388ff9700 1 -- 172.21.15.81:0/2990062712 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd37c007e10 con 0x7fd38c0fb720 2024-03-22T07:18:54.964 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.963+0000 7fd388ff9700 1 -- 172.21.15.81:0/2990062712 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd37c013e80 con 0x7fd38c0fb720 2024-03-22T07:18:54.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.965+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3780052b0 con 0x7fd38c0fb720 2024-03-22T07:18:54.965 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.965+0000 7fd388ff9700 1 -- 172.21.15.81:0/2990062712 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fd37c0136b0 con 0x7fd38c0fb720 2024-03-22T07:18:54.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.966+0000 7fd388ff9700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd37406d560 0x7fd37406fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:54.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.966+0000 7fd388ff9700 1 -- 172.21.15.81:0/2990062712 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd37c08d720 con 0x7fd38c0fb720 2024-03-22T07:18:54.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.966+0000 7fd38b7fe700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd37406d560 0x7fd37406fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:54.967 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.967+0000 7fd38b7fe700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd37406d560 0x7fd37406fa10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd38c109c90 tx=0x7fd38000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:54.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:54.972+0000 7fd388ff9700 1 -- 172.21.15.81:0/2990062712 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd37c0571b0 con 0x7fd38c0fb720 2024-03-22T07:18:55.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.285+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd378005560 con 0x7fd38c0fb720 2024-03-22T07:18:55.286 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:54 smithi081 ceph-mon[113965]: pgmap v223: 65 pgs: 65 active+clean; 2.0 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 110 MiB/s rd, 136 MiB/s wr, 245 op/s 2024-03-22T07:18:55.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.286+0000 7fd388ff9700 1 -- 172.21.15.81:0/2990062712 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd37c05a870 con 0x7fd38c0fb720 2024-03-22T07:18:55.287 INFO:teuthology.orchestra.run.smithi081.stdout:HEALTH_OK 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd37406d560 msgr2=0x7fd37406fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd37406d560 0x7fd37406fa10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd38c109c90 tx=0x7fd38000b040 comp rx=0 tx=0).stop 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c0fb720 msgr2=0x7fd38c10d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c0fb720 0x7fd38c10d3e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b370 tx=0x7fd37c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 shutdown_connections 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd37406d560 0x7fd37406fa10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd38c0fb720 0x7fd38c10d3e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 --2- 172.21.15.81:0/2990062712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd38c0fc030 0x7fd38c108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 >> 172.21.15.81:0/2990062712 conn(0x7fd38c0714c0 msgr2=0x7fd38c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 shutdown_connections 2024-03-22T07:18:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:55.289+0000 7fd39240a700 1 -- 172.21.15.81:0/2990062712 wait complete. 2024-03-22T07:18:55.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:54 smithi149 ceph-mon[83447]: pgmap v223: 65 pgs: 65 active+clean; 2.0 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 110 MiB/s rd, 136 MiB/s wr, 245 op/s 2024-03-22T07:18:56.178 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions' 2024-03-22T07:18:56.214 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:55 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2990062712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:18:56.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:55 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2990062712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:18:56.575 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:18:57.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:56 smithi081 ceph-mon[113965]: pgmap v224: 65 pgs: 65 active+clean; 2.0 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 69 MiB/s rd, 81 MiB/s wr, 166 op/s 2024-03-22T07:18:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:56 smithi149 ceph-mon[83447]: pgmap v224: 65 pgs: 65 active+clean; 2.0 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 69 MiB/s rd, 81 MiB/s wr, 166 op/s 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.454+0000 7f5cc221a700 1 -- 172.21.15.81:0/18986299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 msgr2=0x7f5cbc102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.454+0000 7f5cc221a700 1 --2- 172.21.15.81:0/18986299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 0x7f5cbc102140 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5cac004940 tx=0x7f5cac009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.454+0000 7f5cc221a700 1 -- 172.21.15.81:0/18986299 shutdown_connections 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.454+0000 7f5cc221a700 1 --2- 172.21.15.81:0/18986299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cbc0fc5b0 0x7f5cbc0fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.454+0000 7f5cc221a700 1 --2- 172.21.15.81:0/18986299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 0x7f5cbc102140 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.454+0000 7f5cc221a700 1 -- 172.21.15.81:0/18986299 >> 172.21.15.81:0/18986299 conn(0x7f5cbc0714c0 msgr2=0x7f5cbc0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.454+0000 7f5cc221a700 1 -- 172.21.15.81:0/18986299 shutdown_connections 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.454+0000 7f5cc221a700 1 -- 172.21.15.81:0/18986299 wait complete. 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cc221a700 1 Processor -- start 2024-03-22T07:18:58.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cc221a700 1 -- start start 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cc221a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cbc0fc5b0 0x7f5cbc113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cc221a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 0x7f5cbc1140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cc221a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc114740 con 0x7f5cbc101d70 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cc221a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc10db30 con 0x7f5cbc0fc5b0 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cbaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 0x7f5cbc1140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cbaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 0x7f5cbc1140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39460/0 (socket says 172.21.15.81:39460) 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cbaffd700 1 -- 172.21.15.81:0/1862986705 learned_addr learned my addr 172.21.15.81:0/1862986705 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cbb7fe700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cbc0fc5b0 0x7f5cbc113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cbaffd700 1 -- 172.21.15.81:0/1862986705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cbc0fc5b0 msgr2=0x7f5cbc113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.455+0000 7f5cbaffd700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cbc0fc5b0 0x7f5cbc113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.456+0000 7f5cbaffd700 1 -- 172.21.15.81:0/1862986705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac005040 con 0x7f5cbc101d70 2024-03-22T07:18:58.457 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.456+0000 7f5cbaffd700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 0x7f5cbc1140f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0002a50 tx=0x7f5cb00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.456+0000 7f5ca3fff700 1 -- 172.21.15.81:0/1862986705 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb0007cc0 con 0x7f5cbc101d70 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.456+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc10de10 con 0x7f5cbc101d70 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.456+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc10e3d0 con 0x7f5cbc101d70 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.456+0000 7f5ca3fff700 1 -- 172.21.15.81:0/1862986705 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cb000f040 con 0x7f5cbc101d70 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.456+0000 7f5ca3fff700 1 -- 172.21.15.81:0/1862986705 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb000da30 con 0x7f5cbc101d70 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.457+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc0fd5c0 con 0x7f5cbc101d70 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.458+0000 7f5ca3fff700 1 -- 172.21.15.81:0/1862986705 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f5cb000dc70 con 0x7f5cbc101d70 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.458+0000 7f5ca3fff700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5ca406d380 0x7f5ca406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.458+0000 7f5ca3fff700 1 -- 172.21.15.81:0/1862986705 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5cb008c630 con 0x7f5cbc101d70 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.458+0000 7f5cbb7fe700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5ca406d380 0x7f5ca406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:58.459 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.459+0000 7f5cbb7fe700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5ca406d380 0x7f5ca406f830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5cac004890 tx=0x7f5cac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:58.467 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.467+0000 7f5ca3fff700 1 -- 172.21.15.81:0/1862986705 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5cb0059c10 con 0x7f5cbc101d70 2024-03-22T07:18:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.870+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5cbc10f230 con 0x7f5cbc101d70 2024-03-22T07:18:58.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.871+0000 7f5ca3fff700 1 -- 172.21.15.81:0/1862986705 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5cb00597a0 con 0x7f5cbc101d70 2024-03-22T07:18:58.871 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:18:58.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:18:58.871 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 14 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:18:58.872 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:18:58.873 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5ca406d380 msgr2=0x7f5ca406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5ca406d380 0x7f5ca406f830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5cac004890 tx=0x7f5cac009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 msgr2=0x7f5cbc1140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 0x7f5cbc1140f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0002a50 tx=0x7f5cb00046b0 comp rx=0 tx=0).stop 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 shutdown_connections 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5ca406d380 0x7f5ca406f830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5cbc0fc5b0 0x7f5cbc113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 --2- 172.21.15.81:0/1862986705 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cbc101d70 0x7f5cbc1140f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 >> 172.21.15.81:0/1862986705 conn(0x7f5cbc0714c0 msgr2=0x7f5cbc0fb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 shutdown_connections 2024-03-22T07:18:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:18:58.873+0000 7f5cc221a700 1 -- 172.21.15.81:0/1862986705 wait complete. 2024-03-22T07:18:59.200 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:58 smithi081 ceph-mon[113965]: pgmap v225: 65 pgs: 65 active+clean; 1.7 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 69 MiB/s rd, 82 MiB/s wr, 188 op/s 2024-03-22T07:18:59.200 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:58 smithi081 ceph-mon[113965]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi081.zfgxzy=up:active} 2 up:standby 2024-03-22T07:18:59.200 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:58 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1862986705' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:58 smithi149 ceph-mon[83447]: pgmap v225: 65 pgs: 65 active+clean; 1.7 GiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 69 MiB/s rd, 82 MiB/s wr, 188 op/s 2024-03-22T07:18:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:58 smithi149 ceph-mon[83447]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi081.zfgxzy=up:active} 2 up:standby 2024-03-22T07:18:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:58 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1862986705' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:59.660 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-22T07:18:59.660 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-22T07:18:59.669 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-22T07:19:00.051 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:19:00.227 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:18:59 smithi081 ceph-mon[113965]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi081.zfgxzy=up:active} 2 up:standby 2024-03-22T07:19:00.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:18:59 smithi149 ceph-mon[83447]: fsmap cephfs:2 {0=cephfs.smithi081.bnzfrv=up:active,1=cephfs.smithi081.zfgxzy=up:active} 2 up:standby 2024-03-22T07:19:01.232 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:00 smithi081 ceph-mon[113965]: pgmap v226: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 47 MiB/s rd, 56 MiB/s wr, 197 op/s 2024-03-22T07:19:01.232 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:00 smithi149 ceph-mon[83447]: pgmap v226: 65 pgs: 65 active+clean; 1.0 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 47 MiB/s rd, 56 MiB/s wr, 197 op/s 2024-03-22T07:19:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:02.042 INFO:teuthology.orchestra.run.smithi081.stdout:wait for servicemap items w/ changing names to refresh 2024-03-22T07:19:02.725 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'sleep 60' 2024-03-22T07:19:03.065 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:19:03.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:02 smithi081 ceph-mon[113965]: pgmap v227: 65 pgs: 65 active+clean; 648 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 20 MiB/s wr, 200 op/s 2024-03-22T07:19:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:02 smithi149 ceph-mon[83447]: pgmap v227: 65 pgs: 65 active+clean; 648 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 20 MiB/s wr, 200 op/s 2024-03-22T07:19:05.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:04 smithi081 ceph-mon[113965]: pgmap v228: 65 pgs: 65 active+clean; 307 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 4.5 MiB/s rd, 9.6 MiB/s wr, 205 op/s 2024-03-22T07:19:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:04 smithi149 ceph-mon[83447]: pgmap v228: 65 pgs: 65 active+clean; 307 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 4.5 MiB/s rd, 9.6 MiB/s wr, 205 op/s 2024-03-22T07:19:07.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:06 smithi081 ceph-mon[113965]: pgmap v229: 65 pgs: 65 active+clean; 307 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 888 KiB/s wr, 166 op/s 2024-03-22T07:19:07.353 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:06 smithi149 ceph-mon[83447]: pgmap v229: 65 pgs: 65 active+clean; 307 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 888 KiB/s wr, 166 op/s 2024-03-22T07:19:09.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:08 smithi081 ceph-mon[113965]: pgmap v230: 65 pgs: 65 active+clean; 312 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-03-22T07:19:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:08 smithi149 ceph-mon[83447]: pgmap v230: 65 pgs: 65 active+clean; 312 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 1.4 MiB/s wr, 230 op/s 2024-03-22T07:19:11.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:10 smithi081 ceph-mon[113965]: pgmap v231: 65 pgs: 65 active+clean; 311 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-03-22T07:19:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:10 smithi149 ceph-mon[83447]: pgmap v231: 65 pgs: 65 active+clean; 311 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.4 MiB/s wr, 278 op/s 2024-03-22T07:19:13.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:12 smithi081 ceph-mon[113965]: pgmap v232: 65 pgs: 65 active+clean; 311 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 281 op/s 2024-03-22T07:19:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:12 smithi149 ceph-mon[83447]: pgmap v232: 65 pgs: 65 active+clean; 311 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 281 op/s 2024-03-22T07:19:15.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:14 smithi081 ceph-mon[113965]: pgmap v233: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.4 MiB/s wr, 354 op/s 2024-03-22T07:19:15.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:14 smithi149 ceph-mon[83447]: pgmap v233: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.4 MiB/s wr, 354 op/s 2024-03-22T07:19:16.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:15 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:16.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:15 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:17.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:17 smithi081 ceph-mon[113965]: pgmap v234: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 299 op/s 2024-03-22T07:19:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:17 smithi149 ceph-mon[83447]: pgmap v234: 65 pgs: 65 active+clean; 313 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 299 op/s 2024-03-22T07:19:18.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:18 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:18.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:18 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:19.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:19 smithi081 ceph-mon[113965]: pgmap v235: 65 pgs: 65 active+clean; 305 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 352 op/s 2024-03-22T07:19:19.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:19 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:19.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:19 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:19.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:19 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:19:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:19 smithi149 ceph-mon[83447]: pgmap v235: 65 pgs: 65 active+clean; 305 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 352 op/s 2024-03-22T07:19:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:19 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:19 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:19 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:19:21.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:21 smithi081 ceph-mon[113965]: pgmap v236: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 382 op/s 2024-03-22T07:19:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:21 smithi149 ceph-mon[83447]: pgmap v236: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 382 op/s 2024-03-22T07:19:23.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:23 smithi081 ceph-mon[113965]: pgmap v237: 65 pgs: 65 active+clean; 305 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-03-22T07:19:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:23 smithi149 ceph-mon[83447]: pgmap v237: 65 pgs: 65 active+clean; 305 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 1.7 MiB/s wr, 378 op/s 2024-03-22T07:19:25.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:25 smithi081 ceph-mon[113965]: pgmap v238: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 1.9 MiB/s wr, 414 op/s 2024-03-22T07:19:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:25 smithi149 ceph-mon[83447]: pgmap v238: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 47 KiB/s rd, 1.9 MiB/s wr, 414 op/s 2024-03-22T07:19:27.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:27 smithi081 ceph-mon[113965]: pgmap v239: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.2 MiB/s wr, 301 op/s 2024-03-22T07:19:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:27 smithi149 ceph-mon[83447]: pgmap v239: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.2 MiB/s wr, 301 op/s 2024-03-22T07:19:29.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:29 smithi081 ceph-mon[113965]: pgmap v240: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-03-22T07:19:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:29 smithi149 ceph-mon[83447]: pgmap v240: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-03-22T07:19:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:31 smithi081 ceph-mon[113965]: pgmap v241: 65 pgs: 65 active+clean; 303 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 380 op/s 2024-03-22T07:19:31.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:31 smithi149 ceph-mon[83447]: pgmap v241: 65 pgs: 65 active+clean; 303 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 380 op/s 2024-03-22T07:19:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:33.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:33 smithi081 ceph-mon[113965]: pgmap v242: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 353 op/s 2024-03-22T07:19:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:33 smithi149 ceph-mon[83447]: pgmap v242: 65 pgs: 65 active+clean; 306 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 353 op/s 2024-03-22T07:19:35.260 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-22T07:19:35.261 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-22T07:19:35.285 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:35 smithi081 ceph-mon[113965]: pgmap v243: 65 pgs: 65 active+clean; 303 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.4 MiB/s wr, 372 op/s 2024-03-22T07:19:35.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:35 smithi149 ceph-mon[83447]: pgmap v243: 65 pgs: 65 active+clean; 303 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 1.4 MiB/s wr, 372 op/s 2024-03-22T07:19:35.597 DEBUG:teuthology.parallel:result is None 2024-03-22T07:19:35.598 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T07:19:35.692 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T07:19:35.692 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-22T07:19:35.723 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-22T07:19:35.724 DEBUG:teuthology.parallel:result is None 2024-03-22T07:19:37.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:37 smithi081 ceph-mon[113965]: pgmap v244: 65 pgs: 65 active+clean; 303 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-03-22T07:19:37.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:37 smithi149 ceph-mon[83447]: pgmap v244: 65 pgs: 65 active+clean; 303 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.2 MiB/s wr, 284 op/s 2024-03-22T07:19:39.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:39 smithi081 ceph-mon[113965]: pgmap v245: 65 pgs: 65 active+clean; 302 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.3 MiB/s wr, 330 op/s 2024-03-22T07:19:39.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:39 smithi149 ceph-mon[83447]: pgmap v245: 65 pgs: 65 active+clean; 302 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.3 MiB/s wr, 330 op/s 2024-03-22T07:19:41.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:41 smithi081 ceph-mon[113965]: pgmap v246: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.2 MiB/s wr, 324 op/s 2024-03-22T07:19:41.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:41 smithi149 ceph-mon[83447]: pgmap v246: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.2 MiB/s wr, 324 op/s 2024-03-22T07:19:43.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:43 smithi081 ceph-mon[113965]: pgmap v247: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 244 op/s 2024-03-22T07:19:43.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:43 smithi149 ceph-mon[83447]: pgmap v247: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 1.0 MiB/s wr, 244 op/s 2024-03-22T07:19:45.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:45 smithi081 ceph-mon[113965]: pgmap v248: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 545 KiB/s wr, 177 op/s 2024-03-22T07:19:45.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:45 smithi149 ceph-mon[83447]: pgmap v248: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 545 KiB/s wr, 177 op/s 2024-03-22T07:19:46.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:46 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:46.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:46 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:47.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:47 smithi081 ceph-mon[113965]: pgmap v249: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 216 KiB/s wr, 91 op/s 2024-03-22T07:19:47.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:47 smithi149 ceph-mon[83447]: pgmap v249: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 216 KiB/s wr, 91 op/s 2024-03-22T07:19:49.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:49 smithi081 ceph-mon[113965]: pgmap v250: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 216 KiB/s wr, 92 op/s 2024-03-22T07:19:49.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:49 smithi149 ceph-mon[83447]: pgmap v250: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 216 KiB/s wr, 92 op/s 2024-03-22T07:19:50.569 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:50 smithi081 ceph-mon[113965]: pgmap v251: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 44 KiB/s wr, 46 op/s 2024-03-22T07:19:50.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:50 smithi149 ceph-mon[83447]: pgmap v251: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s rd, 44 KiB/s wr, 46 op/s 2024-03-22T07:19:53.305 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:52 smithi149 ceph-mon[83447]: pgmap v252: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 5.4 KiB/s wr, 3 op/s 2024-03-22T07:19:53.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:52 smithi081 ceph-mon[113965]: pgmap v252: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 170 B/s rd, 5.4 KiB/s wr, 3 op/s 2024-03-22T07:19:55.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:54 smithi081 ceph-mon[113965]: pgmap v253: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T07:19:55.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:54 smithi149 ceph-mon[83447]: pgmap v253: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T07:19:57.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:56 smithi081 ceph-mon[113965]: pgmap v254: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T07:19:57.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:56 smithi149 ceph-mon[83447]: pgmap v254: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T07:19:59.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:19:58 smithi081 ceph-mon[113965]: pgmap v255: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T07:19:59.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:19:58 smithi149 ceph-mon[83447]: pgmap v255: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-03-22T07:20:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:00 smithi081 ceph-mon[113965]: pgmap v256: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:00 smithi081 ceph-mon[113965]: overall HEALTH_OK 2024-03-22T07:20:01.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:00 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:00 smithi149 ceph-mon[83447]: pgmap v256: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:00 smithi149 ceph-mon[83447]: overall HEALTH_OK 2024-03-22T07:20:01.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:00 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:03.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:02 smithi081 ceph-mon[113965]: pgmap v257: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:03.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:02 smithi149 ceph-mon[83447]: pgmap v257: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:05.229 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:04 smithi081 ceph-mon[113965]: pgmap v258: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:05.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:04 smithi149 ceph-mon[83447]: pgmap v258: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:05.643 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch ps' 2024-03-22T07:20:05.916 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:20:07.282 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:06 smithi081 ceph-mon[113965]: pgmap v259: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:07.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:06 smithi149 ceph-mon[83447]: pgmap v259: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:07.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.504+0000 7f41d109d700 1 -- 172.21.15.81:0/389404512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 msgr2=0x7f41cc103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:07.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.504+0000 7f41d109d700 1 --2- 172.21.15.81:0/389404512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 0x7f41cc103320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f41b4004940 tx=0x7f41b4009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:07.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.504+0000 7f41d109d700 1 -- 172.21.15.81:0/389404512 shutdown_connections 2024-03-22T07:20:07.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.504+0000 7f41d109d700 1 --2- 172.21.15.81:0/389404512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41cc0fb4d0 0x7f41cc0fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.504+0000 7f41d109d700 1 --2- 172.21.15.81:0/389404512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 0x7f41cc103320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.504+0000 7f41d109d700 1 -- 172.21.15.81:0/389404512 >> 172.21.15.81:0/389404512 conn(0x7f41cc0714c0 msgr2=0x7f41cc0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.504+0000 7f41d109d700 1 -- 172.21.15.81:0/389404512 shutdown_connections 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.504+0000 7f41d109d700 1 -- 172.21.15.81:0/389404512 wait complete. 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41d109d700 1 Processor -- start 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41d109d700 1 -- start start 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41d109d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41cc0fb4d0 0x7f41cc113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41d109d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 0x7f41cc113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41d109d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc1145b0 con 0x7f41cc102f50 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41d109d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc10db90 con 0x7f41cc0fb4d0 2024-03-22T07:20:07.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41ca59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 0x7f41cc113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41ca59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 0x7f41cc113f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57806/0 (socket says 172.21.15.81:57806) 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41ca59c700 1 -- 172.21.15.81:0/2095800323 learned_addr learned my addr 172.21.15.81:0/2095800323 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.505+0000 7f41cad9d700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41cc0fb4d0 0x7f41cc113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41ca59c700 1 -- 172.21.15.81:0/2095800323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41cc0fb4d0 msgr2=0x7f41cc113a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41ca59c700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41cc0fb4d0 0x7f41cc113a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41ca59c700 1 -- 172.21.15.81:0/2095800323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4005040 con 0x7f41cc102f50 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41cad9d700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41cc0fb4d0 0x7f41cc113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41ca59c700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 0x7f41cc113f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00c830 tx=0x7f41bc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41c37fe700 1 -- 172.21.15.81:0/2095800323 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41bc00a760 con 0x7f41cc102f50 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc10de70 con 0x7f41cc102f50 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41c37fe700 1 -- 172.21.15.81:0/2095800323 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41bc00ad90 con 0x7f41cc102f50 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41c37fe700 1 -- 172.21.15.81:0/2095800323 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41bc0113a0 con 0x7f41cc102f50 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.506+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc10e3b0 con 0x7f41cc102f50 2024-03-22T07:20:07.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.507+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc100200 con 0x7f41cc102f50 2024-03-22T07:20:07.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.507+0000 7f41c37fe700 1 -- 172.21.15.81:0/2095800323 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f41bc012400 con 0x7f41cc102f50 2024-03-22T07:20:07.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.508+0000 7f41c37fe700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f41b806d380 0x7f41b806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:07.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.508+0000 7f41c37fe700 1 -- 172.21.15.81:0/2095800323 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f41bc08c9b0 con 0x7f41cc102f50 2024-03-22T07:20:07.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.508+0000 7f41cad9d700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f41b806d380 0x7f41b806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:07.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.508+0000 7f41cad9d700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f41b806d380 0x7f41b806f830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f41b4004f90 tx=0x7f41b4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:07.513 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.512+0000 7f41c37fe700 1 -- 172.21.15.81:0/2095800323 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f41bc059f90 con 0x7f41cc102f50 2024-03-22T07:20:07.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.715+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 --> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f41cc10f070 con 0x7f41b806d380 2024-03-22T07:20:07.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.725+0000 7f41c37fe700 1 -- 172.21.15.81:0/2095800323 <== mgr.34104 v2:172.21.15.81:6800/2090479429 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f41cc10f070 con 0x7f41b806d380 2024-03-22T07:20:07.726 INFO:teuthology.orchestra.run.smithi081.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:20:07.726 INFO:teuthology.orchestra.run.smithi081.stdout:alertmanager.smithi081 smithi081 *:9093,9094 running (8m) 2m ago 19m 24.4M - 0.25.0 c8568f914cd2 9f22b3f2e7a1 2024-03-22T07:20:07.726 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi081 smithi081 running (6m) 2m ago 19m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 2d986dcafa45 2024-03-22T07:20:07.726 INFO:teuthology.orchestra.run.smithi081.stdout:crash.smithi149 smithi149 running (6m) 116s ago 18m 6660k - 19.0.0-1667-gdb0330b1 4f325f8440e9 68e683f699e2 2024-03-22T07:20:07.726 INFO:teuthology.orchestra.run.smithi081.stdout:grafana.smithi081 smithi081 *:3000 running (8m) 2m ago 18m 87.4M - 9.4.12 a0ea5e3c46d9 0ccd99509365 2024-03-22T07:20:07.726 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.bnzfrv smithi081 running (2m) 2m ago 13m 12.3M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5405937045b2 2024-03-22T07:20:07.726 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi081.zfgxzy smithi081 running (2m) 2m ago 13m 13.7M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5080d1e80a87 2024-03-22T07:20:07.726 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.ffewvr smithi149 running (119s) 116s ago 13m 14.6M - 19.0.0-1667-gdb0330b1 4f325f8440e9 6e2fdd449b68 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:mds.cephfs.smithi149.kdtxfw smithi149 running (2m) 116s ago 13m 14.9M - 19.0.0-1667-gdb0330b1 4f325f8440e9 90fe0ec77272 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi081.bcxarw smithi081 *:8443,9283,8765 running (10m) 2m ago 20m 510M - 19.0.0-1667-gdb0330b1 4f325f8440e9 94ba6ba4cbcf 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:mgr.smithi149.sgmtoa smithi149 *:8443,9283,8765 running (9m) 116s ago 18m 451M - 19.0.0-1667-gdb0330b1 4f325f8440e9 3e507f386a7c 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi081 smithi081 running (7m) 2m ago 20m 49.2M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 8f07a753bc94 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:mon.smithi149 smithi149 running (7m) 116s ago 18m 40.3M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 2c438525329c 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi081 smithi081 *:9100 running (9m) 2m ago 19m 17.1M - 1.5.0 0da6a335fe13 1325e0d1c0e8 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:node-exporter.smithi149 smithi149 *:9100 running (9m) 116s ago 18m 16.5M - 1.5.0 0da6a335fe13 e851a8071223 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 smithi081 running (5m) 2m ago 17m 98.0M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 033c080308b6 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 smithi081 running (5m) 2m ago 16m 108M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 8341fe32f3fa 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 smithi081 running (4m) 2m ago 16m 84.7M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 0b8c514dced1 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 smithi149 running (4m) 116s ago 16m 107M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 c8234f7ed003 2024-03-22T07:20:07.727 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 smithi149 running (3m) 116s ago 15m 103M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 437455ba97c0 2024-03-22T07:20:07.728 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 smithi149 running (3m) 116s ago 15m 89.6M 4096M 19.0.0-1667-gdb0330b1 4f325f8440e9 0d8d31a867d9 2024-03-22T07:20:07.728 INFO:teuthology.orchestra.run.smithi081.stdout:prometheus.smithi081 smithi081 *:9095 running (8m) 2m ago 18m 63.4M - 2.43.0 a07b618ecd1d fa68391c8279 2024-03-22T07:20:07.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.727+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f41b806d380 msgr2=0x7f41b806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:07.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.727+0000 7f41d109d700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f41b806d380 0x7f41b806f830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f41b4004f90 tx=0x7f41b4009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:07.730 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.727+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 msgr2=0x7f41cc113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:07.730 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.727+0000 7f41d109d700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 0x7f41cc113f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00c830 tx=0x7f41bc004470 comp rx=0 tx=0).stop 2024-03-22T07:20:07.730 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.728+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 shutdown_connections 2024-03-22T07:20:07.730 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.728+0000 7f41d109d700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f41b806d380 0x7f41b806f830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.730 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.728+0000 7f41d109d700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41cc0fb4d0 0x7f41cc113a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.730 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.728+0000 7f41d109d700 1 --2- 172.21.15.81:0/2095800323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41cc102f50 0x7f41cc113f60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.730 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.728+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 >> 172.21.15.81:0/2095800323 conn(0x7f41cc0714c0 msgr2=0x7f41cc0fa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:07.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.728+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 shutdown_connections 2024-03-22T07:20:07.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:07.728+0000 7f41d109d700 1 -- 172.21.15.81:0/2095800323 wait complete. 2024-03-22T07:20:08.430 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions' 2024-03-22T07:20:08.693 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:20:09.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:08 smithi081 ceph-mon[113965]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:09.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:08 smithi081 ceph-mon[113965]: pgmap v260: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:08 smithi149 ceph-mon[83447]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:09.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:08 smithi149 ceph-mon[83447]: pgmap v260: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.500+0000 7f5fdfc70700 1 -- 172.21.15.81:0/105207347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 msgr2=0x7f5fd8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.500+0000 7f5fdfc70700 1 --2- 172.21.15.81:0/105207347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 0x7f5fd8106f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8004940 tx=0x7f5fc8009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.500+0000 7f5fdfc70700 1 -- 172.21.15.81:0/105207347 shutdown_connections 2024-03-22T07:20:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.500+0000 7f5fdfc70700 1 --2- 172.21.15.81:0/105207347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 0x7f5fd8106f00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.500+0000 7f5fdfc70700 1 --2- 172.21.15.81:0/105207347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fd80fde90 0x7f5fd80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.500+0000 7f5fdfc70700 1 -- 172.21.15.81:0/105207347 >> 172.21.15.81:0/105207347 conn(0x7f5fd8072bc0 msgr2=0x7f5fd8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.500+0000 7f5fdfc70700 1 -- 172.21.15.81:0/105207347 shutdown_connections 2024-03-22T07:20:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.500+0000 7f5fdfc70700 1 -- 172.21.15.81:0/105207347 wait complete. 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.501+0000 7f5fdfc70700 1 Processor -- start 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.501+0000 7f5fdfc70700 1 -- start start 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.501+0000 7f5fdfc70700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fd80fde90 0x7f5fd8113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.501+0000 7f5fdfc70700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 0x7f5fd81140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.501+0000 7f5fdfc70700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8114730 con 0x7f5fd80fe830 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.501+0000 7f5fdfc70700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810db20 con 0x7f5fd80fde90 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.501+0000 7f5fdd20b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 0x7f5fd81140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.501+0000 7f5fdd20b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 0x7f5fd81140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57822/0 (socket says 172.21.15.81:57822) 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.501+0000 7f5fdd20b700 1 -- 172.21.15.81:0/1131233761 learned_addr learned my addr 172.21.15.81:0/1131233761 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fdda0c700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fd80fde90 0x7f5fd8113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fdd20b700 1 -- 172.21.15.81:0/1131233761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fd80fde90 msgr2=0x7f5fd8113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fdd20b700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fd80fde90 0x7f5fd8113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fdd20b700 1 -- 172.21.15.81:0/1131233761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc8005040 con 0x7f5fd80fe830 2024-03-22T07:20:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fdd20b700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 0x7f5fd81140e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800b3b0 tx=0x7f5fc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fce7fc700 1 -- 172.21.15.81:0/1131233761 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fc800e040 con 0x7f5fd80fe830 2024-03-22T07:20:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fce7fc700 1 -- 172.21.15.81:0/1131233761 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fc8013420 con 0x7f5fd80fe830 2024-03-22T07:20:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd810dda0 con 0x7f5fd80fe830 2024-03-22T07:20:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fce7fc700 1 -- 172.21.15.81:0/1131233761 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fc801c610 con 0x7f5fd80fe830 2024-03-22T07:20:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.502+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd810e2e0 con 0x7f5fd80fe830 2024-03-22T07:20:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.503+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd8065e30 con 0x7f5fd80fe830 2024-03-22T07:20:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.503+0000 7f5fce7fc700 1 -- 172.21.15.81:0/1131233761 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f5fc8013590 con 0x7f5fd80fe830 2024-03-22T07:20:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.504+0000 7f5fce7fc700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5fc406d380 0x7f5fc406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.504+0000 7f5fce7fc700 1 -- 172.21.15.81:0/1131233761 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5fc808db20 con 0x7f5fd80fe830 2024-03-22T07:20:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.504+0000 7f5fdda0c700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5fc406d380 0x7f5fc406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.504+0000 7f5fdda0c700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5fc406d380 0x7f5fc406f830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0004d60 tx=0x7f5fd000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:10.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.509+0000 7f5fce7fc700 1 -- 172.21.15.81:0/1131233761 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5fc805b160 con 0x7f5fd80fe830 2024-03-22T07:20:10.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.781+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5fd8109f10 con 0x7f5fd80fe830 2024-03-22T07:20:10.782 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.782+0000 7f5fce7fc700 1 -- 172.21.15.81:0/1131233761 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5fc8013840 con 0x7f5fd80fe830 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": { 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": { 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": { 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-03-22T07:20:10.783 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:20:10.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-03-22T07:20:10.784 INFO:teuthology.orchestra.run.smithi081.stdout: "overall": { 2024-03-22T07:20:10.784 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 14 2024-03-22T07:20:10.784 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-03-22T07:20:10.784 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5fc406d380 msgr2=0x7f5fc406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5fc406d380 0x7f5fc406f830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0004d60 tx=0x7f5fd000b040 comp rx=0 tx=0).stop 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 msgr2=0x7f5fd81140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 0x7f5fd81140e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800b3b0 tx=0x7f5fc8009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 shutdown_connections 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f5fc406d380 0x7f5fc406f830 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fd80fde90 0x7f5fd8113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 --2- 172.21.15.81:0/1131233761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5fd80fe830 0x7f5fd81140e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 >> 172.21.15.81:0/1131233761 conn(0x7f5fd8072bc0 msgr2=0x7f5fd81037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 shutdown_connections 2024-03-22T07:20:10.786 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:10.784+0000 7f5fdfc70700 1 -- 172.21.15.81:0/1131233761 wait complete. 2024-03-22T07:20:11.068 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:10 smithi081 ceph-mon[113965]: pgmap v261: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:11.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:10 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1131233761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:10 smithi149 ceph-mon[83447]: pgmap v261: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:11.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:10 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1131233761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:11.599 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-22T07:20:11.858 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:20:13.249 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:12 smithi081 ceph-mon[113965]: pgmap v262: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:13.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:12 smithi149 ceph-mon[83447]: pgmap v262: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:13.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.616+0000 7f12a6448700 1 -- 172.21.15.81:0/3507673562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fe7a0 msgr2=0x7f12a0106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:13.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.616+0000 7f12a6448700 1 --2- 172.21.15.81:0/3507673562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fe7a0 0x7f12a0106de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1294004940 tx=0x7f1294009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:13.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.616+0000 7f12a6448700 1 -- 172.21.15.81:0/3507673562 shutdown_connections 2024-03-22T07:20:13.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.616+0000 7f12a6448700 1 --2- 172.21.15.81:0/3507673562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fe7a0 0x7f12a0106de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:13.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.616+0000 7f12a6448700 1 --2- 172.21.15.81:0/3507673562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12a00fde00 0x7f12a00fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:13.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.616+0000 7f12a6448700 1 -- 172.21.15.81:0/3507673562 >> 172.21.15.81:0/3507673562 conn(0x7f12a00f7850 msgr2=0x7f12a00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:13.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.616+0000 7f12a6448700 1 -- 172.21.15.81:0/3507673562 shutdown_connections 2024-03-22T07:20:13.617 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.616+0000 7f12a6448700 1 -- 172.21.15.81:0/3507673562 wait complete. 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f12a6448700 1 Processor -- start 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f12a6448700 1 -- start start 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f12a6448700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fde00 0x7f12a0113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f12a6448700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12a00fe7a0 0x7f12a0113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f12a6448700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0114550 con 0x7f12a00fde00 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f12a6448700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a010dac0 con 0x7f12a00fe7a0 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f129ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fde00 0x7f12a0113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f129ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fde00 0x7f12a0113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57836/0 (socket says 172.21.15.81:57836) 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f129ffff700 1 -- 172.21.15.81:0/1890482907 learned_addr learned my addr 172.21.15.81:0/1890482907 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f129f7fe700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12a00fe7a0 0x7f12a0113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:13.619 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f129ffff700 1 -- 172.21.15.81:0/1890482907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12a00fe7a0 msgr2=0x7f12a0113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:13.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f129ffff700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12a00fe7a0 0x7f12a0113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:13.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.617+0000 7f129ffff700 1 -- 172.21.15.81:0/1890482907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294005040 con 0x7f12a00fde00 2024-03-22T07:20:13.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.618+0000 7f129ffff700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fde00 0x7f12a0113930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f129000b790 tx=0x7f1290009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:13.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.618+0000 7f129cff9700 1 -- 172.21.15.81:0/1890482907 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f129000f6f0 con 0x7f12a00fde00 2024-03-22T07:20:13.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.618+0000 7f129cff9700 1 -- 172.21.15.81:0/1890482907 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f129000fd20 con 0x7f12a00fde00 2024-03-22T07:20:13.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.618+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a010dda0 con 0x7f12a00fde00 2024-03-22T07:20:13.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.618+0000 7f129cff9700 1 -- 172.21.15.81:0/1890482907 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12900103a0 con 0x7f12a00fde00 2024-03-22T07:20:13.620 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.618+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a010e2b0 con 0x7f12a00fde00 2024-03-22T07:20:13.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.619+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a0066b90 con 0x7f12a00fde00 2024-03-22T07:20:13.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.619+0000 7f129cff9700 1 -- 172.21.15.81:0/1890482907 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f1290011400 con 0x7f12a00fde00 2024-03-22T07:20:13.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.620+0000 7f129cff9700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f128806d380 0x7f128806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:13.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.620+0000 7f129cff9700 1 -- 172.21.15.81:0/1890482907 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f129008c750 con 0x7f12a00fde00 2024-03-22T07:20:13.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.620+0000 7f129f7fe700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f128806d380 0x7f128806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:13.621 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.620+0000 7f129f7fe700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f128806d380 0x7f128806f830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f12a010eda0 tx=0x7f1294009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:13.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.624+0000 7f129cff9700 1 -- 172.21.15.81:0/1890482907 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1290059d30 con 0x7f12a00fde00 2024-03-22T07:20:13.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.900+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f12a0109f10 con 0x7f12a00fde00 2024-03-22T07:20:13.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.901+0000 7f129cff9700 1 -- 172.21.15.81:0/1890482907 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f12900598c0 con 0x7f12a00fde00 2024-03-22T07:20:13.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f128806d380 msgr2=0x7f128806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:13.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f128806d380 0x7f128806f830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f12a010eda0 tx=0x7f1294009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:13.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fde00 msgr2=0x7f12a0113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:13.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fde00 0x7f12a0113930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f129000b790 tx=0x7f1290009250 comp rx=0 tx=0).stop 2024-03-22T07:20:13.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 shutdown_connections 2024-03-22T07:20:13.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f128806d380 0x7f128806f830 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:13.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00fde00 0x7f12a0113930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:13.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 --2- 172.21.15.81:0/1890482907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f12a00fe7a0 0x7f12a0113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:13.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 >> 172.21.15.81:0/1890482907 conn(0x7f12a00f7850 msgr2=0x7f12a0103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:13.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 shutdown_connections 2024-03-22T07:20:13.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:13.903+0000 7f12a6448700 1 -- 172.21.15.81:0/1890482907 wait complete. 2024-03-22T07:20:13.914 INFO:teuthology.orchestra.run.smithi081.stdout:true 2024-03-22T07:20:14.282 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:13 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1890482907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:14.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:13 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1890482907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:14.715 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-22T07:20:14.979 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:20:15.274 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:14 smithi081 ceph-mon[113965]: pgmap v263: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:15.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:14 smithi149 ceph-mon[83447]: pgmap v263: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:16.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:15 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:16.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:15 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.671+0000 7ff762854700 1 -- 172.21.15.81:0/1629404831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fde20 msgr2=0x7ff75c0fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.671+0000 7ff762854700 1 --2- 172.21.15.81:0/1629404831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fde20 0x7ff75c0fe1f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff750004940 tx=0x7ff750009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.671+0000 7ff762854700 1 -- 172.21.15.81:0/1629404831 shutdown_connections 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.671+0000 7ff762854700 1 --2- 172.21.15.81:0/1629404831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff75c0fe7c0 0x7ff75c106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.671+0000 7ff762854700 1 --2- 172.21.15.81:0/1629404831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fde20 0x7ff75c0fe1f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.671+0000 7ff762854700 1 -- 172.21.15.81:0/1629404831 >> 172.21.15.81:0/1629404831 conn(0x7ff75c0f7850 msgr2=0x7ff75c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.671+0000 7ff762854700 1 -- 172.21.15.81:0/1629404831 shutdown_connections 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.671+0000 7ff762854700 1 -- 172.21.15.81:0/1629404831 wait complete. 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.672+0000 7ff762854700 1 Processor -- start 2024-03-22T07:20:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.672+0000 7ff762854700 1 -- start start 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.672+0000 7ff762854700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff75c0fde20 0x7ff75c113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.672+0000 7ff762854700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fe7c0 0x7ff75c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.672+0000 7ff762854700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c114570 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.672+0000 7ff762854700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c10dae0 con 0x7ff75c0fde20 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.672+0000 7ff75b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fe7c0 0x7ff75c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.672+0000 7ff75b7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fe7c0 0x7ff75c113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55442/0 (socket says 172.21.15.81:55442) 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff75b7fe700 1 -- 172.21.15.81:0/1198925830 learned_addr learned my addr 172.21.15.81:0/1198925830 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff75bfff700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff75c0fde20 0x7ff75c113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff75b7fe700 1 -- 172.21.15.81:0/1198925830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff75c0fde20 msgr2=0x7ff75c113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff75b7fe700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff75c0fde20 0x7ff75c113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff75b7fe700 1 -- 172.21.15.81:0/1198925830 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff750005040 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff75b7fe700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fe7c0 0x7ff75c113e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00b790 tx=0x7ff74c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff758ff9700 1 -- 172.21.15.81:0/1198925830 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c00f6f0 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff758ff9700 1 -- 172.21.15.81:0/1198925830 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff74c00fd20 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c10ddc0 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff758ff9700 1 -- 172.21.15.81:0/1198925830 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74c0103a0 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.673+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c10e300 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.674+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c066b50 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.675+0000 7ff758ff9700 1 -- 172.21.15.81:0/1198925830 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7ff74c011400 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.675+0000 7ff758ff9700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff74406d380 0x7ff74406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:16.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.675+0000 7ff758ff9700 1 -- 172.21.15.81:0/1198925830 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff74c08c6d0 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.675+0000 7ff75bfff700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff74406d380 0x7ff74406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:16.677 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.676+0000 7ff75bfff700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff74406d380 0x7ff74406f830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff750009fd0 tx=0x7ff750009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:16.680 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.680+0000 7ff758ff9700 1 -- 172.21.15.81:0/1198925830 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff74c059cd0 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.956+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff75c109f30 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.956 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.956+0000 7ff758ff9700 1 -- 172.21.15.81:0/1198925830 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff74c059860 con 0x7ff75c0fe7c0 2024-03-22T07:20:16.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff74406d380 msgr2=0x7ff74406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:16.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff74406d380 0x7ff74406f830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff750009fd0 tx=0x7ff750009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:16.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fe7c0 msgr2=0x7ff75c113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:16.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fe7c0 0x7ff75c113e90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00b790 tx=0x7ff74c009250 comp rx=0 tx=0).stop 2024-03-22T07:20:16.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 shutdown_connections 2024-03-22T07:20:16.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7ff74406d380 0x7ff74406f830 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:16.959 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff75c0fde20 0x7ff75c113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:16.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 --2- 172.21.15.81:0/1198925830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff75c0fe7c0 0x7ff75c113e90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:16.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 >> 172.21.15.81:0/1198925830 conn(0x7ff75c0f7850 msgr2=0x7ff75c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:16.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.958+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 shutdown_connections 2024-03-22T07:20:16.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:16.959+0000 7ff762854700 1 -- 172.21.15.81:0/1198925830 wait complete. 2024-03-22T07:20:16.969 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)" 2024-03-22T07:20:17.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:16 smithi081 ceph-mon[113965]: pgmap v264: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:17.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:16 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1198925830' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:16 smithi149 ceph-mon[83447]: pgmap v264: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:17.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:16 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1198925830' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:17.691 DEBUG:teuthology.parallel:result is None 2024-03-22T07:20:17.691 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:20:17.702 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:20:17.702 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a42e8632-e819-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T07:20:17.968 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/mon.smithi081/config 2024-03-22T07:20:19.161 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:19 smithi081 ceph-mon[113965]: pgmap v265: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:19.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:19 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:19.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:19 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:19.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:19 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:19.162 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:19 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:20:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:19 smithi149 ceph-mon[83447]: pgmap v265: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:19 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:19 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:19 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:19.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:19 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.687+0000 7f212dd3d700 1 -- 172.21.15.81:0/2773675984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 msgr2=0x7f21281031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.687+0000 7f212dd3d700 1 --2- 172.21.15.81:0/2773675984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 0x7f21281031f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2118004940 tx=0x7f2118009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.687+0000 7f212dd3d700 1 -- 172.21.15.81:0/2773675984 shutdown_connections 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.687+0000 7f212dd3d700 1 --2- 172.21.15.81:0/2773675984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21281037c0 0x7f21280fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.687+0000 7f212dd3d700 1 --2- 172.21.15.81:0/2773675984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 0x7f21281031f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.687+0000 7f212dd3d700 1 -- 172.21.15.81:0/2773675984 >> 172.21.15.81:0/2773675984 conn(0x7f21280f89b0 msgr2=0x7f21280fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.687+0000 7f212dd3d700 1 -- 172.21.15.81:0/2773675984 shutdown_connections 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.687+0000 7f212dd3d700 1 -- 172.21.15.81:0/2773675984 wait complete. 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.688+0000 7f212dd3d700 1 Processor -- start 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.688+0000 7f212dd3d700 1 -- start start 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.688+0000 7f212dd3d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 0x7f2128113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.688+0000 7f212dd3d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21281037c0 0x7f2128113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.688+0000 7f212dd3d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2128114500 con 0x7f2128102e20 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.688+0000 7f212dd3d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212810db50 con 0x7f21281037c0 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.688+0000 7f21277fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 0x7f2128113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f21277fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 0x7f2128113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55466/0 (socket says 172.21.15.81:55466) 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f21277fe700 1 -- 172.21.15.81:0/2193254894 learned_addr learned my addr 172.21.15.81:0/2193254894 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f2126ffd700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21281037c0 0x7f2128113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f21277fe700 1 -- 172.21.15.81:0/2193254894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21281037c0 msgr2=0x7f2128113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f21277fe700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21281037c0 0x7f2128113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f21277fe700 1 -- 172.21.15.81:0/2193254894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2118005040 con 0x7f2128102e20 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f21277fe700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 0x7f2128113970 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2118009180 tx=0x7f2118009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f210ffff700 1 -- 172.21.15.81:0/2193254894 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f211800e040 con 0x7f2128102e20 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f210ffff700 1 -- 172.21.15.81:0/2193254894 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f211800bbc0 con 0x7f2128102e20 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212810ddd0 con 0x7f2128102e20 2024-03-22T07:20:19.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f210ffff700 1 -- 172.21.15.81:0/2193254894 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f211801c630 con 0x7f2128102e20 2024-03-22T07:20:19.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.689+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212810e390 con 0x7f2128102e20 2024-03-22T07:20:19.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.690+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2128100130 con 0x7f2128102e20 2024-03-22T07:20:19.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.691+0000 7f210ffff700 1 -- 172.21.15.81:0/2193254894 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f2118013420 con 0x7f2128102e20 2024-03-22T07:20:19.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.691+0000 7f210ffff700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f211006d380 0x7f211006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:19.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.691+0000 7f210ffff700 1 -- 172.21.15.81:0/2193254894 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f211808dc90 con 0x7f2128102e20 2024-03-22T07:20:19.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.691+0000 7f2126ffd700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f211006d380 0x7f211006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:19.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.691+0000 7f2126ffd700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f211006d380 0x7f211006f830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f212810ecc0 tx=0x7f211c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:19.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.696+0000 7f210ffff700 1 -- 172.21.15.81:0/2193254894 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f211805b310 con 0x7f2128102e20 2024-03-22T07:20:19.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.932+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2128100130 con 0x7f2128102e20 2024-03-22T07:20:19.933 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.932+0000 7f210ffff700 1 -- 172.21.15.81:0/2193254894 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 41 v41) v1 ==== 76+0+1914 (secure 0 0 0) 0x7f211805aea0 con 0x7f2128102e20 2024-03-22T07:20:19.933 INFO:teuthology.orchestra.run.smithi081.stdout:e41 2024-03-22T07:20:19.933 INFO:teuthology.orchestra.run.smithi081.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:20:19.933 INFO:teuthology.orchestra.run.smithi081.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:20:19.933 INFO:teuthology.orchestra.run.smithi081.stdout:legacy client fscid: 1 2024-03-22T07:20:19.933 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:fs_name cephfs 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 41 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-03-22T07:06:21.658288+0000 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-03-22T07:18:58.775150+0000 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:tableserver 0 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:root 0 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:session_timeout 60 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:session_autoclose 300 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:max_file_size 1099511627776 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:max_xattr_size 65536 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:required_client_features {} 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure 0 2024-03-22T07:20:19.934 INFO:teuthology.orchestra.run.smithi081.stdout:last_failure_osd_epoch 81 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:max_mds 2 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:in 0,1 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:up {0=34378,1=34410} 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:failed 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:damaged 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:stopped 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:data_pools [3] 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:metadata_pool 2 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:inline_data disabled 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:balancer 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:bal_rank_mask -1 2024-03-22T07:20:19.935 INFO:teuthology.orchestra.run.smithi081.stdout:standby_count_wanted 1 2024-03-22T07:20:19.936 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.bnzfrv{0:34378} state up:active seq 20 export targets 1 join_fscid=1 addr [v2:172.21.15.81:6826/235705027,v1:172.21.15.81:6827/235705027] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:20:19.936 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi081.zfgxzy{1:34410} state up:active seq 15 export targets 0 join_fscid=1 addr [v2:172.21.15.81:6828/2467232405,v1:172.21.15.81:6829/2467232405] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:20:19.936 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:19.936 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:19.936 INFO:teuthology.orchestra.run.smithi081.stdout:Standby daemons: 2024-03-22T07:20:19.936 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:19.936 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.kdtxfw{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6824/3255112562,v1:172.21.15.149:6825/3255112562] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:20:19.936 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.cephfs.smithi149.ffewvr{-1:44291} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.149:6826/1411110570,v1:172.21.15.149:6827/1411110570] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f211006d380 msgr2=0x7f211006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f211006d380 0x7f211006f830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f212810ecc0 tx=0x7f211c009ee0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 msgr2=0x7f2128113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 0x7f2128113970 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2118009180 tx=0x7f2118009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 shutdown_connections 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f211006d380 0x7f211006f830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2128102e20 0x7f2128113970 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 --2- 172.21.15.81:0/2193254894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21281037c0 0x7f2128113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 >> 172.21.15.81:0/2193254894 conn(0x7f21280f89b0 msgr2=0x7f2128107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:19.937 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 shutdown_connections 2024-03-22T07:20:19.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:19.935+0000 7f212dd3d700 1 -- 172.21.15.81:0/2193254894 wait complete. 2024-03-22T07:20:19.938 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 41 2024-03-22T07:20:20.252 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:20 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2193254894' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:20:20.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:20 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2193254894' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:20:20.720 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-22T07:20:20.731 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T07:20:20.832 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.830+0000 7fbca602e640 1 -- 172.21.15.81:0/4714445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fd200 msgr2=0x7fbca00fd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.830+0000 7fbca602e640 1 --2- 172.21.15.81:0/4714445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fd200 0x7fbca00fd5d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009c50 tx=0x7fbc90004d10 comp rx=0 tx=0).stop 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.830+0000 7fbca602e640 1 -- 172.21.15.81:0/4714445 shutdown_connections 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.830+0000 7fbca602e640 1 --2- 172.21.15.81:0/4714445 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca00fdba0 0x7fbca01061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.830+0000 7fbca602e640 1 --2- 172.21.15.81:0/4714445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fd200 0x7fbca00fd5d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.830+0000 7fbca602e640 1 -- 172.21.15.81:0/4714445 >> 172.21.15.81:0/4714445 conn(0x7fbca00f6c70 msgr2=0x7fbca00f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.830+0000 7fbca602e640 1 -- 172.21.15.81:0/4714445 shutdown_connections 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.830+0000 7fbca602e640 1 -- 172.21.15.81:0/4714445 wait complete. 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.831+0000 7fbca602e640 1 Processor -- start 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.831+0000 7fbca602e640 1 -- start start 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.831+0000 7fbca602e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca00fd200 0x7fbca0114f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.831+0000 7fbca602e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fdba0 0x7fbca01154b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.831+0000 7fbca602e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0115ab0 con 0x7fbca00fdba0 2024-03-22T07:20:20.833 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.831+0000 7fbca602e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca010f100 con 0x7fbca00fd200 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc9effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fdba0 0x7fbca01154b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc9effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fdba0 0x7fbca01154b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55474/0 (socket says 172.21.15.81:55474) 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc9effd640 1 -- 172.21.15.81:0/476709415 learned_addr learned my addr 172.21.15.81:0/476709415 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc9f7fe640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca00fd200 0x7fbca0114f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc9effd640 1 -- 172.21.15.81:0/476709415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca00fd200 msgr2=0x7fbca0114f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc9effd640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca00fd200 0x7fbca0114f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc9effd640 1 -- 172.21.15.81:0/476709415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900098b0 con 0x7fbca00fdba0 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc9effd640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fdba0 0x7fbca01154b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbc94002a50 tx=0x7fbc94004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc87fff640 1 -- 172.21.15.81:0/476709415 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc94007c30 con 0x7fbca00fdba0 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca010f3e0 con 0x7fbca00fdba0 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc87fff640 1 -- 172.21.15.81:0/476709415 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc9400f040 con 0x7fbca00fdba0 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbc87fff640 1 -- 172.21.15.81:0/476709415 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9400db50 con 0x7fbca00fdba0 2024-03-22T07:20:20.834 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.832+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca010f900 con 0x7fbca00fdba0 2024-03-22T07:20:20.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.833+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca0065e30 con 0x7fbca00fdba0 2024-03-22T07:20:20.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.834+0000 7fbc87fff640 1 -- 172.21.15.81:0/476709415 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbc9401c400 con 0x7fbca00fdba0 2024-03-22T07:20:20.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.834+0000 7fbc87fff640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbc8806d2f0 0x7fbc8806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:20.835 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.834+0000 7fbc87fff640 1 -- 172.21.15.81:0/476709415 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbc9408c240 con 0x7fbca00fdba0 2024-03-22T07:20:20.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.834+0000 7fbc9f7fe640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbc8806d2f0 0x7fbc8806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:20.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.835+0000 7fbc9f7fe640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbc8806d2f0 0x7fbc8806f7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbc90004a30 tx=0x7fbc9002e040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:20.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:20.839+0000 7fbc87fff640 1 -- 172.21.15.81:0/476709415 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc94055c70 con 0x7fbca00fdba0 2024-03-22T07:20:21.014 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.014+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbca0109360 con 0x7fbca00fdba0 2024-03-22T07:20:21.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.015+0000 7fbc87fff640 1 -- 172.21.15.81:0/476709415 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16642 (secure 0 0 0) 0x7fbc94059330 con 0x7fbca00fdba0 2024-03-22T07:20:21.016 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:21.016 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":85,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","created":"2024-03-22T06:59:35.415160+0000","modified":"2024-03-22T07:18:04.460156+0000","last_up_change":"2024-03-22T07:17:09.537521+0000","last_in_change":"2024-03-22T07:04:44.637120+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:03:49.371549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T07:06:20.045033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T07:06:20.879130+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69addaec-0295-4219-9e22-5ebf831a65d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6803","nonce":2753302858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6805","nonce":2753302858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6809","nonce":2753302858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6807","nonce":2753302858}]},"public_addr":"172.21.15.81:6803/2753302858","cluster_addr":"172.21.15.81:6805/2753302858","heartbeat_back_addr":"172.21.15.81:6809/2753302858","heartbeat_front_addr":"172.21.15.81:6807/2753302858","state":["exists","up"]},{"osd":1,"uuid":"cd572358-1fd5-4e96-8caf-3d00d6e0608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6811","nonce":356075948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6813","nonce":356075948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6817","nonce":356075948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6815","nonce":356075948}]},"public_addr":"172.21.15.81:6811/356075948","cluster_addr":"172.21.15.81:6813/356075948","heartbeat_back_addr":"172.21.15.81:6817/356075948","heartbeat_front_addr":"172.21.15.81:6815/356075948","state":["exists","up"]},{"osd":2,"uuid":"ba6f9b7e-712d-4f4b-8b33-6e051daf2875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6819","nonce":3743751332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6821","nonce":3743751332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6825","nonce":3743751332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6823","nonce":3743751332}]},"public_addr":"172.21.15.81:6819/3743751332","cluster_addr":"172.21.15.81:6821/3743751332","heartbeat_back_addr":"172.21.15.81:6825/3743751332","heartbeat_front_addr":"172.21.15.81:6823/3743751332","state":["exists","up"]},{"osd":3,"uuid":"e5adfd7b-e190-45b4-b8ca-da7c45d21d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6801","nonce":1325806457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6803","nonce":1325806457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6807","nonce":1325806457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6805","nonce":1325806457}]},"public_addr":"172.21.15.149:6801/1325806457","cluster_addr":"172.21.15.149:6803/1325806457","heartbeat_back_addr":"172.21.15.149:6807/1325806457","heartbeat_front_addr":"172.21.15.149:6805/1325806457","state":["exists","up"]},{"osd":4,"uuid":"a85bb681-b621-4fbf-b77a-cbd090a41a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6809","nonce":1122192339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6811","nonce":1122192339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6815","nonce":1122192339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6813","nonce":1122192339}]},"public_addr":"172.21.15.149:6809/1122192339","cluster_addr":"172.21.15.149:6811/1122192339","heartbeat_back_addr":"172.21.15.149:6815/1122192339","heartbeat_front_addr":"172.21.15.149:6813/1122192339","state":["exists","up"]},{"osd":5,"uuid":"0e83552d-4e6c-4426-9b7c-99d9ad5c45ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6817","nonce":530633899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6819","nonce":530633899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6823","nonce":530633899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6821","nonce":530633899}]},"public_addr":"172.21.15.149:6817/530633899","cluster_addr":"172.21.15.149:6819/530633899","heartbeat_back_addr":"172.21.15.149:6823/530633899","heartbeat_front_addr":"172.21.15.149:6821/530633899","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:02:52.797148+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:18.980557+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:46.975511+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:10.893766+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:36.048181+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:05:01.848799+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6829/268578316":"2024-03-23T07:17:11.575629+0000","172.21.15.149:6824/3521694486":"2024-03-23T07:17:11.575646+0000","172.21.15.81:0/1030883466":"2024-03-23T07:00:35.069960+0000","172.21.15.149:6829/18108693":"2024-03-23T07:10:12.272683+0000","172.21.15.81:0/3192935300":"2024-03-23T07:08:38.778309+0000","172.21.15.149:6828/18108693":"2024-03-23T07:10:12.272683+0000","172.21.15.81:6828/268578316":"2024-03-23T07:17:11.575629+0000","172.21.15.81:6801/742540357":"2024-03-23T07:08:38.778309+0000","172.21.15.81:0/1980154994":"2024-03-23T07:08:38.778309+0000","172.21.15.81:0/2249539268":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6800/2774978653":"2024-03-23T07:12:45.787752+0000","172.21.15.81:6827/2831536232":"2024-03-23T07:06:37.046400+0000","172.21.15.81:6801/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.149:0/3423294147":"2024-03-23T07:10:12.272683+0000","172.21.15.81:6800/742540357":"2024-03-23T07:08:38.778309+0000","172.21.15.81:0/2438362157":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/473242716":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/1138486583":"2024-03-23T07:12:45.787752+0000","172.21.15.149:6825/3521694486":"2024-03-23T07:17:11.575646+0000","172.21.15.81:6800/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6800/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.149:0/3477455875":"2024-03-23T07:10:12.272683+0000","172.21.15.81:6826/2831536232":"2024-03-23T07:06:37.046400+0000","172.21.15.81:0/683056633":"2024-03-23T07:12:45.787752+0000","172.21.15.81:6801/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/2809233037":"2024-03-23T07:08:38.778309+0000","172.21.15.81:6800/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6801/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/1939667879":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/4180230166":"2024-03-23T07:08:38.778309+0000","172.21.15.149:0/594572994":"2024-03-23T07:10:12.272683+0000","172.21.15.81:0/3756300008":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/2472307158":"2024-03-23T07:00:01.117993+0000","172.21.15.149:0/749736269":"2024-03-23T07:10:12.272683+0000","172.21.15.81:0/988274144":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/770908409":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/1704456246":"2024-03-23T07:12:45.787752+0000","172.21.15.81:6801/2774978653":"2024-03-23T07:12:45.787752+0000","172.21.15.81:0/1143393440":"2024-03-23T07:12:45.787752+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:20:21.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.017+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbc8806d2f0 msgr2=0x7fbc8806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.017+0000 7fbca602e640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbc8806d2f0 0x7fbc8806f7a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbc90004a30 tx=0x7fbc9002e040 comp rx=0 tx=0).stop 2024-03-22T07:20:21.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.017+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fdba0 msgr2=0x7fbca01154b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.017+0000 7fbca602e640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fdba0 0x7fbca01154b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbc94002a50 tx=0x7fbc94004600 comp rx=0 tx=0).stop 2024-03-22T07:20:21.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.017+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 shutdown_connections 2024-03-22T07:20:21.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.017+0000 7fbca602e640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fbc8806d2f0 0x7fbc8806f7a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.017+0000 7fbca602e640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbca00fdba0 0x7fbca01154b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.017+0000 7fbca602e640 1 --2- 172.21.15.81:0/476709415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbca00fd200 0x7fbca0114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.017+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 >> 172.21.15.81:0/476709415 conn(0x7fbca00f6c70 msgr2=0x7fbca00f80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:21.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.018+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 shutdown_connections 2024-03-22T07:20:21.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.018+0000 7fbca602e640 1 -- 172.21.15.81:0/476709415 wait complete. 2024-03-22T07:20:21.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:03:49.371549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T07:06:20.045033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T07:06:20.879130+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T07:20:21.030 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T07:20:21.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.129+0000 7f1213b1d640 1 -- 172.21.15.81:0/1346393407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102200 msgr2=0x7f120c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.131 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.129+0000 7f1213b1d640 1 --2- 172.21.15.81:0/1346393407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102200 0x7f120c1025d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f12080048d0 tx=0x7f1208009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.129+0000 7f1213b1d640 1 -- 172.21.15.81:0/1346393407 shutdown_connections 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.129+0000 7f1213b1d640 1 --2- 172.21.15.81:0/1346393407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f120c102ba0 0x7f120c0fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.129+0000 7f1213b1d640 1 --2- 172.21.15.81:0/1346393407 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102200 0x7f120c1025d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.129+0000 7f1213b1d640 1 -- 172.21.15.81:0/1346393407 >> 172.21.15.81:0/1346393407 conn(0x7f120c0708f0 msgr2=0x7f120c070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.129+0000 7f1213b1d640 1 -- 172.21.15.81:0/1346393407 shutdown_connections 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.129+0000 7f1213b1d640 1 -- 172.21.15.81:0/1346393407 wait complete. 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f1213b1d640 1 Processor -- start 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f1213b1d640 1 -- start start 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f1213b1d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f120c102200 0x7f120c112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f1213b1d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102ba0 0x7f120c113320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f1213b1d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c113920 con 0x7f120c102ba0 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f1213b1d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c10cfc0 con 0x7f120c102200 2024-03-22T07:20:21.132 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f12110b8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102ba0 0x7f120c113320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f12110b8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102ba0 0x7f120c113320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55500/0 (socket says 172.21.15.81:55500) 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f12110b8640 1 -- 172.21.15.81:0/3931371556 learned_addr learned my addr 172.21.15.81:0/3931371556 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f12118b9640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f120c102200 0x7f120c112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f12110b8640 1 -- 172.21.15.81:0/3931371556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f120c102200 msgr2=0x7f120c112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f12110b8640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f120c102200 0x7f120c112de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f12110b8640 1 -- 172.21.15.81:0/3931371556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208005040 con 0x7f120c102ba0 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.130+0000 7f12110b8640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102ba0 0x7f120c113320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c760 tx=0x7f11fc00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.131+0000 7f12027fc640 1 -- 172.21.15.81:0/3931371556 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11fc00a820 con 0x7f120c102ba0 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.131+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c10d2a0 con 0x7f120c102ba0 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.131+0000 7f12027fc640 1 -- 172.21.15.81:0/3931371556 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11fc00ae50 con 0x7f120c102ba0 2024-03-22T07:20:21.133 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.131+0000 7f12027fc640 1 -- 172.21.15.81:0/3931371556 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11fc0115e0 con 0x7f120c102ba0 2024-03-22T07:20:21.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.131+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c10d7c0 con 0x7f120c102ba0 2024-03-22T07:20:21.134 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.131+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c0652a0 con 0x7f120c102ba0 2024-03-22T07:20:21.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.132+0000 7f12027fc640 1 -- 172.21.15.81:0/3931371556 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f11fc012400 con 0x7f120c102ba0 2024-03-22T07:20:21.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.132+0000 7f12027fc640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f11f806d2f0 0x7f11f806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:21.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.132+0000 7f12027fc640 1 -- 172.21.15.81:0/3931371556 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f11fc08cb50 con 0x7f120c102ba0 2024-03-22T07:20:21.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.132+0000 7f12118b9640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f11f806d2f0 0x7f11f806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:21.135 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.133+0000 7f12118b9640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f11f806d2f0 0x7f11f806f7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1208004540 tx=0x7f1208009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:21.137 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.137+0000 7f12027fc640 1 -- 172.21.15.81:0/3931371556 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11fc056600 con 0x7f120c102ba0 2024-03-22T07:20:21.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.310+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f120c109390 con 0x7f120c102ba0 2024-03-22T07:20:21.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.310+0000 7f12027fc640 1 -- 172.21.15.81:0/3931371556 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11fc08c0a0 con 0x7f120c102ba0 2024-03-22T07:20:21.311 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-03-22T07:20:21.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f11f806d2f0 msgr2=0x7f11f806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f11f806d2f0 0x7f11f806f7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1208004540 tx=0x7f1208009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102ba0 msgr2=0x7f120c113320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102ba0 0x7f120c113320 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c760 tx=0x7f11fc00cb60 comp rx=0 tx=0).stop 2024-03-22T07:20:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 shutdown_connections 2024-03-22T07:20:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f11f806d2f0 0x7f11f806f7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f120c102ba0 0x7f120c113320 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 --2- 172.21.15.81:0/3931371556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f120c102200 0x7f120c112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 >> 172.21.15.81:0/3931371556 conn(0x7f120c0708f0 msgr2=0x7f120c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 shutdown_connections 2024-03-22T07:20:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.312+0000 7f1213b1d640 1 -- 172.21.15.81:0/3931371556 wait complete. 2024-03-22T07:20:21.318 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:21 smithi081 ceph-mon[113965]: pgmap v266: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:21.323 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T07:20:21.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:21 smithi149 ceph-mon[83447]: pgmap v266: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.422+0000 7f78ef31d640 1 -- 172.21.15.81:0/1568834231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e8102240 msgr2=0x7f78e8102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.422+0000 7f78ef31d640 1 --2- 172.21.15.81:0/1568834231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e8102240 0x7f78e8102610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009c50 tx=0x7f78e4004d10 comp rx=0 tx=0).stop 2024-03-22T07:20:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 -- 172.21.15.81:0/1568834231 shutdown_connections 2024-03-22T07:20:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 --2- 172.21.15.81:0/1568834231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e80643a0 0x7f78e80647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 --2- 172.21.15.81:0/1568834231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e8102240 0x7f78e8102610 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 -- 172.21.15.81:0/1568834231 >> 172.21.15.81:0/1568834231 conn(0x7f78e80710b0 msgr2=0x7f78e80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 -- 172.21.15.81:0/1568834231 shutdown_connections 2024-03-22T07:20:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 -- 172.21.15.81:0/1568834231 wait complete. 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 Processor -- start 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 -- start start 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e80643a0 0x7f78e80fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8102240 0x7f78e80fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80fd0c0 con 0x7f78e80643a0 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ef31d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80fbc60 con 0x7f78e8102240 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ed0b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e80643a0 0x7f78e80fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ed0b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e80643a0 0x7f78e80fb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55508/0 (socket says 172.21.15.81:55508) 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.423+0000 7f78ed0b9640 1 -- 172.21.15.81:0/2306701396 learned_addr learned my addr 172.21.15.81:0/2306701396 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ec8b8640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8102240 0x7f78e80fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ed0b9640 1 -- 172.21.15.81:0/2306701396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8102240 msgr2=0x7f78e80fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ed0b9640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8102240 0x7f78e80fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ed0b9640 1 -- 172.21.15.81:0/2306701396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e40098b0 con 0x7f78e80643a0 2024-03-22T07:20:21.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ed0b9640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e80643a0 0x7f78e80fb1e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78e402c1f0 tx=0x7f78e4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:21.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ddffb640 1 -- 172.21.15.81:0/2306701396 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e403a070 con 0x7f78e80643a0 2024-03-22T07:20:21.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e80fbee0 con 0x7f78e80643a0 2024-03-22T07:20:21.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ddffb640 1 -- 172.21.15.81:0/2306701396 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78e4004610 con 0x7f78e80643a0 2024-03-22T07:20:21.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ddffb640 1 -- 172.21.15.81:0/2306701396 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e403ebe0 con 0x7f78e80643a0 2024-03-22T07:20:21.427 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.424+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e8119420 con 0x7f78e80643a0 2024-03-22T07:20:21.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.425+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e80fd940 con 0x7f78e80643a0 2024-03-22T07:20:21.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.426+0000 7f78ddffb640 1 -- 172.21.15.81:0/2306701396 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f78e4032070 con 0x7f78e80643a0 2024-03-22T07:20:21.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.426+0000 7f78ddffb640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f78d406d300 0x7f78d406f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:21.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.426+0000 7f78ddffb640 1 -- 172.21.15.81:0/2306701396 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f78e40aee30 con 0x7f78e80643a0 2024-03-22T07:20:21.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.426+0000 7f78ec8b8640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f78d406d300 0x7f78d406f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:21.428 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.426+0000 7f78ec8b8640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f78d406d300 0x7f78d406f7b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f78d8007fd0 tx=0x7f78d8009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:21.431 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.431+0000 7f78ddffb640 1 -- 172.21.15.81:0/2306701396 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78e4078910 con 0x7f78e80643a0 2024-03-22T07:20:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.604+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f78e804bcc0 con 0x7f78e80643a0 2024-03-22T07:20:21.605 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.604+0000 7f78ddffb640 1 -- 172.21.15.81:0/2306701396 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f78e407bfd0 con 0x7f78e80643a0 2024-03-22T07:20:21.605 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-03-22T07:20:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f78d406d300 msgr2=0x7f78d406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f78d406d300 0x7f78d406f7b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f78d8007fd0 tx=0x7f78d8009e90 comp rx=0 tx=0).stop 2024-03-22T07:20:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e80643a0 msgr2=0x7f78e80fb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e80643a0 0x7f78e80fb1e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78e402c1f0 tx=0x7f78e4004d10 comp rx=0 tx=0).stop 2024-03-22T07:20:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 shutdown_connections 2024-03-22T07:20:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f78d406d300 0x7f78d406f7b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78e8102240 0x7f78e80fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 --2- 172.21.15.81:0/2306701396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e80643a0 0x7f78e80fb1e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 >> 172.21.15.81:0/2306701396 conn(0x7f78e80710b0 msgr2=0x7f78e80f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:21.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 shutdown_connections 2024-03-22T07:20:21.608 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.606+0000 7f78ef31d640 1 -- 172.21.15.81:0/2306701396 wait complete. 2024-03-22T07:20:21.618 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T07:20:21.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.717+0000 7fdb52aae640 1 -- 172.21.15.81:0/3083397187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c101c70 msgr2=0x7fdb4c0fb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.717+0000 7fdb52aae640 1 --2- 172.21.15.81:0/3083397187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c101c70 0x7fdb4c0fb300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdb340048d0 tx=0x7fdb34009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 -- 172.21.15.81:0/3083397187 shutdown_connections 2024-03-22T07:20:21.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 --2- 172.21.15.81:0/3083397187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c101c70 0x7fdb4c0fb300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 --2- 172.21.15.81:0/3083397187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c1012d0 0x7fdb4c1016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 -- 172.21.15.81:0/3083397187 >> 172.21.15.81:0/3083397187 conn(0x7fdb4c0f6ec0 msgr2=0x7fdb4c0f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:21.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 -- 172.21.15.81:0/3083397187 shutdown_connections 2024-03-22T07:20:21.720 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 -- 172.21.15.81:0/3083397187 wait complete. 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 Processor -- start 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 -- start start 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c1012d0 0x7fdb4c111e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c101c70 0x7fdb4c112370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c112970 con 0x7fdb4c1012d0 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.718+0000 7fdb52aae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c10bfc0 con 0x7fdb4c101c70 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb5084a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c1012d0 0x7fdb4c111e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb5084a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c1012d0 0x7fdb4c111e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55524/0 (socket says 172.21.15.81:55524) 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb5084a640 1 -- 172.21.15.81:0/1822588630 learned_addr learned my addr 172.21.15.81:0/1822588630 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb5084a640 1 -- 172.21.15.81:0/1822588630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c101c70 msgr2=0x7fdb4c112370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb5084a640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c101c70 0x7fdb4c112370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.721 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb5084a640 1 -- 172.21.15.81:0/1822588630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34005040 con 0x7fdb4c1012d0 2024-03-22T07:20:21.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb5084a640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c1012d0 0x7fdb4c111e30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c004e60 tx=0x7fdb3c007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:21.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb497fa640 1 -- 172.21.15.81:0/1822588630 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3c004200 con 0x7fdb4c1012d0 2024-03-22T07:20:21.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb497fa640 1 -- 172.21.15.81:0/1822588630 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb3c00acc0 con 0x7fdb4c1012d0 2024-03-22T07:20:21.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c10c240 con 0x7fdb4c1012d0 2024-03-22T07:20:21.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb497fa640 1 -- 172.21.15.81:0/1822588630 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3c011e40 con 0x7fdb4c1012d0 2024-03-22T07:20:21.722 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.719+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c10c760 con 0x7fdb4c1012d0 2024-03-22T07:20:21.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.720+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c0fca40 con 0x7fdb4c1012d0 2024-03-22T07:20:21.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.721+0000 7fdb497fa640 1 -- 172.21.15.81:0/1822588630 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fdb3c00a7f0 con 0x7fdb4c1012d0 2024-03-22T07:20:21.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.721+0000 7fdb497fa640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb3806d2f0 0x7fdb3806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:21.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.721+0000 7fdb4bfff640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb3806d2f0 0x7fdb3806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:21.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.721+0000 7fdb497fa640 1 -- 172.21.15.81:0/1822588630 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdb3c015070 con 0x7fdb4c1012d0 2024-03-22T07:20:21.723 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.722+0000 7fdb4bfff640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb3806d2f0 0x7fdb3806f7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdb340043d0 tx=0x7fdb34009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:21.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.726+0000 7fdb497fa640 1 -- 172.21.15.81:0/1822588630 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb3c0588b0 con 0x7fdb4c1012d0 2024-03-22T07:20:21.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.900+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fdb4c10d4a0 con 0x7fdb4c1012d0 2024-03-22T07:20:21.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.900+0000 7fdb497fa640 1 -- 172.21.15.81:0/1822588630 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdb3c05bf70 con 0x7fdb4c1012d0 2024-03-22T07:20:21.901 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb3806d2f0 msgr2=0x7fdb3806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb3806d2f0 0x7fdb3806f7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdb340043d0 tx=0x7fdb34009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c1012d0 msgr2=0x7fdb4c111e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c1012d0 0x7fdb4c111e30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c004e60 tx=0x7fdb3c007ee0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 shutdown_connections 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fdb3806d2f0 0x7fdb3806f7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c101c70 0x7fdb4c112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 --2- 172.21.15.81:0/1822588630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4c1012d0 0x7fdb4c111e30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 >> 172.21.15.81:0/1822588630 conn(0x7fdb4c0f6ec0 msgr2=0x7fdb4c0f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:21.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 shutdown_connections 2024-03-22T07:20:21.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:21.902+0000 7fdb52aae640 1 -- 172.21.15.81:0/1822588630 wait complete. 2024-03-22T07:20:21.914 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T07:20:22.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.010+0000 7f32bee14640 1 -- 172.21.15.81:0/2833597527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b81010c0 msgr2=0x7f32b8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.016 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.010+0000 7f32bee14640 1 --2- 172.21.15.81:0/2833597527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b81010c0 0x7f32b8101490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0048d0 tx=0x7f32ac009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.010+0000 7f32bee14640 1 -- 172.21.15.81:0/2833597527 shutdown_connections 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.010+0000 7f32bee14640 1 --2- 172.21.15.81:0/2833597527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b8101a60 0x7f32b80fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.010+0000 7f32bee14640 1 --2- 172.21.15.81:0/2833597527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b81010c0 0x7f32b8101490 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.010+0000 7f32bee14640 1 -- 172.21.15.81:0/2833597527 >> 172.21.15.81:0/2833597527 conn(0x7f32b80f6c70 msgr2=0x7f32b80f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.011+0000 7f32bee14640 1 -- 172.21.15.81:0/2833597527 shutdown_connections 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.011+0000 7f32bee14640 1 -- 172.21.15.81:0/2833597527 wait complete. 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.011+0000 7f32bee14640 1 Processor -- start 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.011+0000 7f32bee14640 1 -- start start 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.011+0000 7f32bee14640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b81010c0 0x7f32b810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.011+0000 7f32bee14640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b8101a60 0x7f32b810e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.011+0000 7f32bee14640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b810e8e0 con 0x7f32b8101a60 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.011+0000 7f32bee14640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b810ea50 con 0x7f32b81010c0 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.011+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b8101a60 0x7f32b810e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b8101a60 0x7f32b810e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55548/0 (socket says 172.21.15.81:55548) 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32b7fff640 1 -- 172.21.15.81:0/2299197173 learned_addr learned my addr 172.21.15.81:0/2299197173 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32bcbb0640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b81010c0 0x7f32b810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32b7fff640 1 -- 172.21.15.81:0/2299197173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b81010c0 msgr2=0x7f32b810de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32b7fff640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b81010c0 0x7f32b810de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32b7fff640 1 -- 172.21.15.81:0/2299197173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac005040 con 0x7f32b8101a60 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32b7fff640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b8101a60 0x7f32b810e3a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f32a4002a50 tx=0x7f32a4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32b57fa640 1 -- 172.21.15.81:0/2299197173 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a4007c30 con 0x7f32b8101a60 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32b57fa640 1 -- 172.21.15.81:0/2299197173 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32a400f040 con 0x7f32b8101a60 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b8116ed0 con 0x7f32b8101a60 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32b57fa640 1 -- 172.21.15.81:0/2299197173 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32a400dad0 con 0x7f32b8101a60 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.012+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b8117200 con 0x7f32b8101a60 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.014+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b80fc830 con 0x7f32b8101a60 2024-03-22T07:20:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.014+0000 7f32b57fa640 1 -- 172.21.15.81:0/2299197173 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f32a400dc70 con 0x7f32b8101a60 2024-03-22T07:20:22.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.014+0000 7f32b57fa640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32a806d2f0 0x7f32a806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:22.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.014+0000 7f32b57fa640 1 -- 172.21.15.81:0/2299197173 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f32a408c030 con 0x7f32b8101a60 2024-03-22T07:20:22.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.014+0000 7f32bcbb0640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32a806d2f0 0x7f32a806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:22.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.015+0000 7f32bcbb0640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32a806d2f0 0x7f32a806f7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32ac000c00 tx=0x7f32ac009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:22.019 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.019+0000 7f32b57fa640 1 -- 172.21.15.81:0/2299197173 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32a4055b90 con 0x7f32b8101a60 2024-03-22T07:20:22.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.226+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32b804bcc0 con 0x7f32b8101a60 2024-03-22T07:20:22.227 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.227+0000 7f32b57fa640 1 -- 172.21.15.81:0/2299197173 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5018 (secure 0 0 0) 0x7f32a4059250 con 0x7f32b8101a60 2024-03-22T07:20:22.228 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:22.228 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:18:58.775150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34378,"mds_1":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34378":{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":0,"incarnation":34,"state":"up:active","state_seq":20,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":1,"incarnation":34,"state":"up:active","state_seq":15,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:22.228 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 41 2024-03-22T07:20:22.229 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32a806d2f0 msgr2=0x7f32a806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32a806d2f0 0x7f32a806f7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32ac000c00 tx=0x7f32ac009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b8101a60 msgr2=0x7f32b810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b8101a60 0x7f32b810e3a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f32a4002a50 tx=0x7f32a4004600 comp rx=0 tx=0).stop 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 shutdown_connections 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f32a806d2f0 0x7f32a806f7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32b8101a60 0x7f32b810e3a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 --2- 172.21.15.81:0/2299197173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32b81010c0 0x7f32b810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 >> 172.21.15.81:0/2299197173 conn(0x7f32b80f6c70 msgr2=0x7f32b80f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 shutdown_connections 2024-03-22T07:20:22.230 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.229+0000 7f32bee14640 1 -- 172.21.15.81:0/2299197173 wait complete. 2024-03-22T07:20:22.240 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:22 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/476709415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:20:22.240 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:22 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3931371556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:20:22.240 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:22 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2306701396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:20:22.240 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:22 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1822588630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:20:22.241 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 18} 2024-03-22T07:20:22.241 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-03-22T07:20:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.339+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/3701664053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 msgr2=0x7f6f9c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.339+0000 7f6fa2f5d640 1 --2- 172.21.15.81:0/3701664053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 0x7f6f9c101510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0048d0 tx=0x7f6f8c009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/3701664053 shutdown_connections 2024-03-22T07:20:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 --2- 172.21.15.81:0/3701664053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f9c0fb110 0x7f6f9c0fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 --2- 172.21.15.81:0/3701664053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 0x7f6f9c101510 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/3701664053 >> 172.21.15.81:0/3701664053 conn(0x7f6f9c0710b0 msgr2=0x7f6f9c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/3701664053 shutdown_connections 2024-03-22T07:20:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/3701664053 wait complete. 2024-03-22T07:20:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 Processor -- start 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 -- start start 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f9c0fb110 0x7f6f9c112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 0x7f6f9c1131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c1137f0 con 0x7f6f9c101140 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6fa2f5d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c10ce40 con 0x7f6f9c0fb110 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 0x7f6f9c1131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.340+0000 7f6f9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 0x7f6f9c1131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55564/0 (socket says 172.21.15.81:55564) 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6f9bfff640 1 -- 172.21.15.81:0/2915367306 learned_addr learned my addr 172.21.15.81:0/2915367306 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6fa0cf9640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f9c0fb110 0x7f6f9c112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6f9bfff640 1 -- 172.21.15.81:0/2915367306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f9c0fb110 msgr2=0x7f6f9c112cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6f9bfff640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f9c0fb110 0x7f6f9c112cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6f9bfff640 1 -- 172.21.15.81:0/2915367306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8c005040 con 0x7f6f9c101140 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6f9bfff640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 0x7f6f9c1131f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000eb70 tx=0x7f6f9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6f997fa640 1 -- 172.21.15.81:0/2915367306 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f90007c30 con 0x7f6f9c101140 2024-03-22T07:20:22.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c10d0c0 con 0x7f6f9c101140 2024-03-22T07:20:22.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6f997fa640 1 -- 172.21.15.81:0/2915367306 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f9000f040 con 0x7f6f9c101140 2024-03-22T07:20:22.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6f997fa640 1 -- 172.21.15.81:0/2915367306 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f900095b0 con 0x7f6f9c101140 2024-03-22T07:20:22.344 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.341+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c10d5e0 con 0x7f6f9c101140 2024-03-22T07:20:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.342+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c0fc800 con 0x7f6f9c101140 2024-03-22T07:20:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.342+0000 7f6f997fa640 1 -- 172.21.15.81:0/2915367306 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f6f90009730 con 0x7f6f9c101140 2024-03-22T07:20:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.343+0000 7f6f997fa640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6f8406d2f0 0x7f6f8406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.343+0000 7f6f997fa640 1 -- 172.21.15.81:0/2915367306 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6f9008c6f0 con 0x7f6f9c101140 2024-03-22T07:20:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.343+0000 7f6fa0cf9640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6f8406d2f0 0x7f6f8406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.343+0000 7f6fa0cf9640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6f8406d2f0 0x7f6f8406f7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009990 tx=0x7f6f8c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:22.348 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.347+0000 7f6f997fa640 1 -- 172.21.15.81:0/2915367306 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6f90056120 con 0x7f6f9c101140 2024-03-22T07:20:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:22 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/476709415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:20:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:22 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3931371556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:20:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:22 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2306701396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:20:22.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:22 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1822588630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:20:22.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.556+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f6f9c108220 con 0x7f6f9c101140 2024-03-22T07:20:22.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.556+0000 7f6f997fa640 1 -- 172.21.15.81:0/2915367306 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v41) v1 ==== 107+0+4844 (secure 0 0 0) 0x7f6f900597e0 con 0x7f6f9c101140 2024-03-22T07:20:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:22.557 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14496,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/4012792796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":4012792796},{"type":"v1","addr":"172.21.15.81:6827","nonce":4012792796}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:08:18.404040+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi081.zfgxzy","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.81:6829/268578316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":268578316},{"type":"v1","addr":"172.21.15.81:6829","nonce":268578316}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24251":{"gid":24251,"name":"cephfs.smithi149.kdtxfw","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6825/3521694486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3521694486},{"type":"v1","addr":"172.21.15.149:6825","nonce":3521694486}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:22.557 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 17 2024-03-22T07:20:22.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6f8406d2f0 msgr2=0x7f6f8406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6f8406d2f0 0x7f6f8406f7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009990 tx=0x7f6f8c009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 msgr2=0x7f6f9c1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 0x7f6f9c1131f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000eb70 tx=0x7f6f9000b040 comp rx=0 tx=0).stop 2024-03-22T07:20:22.559 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 shutdown_connections 2024-03-22T07:20:22.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6f8406d2f0 0x7f6f8406f7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f9c101140 0x7f6f9c1131f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 --2- 172.21.15.81:0/2915367306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f9c0fb110 0x7f6f9c112cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 >> 172.21.15.81:0/2915367306 conn(0x7f6f9c0710b0 msgr2=0x7f6f9c0f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:22.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 shutdown_connections 2024-03-22T07:20:22.560 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.559+0000 7f6fa2f5d640 1 -- 172.21.15.81:0/2915367306 wait complete. 2024-03-22T07:20:22.570 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-22T07:20:22.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 -- 172.21.15.81:0/4028245566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac54101c60 msgr2=0x7fac540fb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.670 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 --2- 172.21.15.81:0/4028245566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac54101c60 0x7fac540fb2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac480048d0 tx=0x7fac48009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 -- 172.21.15.81:0/4028245566 shutdown_connections 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 --2- 172.21.15.81:0/4028245566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac54101c60 0x7fac540fb2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 --2- 172.21.15.81:0/4028245566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac541012c0 0x7fac54101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 -- 172.21.15.81:0/4028245566 >> 172.21.15.81:0/4028245566 conn(0x7fac540f6eb0 msgr2=0x7fac540f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 -- 172.21.15.81:0/4028245566 shutdown_connections 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 -- 172.21.15.81:0/4028245566 wait complete. 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 Processor -- start 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.668+0000 7fac5a425640 1 -- start start 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac5a425640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac541012c0 0x7fac5410e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac5a425640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54101c60 0x7fac5410e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac5a425640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5410ebd0 con 0x7fac541012c0 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac5a425640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5410ed40 con 0x7fac54101c60 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac53fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac541012c0 0x7fac5410e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:22.671 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac53fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac541012c0 0x7fac5410e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55578/0 (socket says 172.21.15.81:55578) 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac53fff640 1 -- 172.21.15.81:0/1098068559 learned_addr learned my addr 172.21.15.81:0/1098068559 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac537fe640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54101c60 0x7fac5410e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac53fff640 1 -- 172.21.15.81:0/1098068559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54101c60 msgr2=0x7fac5410e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac53fff640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54101c60 0x7fac5410e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac53fff640 1 -- 172.21.15.81:0/1098068559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac48005040 con 0x7fac541012c0 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac537fe640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54101c60 0x7fac5410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac53fff640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac541012c0 0x7fac5410e150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b790 tx=0x7fac440091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac50ff9640 1 -- 172.21.15.81:0/1098068559 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac4400c700 con 0x7fac541012c0 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac50ff9640 1 -- 172.21.15.81:0/1098068559 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac4400cd30 con 0x7fac541012c0 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac541171c0 con 0x7fac541012c0 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac50ff9640 1 -- 172.21.15.81:0/1098068559 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac44013370 con 0x7fac541012c0 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.669+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac541174f0 con 0x7fac541012c0 2024-03-22T07:20:22.672 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.670+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac540fca30 con 0x7fac541012c0 2024-03-22T07:20:22.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.671+0000 7fac50ff9640 1 -- 172.21.15.81:0/1098068559 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fac44012070 con 0x7fac541012c0 2024-03-22T07:20:22.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.671+0000 7fac50ff9640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fac3c06d2f0 0x7fac3c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:22.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.671+0000 7fac50ff9640 1 -- 172.21.15.81:0/1098068559 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fac4408c5e0 con 0x7fac541012c0 2024-03-22T07:20:22.673 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.671+0000 7fac537fe640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fac3c06d2f0 0x7fac3c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:22.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.672+0000 7fac537fe640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fac3c06d2f0 0x7fac3c06f7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fac54065dc0 tx=0x7fac48009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:22.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.676+0000 7fac50ff9640 1 -- 172.21.15.81:0/1098068559 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac44056010 con 0x7fac541012c0 2024-03-22T07:20:22.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.884+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7fac5404bcc0 con 0x7fac541012c0 2024-03-22T07:20:22.885 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.885+0000 7fac50ff9640 1 -- 172.21.15.81:0/1098068559 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v41) v1 ==== 107+0+4845 (secure 0 0 0) 0x7fac440596d0 con 0x7fac541012c0 2024-03-22T07:20:22.886 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:22.886 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14496,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/4012792796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":4012792796},{"type":"v1","addr":"172.21.15.81:6827","nonce":4012792796}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:09:08.725410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi081.zfgxzy","rank":0,"incarnation":12,"state":"up:active","state_seq":6,"addr":"172.21.15.81:6829/268578316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":268578316},{"type":"v1","addr":"172.21.15.81:6829","nonce":268578316}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24251":{"gid":24251,"name":"cephfs.smithi149.kdtxfw","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.149:6825/3521694486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3521694486},{"type":"v1","addr":"172.21.15.149:6825","nonce":3521694486}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:22.886 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 18 2024-03-22T07:20:22.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fac3c06d2f0 msgr2=0x7fac3c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fac3c06d2f0 0x7fac3c06f7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fac54065dc0 tx=0x7fac48009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac541012c0 msgr2=0x7fac5410e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac541012c0 0x7fac5410e150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac4400b790 tx=0x7fac440091d0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 shutdown_connections 2024-03-22T07:20:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fac3c06d2f0 0x7fac3c06f7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac54101c60 0x7fac5410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 --2- 172.21.15.81:0/1098068559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac541012c0 0x7fac5410e150 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 >> 172.21.15.81:0/1098068559 conn(0x7fac540f6eb0 msgr2=0x7fac540f7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 shutdown_connections 2024-03-22T07:20:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.887+0000 7fac5a425640 1 -- 172.21.15.81:0/1098068559 wait complete. 2024-03-22T07:20:22.899 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-22T07:20:22.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 -- 172.21.15.81:0/1456470928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc1012e0 msgr2=0x7f46fc1016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:22.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 --2- 172.21.15.81:0/1456470928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc1012e0 0x7f46fc1016b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0048d0 tx=0x7f46ec009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:22.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 -- 172.21.15.81:0/1456470928 shutdown_connections 2024-03-22T07:20:22.999 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 --2- 172.21.15.81:0/1456470928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc101c80 0x7f46fc0fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 --2- 172.21.15.81:0/1456470928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc1012e0 0x7f46fc1016b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 -- 172.21.15.81:0/1456470928 >> 172.21.15.81:0/1456470928 conn(0x7f46fc0f6e70 msgr2=0x7f46fc0f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 -- 172.21.15.81:0/1456470928 shutdown_connections 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 -- 172.21.15.81:0/1456470928 wait complete. 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 Processor -- start 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.997+0000 7f4702658640 1 -- start start 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f4702658640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc1012e0 0x7f46fc111df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f4702658640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc101c80 0x7f46fc112330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f4702658640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc112930 con 0x7f46fc101c80 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f4702658640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc112aa0 con 0x7f46fc1012e0 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46fb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc101c80 0x7f46fc112330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46fb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc101c80 0x7f46fc112330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55602/0 (socket says 172.21.15.81:55602) 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46fb7fe640 1 -- 172.21.15.81:0/2712456152 learned_addr learned my addr 172.21.15.81:0/2712456152 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46fbfff640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc1012e0 0x7f46fc111df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46fb7fe640 1 -- 172.21.15.81:0/2712456152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc1012e0 msgr2=0x7f46fc111df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46fb7fe640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc1012e0 0x7f46fc111df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46fb7fe640 1 -- 172.21.15.81:0/2712456152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec005040 con 0x7f46fc101c80 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46fb7fe640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc101c80 0x7f46fc112330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c790 tx=0x7f46f00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46f8ff9640 1 -- 172.21.15.81:0/2712456152 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f000a820 con 0x7f46fc101c80 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46f8ff9640 1 -- 172.21.15.81:0/2712456152 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46f000ae50 con 0x7f46fc101c80 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc10c130 con 0x7f46fc101c80 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f46f8ff9640 1 -- 172.21.15.81:0/2712456152 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f00115e0 con 0x7f46fc101c80 2024-03-22T07:20:23.001 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.998+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc10c650 con 0x7f46fc101c80 2024-03-22T07:20:23.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:22.999+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc0fca50 con 0x7f46fc101c80 2024-03-22T07:20:23.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.000+0000 7f46f8ff9640 1 -- 172.21.15.81:0/2712456152 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f46f0012400 con 0x7f46fc101c80 2024-03-22T07:20:23.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.000+0000 7f46f8ff9640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f46e406d2f0 0x7f46e406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.002 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.000+0000 7f46f8ff9640 1 -- 172.21.15.81:0/2712456152 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f46f008cae0 con 0x7f46fc101c80 2024-03-22T07:20:23.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.000+0000 7f46fbfff640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f46e406d2f0 0x7f46e406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.001+0000 7f46fbfff640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f46e406d2f0 0x7f46e406f7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46ec000c00 tx=0x7f46ec009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:23.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.005+0000 7f46f8ff9640 1 -- 172.21.15.81:0/2712456152 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f46f0056590 con 0x7f46fc101c80 2024-03-22T07:20:23.213 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.213+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f46fc04bcc0 con 0x7f46fc101c80 2024-03-22T07:20:23.214 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.214+0000 7f46f8ff9640 1 -- 172.21.15.81:0/2712456152 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v41) v1 ==== 107+0+4847 (secure 0 0 0) 0x7f46f0054e60 con 0x7f46fc101c80 2024-03-22T07:20:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:23.215 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14496,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/4012792796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":4012792796},{"type":"v1","addr":"172.21.15.81:6827","nonce":4012792796}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:11:44.962347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi081.zfgxzy","rank":0,"incarnation":12,"state":"up:active","state_seq":82,"addr":"172.21.15.81:6829/268578316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":268578316},{"type":"v1","addr":"172.21.15.81:6829","nonce":268578316}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24251":{"gid":24251,"name":"cephfs.smithi149.kdtxfw","rank":1,"incarnation":7,"state":"up:active","state_seq":81,"addr":"172.21.15.149:6825/3521694486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3521694486},{"type":"v1","addr":"172.21.15.149:6825","nonce":3521694486}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:23.215 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 19 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f46e406d2f0 msgr2=0x7f46e406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f46e406d2f0 0x7f46e406f7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46ec000c00 tx=0x7f46ec009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc101c80 msgr2=0x7f46fc112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc101c80 0x7f46fc112330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c790 tx=0x7f46f00043c0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 shutdown_connections 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f46e406d2f0 0x7f46e406f7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46fc101c80 0x7f46fc112330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 --2- 172.21.15.81:0/2712456152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46fc1012e0 0x7f46fc111df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 >> 172.21.15.81:0/2712456152 conn(0x7f46fc0f6e70 msgr2=0x7f46fc0f7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:23.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 shutdown_connections 2024-03-22T07:20:23.218 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.216+0000 7f4702658640 1 -- 172.21.15.81:0/2712456152 wait complete. 2024-03-22T07:20:23.226 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:23 smithi081 ceph-mon[113965]: pgmap v267: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:23.226 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:23 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2299197173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:20:23.226 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:23 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2915367306' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-22T07:20:23.226 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:23 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1098068559' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-22T07:20:23.227 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-22T07:20:23.327 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 -- 172.21.15.81:0/2080986733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f84101c80 msgr2=0x7f8f840fb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 --2- 172.21.15.81:0/2080986733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f84101c80 0x7f8f840fb310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f740048d0 tx=0x7f8f74009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 -- 172.21.15.81:0/2080986733 shutdown_connections 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 --2- 172.21.15.81:0/2080986733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f84101c80 0x7f8f840fb310 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 --2- 172.21.15.81:0/2080986733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f841012e0 0x7f8f841016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 -- 172.21.15.81:0/2080986733 >> 172.21.15.81:0/2080986733 conn(0x7f8f840f6e70 msgr2=0x7f8f840f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 -- 172.21.15.81:0/2080986733 shutdown_connections 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 -- 172.21.15.81:0/2080986733 wait complete. 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 Processor -- start 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 -- start start 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f841012e0 0x7f8f8410e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f84101c80 0x7f8f8410e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.328 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8410ec30 con 0x7f8f841012e0 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.325+0000 7f8f890ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8410eda0 con 0x7f8f84101c80 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f82d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f841012e0 0x7f8f8410e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f82d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f841012e0 0x7f8f8410e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55634/0 (socket says 172.21.15.81:55634) 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f82d9d640 1 -- 172.21.15.81:0/3792994987 learned_addr learned my addr 172.21.15.81:0/3792994987 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f8259c640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f84101c80 0x7f8f8410e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f82d9d640 1 -- 172.21.15.81:0/3792994987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f84101c80 msgr2=0x7f8f8410e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f82d9d640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f84101c80 0x7f8f8410e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f82d9d640 1 -- 172.21.15.81:0/3792994987 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74005040 con 0x7f8f841012e0 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f8259c640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f84101c80 0x7f8f8410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f82d9d640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f841012e0 0x7f8f8410e1b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800b790 tx=0x7f8f780091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f6b7fe640 1 -- 172.21.15.81:0/3792994987 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f7800c700 con 0x7f8f841012e0 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f84117220 con 0x7f8f841012e0 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f6b7fe640 1 -- 172.21.15.81:0/3792994987 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f7800cd30 con 0x7f8f841012e0 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f6b7fe640 1 -- 172.21.15.81:0/3792994987 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f78013370 con 0x7f8f841012e0 2024-03-22T07:20:23.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.326+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f84117550 con 0x7f8f841012e0 2024-03-22T07:20:23.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.327+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f840fca50 con 0x7f8f841012e0 2024-03-22T07:20:23.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.328+0000 7f8f6b7fe640 1 -- 172.21.15.81:0/3792994987 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f8f78012070 con 0x7f8f841012e0 2024-03-22T07:20:23.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.328+0000 7f8f6b7fe640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8f6c06d2f0 0x7f8f6c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.328+0000 7f8f6b7fe640 1 -- 172.21.15.81:0/3792994987 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8f7808c5b0 con 0x7f8f841012e0 2024-03-22T07:20:23.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.328+0000 7f8f8259c640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8f6c06d2f0 0x7f8f6c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.329+0000 7f8f8259c640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8f6c06d2f0 0x7f8f6c06f7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f84065d40 tx=0x7f8f74009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:23.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.333+0000 7f8f6b7fe640 1 -- 172.21.15.81:0/3792994987 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f78055fe0 con 0x7f8f841012e0 2024-03-22T07:20:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:23 smithi149 ceph-mon[83447]: pgmap v267: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:23 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2299197173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:20:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:23 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2915367306' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-03-22T07:20:23.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:23 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1098068559' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-22T07:20:23.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.540+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f8f8410ef60 con 0x7f8f841012e0 2024-03-22T07:20:23.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.541+0000 7f8f6b7fe640 1 -- 172.21.15.81:0/3792994987 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v41) v1 ==== 107+0+4846 (secure 0 0 0) 0x7f8f8410ef60 con 0x7f8f841012e0 2024-03-22T07:20:23.542 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:23.542 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14496,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/4012792796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":4012792796},{"type":"v1","addr":"172.21.15.81:6827","nonce":4012792796}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:11:58.384645+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi081.zfgxzy","rank":0,"incarnation":12,"state":"up:active","state_seq":82,"addr":"172.21.15.81:6829/268578316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":268578316},{"type":"v1","addr":"172.21.15.81:6829","nonce":268578316}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24251":{"gid":24251,"name":"cephfs.smithi149.kdtxfw","rank":1,"incarnation":7,"state":"up:active","state_seq":81,"addr":"172.21.15.149:6825/3521694486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3521694486},{"type":"v1","addr":"172.21.15.149:6825","nonce":3521694486}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:23.542 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 20 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8f6c06d2f0 msgr2=0x7f8f6c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8f6c06d2f0 0x7f8f6c06f7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f84065d40 tx=0x7f8f74009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f841012e0 msgr2=0x7f8f8410e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f841012e0 0x7f8f8410e1b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800b790 tx=0x7f8f780091d0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 shutdown_connections 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f8f6c06d2f0 0x7f8f6c06f7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f84101c80 0x7f8f8410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 --2- 172.21.15.81:0/3792994987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f841012e0 0x7f8f8410e1b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 >> 172.21.15.81:0/3792994987 conn(0x7f8f840f6e70 msgr2=0x7f8f840f7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 shutdown_connections 2024-03-22T07:20:23.544 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.543+0000 7f8f890ae640 1 -- 172.21.15.81:0/3792994987 wait complete. 2024-03-22T07:20:23.554 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-22T07:20:23.654 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.652+0000 7f769a2ae640 1 -- 172.21.15.81:0/472952398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fdd10 msgr2=0x7f76941063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.652+0000 7f769a2ae640 1 --2- 172.21.15.81:0/472952398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fdd10 0x7f76941063e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f76880048d0 tx=0x7f7688009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.652+0000 7f769a2ae640 1 -- 172.21.15.81:0/472952398 shutdown_connections 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.652+0000 7f769a2ae640 1 --2- 172.21.15.81:0/472952398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fdd10 0x7f76941063e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.652+0000 7f769a2ae640 1 --2- 172.21.15.81:0/472952398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76940fd400 0x7f76940fd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.652+0000 7f769a2ae640 1 -- 172.21.15.81:0/472952398 >> 172.21.15.81:0/472952398 conn(0x7f76940f6eb0 msgr2=0x7f76940f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.652+0000 7f769a2ae640 1 -- 172.21.15.81:0/472952398 shutdown_connections 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.652+0000 7f769a2ae640 1 -- 172.21.15.81:0/472952398 wait complete. 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f769a2ae640 1 Processor -- start 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f769a2ae640 1 -- start start 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f769a2ae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fd400 0x7f7694112fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f769a2ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76940fdd10 0x7f76941134e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.655 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f769a2ae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7694113ae0 con 0x7f76940fd400 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f769a2ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769410d130 con 0x7f76940fdd10 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f7693fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fd400 0x7f7694112fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f7693fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fd400 0x7f7694112fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55656/0 (socket says 172.21.15.81:55656) 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f7693fff640 1 -- 172.21.15.81:0/3259129351 learned_addr learned my addr 172.21.15.81:0/3259129351 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.653+0000 7f76937fe640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76940fdd10 0x7f76941134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f7693fff640 1 -- 172.21.15.81:0/3259129351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76940fdd10 msgr2=0x7f76941134e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f7693fff640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76940fdd10 0x7f76941134e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f7693fff640 1 -- 172.21.15.81:0/3259129351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7688005040 con 0x7f76940fd400 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f76937fe640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76940fdd10 0x7f76941134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f7693fff640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fd400 0x7f7694112fa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f768400c790 tx=0x7f76840043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f7690ff9640 1 -- 172.21.15.81:0/3259129351 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f768400a820 con 0x7f76940fd400 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769410d410 con 0x7f76940fd400 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f7690ff9640 1 -- 172.21.15.81:0/3259129351 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f768400ae50 con 0x7f76940fd400 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f7690ff9640 1 -- 172.21.15.81:0/3259129351 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76840115e0 con 0x7f76940fd400 2024-03-22T07:20:23.656 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.654+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769410d930 con 0x7f76940fd400 2024-03-22T07:20:23.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.655+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7694065e50 con 0x7f76940fd400 2024-03-22T07:20:23.657 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.655+0000 7f7690ff9640 1 -- 172.21.15.81:0/3259129351 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f7684012400 con 0x7f76940fd400 2024-03-22T07:20:23.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.656+0000 7f7690ff9640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f767c06d2f0 0x7f767c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.656+0000 7f76937fe640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f767c06d2f0 0x7f767c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.656+0000 7f7690ff9640 1 -- 172.21.15.81:0/3259129351 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f768408cae0 con 0x7f76940fd400 2024-03-22T07:20:23.658 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.656+0000 7f76937fe640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f767c06d2f0 0x7f767c06f7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f769410e860 tx=0x7f7688009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:23.661 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.660+0000 7f7690ff9640 1 -- 172.21.15.81:0/3259129351 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7684056590 con 0x7f76940fd400 2024-03-22T07:20:23.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.870+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f76940638c0 con 0x7f76940fd400 2024-03-22T07:20:23.871 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.871+0000 7f7690ff9640 1 -- 172.21.15.81:0/3259129351 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v41) v1 ==== 107+0+4845 (secure 0 0 0) 0x7f7684054e60 con 0x7f76940fd400 2024-03-22T07:20:23.871 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:23.872 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14496,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/4012792796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":4012792796},{"type":"v1","addr":"172.21.15.81:6827","nonce":4012792796}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:12:23.482585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi081.zfgxzy","rank":0,"incarnation":12,"state":"up:active","state_seq":82,"addr":"172.21.15.81:6829/268578316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":268578316},{"type":"v1","addr":"172.21.15.81:6829","nonce":268578316}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24251":{"gid":24251,"name":"cephfs.smithi149.kdtxfw","rank":1,"incarnation":7,"state":"up:active","state_seq":81,"addr":"172.21.15.149:6825/3521694486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3521694486},{"type":"v1","addr":"172.21.15.149:6825","nonce":3521694486}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:23.872 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 21 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f767c06d2f0 msgr2=0x7f767c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f767c06d2f0 0x7f767c06f7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f769410e860 tx=0x7f7688009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fd400 msgr2=0x7f7694112fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fd400 0x7f7694112fa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f768400c790 tx=0x7f76840043c0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 shutdown_connections 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f767c06d2f0 0x7f767c06f7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76940fdd10 0x7f76941134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 --2- 172.21.15.81:0/3259129351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76940fd400 0x7f7694112fa0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 >> 172.21.15.81:0/3259129351 conn(0x7f76940f6eb0 msgr2=0x7f76940f8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:23.874 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 shutdown_connections 2024-03-22T07:20:23.875 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.873+0000 7f769a2ae640 1 -- 172.21.15.81:0/3259129351 wait complete. 2024-03-22T07:20:23.885 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-22T07:20:23.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 -- 172.21.15.81:0/4049991686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c4102b50 msgr2=0x7f23c40fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 --2- 172.21.15.81:0/4049991686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c4102b50 0x7f23c40fc1e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0048d0 tx=0x7f23ac009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.985 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 -- 172.21.15.81:0/4049991686 shutdown_connections 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 --2- 172.21.15.81:0/4049991686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c4102b50 0x7f23c40fc1e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 --2- 172.21.15.81:0/4049991686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c41021b0 0x7f23c4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 -- 172.21.15.81:0/4049991686 >> 172.21.15.81:0/4049991686 conn(0x7f23c40f7dc0 msgr2=0x7f23c40fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 -- 172.21.15.81:0/4049991686 shutdown_connections 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 -- 172.21.15.81:0/4049991686 wait complete. 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 Processor -- start 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.983+0000 7f23ca22e640 1 -- start start 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23ca22e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c41021b0 0x7f23c4112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23ca22e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4102b50 0x7f23c4113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23ca22e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c4113810 con 0x7f23c41021b0 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23ca22e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c410ce60 con 0x7f23c4102b50 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c41021b0 0x7f23c4112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.986 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23c37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c41021b0 0x7f23c4112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55676/0 (socket says 172.21.15.81:55676) 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23c37fe640 1 -- 172.21.15.81:0/3168778116 learned_addr learned my addr 172.21.15.81:0/3168778116 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23c2ffd640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4102b50 0x7f23c4113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23c37fe640 1 -- 172.21.15.81:0/3168778116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4102b50 msgr2=0x7f23c4113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23c37fe640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4102b50 0x7f23c4113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23c37fe640 1 -- 172.21.15.81:0/3168778116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac005040 con 0x7f23c41021b0 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23c37fe640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c41021b0 0x7f23c4112cd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23b400cae0 tx=0x7f23b4007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23bbfff640 1 -- 172.21.15.81:0/3168778116 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b400ab00 con 0x7f23c41021b0 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23bbfff640 1 -- 172.21.15.81:0/3168778116 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23b400ac60 con 0x7f23c41021b0 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c410d140 con 0x7f23c41021b0 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23bbfff640 1 -- 172.21.15.81:0/3168778116 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b40116f0 con 0x7f23c41021b0 2024-03-22T07:20:23.987 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.984+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c410d630 con 0x7f23c41021b0 2024-03-22T07:20:23.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.985+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c40fd920 con 0x7f23c41021b0 2024-03-22T07:20:23.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.986+0000 7f23bbfff640 1 -- 172.21.15.81:0/3168778116 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f23b4012440 con 0x7f23c41021b0 2024-03-22T07:20:23.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.986+0000 7f23bbfff640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f23b006d2f0 0x7f23b006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:23.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.986+0000 7f23c2ffd640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f23b006d2f0 0x7f23b006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:23.988 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.986+0000 7f23bbfff640 1 -- 172.21.15.81:0/3168778116 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f23b408d290 con 0x7f23c41021b0 2024-03-22T07:20:23.989 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.987+0000 7f23c2ffd640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f23b006d2f0 0x7f23b006f7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23c40fa830 tx=0x7f23ac009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:23.991 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:23.991+0000 7f23bbfff640 1 -- 172.21.15.81:0/3168778116 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23b4056cc0 con 0x7f23c41021b0 2024-03-22T07:20:24.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.201+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f23c410e340 con 0x7f23c41021b0 2024-03-22T07:20:24.202 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.202+0000 7f23bbfff640 1 -- 172.21.15.81:0/3168778116 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v41) v1 ==== 107+0+4845 (secure 0 0 0) 0x7f23b405a380 con 0x7f23c41021b0 2024-03-22T07:20:24.203 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:24.203 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14496,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/4012792796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":4012792796},{"type":"v1","addr":"172.21.15.81:6827","nonce":4012792796}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:12:38.916315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi081.zfgxzy","rank":0,"incarnation":12,"state":"up:active","state_seq":95,"addr":"172.21.15.81:6829/268578316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":268578316},{"type":"v1","addr":"172.21.15.81:6829","nonce":268578316}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24251":{"gid":24251,"name":"cephfs.smithi149.kdtxfw","rank":1,"incarnation":7,"state":"up:active","state_seq":94,"addr":"172.21.15.149:6825/3521694486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3521694486},{"type":"v1","addr":"172.21.15.149:6825","nonce":3521694486}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:24.203 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 22 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f23b006d2f0 msgr2=0x7f23b006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f23b006d2f0 0x7f23b006f7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23c40fa830 tx=0x7f23ac009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c41021b0 msgr2=0x7f23c4112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c41021b0 0x7f23c4112cd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23b400cae0 tx=0x7f23b4007e50 comp rx=0 tx=0).stop 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 shutdown_connections 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f23b006d2f0 0x7f23b006f7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4102b50 0x7f23c4113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 --2- 172.21.15.81:0/3168778116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23c41021b0 0x7f23c4112cd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 >> 172.21.15.81:0/3168778116 conn(0x7f23c40f7dc0 msgr2=0x7f23c40f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 shutdown_connections 2024-03-22T07:20:24.205 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.204+0000 7f23ca22e640 1 -- 172.21.15.81:0/3168778116 wait complete. 2024-03-22T07:20:24.214 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:24 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2712456152' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-22T07:20:24.214 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:24 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3792994987' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-22T07:20:24.214 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:24 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3259129351' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-22T07:20:24.216 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-22T07:20:24.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.313+0000 7f3f002ae640 1 -- 172.21.15.81:0/3735865336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef80fa880 msgr2=0x7f3ef80facd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.313+0000 7f3f002ae640 1 --2- 172.21.15.81:0/3735865336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef80fa880 0x7f3ef80facd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80048d0 tx=0x7f3ee8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.313+0000 7f3f002ae640 1 -- 172.21.15.81:0/3735865336 shutdown_connections 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.313+0000 7f3f002ae640 1 --2- 172.21.15.81:0/3735865336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef80fa880 0x7f3ef80facd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.313+0000 7f3f002ae640 1 --2- 172.21.15.81:0/3735865336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef8102250 0x7f3ef8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.313+0000 7f3f002ae640 1 -- 172.21.15.81:0/3735865336 >> 172.21.15.81:0/3735865336 conn(0x7f3ef8070880 msgr2=0x7f3ef8070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.313+0000 7f3f002ae640 1 -- 172.21.15.81:0/3735865336 shutdown_connections 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.313+0000 7f3f002ae640 1 -- 172.21.15.81:0/3735865336 wait complete. 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3f002ae640 1 Processor -- start 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3f002ae640 1 -- start start 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3f002ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef80fa880 0x7f3ef8112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3f002ae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef8102250 0x7f3ef81132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3f002ae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef8113950 con 0x7f3ef8102250 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3f002ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef810cf10 con 0x7f3ef80fa880 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3efd849640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef8102250 0x7f3ef81132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:24.317 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3efd849640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef8102250 0x7f3ef81132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55696/0 (socket says 172.21.15.81:55696) 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3efd849640 1 -- 172.21.15.81:0/1709076825 learned_addr learned my addr 172.21.15.81:0/1709076825 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3efe04a640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef80fa880 0x7f3ef8112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3efd849640 1 -- 172.21.15.81:0/1709076825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef80fa880 msgr2=0x7f3ef8112d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3efd849640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef80fa880 0x7f3ef8112d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3efd849640 1 -- 172.21.15.81:0/1709076825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8005040 con 0x7f3ef8102250 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.314+0000 7f3efe04a640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef80fa880 0x7f3ef8112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.315+0000 7f3efd849640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef8102250 0x7f3ef81132c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80047f0 tx=0x7f3ee8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.315+0000 7f3eeeffd640 1 -- 172.21.15.81:0/1709076825 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ee80094c0 con 0x7f3ef8102250 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.315+0000 7f3eeeffd640 1 -- 172.21.15.81:0/1709076825 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ee8032e50 con 0x7f3ef8102250 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.315+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef810d190 con 0x7f3ef8102250 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.315+0000 7f3eeeffd640 1 -- 172.21.15.81:0/1709076825 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ee80435f0 con 0x7f3ef8102250 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.315+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef810d650 con 0x7f3ef8102250 2024-03-22T07:20:24.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.316+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef804bcc0 con 0x7f3ef8102250 2024-03-22T07:20:24.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.316+0000 7f3eeeffd640 1 -- 172.21.15.81:0/1709076825 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f3ee802f070 con 0x7f3ef8102250 2024-03-22T07:20:24.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.317+0000 7f3eeeffd640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3ee406d300 0x7f3ee406f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:24.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.317+0000 7f3efe04a640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3ee406d300 0x7f3ee406f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:24.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.317+0000 7f3eeeffd640 1 -- 172.21.15.81:0/1709076825 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3ee80ae760 con 0x7f3ef8102250 2024-03-22T07:20:24.319 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.317+0000 7f3efe04a640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3ee406d300 0x7f3ee406f7b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0007fd0 tx=0x7f3ef00074f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:24.321 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.321+0000 7f3eeeffd640 1 -- 172.21.15.81:0/1709076825 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ee8078190 con 0x7f3ef8102250 2024-03-22T07:20:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:24 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2712456152' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-22T07:20:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:24 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3792994987' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-22T07:20:24.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:24 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3259129351' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-22T07:20:24.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.532+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f3ef80653e0 con 0x7f3ef8102250 2024-03-22T07:20:24.533 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.532+0000 7f3eeeffd640 1 -- 172.21.15.81:0/1709076825 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v41) v1 ==== 107+0+3264 (secure 0 0 0) 0x7f3ee803fcd0 con 0x7f3ef8102250 2024-03-22T07:20:24.533 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:24.533 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14496,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/4012792796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":4012792796},{"type":"v1","addr":"172.21.15.81:6827","nonce":4012792796}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:24.534 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 23 2024-03-22T07:20:24.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3ee406d300 msgr2=0x7f3ee406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.535 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3ee406d300 0x7f3ee406f7b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0007fd0 tx=0x7f3ef00074f0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef8102250 msgr2=0x7f3ef81132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef8102250 0x7f3ef81132c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80047f0 tx=0x7f3ee8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 shutdown_connections 2024-03-22T07:20:24.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f3ee406d300 0x7f3ee406f7b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ef8102250 0x7f3ef81132c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 --2- 172.21.15.81:0/1709076825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ef80fa880 0x7f3ef8112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 >> 172.21.15.81:0/1709076825 conn(0x7f3ef8070880 msgr2=0x7f3ef80f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:24.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 shutdown_connections 2024-03-22T07:20:24.536 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.535+0000 7f3f002ae640 1 -- 172.21.15.81:0/1709076825 wait complete. 2024-03-22T07:20:24.546 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:24.546 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-22T07:20:24.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.644+0000 7f69a3618640 1 -- 172.21.15.81:0/1087623724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c101140 msgr2=0x7f699c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.644+0000 7f69a3618640 1 --2- 172.21.15.81:0/1087623724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c101140 0x7f699c101510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f69980048d0 tx=0x7f6998009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.644+0000 7f69a3618640 1 -- 172.21.15.81:0/1087623724 shutdown_connections 2024-03-22T07:20:24.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.644+0000 7f69a3618640 1 --2- 172.21.15.81:0/1087623724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c0fb110 0x7f699c0fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.644+0000 7f69a3618640 1 --2- 172.21.15.81:0/1087623724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c101140 0x7f699c101510 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.644+0000 7f69a3618640 1 -- 172.21.15.81:0/1087623724 >> 172.21.15.81:0/1087623724 conn(0x7f699c0710b0 msgr2=0x7f699c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:24.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a3618640 1 -- 172.21.15.81:0/1087623724 shutdown_connections 2024-03-22T07:20:24.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a3618640 1 -- 172.21.15.81:0/1087623724 wait complete. 2024-03-22T07:20:24.647 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a3618640 1 Processor -- start 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a3618640 1 -- start start 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a3618640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c0fb110 0x7f699c112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a3618640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c101140 0x7f699c1131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a3618640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c1137f0 con 0x7f699c0fb110 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a3618640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c10ce40 con 0x7f699c101140 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a13b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c0fb110 0x7f699c112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a13b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c0fb110 0x7f699c112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55354/0 (socket says 172.21.15.81:55354) 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a13b4640 1 -- 172.21.15.81:0/4056105376 learned_addr learned my addr 172.21.15.81:0/4056105376 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a0bb3640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c101140 0x7f699c1131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a13b4640 1 -- 172.21.15.81:0/4056105376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c101140 msgr2=0x7f699c1131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a13b4640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c101140 0x7f699c1131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.645+0000 7f69a13b4640 1 -- 172.21.15.81:0/4056105376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6998005040 con 0x7f699c0fb110 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.646+0000 7f69a13b4640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c0fb110 0x7f699c112cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f69980041d0 tx=0x7f6998009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.646+0000 7f6991ffb640 1 -- 172.21.15.81:0/4056105376 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6998030870 con 0x7f699c0fb110 2024-03-22T07:20:24.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.646+0000 7f6991ffb640 1 -- 172.21.15.81:0/4056105376 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69980092e0 con 0x7f699c0fb110 2024-03-22T07:20:24.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.646+0000 7f6991ffb640 1 -- 172.21.15.81:0/4056105376 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6998043400 con 0x7f699c0fb110 2024-03-22T07:20:24.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.646+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699c10d0c0 con 0x7f699c0fb110 2024-03-22T07:20:24.649 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.646+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c10d5e0 con 0x7f699c0fb110 2024-03-22T07:20:24.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.647+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699c0fc800 con 0x7f699c0fb110 2024-03-22T07:20:24.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.647+0000 7f6991ffb640 1 -- 172.21.15.81:0/4056105376 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f699803b420 con 0x7f699c0fb110 2024-03-22T07:20:24.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.648+0000 7f6991ffb640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f698806d2f0 0x7f698806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:24.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.648+0000 7f69a0bb3640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f698806d2f0 0x7f698806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:24.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.648+0000 7f6991ffb640 1 -- 172.21.15.81:0/4056105376 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f69980af7f0 con 0x7f699c0fb110 2024-03-22T07:20:24.650 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.648+0000 7f69a0bb3640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f698806d2f0 0x7f698806f7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f698c007fd0 tx=0x7f698c009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:24.653 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.653+0000 7f6991ffb640 1 -- 172.21.15.81:0/4056105376 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f69980792a0 con 0x7f699c0fb110 2024-03-22T07:20:24.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.858+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f699c10e320 con 0x7f699c0fb110 2024-03-22T07:20:24.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.859+0000 7f6991ffb640 1 -- 172.21.15.81:0/4056105376 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v41) v1 ==== 107+0+4825 (secure 0 0 0) 0x7f699807c960 con 0x7f699c0fb110 2024-03-22T07:20:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:24.860 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14496,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/4012792796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":4012792796},{"type":"v1","addr":"172.21.15.81:6827","nonce":4012792796}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":44243,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/2789599242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2789599242},{"type":"v1","addr":"172.21.15.149:6825","nonce":2789599242}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44245,"name":"cephfs.smithi081.zfgxzy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6829/3973077481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3973077481},{"type":"v1","addr":"172.21.15.81:6829","nonce":3973077481}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:24.860 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 24 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f698806d2f0 msgr2=0x7f698806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f698806d2f0 0x7f698806f7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f698c007fd0 tx=0x7f698c009e90 comp rx=0 tx=0).stop 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c0fb110 msgr2=0x7f699c112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c0fb110 0x7f699c112cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f69980041d0 tx=0x7f6998009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 shutdown_connections 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f698806d2f0 0x7f698806f7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c101140 0x7f699c1131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 --2- 172.21.15.81:0/4056105376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f699c0fb110 0x7f699c112cb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 >> 172.21.15.81:0/4056105376 conn(0x7f699c0710b0 msgr2=0x7f699c0f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 shutdown_connections 2024-03-22T07:20:24.862 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.861+0000 7f69a3618640 1 -- 172.21.15.81:0/4056105376 wait complete. 2024-03-22T07:20:24.873 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:24.873 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-22T07:20:24.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.971+0000 7f680251d640 1 -- 172.21.15.81:0/410546821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc101a90 msgr2=0x7f67fc0fb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.971+0000 7f680251d640 1 --2- 172.21.15.81:0/410546821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc101a90 0x7f67fc0fb120 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67f00048d0 tx=0x7f67f0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.973 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.971+0000 7f680251d640 1 -- 172.21.15.81:0/410546821 shutdown_connections 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.971+0000 7f680251d640 1 --2- 172.21.15.81:0/410546821 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc101a90 0x7f67fc0fb120 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.971+0000 7f680251d640 1 --2- 172.21.15.81:0/410546821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67fc1010f0 0x7f67fc1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.971+0000 7f680251d640 1 -- 172.21.15.81:0/410546821 >> 172.21.15.81:0/410546821 conn(0x7f67fc0f6d00 msgr2=0x7f67fc0f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.971+0000 7f680251d640 1 -- 172.21.15.81:0/410546821 shutdown_connections 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.971+0000 7f680251d640 1 -- 172.21.15.81:0/410546821 wait complete. 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.971+0000 7f680251d640 1 Processor -- start 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f680251d640 1 -- start start 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f680251d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc1010f0 0x7f67fc113030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f680251d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67fc101a90 0x7f67fc113570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f680251d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc113b70 con 0x7f67fc1010f0 2024-03-22T07:20:24.974 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f680251d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc10d1c0 con 0x7f67fc101a90 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67fbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc1010f0 0x7f67fc113030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67fbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc1010f0 0x7f67fc113030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55370/0 (socket says 172.21.15.81:55370) 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67fbfff640 1 -- 172.21.15.81:0/2366728209 learned_addr learned my addr 172.21.15.81:0/2366728209 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67fb7fe640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67fc101a90 0x7f67fc113570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67fbfff640 1 -- 172.21.15.81:0/2366728209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67fc101a90 msgr2=0x7f67fc113570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67fbfff640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67fc101a90 0x7f67fc113570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67fbfff640 1 -- 172.21.15.81:0/2366728209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0005040 con 0x7f67fc1010f0 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67fbfff640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc1010f0 0x7f67fc113030 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f67ec002a50 tx=0x7f67ec004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67f8ff9640 1 -- 172.21.15.81:0/2366728209 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67ec007c30 con 0x7f67fc1010f0 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.972+0000 7f67f8ff9640 1 -- 172.21.15.81:0/2366728209 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67ec00f040 con 0x7f67fc1010f0 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.973+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc10d4a0 con 0x7f67fc1010f0 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.973+0000 7f67f8ff9640 1 -- 172.21.15.81:0/2366728209 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67ec00db50 con 0x7f67fc1010f0 2024-03-22T07:20:24.975 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.973+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc10d990 con 0x7f67fc1010f0 2024-03-22T07:20:24.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.973+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc0fc860 con 0x7f67fc1010f0 2024-03-22T07:20:24.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.974+0000 7f67f8ff9640 1 -- 172.21.15.81:0/2366728209 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f67ec00dcf0 con 0x7f67fc1010f0 2024-03-22T07:20:24.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.974+0000 7f67f8ff9640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67e406d2f0 0x7f67e406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:24.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.974+0000 7f67f8ff9640 1 -- 172.21.15.81:0/2366728209 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f67ec08c110 con 0x7f67fc1010f0 2024-03-22T07:20:24.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.974+0000 7f67fb7fe640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67e406d2f0 0x7f67e406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:24.977 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.975+0000 7f67fb7fe640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67e406d2f0 0x7f67e406f7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0f96b0 tx=0x7f67f0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:24.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:24.979+0000 7f67f8ff9640 1 -- 172.21.15.81:0/2366728209 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67ec055c70 con 0x7f67fc1010f0 2024-03-22T07:20:25.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.188+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f67fc10e6a0 con 0x7f67fc1010f0 2024-03-22T07:20:25.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.189+0000 7f67f8ff9640 1 -- 172.21.15.81:0/2366728209 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v41) v1 ==== 107+0+4046 (secure 0 0 0) 0x7f67ec059330 con 0x7f67fc1010f0 2024-03-22T07:20:25.190 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:25.190 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":44243,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/2789599242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2789599242},{"type":"v1","addr":"172.21.15.149:6825","nonce":2789599242}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44245,"name":"cephfs.smithi081.zfgxzy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6829/3973077481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3973077481},{"type":"v1","addr":"172.21.15.81:6829","nonce":3973077481}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:25.190 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 25 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67e406d2f0 msgr2=0x7f67e406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67e406d2f0 0x7f67e406f7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f67fc0f96b0 tx=0x7f67f0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc1010f0 msgr2=0x7f67fc113030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc1010f0 0x7f67fc113030 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f67ec002a50 tx=0x7f67ec004600 comp rx=0 tx=0).stop 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 shutdown_connections 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67e406d2f0 0x7f67e406f7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67fc101a90 0x7f67fc113570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 --2- 172.21.15.81:0/2366728209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67fc1010f0 0x7f67fc113030 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 >> 172.21.15.81:0/2366728209 conn(0x7f67fc0f6d00 msgr2=0x7f67fc0f8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 shutdown_connections 2024-03-22T07:20:25.192 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.191+0000 7f680251d640 1 -- 172.21.15.81:0/2366728209 wait complete. 2024-03-22T07:20:25.201 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:25 smithi081 ceph-mon[113965]: pgmap v268: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:25.201 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:25 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3168778116' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-22T07:20:25.201 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:25 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1709076825' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-22T07:20:25.201 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:25 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/4056105376' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-22T07:20:25.203 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:25.203 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-22T07:20:25.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.300+0000 7f9f9fe14640 1 -- 172.21.15.81:0/2476199889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 msgr2=0x7f9f980facf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.300+0000 7f9f9fe14640 1 --2- 172.21.15.81:0/2476199889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 0x7f9f980facf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f880049c0 tx=0x7f9f8800aeb0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.300+0000 7f9f9fe14640 1 -- 172.21.15.81:0/2476199889 shutdown_connections 2024-03-22T07:20:25.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.300+0000 7f9f9fe14640 1 --2- 172.21.15.81:0/2476199889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 0x7f9f980facf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.300+0000 7f9f9fe14640 1 --2- 172.21.15.81:0/2476199889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f981008a0 0x7f9f98100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.300+0000 7f9f9fe14640 1 -- 172.21.15.81:0/2476199889 >> 172.21.15.81:0/2476199889 conn(0x7f9f980714a0 msgr2=0x7f9f980f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:25.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.300+0000 7f9f9fe14640 1 -- 172.21.15.81:0/2476199889 shutdown_connections 2024-03-22T07:20:25.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.300+0000 7f9f9fe14640 1 -- 172.21.15.81:0/2476199889 wait complete. 2024-03-22T07:20:25.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.300+0000 7f9f9fe14640 1 Processor -- start 2024-03-22T07:20:25.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9fe14640 1 -- start start 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9fe14640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 0x7f9f98112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9fe14640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f981008a0 0x7f9f98113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9fe14640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98113850 con 0x7f9f980fa880 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9fe14640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9810cea0 con 0x7f9f981008a0 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9dbb0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 0x7f9f98112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9dbb0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 0x7f9f98112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55398/0 (socket says 172.21.15.81:55398) 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9dbb0640 1 -- 172.21.15.81:0/3560341622 learned_addr learned my addr 172.21.15.81:0/3560341622 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9dbb0640 1 -- 172.21.15.81:0/3560341622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f981008a0 msgr2=0x7f9f98113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9dbb0640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f981008a0 0x7f9f98113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9dbb0640 1 -- 172.21.15.81:0/3560341622 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88005040 con 0x7f9f980fa880 2024-03-22T07:20:25.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9dbb0640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 0x7f9f98112d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000db00 tx=0x7f9f90004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f8e7fc640 1 -- 172.21.15.81:0/3560341622 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f90017070 con 0x7f9f980fa880 2024-03-22T07:20:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9810d180 con 0x7f9f980fa880 2024-03-22T07:20:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f8e7fc640 1 -- 172.21.15.81:0/3560341622 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f9000f040 con 0x7f9f980fa880 2024-03-22T07:20:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f8e7fc640 1 -- 172.21.15.81:0/3560341622 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f90003ae0 con 0x7f9f980fa880 2024-03-22T07:20:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.301+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9810d720 con 0x7f9f980fa880 2024-03-22T07:20:25.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.302+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f980fbfc0 con 0x7f9f980fa880 2024-03-22T07:20:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.303+0000 7f9f8e7fc640 1 -- 172.21.15.81:0/3560341622 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f9f90003c40 con 0x7f9f980fa880 2024-03-22T07:20:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.303+0000 7f9f8e7fc640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9f8406d2f0 0x7f9f8406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.303+0000 7f9f8e7fc640 1 -- 172.21.15.81:0/3560341622 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9f9008cae0 con 0x7f9f980fa880 2024-03-22T07:20:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.303+0000 7f9f9d3af640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9f8406d2f0 0x7f9f8406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:25.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.304+0000 7f9f9d3af640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9f8406d2f0 0x7f9f8406f7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800abd0 tx=0x7f9f8800aeb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:25.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.308+0000 7f9f8e7fc640 1 -- 172.21.15.81:0/3560341622 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f90056510 con 0x7f9f980fa880 2024-03-22T07:20:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:25 smithi149 ceph-mon[83447]: pgmap v268: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:25 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3168778116' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-22T07:20:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:25 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1709076825' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-22T07:20:25.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:25 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/4056105376' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-22T07:20:25.516 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.516+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f9f9804bcc0 con 0x7f9f980fa880 2024-03-22T07:20:25.517 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.517+0000 7f9f8e7fc640 1 -- 172.21.15.81:0/3560341622 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v41) v1 ==== 107+0+4856 (secure 0 0 0) 0x7f9f90059bd0 con 0x7f9f980fa880 2024-03-22T07:20:25.517 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:25.517 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44243,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/2789599242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2789599242},{"type":"v1","addr":"172.21.15.149:6825","nonce":2789599242}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44245,"name":"cephfs.smithi081.zfgxzy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6829/3973077481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3973077481},{"type":"v1","addr":"172.21.15.81:6829","nonce":3973077481}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:25.518 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 26 2024-03-22T07:20:25.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9f8406d2f0 msgr2=0x7f9f8406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9f8406d2f0 0x7f9f8406f7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800abd0 tx=0x7f9f8800aeb0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.519 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 msgr2=0x7f9f98112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 0x7f9f98112d10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000db00 tx=0x7f9f90004e60 comp rx=0 tx=0).stop 2024-03-22T07:20:25.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 shutdown_connections 2024-03-22T07:20:25.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f9f8406d2f0 0x7f9f8406f7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f981008a0 0x7f9f98113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 --2- 172.21.15.81:0/3560341622 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f980fa880 0x7f9f98112d10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 >> 172.21.15.81:0/3560341622 conn(0x7f9f980714a0 msgr2=0x7f9f98103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:25.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 shutdown_connections 2024-03-22T07:20:25.520 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.519+0000 7f9f9fe14640 1 -- 172.21.15.81:0/3560341622 wait complete. 2024-03-22T07:20:25.530 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:25.530 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-22T07:20:25.631 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 -- 172.21.15.81:0/4045993725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0044101c80 msgr2=0x7f00440fb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 --2- 172.21.15.81:0/4045993725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0044101c80 0x7f00440fb310 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f00300048d0 tx=0x7f0030009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 -- 172.21.15.81:0/4045993725 shutdown_connections 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 --2- 172.21.15.81:0/4045993725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0044101c80 0x7f00440fb310 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 --2- 172.21.15.81:0/4045993725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00441012e0 0x7f00441016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 -- 172.21.15.81:0/4045993725 >> 172.21.15.81:0/4045993725 conn(0x7f00440f6eb0 msgr2=0x7f00440f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 -- 172.21.15.81:0/4045993725 shutdown_connections 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 -- 172.21.15.81:0/4045993725 wait complete. 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 Processor -- start 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 -- start start 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00441012e0 0x7f0044111e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:25.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0044101c80 0x7f00441123a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0044112a30 con 0x7f00441012e0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.629+0000 7f004aaae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004410c040 con 0x7f0044101c80 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f004884a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00441012e0 0x7f0044111e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f004884a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00441012e0 0x7f0044111e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55408/0 (socket says 172.21.15.81:55408) 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f004884a640 1 -- 172.21.15.81:0/4103608673 learned_addr learned my addr 172.21.15.81:0/4103608673 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f004884a640 1 -- 172.21.15.81:0/4103608673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0044101c80 msgr2=0x7f00441123a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f004884a640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0044101c80 0x7f00441123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f004884a640 1 -- 172.21.15.81:0/4103608673 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0030005040 con 0x7f00441012e0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f004884a640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00441012e0 0x7f0044111e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f003800daf0 tx=0x7f003800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f00417fa640 1 -- 172.21.15.81:0/4103608673 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f003800edb0 con 0x7f00441012e0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f00417fa640 1 -- 172.21.15.81:0/4103608673 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0038017940 con 0x7f00441012e0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004410c2c0 con 0x7f00441012e0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f00417fa640 1 -- 172.21.15.81:0/4103608673 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0038016b00 con 0x7f00441012e0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.630+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004410c7e0 con 0x7f00441012e0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.631+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00440fca50 con 0x7f00441012e0 2024-03-22T07:20:25.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.632+0000 7f00417fa640 1 -- 172.21.15.81:0/4103608673 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f0038014070 con 0x7f00441012e0 2024-03-22T07:20:25.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.632+0000 7f00417fa640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f003406d2f0 0x7f003406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:25.634 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.632+0000 7f00417fa640 1 -- 172.21.15.81:0/4103608673 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f003808ec40 con 0x7f00441012e0 2024-03-22T07:20:25.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.632+0000 7f0043fff640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f003406d2f0 0x7f003406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:25.635 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.632+0000 7f0043fff640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f003406d2f0 0x7f003406f7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00300043d0 tx=0x7f0030009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:25.637 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.636+0000 7f00417fa640 1 -- 172.21.15.81:0/4103608673 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00380586f0 con 0x7f00441012e0 2024-03-22T07:20:25.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.844+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f0044108530 con 0x7f00441012e0 2024-03-22T07:20:25.845 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.845+0000 7f00417fa640 1 -- 172.21.15.81:0/4103608673 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v41) v1 ==== 107+0+4077 (secure 0 0 0) 0x7f003805bdb0 con 0x7f00441012e0 2024-03-22T07:20:25.845 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:25.846 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44243,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/2789599242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2789599242},{"type":"v1","addr":"172.21.15.149:6825","nonce":2789599242}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:25.846 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 27 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f003406d2f0 msgr2=0x7f003406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f003406d2f0 0x7f003406f7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00300043d0 tx=0x7f0030009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00441012e0 msgr2=0x7f0044111e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00441012e0 0x7f0044111e60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f003800daf0 tx=0x7f003800b460 comp rx=0 tx=0).stop 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 shutdown_connections 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f003406d2f0 0x7f003406f7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0044101c80 0x7f00441123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 --2- 172.21.15.81:0/4103608673 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00441012e0 0x7f0044111e60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 >> 172.21.15.81:0/4103608673 conn(0x7f00440f6eb0 msgr2=0x7f00440f7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 shutdown_connections 2024-03-22T07:20:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.847+0000 7f004aaae640 1 -- 172.21.15.81:0/4103608673 wait complete. 2024-03-22T07:20:25.859 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:25.859 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-22T07:20:25.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 -- 172.21.15.81:0/2039125859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 msgr2=0x7f24500fb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 --2- 172.21.15.81:0/2039125859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 0x7f24500fb2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2444009c50 tx=0x7f2444004d10 comp rx=0 tx=0).stop 2024-03-22T07:20:25.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 -- 172.21.15.81:0/2039125859 shutdown_connections 2024-03-22T07:20:25.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 --2- 172.21.15.81:0/2039125859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 0x7f24500fb2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.960 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 --2- 172.21.15.81:0/2039125859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501012c0 0x7f2450101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 -- 172.21.15.81:0/2039125859 >> 172.21.15.81:0/2039125859 conn(0x7f24500f6e90 msgr2=0x7f24500f92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 -- 172.21.15.81:0/2039125859 shutdown_connections 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 -- 172.21.15.81:0/2039125859 wait complete. 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 Processor -- start 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 -- start start 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501012c0 0x7f2450112f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 0x7f2450113440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450113a40 con 0x7f2450101c60 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.958+0000 7f245539d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245010d090 con 0x7f24501012c0 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f244e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 0x7f2450113440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f244e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 0x7f2450113440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55434/0 (socket says 172.21.15.81:55434) 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f244e7fc640 1 -- 172.21.15.81:0/1663670386 learned_addr learned my addr 172.21.15.81:0/1663670386 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f244effd640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501012c0 0x7f2450112f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:25.961 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f244e7fc640 1 -- 172.21.15.81:0/1663670386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501012c0 msgr2=0x7f2450112f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:25.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f244e7fc640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501012c0 0x7f2450112f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:25.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f244e7fc640 1 -- 172.21.15.81:0/1663670386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440098b0 con 0x7f2450101c60 2024-03-22T07:20:25.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f244e7fc640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 0x7f2450113440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2444004a30 tx=0x7f2444004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:25.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f24377fe640 1 -- 172.21.15.81:0/1663670386 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244403a070 con 0x7f2450101c60 2024-03-22T07:20:25.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f24377fe640 1 -- 172.21.15.81:0/1663670386 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2444035b20 con 0x7f2450101c60 2024-03-22T07:20:25.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f24377fe640 1 -- 172.21.15.81:0/1663670386 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244403eaa0 con 0x7f2450101c60 2024-03-22T07:20:25.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245010d310 con 0x7f2450101c60 2024-03-22T07:20:25.962 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.959+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245010d8b0 con 0x7f2450101c60 2024-03-22T07:20:25.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.960+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24500fca30 con 0x7f2450101c60 2024-03-22T07:20:25.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.961+0000 7f24377fe640 1 -- 172.21.15.81:0/1663670386 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f2444032070 con 0x7f2450101c60 2024-03-22T07:20:25.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.961+0000 7f24377fe640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f243806d2f0 0x7f243806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:25.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.961+0000 7f24377fe640 1 -- 172.21.15.81:0/1663670386 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f24440af120 con 0x7f2450101c60 2024-03-22T07:20:25.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.961+0000 7f244effd640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f243806d2f0 0x7f243806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:25.963 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.962+0000 7f244effd640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f243806d2f0 0x7f243806f7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24480041d0 tx=0x7f2448009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:25.966 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:25.966+0000 7f24377fe640 1 -- 172.21.15.81:0/1663670386 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2444078bd0 con 0x7f2450101c60 2024-03-22T07:20:26.176 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.176+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f2450108420 con 0x7f2450101c60 2024-03-22T07:20:26.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.176+0000 7f24377fe640 1 -- 172.21.15.81:0/1663670386 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v41) v1 ==== 107+0+4890 (secure 0 0 0) 0x7f244407c290 con 0x7f2450101c60 2024-03-22T07:20:26.177 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:26.177 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44243,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/2789599242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2789599242},{"type":"v1","addr":"172.21.15.149:6825","nonce":2789599242}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:26.177 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 28 2024-03-22T07:20:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f243806d2f0 msgr2=0x7f243806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f243806d2f0 0x7f243806f7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f24480041d0 tx=0x7f2448009040 comp rx=0 tx=0).stop 2024-03-22T07:20:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 msgr2=0x7f2450113440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 0x7f2450113440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2444004a30 tx=0x7f2444004d10 comp rx=0 tx=0).stop 2024-03-22T07:20:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 shutdown_connections 2024-03-22T07:20:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f243806d2f0 0x7f243806f7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2450101c60 0x7f2450113440 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.179 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 --2- 172.21.15.81:0/1663670386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24501012c0 0x7f2450112f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 >> 172.21.15.81:0/1663670386 conn(0x7f24500f6e90 msgr2=0x7f24500f7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:26.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.178+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 shutdown_connections 2024-03-22T07:20:26.180 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.179+0000 7f245539d640 1 -- 172.21.15.81:0/1663670386 wait complete. 2024-03-22T07:20:26.189 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:26 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2366728209' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-22T07:20:26.189 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:26 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3560341622' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-22T07:20:26.189 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:26 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/4103608673' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-22T07:20:26.190 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:26.190 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-22T07:20:26.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 -- 172.21.15.81:0/3273895471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 msgr2=0x7febe80faac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 --2- 172.21.15.81:0/3273895471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 0x7febe80faac0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febd40048d0 tx=0x7febd4009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 -- 172.21.15.81:0/3273895471 shutdown_connections 2024-03-22T07:20:26.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 --2- 172.21.15.81:0/3273895471 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 0x7febe80faac0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 --2- 172.21.15.81:0/3273895471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe8102260 0x7febe8102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 -- 172.21.15.81:0/3273895471 >> 172.21.15.81:0/3273895471 conn(0x7febe80714a0 msgr2=0x7febe80f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 -- 172.21.15.81:0/3273895471 shutdown_connections 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 -- 172.21.15.81:0/3273895471 wait complete. 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 Processor -- start 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.288+0000 7febeeaae640 1 -- start start 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febeeaae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 0x7febe8112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febeeaae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe8102260 0x7febe8113320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febeeaae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe81139b0 con 0x7febe80fa650 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febeeaae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe810cf70 con 0x7febe8102260 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febec84a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 0x7febe8112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:26.291 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febec84a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 0x7febe8112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55454/0 (socket says 172.21.15.81:55454) 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febec84a640 1 -- 172.21.15.81:0/2524138908 learned_addr learned my addr 172.21.15.81:0/2524138908 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febe7fff640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe8102260 0x7febe8113320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febec84a640 1 -- 172.21.15.81:0/2524138908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe8102260 msgr2=0x7febe8113320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febec84a640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe8102260 0x7febe8113320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febec84a640 1 -- 172.21.15.81:0/2524138908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd4005040 con 0x7febe80fa650 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febec84a640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 0x7febe8112de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7febdc00b580 tx=0x7febdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febe57fa640 1 -- 172.21.15.81:0/2524138908 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febdc00f6f0 con 0x7febe80fa650 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.289+0000 7febe57fa640 1 -- 172.21.15.81:0/2524138908 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febdc00fd20 con 0x7febe80fa650 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.290+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe810d250 con 0x7febe80fa650 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.290+0000 7febe57fa640 1 -- 172.21.15.81:0/2524138908 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febdc0103a0 con 0x7febe80fa650 2024-03-22T07:20:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.290+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe810d740 con 0x7febe80fa650 2024-03-22T07:20:26.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.290+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe80fd970 con 0x7febe80fa650 2024-03-22T07:20:26.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.291+0000 7febe57fa640 1 -- 172.21.15.81:0/2524138908 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7febdc011400 con 0x7febe80fa650 2024-03-22T07:20:26.293 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.291+0000 7febe57fa640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7febd806d300 0x7febd806f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:26.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.291+0000 7febe57fa640 1 -- 172.21.15.81:0/2524138908 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7febdc08c8e0 con 0x7febe80fa650 2024-03-22T07:20:26.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.291+0000 7febe7fff640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7febd806d300 0x7febd806f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:26.294 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.292+0000 7febe7fff640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7febd806d300 0x7febd806f7b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febd40048a0 tx=0x7febd4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:26.296 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.296+0000 7febe57fa640 1 -- 172.21.15.81:0/2524138908 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7febdc056390 con 0x7febe80fa650 2024-03-22T07:20:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:26 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2366728209' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-22T07:20:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:26 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3560341622' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-22T07:20:26.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:26 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/4103608673' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-22T07:20:26.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.508+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7febe804bcc0 con 0x7febe80fa650 2024-03-22T07:20:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.508+0000 7febe57fa640 1 -- 172.21.15.81:0/2524138908 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v41) v1 ==== 107+0+4108 (secure 0 0 0) 0x7febdc059a50 con 0x7febe80fa650 2024-03-22T07:20:26.509 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:26.509 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:26.509 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 29 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7febd806d300 msgr2=0x7febd806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7febd806d300 0x7febd806f7b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febd40048a0 tx=0x7febd4009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 msgr2=0x7febe8112de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 0x7febe8112de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7febdc00b580 tx=0x7febdc0091a0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 shutdown_connections 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7febd806d300 0x7febd806f7b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe8102260 0x7febe8113320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 --2- 172.21.15.81:0/2524138908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febe80fa650 0x7febe8112de0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 >> 172.21.15.81:0/2524138908 conn(0x7febe80714a0 msgr2=0x7febe8072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:26.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 shutdown_connections 2024-03-22T07:20:26.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.510+0000 7febeeaae640 1 -- 172.21.15.81:0/2524138908 wait complete. 2024-03-22T07:20:26.522 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:26.522 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-22T07:20:26.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.619+0000 7f67e839d640 1 -- 172.21.15.81:0/3591973812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e0101140 msgr2=0x7f67e0101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.619+0000 7f67e839d640 1 --2- 172.21.15.81:0/3591973812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e0101140 0x7f67e0101510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f67d80048d0 tx=0x7f67d8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 -- 172.21.15.81:0/3591973812 shutdown_connections 2024-03-22T07:20:26.622 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 --2- 172.21.15.81:0/3591973812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67e00fb110 0x7f67e00fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 --2- 172.21.15.81:0/3591973812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e0101140 0x7f67e0101510 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 -- 172.21.15.81:0/3591973812 >> 172.21.15.81:0/3591973812 conn(0x7f67e00710b0 msgr2=0x7f67e00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 -- 172.21.15.81:0/3591973812 shutdown_connections 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 -- 172.21.15.81:0/3591973812 wait complete. 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 Processor -- start 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 -- start start 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e00fb110 0x7f67e0111c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67e0101140 0x7f67e0112170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0112800 con 0x7f67e00fb110 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e839d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e010bdc0 con 0x7f67e0101140 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e6139640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e00fb110 0x7f67e0111c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e6139640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e00fb110 0x7f67e0111c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55470/0 (socket says 172.21.15.81:55470) 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.620+0000 7f67e6139640 1 -- 172.21.15.81:0/1046982318 learned_addr learned my addr 172.21.15.81:0/1046982318 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67e5938640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67e0101140 0x7f67e0112170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67e6139640 1 -- 172.21.15.81:0/1046982318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67e0101140 msgr2=0x7f67e0112170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67e6139640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67e0101140 0x7f67e0112170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67e6139640 1 -- 172.21.15.81:0/1046982318 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8005040 con 0x7f67e00fb110 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67e5938640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67e0101140 0x7f67e0112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67e6139640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e00fb110 0x7f67e0111c30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f67d80041d0 tx=0x7f67d8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67d6ffd640 1 -- 172.21.15.81:0/1046982318 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67d80092e0 con 0x7f67e00fb110 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67d6ffd640 1 -- 172.21.15.81:0/1046982318 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67d8009440 con 0x7f67e00fb110 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67d6ffd640 1 -- 172.21.15.81:0/1046982318 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67d8043640 con 0x7f67e00fb110 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e010c040 con 0x7f67e00fb110 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.621+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e010c560 con 0x7f67e00fb110 2024-03-22T07:20:26.624 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.622+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e00fc800 con 0x7f67e00fb110 2024-03-22T07:20:26.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.622+0000 7f67d6ffd640 1 -- 172.21.15.81:0/1046982318 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f67d803b9f0 con 0x7f67e00fb110 2024-03-22T07:20:26.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.623+0000 7f67d6ffd640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67cc06d2f0 0x7f67cc06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:26.625 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.623+0000 7f67d6ffd640 1 -- 172.21.15.81:0/1046982318 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f67d80af730 con 0x7f67e00fb110 2024-03-22T07:20:26.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.623+0000 7f67e5938640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67cc06d2f0 0x7f67cc06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:26.626 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.623+0000 7f67e5938640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67cc06d2f0 0x7f67cc06f7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f67d0006fd0 tx=0x7f67d0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:26.628 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.627+0000 7f67d6ffd640 1 -- 172.21.15.81:0/1046982318 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67d80791e0 con 0x7f67e00fb110 2024-03-22T07:20:26.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.835+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f67e010c1d0 con 0x7f67e00fb110 2024-03-22T07:20:26.836 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.836+0000 7f67d6ffd640 1 -- 172.21.15.81:0/1046982318 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v41) v1 ==== 107+0+4924 (secure 0 0 0) 0x7f67e010c1d0 con 0x7f67e00fb110 2024-03-22T07:20:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:26.837 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24259,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.149:6827/3446640029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3446640029},{"type":"v1","addr":"172.21.15.149:6827","nonce":3446640029}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:26.837 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 30 2024-03-22T07:20:26.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67cc06d2f0 msgr2=0x7f67cc06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67cc06d2f0 0x7f67cc06f7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f67d0006fd0 tx=0x7f67d0008040 comp rx=0 tx=0).stop 2024-03-22T07:20:26.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e00fb110 msgr2=0x7f67e0111c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e00fb110 0x7f67e0111c30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f67d80041d0 tx=0x7f67d8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 shutdown_connections 2024-03-22T07:20:26.839 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f67cc06d2f0 0x7f67cc06f7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67e0101140 0x7f67e0112170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 --2- 172.21.15.81:0/1046982318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67e00fb110 0x7f67e0111c30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 >> 172.21.15.81:0/1046982318 conn(0x7f67e00710b0 msgr2=0x7f67e00f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:26.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 shutdown_connections 2024-03-22T07:20:26.840 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.838+0000 7f67e839d640 1 -- 172.21.15.81:0/1046982318 wait complete. 2024-03-22T07:20:26.850 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:26.851 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-22T07:20:26.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 -- 172.21.15.81:0/291449713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91000fa650 msgr2=0x7f91000faac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 --2- 172.21.15.81:0/291449713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91000fa650 0x7f91000faac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f90f00048d0 tx=0x7f90f0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.951 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 -- 172.21.15.81:0/291449713 shutdown_connections 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 --2- 172.21.15.81:0/291449713 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91000fa650 0x7f91000faac0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 --2- 172.21.15.81:0/291449713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9100102260 0x7f9100102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 -- 172.21.15.81:0/291449713 >> 172.21.15.81:0/291449713 conn(0x7f91000714a0 msgr2=0x7f91000f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 -- 172.21.15.81:0/291449713 shutdown_connections 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 -- 172.21.15.81:0/291449713 wait complete. 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 Processor -- start 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.949+0000 7f910842e640 1 -- start start 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f910842e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91000fa650 0x7f9100112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f910842e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100102260 0x7f91001132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f910842e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91001138a0 con 0x7f9100102260 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f910842e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910010cef0 con 0x7f91000fa650 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f91059c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100102260 0x7f91001132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:26.952 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f91059c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100102260 0x7f91001132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55482/0 (socket says 172.21.15.81:55482) 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f91059c9640 1 -- 172.21.15.81:0/1538758919 learned_addr learned my addr 172.21.15.81:0/1538758919 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f91061ca640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91000fa650 0x7f9100112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f91059c9640 1 -- 172.21.15.81:0/1538758919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91000fa650 msgr2=0x7f9100112d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f91059c9640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91000fa650 0x7f9100112d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f91059c9640 1 -- 172.21.15.81:0/1538758919 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f0005040 con 0x7f9100102260 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f91059c9640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100102260 0x7f91001132a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f90f0034040 tx=0x7f90f0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f90f6ffd640 1 -- 172.21.15.81:0/1538758919 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f00092b0 con 0x7f9100102260 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910010d170 con 0x7f9100102260 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f90f6ffd640 1 -- 172.21.15.81:0/1538758919 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90f003b400 con 0x7f9100102260 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f90f6ffd640 1 -- 172.21.15.81:0/1538758919 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90f00435d0 con 0x7f9100102260 2024-03-22T07:20:26.953 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.950+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910010d630 con 0x7f9100102260 2024-03-22T07:20:26.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.951+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91000fd970 con 0x7f9100102260 2024-03-22T07:20:26.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.952+0000 7f90f6ffd640 1 -- 172.21.15.81:0/1538758919 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f90f002f070 con 0x7f9100102260 2024-03-22T07:20:26.954 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.952+0000 7f90f6ffd640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f90ec06d2f0 0x7f90ec06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:26.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.952+0000 7f90f6ffd640 1 -- 172.21.15.81:0/1538758919 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f90f00ae640 con 0x7f9100102260 2024-03-22T07:20:26.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.952+0000 7f91061ca640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f90ec06d2f0 0x7f90ec06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:26.955 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.953+0000 7f91061ca640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f90ec06d2f0 0x7f90ec06f7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009af0 tx=0x7f90fc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:26.957 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:26.957+0000 7f90f6ffd640 1 -- 172.21.15.81:0/1538758919 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f90f0078120 con 0x7f9100102260 2024-03-22T07:20:27.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.169+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f9100109440 con 0x7f9100102260 2024-03-22T07:20:27.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.169+0000 7f90f6ffd640 1 -- 172.21.15.81:0/1538758919 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v41) v1 ==== 107+0+4142 (secure 0 0 0) 0x7f90f007b7e0 con 0x7f9100102260 2024-03-22T07:20:27.170 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:27.170 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:27.170 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 31 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f90ec06d2f0 msgr2=0x7f90ec06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f90ec06d2f0 0x7f90ec06f7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009af0 tx=0x7f90fc00b040 comp rx=0 tx=0).stop 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100102260 msgr2=0x7f91001132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100102260 0x7f91001132a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f90f0034040 tx=0x7f90f0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 shutdown_connections 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f90ec06d2f0 0x7f90ec06f7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9100102260 0x7f91001132a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 --2- 172.21.15.81:0/1538758919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91000fa650 0x7f9100112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 >> 172.21.15.81:0/1538758919 conn(0x7f91000714a0 msgr2=0x7f9100072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 shutdown_connections 2024-03-22T07:20:27.172 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.171+0000 7f910842e640 1 -- 172.21.15.81:0/1538758919 wait complete. 2024-03-22T07:20:27.181 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:27 smithi081 ceph-mon[113965]: pgmap v269: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:27.181 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:27 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1663670386' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-22T07:20:27.182 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:27 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2524138908' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-22T07:20:27.182 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:27 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1046982318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-22T07:20:27.183 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:27.183 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.281+0000 7f94f375d640 1 -- 172.21.15.81:0/1322274791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec102240 msgr2=0x7f94ec102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.281+0000 7f94f375d640 1 --2- 172.21.15.81:0/1322274791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec102240 0x7f94ec102610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f94e80048d0 tx=0x7f94e8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 -- 172.21.15.81:0/1322274791 shutdown_connections 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 --2- 172.21.15.81:0/1322274791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ec0643a0 0x7f94ec0647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 --2- 172.21.15.81:0/1322274791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec102240 0x7f94ec102610 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 -- 172.21.15.81:0/1322274791 >> 172.21.15.81:0/1322274791 conn(0x7f94ec0710b0 msgr2=0x7f94ec0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 -- 172.21.15.81:0/1322274791 shutdown_connections 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 -- 172.21.15.81:0/1322274791 wait complete. 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 Processor -- start 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 -- start start 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec0643a0 0x7f94ec112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ec102240 0x7f94ec113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec113810 con 0x7f94ec0643a0 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f375d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec10ce60 con 0x7f94ec102240 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f14f9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec0643a0 0x7f94ec112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f14f9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec0643a0 0x7f94ec112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55490/0 (socket says 172.21.15.81:55490) 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.282+0000 7f94f14f9640 1 -- 172.21.15.81:0/1992589956 learned_addr learned my addr 172.21.15.81:0/1992589956 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94f0cf8640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ec102240 0x7f94ec113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94f14f9640 1 -- 172.21.15.81:0/1992589956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ec102240 msgr2=0x7f94ec113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94f14f9640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ec102240 0x7f94ec113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94f14f9640 1 -- 172.21.15.81:0/1992589956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e8005040 con 0x7f94ec0643a0 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94f0cf8640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ec102240 0x7f94ec113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94f14f9640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec0643a0 0x7f94ec112cd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f94e8004930 tx=0x7f94e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94e1ffb640 1 -- 172.21.15.81:0/1992589956 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e80094c0 con 0x7f94ec0643a0 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec10d0e0 con 0x7f94ec0643a0 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94e1ffb640 1 -- 172.21.15.81:0/1992589956 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94e8032e50 con 0x7f94ec0643a0 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94e1ffb640 1 -- 172.21.15.81:0/1992589956 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e80433b0 con 0x7f94ec0643a0 2024-03-22T07:20:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.283+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec10d5a0 con 0x7f94ec0643a0 2024-03-22T07:20:27.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.284+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec0fd940 con 0x7f94ec0643a0 2024-03-22T07:20:27.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.285+0000 7f94e1ffb640 1 -- 172.21.15.81:0/1992589956 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f94e802f070 con 0x7f94ec0643a0 2024-03-22T07:20:27.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.285+0000 7f94e1ffb640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f94d806d2f0 0x7f94d806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:27.286 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.285+0000 7f94e1ffb640 1 -- 172.21.15.81:0/1992589956 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f94e80ae840 con 0x7f94ec0643a0 2024-03-22T07:20:27.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.285+0000 7f94f0cf8640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f94d806d2f0 0x7f94d806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:27.287 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.286+0000 7f94f0cf8640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f94d806d2f0 0x7f94d806f7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f94ec10e590 tx=0x7f94dc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:27.290 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.289+0000 7f94e1ffb640 1 -- 172.21.15.81:0/1992589956 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94e8078270 con 0x7f94ec0643a0 2024-03-22T07:20:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:27 smithi149 ceph-mon[83447]: pgmap v269: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:27 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1663670386' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-22T07:20:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:27 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2524138908' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-22T07:20:27.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:27 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1046982318' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-22T07:20:27.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.499+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f94ec10e340 con 0x7f94ec0643a0 2024-03-22T07:20:27.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.499+0000 7f94e1ffb640 1 -- 172.21.15.81:0/1992589956 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v41) v1 ==== 107+0+4958 (secure 0 0 0) 0x7f94e807b930 con 0x7f94ec0643a0 2024-03-22T07:20:27.500 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:27.500 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:17:11.575650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:27.500 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 32 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f94d806d2f0 msgr2=0x7f94d806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f94d806d2f0 0x7f94d806f7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f94ec10e590 tx=0x7f94dc00b040 comp rx=0 tx=0).stop 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec0643a0 msgr2=0x7f94ec112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec0643a0 0x7f94ec112cd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f94e8004930 tx=0x7f94e8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 shutdown_connections 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f94d806d2f0 0x7f94d806f7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94ec102240 0x7f94ec113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 --2- 172.21.15.81:0/1992589956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94ec0643a0 0x7f94ec112cd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 >> 172.21.15.81:0/1992589956 conn(0x7f94ec0710b0 msgr2=0x7f94ec0f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 shutdown_connections 2024-03-22T07:20:27.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.501+0000 7f94f375d640 1 -- 172.21.15.81:0/1992589956 wait complete. 2024-03-22T07:20:27.513 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:20:27.513 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-22T07:20:27.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.610+0000 7f03e275d640 1 -- 172.21.15.81:0/356229520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 msgr2=0x7f03dc0fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.610+0000 7f03e275d640 1 --2- 172.21.15.81:0/356229520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 0x7f03dc0fc200 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f03d00048d0 tx=0x7f03d0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.610+0000 7f03e275d640 1 -- 172.21.15.81:0/356229520 shutdown_connections 2024-03-22T07:20:27.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.610+0000 7f03e275d640 1 --2- 172.21.15.81:0/356229520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 0x7f03dc0fc200 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.610+0000 7f03e275d640 1 --2- 172.21.15.81:0/356229520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03dc1021d0 0x7f03dc1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.612 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.610+0000 7f03e275d640 1 -- 172.21.15.81:0/356229520 >> 172.21.15.81:0/356229520 conn(0x7f03dc0f7dc0 msgr2=0x7f03dc0fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.610+0000 7f03e275d640 1 -- 172.21.15.81:0/356229520 shutdown_connections 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.610+0000 7f03e275d640 1 -- 172.21.15.81:0/356229520 wait complete. 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03e275d640 1 Processor -- start 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03e275d640 1 -- start start 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03e275d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03dc1021d0 0x7f03dc112d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03e275d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 0x7f03dc1132b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03e275d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc113940 con 0x7f03dc102b70 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03e275d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc10cf00 con 0x7f03dc1021d0 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03db7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 0x7f03dc1132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03db7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 0x7f03dc1132b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55506/0 (socket says 172.21.15.81:55506) 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03db7fe640 1 -- 172.21.15.81:0/2554493241 learned_addr learned my addr 172.21.15.81:0/2554493241 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03db7fe640 1 -- 172.21.15.81:0/2554493241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03dc1021d0 msgr2=0x7f03dc112d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03db7fe640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03dc1021d0 0x7f03dc112d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.611+0000 7f03db7fe640 1 -- 172.21.15.81:0/2554493241 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d0005040 con 0x7f03dc102b70 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.612+0000 7f03db7fe640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 0x7f03dc1132b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f03d0004930 tx=0x7f03d0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:27.613 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.612+0000 7f03d8ff9640 1 -- 172.21.15.81:0/2554493241 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d00094c0 con 0x7f03dc102b70 2024-03-22T07:20:27.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.612+0000 7f03d8ff9640 1 -- 172.21.15.81:0/2554493241 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03d0032e50 con 0x7f03dc102b70 2024-03-22T07:20:27.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.612+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc10d180 con 0x7f03dc102b70 2024-03-22T07:20:27.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.612+0000 7f03d8ff9640 1 -- 172.21.15.81:0/2554493241 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d00433b0 con 0x7f03dc102b70 2024-03-22T07:20:27.614 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.612+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc10d640 con 0x7f03dc102b70 2024-03-22T07:20:27.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.613+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc0fd940 con 0x7f03dc102b70 2024-03-22T07:20:27.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.613+0000 7f03d8ff9640 1 -- 172.21.15.81:0/2554493241 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f03d002f070 con 0x7f03dc102b70 2024-03-22T07:20:27.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.614+0000 7f03d8ff9640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f03c406d2f0 0x7f03c406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:27.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.614+0000 7f03d8ff9640 1 -- 172.21.15.81:0/2554493241 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f03d00ae900 con 0x7f03dc102b70 2024-03-22T07:20:27.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.614+0000 7f03dbfff640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f03c406d2f0 0x7f03c406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:27.615 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.614+0000 7f03dbfff640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f03c406d2f0 0x7f03c406f7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0097e0 tx=0x7f03cc006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:27.618 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.618+0000 7f03d8ff9640 1 -- 172.21.15.81:0/2554493241 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f03d0078330 con 0x7f03dc102b70 2024-03-22T07:20:27.827 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.826+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f03dc1093f0 con 0x7f03dc102b70 2024-03-22T07:20:27.827 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.827+0000 7f03d8ff9640 1 -- 172.21.15.81:0/2554493241 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v41) v1 ==== 107+0+4957 (secure 0 0 0) 0x7f03d007b9f0 con 0x7f03dc102b70 2024-03-22T07:20:27.827 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:27.828 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:18:15.849607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:27.828 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 33 2024-03-22T07:20:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f03c406d2f0 msgr2=0x7f03c406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f03c406d2f0 0x7f03c406f7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0097e0 tx=0x7f03cc006d20 comp rx=0 tx=0).stop 2024-03-22T07:20:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 msgr2=0x7f03dc1132b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 0x7f03dc1132b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f03d0004930 tx=0x7f03d0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 shutdown_connections 2024-03-22T07:20:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f03c406d2f0 0x7f03c406f7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03dc102b70 0x7f03dc1132b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 --2- 172.21.15.81:0/2554493241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03dc1021d0 0x7f03dc112d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.830 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 >> 172.21.15.81:0/2554493241 conn(0x7f03dc0f7dc0 msgr2=0x7f03dc0f8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:27.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 shutdown_connections 2024-03-22T07:20:27.831 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.829+0000 7f03e275d640 1 -- 172.21.15.81:0/2554493241 wait complete. 2024-03-22T07:20:27.841 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-22T07:20:27.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.939+0000 7f715839d640 1 -- 172.21.15.81:0/3507090276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 msgr2=0x7f71501016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.939+0000 7f715839d640 1 --2- 172.21.15.81:0/3507090276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 0x7f71501016b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f714c0048d0 tx=0x7f714c009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.939+0000 7f715839d640 1 -- 172.21.15.81:0/3507090276 shutdown_connections 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.939+0000 7f715839d640 1 --2- 172.21.15.81:0/3507090276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7150101c80 0x7f71500fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.939+0000 7f715839d640 1 --2- 172.21.15.81:0/3507090276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 0x7f71501016b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.939+0000 7f715839d640 1 -- 172.21.15.81:0/3507090276 >> 172.21.15.81:0/3507090276 conn(0x7f71500f6eb0 msgr2=0x7f71500f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.939+0000 7f715839d640 1 -- 172.21.15.81:0/3507090276 shutdown_connections 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.939+0000 7f715839d640 1 -- 172.21.15.81:0/3507090276 wait complete. 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f715839d640 1 Processor -- start 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f715839d640 1 -- start start 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f715839d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 0x7f715010e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f715839d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7150101c80 0x7f715010e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f715839d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715010ebc0 con 0x7f71501012e0 2024-03-22T07:20:27.942 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f715839d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715010ed30 con 0x7f7150101c80 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7156139640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 0x7f715010e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7156139640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 0x7f715010e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55516/0 (socket says 172.21.15.81:55516) 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7156139640 1 -- 172.21.15.81:0/4231039753 learned_addr learned my addr 172.21.15.81:0/4231039753 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7155938640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7150101c80 0x7f715010e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7156139640 1 -- 172.21.15.81:0/4231039753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7150101c80 msgr2=0x7f715010e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7156139640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7150101c80 0x7f715010e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7156139640 1 -- 172.21.15.81:0/4231039753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c005040 con 0x7f71501012e0 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7155938640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7150101c80 0x7f715010e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7156139640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 0x7f715010e140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f714c004930 tx=0x7f714c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7146ffd640 1 -- 172.21.15.81:0/4231039753 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714c0094c0 con 0x7f71501012e0 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7146ffd640 1 -- 172.21.15.81:0/4231039753 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f714c032e50 con 0x7f71501012e0 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f7146ffd640 1 -- 172.21.15.81:0/4231039753 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f714c0433b0 con 0x7f71501012e0 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.940+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715010efb0 con 0x7f71501012e0 2024-03-22T07:20:27.943 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.941+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7150117460 con 0x7f71501012e0 2024-03-22T07:20:27.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.941+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71500fca50 con 0x7f71501012e0 2024-03-22T07:20:27.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.942+0000 7f7146ffd640 1 -- 172.21.15.81:0/4231039753 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f714c02f070 con 0x7f71501012e0 2024-03-22T07:20:27.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.942+0000 7f7146ffd640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f713c06d2f0 0x7f713c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:27.944 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.942+0000 7f7146ffd640 1 -- 172.21.15.81:0/4231039753 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f714c0ae720 con 0x7f71501012e0 2024-03-22T07:20:27.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.942+0000 7f7155938640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f713c06d2f0 0x7f713c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:27.945 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.943+0000 7f7155938640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f713c06d2f0 0x7f713c06f7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7140006fd0 tx=0x7f7140008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:27.947 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:27.947+0000 7f7146ffd640 1 -- 172.21.15.81:0/4231039753 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f714c078150 con 0x7f71501012e0 2024-03-22T07:20:28.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.157+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f715010efb0 con 0x7f71501012e0 2024-03-22T07:20:28.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.157+0000 7f7146ffd640 1 -- 172.21.15.81:0/4231039753 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v41) v1 ==== 107+0+5014 (secure 0 0 0) 0x7f715010efb0 con 0x7f71501012e0 2024-03-22T07:20:28.158 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:28.158 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:18:16.853345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34378,"mds_1":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34378":{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":1,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:28.158 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 34 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f713c06d2f0 msgr2=0x7f713c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f713c06d2f0 0x7f713c06f7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7140006fd0 tx=0x7f7140008040 comp rx=0 tx=0).stop 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 msgr2=0x7f715010e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 0x7f715010e140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f714c004930 tx=0x7f714c009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 shutdown_connections 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f713c06d2f0 0x7f713c06f7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7150101c80 0x7f715010e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 --2- 172.21.15.81:0/4231039753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71501012e0 0x7f715010e140 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 >> 172.21.15.81:0/4231039753 conn(0x7f71500f6eb0 msgr2=0x7f71500f7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 shutdown_connections 2024-03-22T07:20:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.159+0000 7f715839d640 1 -- 172.21.15.81:0/4231039753 wait complete. 2024-03-22T07:20:28.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:28 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1538758919' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-22T07:20:28.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:28 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1992589956' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-22T07:20:28.169 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:28 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2554493241' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-22T07:20:28.170 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-22T07:20:28.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.267+0000 7f6474d9d640 1 -- 172.21.15.81:0/609814335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6470101140 msgr2=0x7f6470101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.269 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.267+0000 7f6474d9d640 1 --2- 172.21.15.81:0/609814335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6470101140 0x7f6470101510 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f64680048d0 tx=0x7f6468009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 -- 172.21.15.81:0/609814335 shutdown_connections 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 --2- 172.21.15.81:0/609814335 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64700fb110 0x7f64700fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 --2- 172.21.15.81:0/609814335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6470101140 0x7f6470101510 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 -- 172.21.15.81:0/609814335 >> 172.21.15.81:0/609814335 conn(0x7f64700710b0 msgr2=0x7f64700714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 -- 172.21.15.81:0/609814335 shutdown_connections 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 -- 172.21.15.81:0/609814335 wait complete. 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 Processor -- start 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 -- start start 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64700fb110 0x7f647010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6470101140 0x7f647010e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6470065310 con 0x7f64700fb110 2024-03-22T07:20:28.270 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f6474d9d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647010e900 con 0x7f6470101140 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f646e59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64700fb110 0x7f647010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f646e59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64700fb110 0x7f647010de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55532/0 (socket says 172.21.15.81:55532) 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.268+0000 7f646e59c640 1 -- 172.21.15.81:0/3313981727 learned_addr learned my addr 172.21.15.81:0/3313981727 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.269+0000 7f646e59c640 1 -- 172.21.15.81:0/3313981727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6470101140 msgr2=0x7f647010e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.269+0000 7f646e59c640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6470101140 0x7f647010e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.269+0000 7f646e59c640 1 -- 172.21.15.81:0/3313981727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6468005040 con 0x7f64700fb110 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.269+0000 7f646e59c640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64700fb110 0x7f647010de80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64680041d0 tx=0x7f6468009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.269+0000 7f645effd640 1 -- 172.21.15.81:0/3313981727 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64680092e0 con 0x7f64700fb110 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.269+0000 7f645effd640 1 -- 172.21.15.81:0/3313981727 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6468009440 con 0x7f64700fb110 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.269+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647010eb80 con 0x7f64700fb110 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.269+0000 7f645effd640 1 -- 172.21.15.81:0/3313981727 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6468043640 con 0x7f64700fb110 2024-03-22T07:20:28.271 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.269+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6470117140 con 0x7f64700fb110 2024-03-22T07:20:28.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.270+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64700fc800 con 0x7f64700fb110 2024-03-22T07:20:28.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.270+0000 7f645effd640 1 -- 172.21.15.81:0/3313981727 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f646803b9f0 con 0x7f64700fb110 2024-03-22T07:20:28.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.271+0000 7f645effd640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f645806d2f0 0x7f645806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:28.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.271+0000 7f645effd640 1 -- 172.21.15.81:0/3313981727 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f64680af690 con 0x7f64700fb110 2024-03-22T07:20:28.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.271+0000 7f646dd9b640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f645806d2f0 0x7f645806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:28.272 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.271+0000 7f646dd9b640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f645806d2f0 0x7f645806f7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6464007fd0 tx=0x7f6464007b20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:28.275 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.275+0000 7f645effd640 1 -- 172.21.15.81:0/3313981727 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64680791e0 con 0x7f64700fb110 2024-03-22T07:20:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:28 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1538758919' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-22T07:20:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:28 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1992589956' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-22T07:20:28.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:28 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2554493241' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-22T07:20:28.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.484+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f647004bcc0 con 0x7f64700fb110 2024-03-22T07:20:28.485 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.485+0000 7f645effd640 1 -- 172.21.15.81:0/3313981727 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v41) v1 ==== 107+0+5016 (secure 0 0 0) 0x7f646807c8a0 con 0x7f64700fb110 2024-03-22T07:20:28.485 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:28.485 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:18:20.366341+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34378,"mds_1":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34378":{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":1,"incarnation":34,"state":"up:resolve","state_seq":12,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:28.485 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 35 2024-03-22T07:20:28.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.486+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f645806d2f0 msgr2=0x7f645806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.486+0000 7f6474d9d640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f645806d2f0 0x7f645806f7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6464007fd0 tx=0x7f6464007b20 comp rx=0 tx=0).stop 2024-03-22T07:20:28.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.486+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64700fb110 msgr2=0x7f647010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.486+0000 7f6474d9d640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64700fb110 0x7f647010de80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f64680041d0 tx=0x7f6468009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.487+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 shutdown_connections 2024-03-22T07:20:28.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.487+0000 7f6474d9d640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f645806d2f0 0x7f645806f7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.487+0000 7f6474d9d640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6470101140 0x7f647010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.487 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.487+0000 7f6474d9d640 1 --2- 172.21.15.81:0/3313981727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64700fb110 0x7f647010de80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.487+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 >> 172.21.15.81:0/3313981727 conn(0x7f64700710b0 msgr2=0x7f64700f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:28.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.487+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 shutdown_connections 2024-03-22T07:20:28.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.487+0000 7f6474d9d640 1 -- 172.21.15.81:0/3313981727 wait complete. 2024-03-22T07:20:28.498 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-22T07:20:28.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.596+0000 7fc97a39d640 1 -- 172.21.15.81:0/1146544490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 msgr2=0x7fc9740647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.596+0000 7fc97a39d640 1 --2- 172.21.15.81:0/1146544490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 0x7fc9740647d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9680048d0 tx=0x7fc968009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.596+0000 7fc97a39d640 1 -- 172.21.15.81:0/1146544490 shutdown_connections 2024-03-22T07:20:28.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.596+0000 7fc97a39d640 1 --2- 172.21.15.81:0/1146544490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 0x7fc9740647d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.598 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.596+0000 7fc97a39d640 1 --2- 172.21.15.81:0/1146544490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc974102240 0x7fc974102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.596+0000 7fc97a39d640 1 -- 172.21.15.81:0/1146544490 >> 172.21.15.81:0/1146544490 conn(0x7fc9740710b0 msgr2=0x7fc9740714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.596+0000 7fc97a39d640 1 -- 172.21.15.81:0/1146544490 shutdown_connections 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.596+0000 7fc97a39d640 1 -- 172.21.15.81:0/1146544490 wait complete. 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc97a39d640 1 Processor -- start 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc97a39d640 1 -- start start 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc97a39d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 0x7fc9740fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc97a39d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc974102240 0x7fc9740fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc97a39d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740fb720 con 0x7fc9740643a0 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc97a39d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9740fb890 con 0x7fc974102240 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc973fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 0x7fc9740fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc973fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 0x7fc9740fcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55550/0 (socket says 172.21.15.81:55550) 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc973fff640 1 -- 172.21.15.81:0/523466584 learned_addr learned my addr 172.21.15.81:0/523466584 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc9737fe640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc974102240 0x7fc9740fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc973fff640 1 -- 172.21.15.81:0/523466584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc974102240 msgr2=0x7fc9740fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc973fff640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc974102240 0x7fc9740fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.597+0000 7fc973fff640 1 -- 172.21.15.81:0/523466584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968005040 con 0x7fc9740643a0 2024-03-22T07:20:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.598+0000 7fc973fff640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 0x7fc9740fcc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc964002a50 tx=0x7fc964004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.598+0000 7fc970ff9640 1 -- 172.21.15.81:0/523466584 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc964007c30 con 0x7fc9740643a0 2024-03-22T07:20:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.598+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9740fbb10 con 0x7fc9740643a0 2024-03-22T07:20:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.598+0000 7fc970ff9640 1 -- 172.21.15.81:0/523466584 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc96400f040 con 0x7fc9740643a0 2024-03-22T07:20:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.598+0000 7fc970ff9640 1 -- 172.21.15.81:0/523466584 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc96400db50 con 0x7fc9740643a0 2024-03-22T07:20:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.598+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9740fc000 con 0x7fc9740643a0 2024-03-22T07:20:28.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.599+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9740fd940 con 0x7fc9740643a0 2024-03-22T07:20:28.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.599+0000 7fc970ff9640 1 -- 172.21.15.81:0/523466584 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fc96401c400 con 0x7fc9740643a0 2024-03-22T07:20:28.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.600+0000 7fc970ff9640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc95c06d2f0 0x7fc95c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:28.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.600+0000 7fc970ff9640 1 -- 172.21.15.81:0/523466584 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc96408c190 con 0x7fc9740643a0 2024-03-22T07:20:28.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.600+0000 7fc9737fe640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc95c06d2f0 0x7fc95c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:28.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.600+0000 7fc9737fe640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc95c06d2f0 0x7fc95c06f7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc974111170 tx=0x7fc968009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.604+0000 7fc970ff9640 1 -- 172.21.15.81:0/523466584 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc964055c70 con 0x7fc9740643a0 2024-03-22T07:20:28.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.812+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7fc97404bcc0 con 0x7fc9740643a0 2024-03-22T07:20:28.813 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.813+0000 7fc970ff9640 1 -- 172.21.15.81:0/523466584 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v41) v1 ==== 107+0+5018 (secure 0 0 0) 0x7fc964059330 con 0x7fc9740643a0 2024-03-22T07:20:28.813 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:28.813 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:18:22.150288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34378,"mds_1":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34378":{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":0,"incarnation":34,"state":"up:resolve","state_seq":17,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":1,"incarnation":34,"state":"up:resolve","state_seq":12,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:28.814 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 36 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc95c06d2f0 msgr2=0x7fc95c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc95c06d2f0 0x7fc95c06f7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc974111170 tx=0x7fc968009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 msgr2=0x7fc9740fcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 0x7fc9740fcc70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc964002a50 tx=0x7fc964004600 comp rx=0 tx=0).stop 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 shutdown_connections 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc95c06d2f0 0x7fc95c06f7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc974102240 0x7fc9740fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 --2- 172.21.15.81:0/523466584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9740643a0 0x7fc9740fcc70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 >> 172.21.15.81:0/523466584 conn(0x7fc9740710b0 msgr2=0x7fc9740f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 shutdown_connections 2024-03-22T07:20:28.816 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.815+0000 7fc97a39d640 1 -- 172.21.15.81:0/523466584 wait complete. 2024-03-22T07:20:28.826 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-03-22T07:20:28.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.923+0000 7f074139d640 1 -- 172.21.15.81:0/3198111429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c0643a0 msgr2=0x7f073c0647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.923+0000 7f074139d640 1 --2- 172.21.15.81:0/3198111429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c0643a0 0x7f073c0647d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f07300048d0 tx=0x7f0730009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.923+0000 7f074139d640 1 -- 172.21.15.81:0/3198111429 shutdown_connections 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.923+0000 7f074139d640 1 --2- 172.21.15.81:0/3198111429 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c0643a0 0x7f073c0647d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.923+0000 7f074139d640 1 --2- 172.21.15.81:0/3198111429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073c102240 0x7f073c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.923+0000 7f074139d640 1 -- 172.21.15.81:0/3198111429 >> 172.21.15.81:0/3198111429 conn(0x7f073c0710b0 msgr2=0x7f073c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.923+0000 7f074139d640 1 -- 172.21.15.81:0/3198111429 shutdown_connections 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.923+0000 7f074139d640 1 -- 172.21.15.81:0/3198111429 wait complete. 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f074139d640 1 Processor -- start 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f074139d640 1 -- start start 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f074139d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073c0643a0 0x7f073c112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f074139d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c102240 0x7f073c113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f074139d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c113800 con 0x7f073c102240 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f074139d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c10ce50 con 0x7f073c0643a0 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f073a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c102240 0x7f073c113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f073a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c102240 0x7f073c113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55566/0 (socket says 172.21.15.81:55566) 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f073a7fc640 1 -- 172.21.15.81:0/1364678563 learned_addr learned my addr 172.21.15.81:0/1364678563 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f073a7fc640 1 -- 172.21.15.81:0/1364678563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073c0643a0 msgr2=0x7f073c112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f073affd640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073c0643a0 0x7f073c112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f073a7fc640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073c0643a0 0x7f073c112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f073a7fc640 1 -- 172.21.15.81:0/1364678563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0730005040 con 0x7f073c102240 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.924+0000 7f073affd640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073c0643a0 0x7f073c112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.925+0000 7f073a7fc640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c102240 0x7f073c113200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0730004930 tx=0x7f0730009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.925+0000 7f07237fe640 1 -- 172.21.15.81:0/1364678563 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07300094c0 con 0x7f073c102240 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.925+0000 7f07237fe640 1 -- 172.21.15.81:0/1364678563 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0730032e50 con 0x7f073c102240 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.925+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c10d0d0 con 0x7f073c102240 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.925+0000 7f07237fe640 1 -- 172.21.15.81:0/1364678563 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07300433b0 con 0x7f073c102240 2024-03-22T07:20:28.927 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.925+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c10d590 con 0x7f073c102240 2024-03-22T07:20:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.926+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c0fd940 con 0x7f073c102240 2024-03-22T07:20:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.926+0000 7f07237fe640 1 -- 172.21.15.81:0/1364678563 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f073002f070 con 0x7f073c102240 2024-03-22T07:20:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.926+0000 7f07237fe640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f072406d2f0 0x7f072406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.927+0000 7f07237fe640 1 -- 172.21.15.81:0/1364678563 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f07300ae840 con 0x7f073c102240 2024-03-22T07:20:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.927+0000 7f073affd640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f072406d2f0 0x7f072406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.927+0000 7f073affd640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f072406d2f0 0x7f072406f7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f072c007ee0 tx=0x7f072c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:28.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:28.931+0000 7f07237fe640 1 -- 172.21.15.81:0/1364678563 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0730078270 con 0x7f073c102240 2024-03-22T07:20:29.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.141+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f073c109300 con 0x7f073c102240 2024-03-22T07:20:29.141 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.141+0000 7f07237fe640 1 -- 172.21.15.81:0/1364678563 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v41) v1 ==== 107+0+5022 (secure 0 0 0) 0x7f073007b930 con 0x7f073c102240 2024-03-22T07:20:29.142 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:29.142 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:18:23.074835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34378,"mds_1":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34378":{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":1,"incarnation":34,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:29.142 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 37 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f072406d2f0 msgr2=0x7f072406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f072406d2f0 0x7f072406f7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f072c007ee0 tx=0x7f072c00b040 comp rx=0 tx=0).stop 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c102240 msgr2=0x7f073c113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c102240 0x7f073c113200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0730004930 tx=0x7f0730009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 shutdown_connections 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f072406d2f0 0x7f072406f7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f073c102240 0x7f073c113200 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 --2- 172.21.15.81:0/1364678563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073c0643a0 0x7f073c112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 >> 172.21.15.81:0/1364678563 conn(0x7f073c0710b0 msgr2=0x7f073c0f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:29.144 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 shutdown_connections 2024-03-22T07:20:29.145 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.143+0000 7f074139d640 1 -- 172.21.15.81:0/1364678563 wait complete. 2024-03-22T07:20:29.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:29 smithi081 ceph-mon[113965]: pgmap v270: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:29.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:29 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/4231039753' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-22T07:20:29.153 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:29 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3313981727' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-22T07:20:29.154 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:29 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/523466584' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-22T07:20:29.155 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/4065776076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 msgr2=0x7f7b640647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 --2- 172.21.15.81:0/4065776076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 0x7f7b640647f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b600048d0 tx=0x7f7b60009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/4065776076 shutdown_connections 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 --2- 172.21.15.81:0/4065776076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 0x7f7b640647f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 --2- 172.21.15.81:0/4065776076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64102210 0x7f7b641025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/4065776076 >> 172.21.15.81:0/4065776076 conn(0x7f7b640710b0 msgr2=0x7f7b640714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/4065776076 shutdown_connections 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/4065776076 wait complete. 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 Processor -- start 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 -- start start 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 0x7f7b64112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:29.257 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64102210 0x7f7b64113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64113820 con 0x7f7b640643a0 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6ba6e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6410ce70 con 0x7f7b64102210 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.255+0000 7f7b6980a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 0x7f7b64112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b6980a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 0x7f7b64112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55594/0 (socket says 172.21.15.81:55594) 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b6980a640 1 -- 172.21.15.81:0/3842176275 learned_addr learned my addr 172.21.15.81:0/3842176275 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b69009640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64102210 0x7f7b64113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b6980a640 1 -- 172.21.15.81:0/3842176275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64102210 msgr2=0x7f7b64113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b6980a640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64102210 0x7f7b64113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b6980a640 1 -- 172.21.15.81:0/3842176275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b60005040 con 0x7f7b640643a0 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b6980a640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 0x7f7b64112ce0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400eb70 tx=0x7f7b5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b5a7fc640 1 -- 172.21.15.81:0/3842176275 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b54007c30 con 0x7f7b640643a0 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b5a7fc640 1 -- 172.21.15.81:0/3842176275 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b5400f040 con 0x7f7b640643a0 2024-03-22T07:20:29.258 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6410d0f0 con 0x7f7b640643a0 2024-03-22T07:20:29.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b5a7fc640 1 -- 172.21.15.81:0/3842176275 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b540095b0 con 0x7f7b640643a0 2024-03-22T07:20:29.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.256+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6410d5e0 con 0x7f7b640643a0 2024-03-22T07:20:29.259 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.257+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b640fd910 con 0x7f7b640643a0 2024-03-22T07:20:29.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.258+0000 7f7b5a7fc640 1 -- 172.21.15.81:0/3842176275 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f7b54009730 con 0x7f7b640643a0 2024-03-22T07:20:29.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.258+0000 7f7b5a7fc640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7b5006d2f0 0x7f7b5006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:29.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.259+0000 7f7b5a7fc640 1 -- 172.21.15.81:0/3842176275 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7b5408c640 con 0x7f7b640643a0 2024-03-22T07:20:29.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.259+0000 7f7b69009640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7b5006d2f0 0x7f7b5006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:29.260 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.259+0000 7f7b69009640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7b5006d2f0 0x7f7b5006f7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7b640fb090 tx=0x7f7b60009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:29.263 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.263+0000 7f7b5a7fc640 1 -- 172.21.15.81:0/3842176275 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b54056120 con 0x7f7b640643a0 2024-03-22T07:20:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:29 smithi149 ceph-mon[83447]: pgmap v270: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:29 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/4231039753' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-22T07:20:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:29 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3313981727' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-22T07:20:29.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:29 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/523466584' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-22T07:20:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.480+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f7b6410e350 con 0x7f7b640643a0 2024-03-22T07:20:29.481 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.481+0000 7f7b5a7fc640 1 -- 172.21.15.81:0/3842176275 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v41) v1 ==== 107+0+5016 (secure 0 0 0) 0x7f7b540597e0 con 0x7f7b640643a0 2024-03-22T07:20:29.481 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:29.481 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:18:23.981266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34378,"mds_1":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34378":{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":1,"incarnation":34,"state":"up:rejoin","state_seq":14,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:29.482 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 38 2024-03-22T07:20:29.483 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7b5006d2f0 msgr2=0x7f7b5006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7b5006d2f0 0x7f7b5006f7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7b640fb090 tx=0x7f7b60009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 msgr2=0x7f7b64112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 0x7f7b64112ce0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400eb70 tx=0x7f7b5400b040 comp rx=0 tx=0).stop 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 shutdown_connections 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f7b5006d2f0 0x7f7b5006f7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b64102210 0x7f7b64113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 --2- 172.21.15.81:0/3842176275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b640643a0 0x7f7b64112ce0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 >> 172.21.15.81:0/3842176275 conn(0x7f7b640710b0 msgr2=0x7f7b640f95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 shutdown_connections 2024-03-22T07:20:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.483+0000 7f7b6ba6e640 1 -- 172.21.15.81:0/3842176275 wait complete. 2024-03-22T07:20:29.495 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-03-22T07:20:29.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.596+0000 7fe7b491d640 1 -- 172.21.15.81:0/2275306979 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 msgr2=0x7fe7b0102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.596+0000 7fe7b491d640 1 --2- 172.21.15.81:0/2275306979 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 0x7fe7b0102610 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00048d0 tx=0x7fe7a0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.600 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.596+0000 7fe7b491d640 1 -- 172.21.15.81:0/2275306979 shutdown_connections 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.596+0000 7fe7b491d640 1 --2- 172.21.15.81:0/2275306979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b00643a0 0x7fe7b00647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.596+0000 7fe7b491d640 1 --2- 172.21.15.81:0/2275306979 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 0x7fe7b0102610 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.596+0000 7fe7b491d640 1 -- 172.21.15.81:0/2275306979 >> 172.21.15.81:0/2275306979 conn(0x7fe7b00710b0 msgr2=0x7fe7b00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.596+0000 7fe7b491d640 1 -- 172.21.15.81:0/2275306979 shutdown_connections 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.596+0000 7fe7b491d640 1 -- 172.21.15.81:0/2275306979 wait complete. 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7b491d640 1 Processor -- start 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7b491d640 1 -- start start 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7b491d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b00643a0 0x7fe7b0112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7b491d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 0x7fe7b0113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:29.601 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7b491d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b0113810 con 0x7fe7b0102240 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7b491d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b010ce60 con 0x7fe7b00643a0 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 0x7fe7b0113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 0x7fe7b0113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55616/0 (socket says 172.21.15.81:55616) 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7a7fff640 1 -- 172.21.15.81:0/1440243092 learned_addr learned my addr 172.21.15.81:0/1440243092 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7a7fff640 1 -- 172.21.15.81:0/1440243092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b00643a0 msgr2=0x7fe7b0112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7ae59c640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b00643a0 0x7fe7b0112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7a7fff640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b00643a0 0x7fe7b0112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7a7fff640 1 -- 172.21.15.81:0/1440243092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0005040 con 0x7fe7b0102240 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7ae59c640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b00643a0 0x7fe7b0112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7a7fff640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 0x7fe7b0113210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c790 tx=0x7fe7980043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.597+0000 7fe7a6ffd640 1 -- 172.21.15.81:0/1440243092 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79800a820 con 0x7fe7b0102240 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.598+0000 7fe7a6ffd640 1 -- 172.21.15.81:0/1440243092 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe79800ae50 con 0x7fe7b0102240 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.598+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b010d140 con 0x7fe7b0102240 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.598+0000 7fe7a6ffd640 1 -- 172.21.15.81:0/1440243092 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7980115e0 con 0x7fe7b0102240 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.598+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b010d660 con 0x7fe7b0102240 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.599+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b00fd940 con 0x7fe7b0102240 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.599+0000 7fe7a6ffd640 1 -- 172.21.15.81:0/1440243092 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fe798012400 con 0x7fe7b0102240 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.599+0000 7fe7a6ffd640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe79c06d2f0 0x7fe79c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.600+0000 7fe7a6ffd640 1 -- 172.21.15.81:0/1440243092 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe79808ca10 con 0x7fe7b0102240 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.600+0000 7fe7ae59c640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe79c06d2f0 0x7fe79c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.600+0000 7fe7ae59c640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe79c06d2f0 0x7fe79c06f7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0000c00 tx=0x7fe7a0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:29.604 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.604+0000 7fe7a6ffd640 1 -- 172.21.15.81:0/1440243092 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe798056560 con 0x7fe7b0102240 2024-03-22T07:20:29.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.817+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7fe7b0109350 con 0x7fe7b0102240 2024-03-22T07:20:29.818 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.818+0000 7fe7a6ffd640 1 -- 172.21.15.81:0/1440243092 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v41) v1 ==== 107+0+5016 (secure 0 0 0) 0x7fe798059c20 con 0x7fe7b0102240 2024-03-22T07:20:29.819 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:29.819 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:18:25.260639+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34378,"mds_1":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34378":{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":0,"incarnation":34,"state":"up:active","state_seq":20,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":1,"incarnation":34,"state":"up:active","state_seq":15,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:29.819 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 39 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe79c06d2f0 msgr2=0x7fe79c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe79c06d2f0 0x7fe79c06f7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0000c00 tx=0x7fe7a0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 msgr2=0x7fe7b0113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 0x7fe7b0113210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c790 tx=0x7fe7980043c0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 shutdown_connections 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fe79c06d2f0 0x7fe79c06f7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7b0102240 0x7fe7b0113210 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 --2- 172.21.15.81:0/1440243092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7b00643a0 0x7fe7b0112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 >> 172.21.15.81:0/1440243092 conn(0x7fe7b00710b0 msgr2=0x7fe7b00f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 shutdown_connections 2024-03-22T07:20:29.821 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.820+0000 7fe7b491d640 1 -- 172.21.15.81:0/1440243092 wait complete. 2024-03-22T07:20:29.832 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-03-22T07:20:29.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.935+0000 7fabdfc2e640 1 -- 172.21.15.81:0/4093202466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 msgr2=0x7fabd8102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.935+0000 7fabdfc2e640 1 --2- 172.21.15.81:0/4093202466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 0x7fabd8102610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fabc80048d0 tx=0x7fabc8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.935+0000 7fabdfc2e640 1 -- 172.21.15.81:0/4093202466 shutdown_connections 2024-03-22T07:20:29.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.935+0000 7fabdfc2e640 1 --2- 172.21.15.81:0/4093202466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabd80643a0 0x7fabd80647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 --2- 172.21.15.81:0/4093202466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 0x7fabd8102610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.938 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 -- 172.21.15.81:0/4093202466 >> 172.21.15.81:0/4093202466 conn(0x7fabd80710b0 msgr2=0x7fabd80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 -- 172.21.15.81:0/4093202466 shutdown_connections 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 -- 172.21.15.81:0/4093202466 wait complete. 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 Processor -- start 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 -- start start 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabd80643a0 0x7fabd80fb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 0x7fabd80fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80fd0c0 con 0x7fabd8102240 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.936+0000 7fabdfc2e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80fbc60 con 0x7fabd80643a0 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdd1c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 0x7fabd80fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdd1c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 0x7fabd80fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55626/0 (socket says 172.21.15.81:55626) 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdd1c9640 1 -- 172.21.15.81:0/1760455544 learned_addr learned my addr 172.21.15.81:0/1760455544 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdd9ca640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabd80643a0 0x7fabd80fb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdd1c9640 1 -- 172.21.15.81:0/1760455544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabd80643a0 msgr2=0x7fabd80fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:29.939 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdd1c9640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabd80643a0 0x7fabd80fb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:29.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdd1c9640 1 -- 172.21.15.81:0/1760455544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8005040 con 0x7fabd8102240 2024-03-22T07:20:29.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdd1c9640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 0x7fabd80fb720 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fabd400eb70 tx=0x7fabd400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:29.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabce7fc640 1 -- 172.21.15.81:0/1760455544 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabd4007c30 con 0x7fabd8102240 2024-03-22T07:20:29.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabce7fc640 1 -- 172.21.15.81:0/1760455544 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabd400f040 con 0x7fabd8102240 2024-03-22T07:20:29.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd80fbee0 con 0x7fabd8102240 2024-03-22T07:20:29.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabce7fc640 1 -- 172.21.15.81:0/1760455544 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabd40095b0 con 0x7fabd8102240 2024-03-22T07:20:29.940 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.937+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd81193a0 con 0x7fabd8102240 2024-03-22T07:20:29.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.938+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd80fd940 con 0x7fabd8102240 2024-03-22T07:20:29.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.939+0000 7fabce7fc640 1 -- 172.21.15.81:0/1760455544 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fabd4009730 con 0x7fabd8102240 2024-03-22T07:20:29.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.939+0000 7fabce7fc640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fabc406d2f0 0x7fabc406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:29.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.939+0000 7fabce7fc640 1 -- 172.21.15.81:0/1760455544 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fabd408c6c0 con 0x7fabd8102240 2024-03-22T07:20:29.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.939+0000 7fabdd9ca640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fabc406d2f0 0x7fabc406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:29.941 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.939+0000 7fabdd9ca640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fabc406d2f0 0x7fabc406f7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fabc8003080 tx=0x7fabc8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:29.948 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:29.948+0000 7fabce7fc640 1 -- 172.21.15.81:0/1760455544 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fabd40560f0 con 0x7fabd8102240 2024-03-22T07:20:30.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.166+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7fabd804bcc0 con 0x7fabd8102240 2024-03-22T07:20:30.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.166+0000 7fabce7fc640 1 -- 172.21.15.81:0/1760455544 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v41) v1 ==== 107+0+5017 (secure 0 0 0) 0x7fabd40597b0 con 0x7fabd8102240 2024-03-22T07:20:30.167 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:30.167 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44267,"name":"cephfs.smithi149.kdtxfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6825/3255112562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3255112562},{"type":"v1","addr":"172.21.15.149:6825","nonce":3255112562}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi149.ffewvr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6827/1411110570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":1411110570},{"type":"v1","addr":"172.21.15.149:6827","nonce":1411110570}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-22T07:06:21.658288+0000","modified":"2024-03-22T07:18:57.298355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34378,"mds_1":34410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34378":{"gid":34378,"name":"cephfs.smithi081.bnzfrv","rank":0,"incarnation":34,"state":"up:active","state_seq":20,"addr":"172.21.15.81:6827/235705027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":235705027},{"type":"v1","addr":"172.21.15.81:6827","nonce":235705027}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34410":{"gid":34410,"name":"cephfs.smithi081.zfgxzy","rank":1,"incarnation":34,"state":"up:active","state_seq":15,"addr":"172.21.15.81:6829/2467232405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":2467232405},{"type":"v1","addr":"172.21.15.81:6829","nonce":2467232405}]},"join_fscid":1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:20:30.167 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 40 2024-03-22T07:20:30.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fabc406d2f0 msgr2=0x7fabc406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fabc406d2f0 0x7fabc406f7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fabc8003080 tx=0x7fabc8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 msgr2=0x7fabd80fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 0x7fabd80fb720 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fabd400eb70 tx=0x7fabd400b040 comp rx=0 tx=0).stop 2024-03-22T07:20:30.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 shutdown_connections 2024-03-22T07:20:30.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fabc406d2f0 0x7fabc406f7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8102240 0x7fabd80fb720 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 --2- 172.21.15.81:0/1760455544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fabd80643a0 0x7fabd80fb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 >> 172.21.15.81:0/1760455544 conn(0x7fabd80710b0 msgr2=0x7fabd80f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:30.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 shutdown_connections 2024-03-22T07:20:30.170 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.168+0000 7fabdfc2e640 1 -- 172.21.15.81:0/1760455544 wait complete. 2024-03-22T07:20:30.178 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:30 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1364678563' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-22T07:20:30.179 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:30 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3842176275' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-22T07:20:30.179 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:30 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1440243092' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-22T07:20:30.180 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-22T07:20:30.191 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-22T07:20:30.191 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:20:30.191 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:20:30.207 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-03-22T07:20:30.208 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:20:30.264 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T07:20:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:30 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1364678563' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-22T07:20:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:30 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3842176275' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-22T07:20:30.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:30 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1440243092' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-22T07:20:30.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.401+0000 7fb55e19d640 1 -- 172.21.15.81:0/3468961458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 msgr2=0x7fb558106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.401+0000 7fb55e19d640 1 --2- 172.21.15.81:0/3468961458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 0x7fb558106210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb548004900 tx=0x7fb548009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 -- 172.21.15.81:0/3468961458 shutdown_connections 2024-03-22T07:20:30.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 --2- 172.21.15.81:0/3468961458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 0x7fb558106210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 --2- 172.21.15.81:0/3468961458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5580fd1a0 0x7fb5580fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 -- 172.21.15.81:0/3468961458 >> 172.21.15.81:0/3468961458 conn(0x7fb5580f6ca0 msgr2=0x7fb5580f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:30.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 -- 172.21.15.81:0/3468961458 shutdown_connections 2024-03-22T07:20:30.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 -- 172.21.15.81:0/3468961458 wait complete. 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 Processor -- start 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 -- start start 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5580fd1a0 0x7fb5580fb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 0x7fb5580fb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5580fcf40 con 0x7fb5580fdb40 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.402+0000 7fb55e19d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5580fbae0 con 0x7fb5580fd1a0 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb556ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 0x7fb5580fb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb556ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 0x7fb5580fb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55652/0 (socket says 172.21.15.81:55652) 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb556ffd640 1 -- 172.21.15.81:0/4053217038 learned_addr learned my addr 172.21.15.81:0/4053217038 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb5577fe640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5580fd1a0 0x7fb5580fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb556ffd640 1 -- 172.21.15.81:0/4053217038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5580fd1a0 msgr2=0x7fb5580fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.405 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb556ffd640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5580fd1a0 0x7fb5580fb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb556ffd640 1 -- 172.21.15.81:0/4053217038 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb548005040 con 0x7fb5580fdb40 2024-03-22T07:20:30.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb556ffd640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 0x7fb5580fb5a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb548000c00 tx=0x7fb548009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:30.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb53ffff640 1 -- 172.21.15.81:0/4053217038 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5480094c0 con 0x7fb5580fdb40 2024-03-22T07:20:30.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5580fbd60 con 0x7fb5580fdb40 2024-03-22T07:20:30.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb53ffff640 1 -- 172.21.15.81:0/4053217038 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb54803d920 con 0x7fb5580fdb40 2024-03-22T07:20:30.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb53ffff640 1 -- 172.21.15.81:0/4053217038 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54803b930 con 0x7fb5580fdb40 2024-03-22T07:20:30.406 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.403+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb558119430 con 0x7fb5580fdb40 2024-03-22T07:20:30.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.404+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb558065d70 con 0x7fb5580fdb40 2024-03-22T07:20:30.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.405+0000 7fb53ffff640 1 -- 172.21.15.81:0/4053217038 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fb54803da90 con 0x7fb5580fdb40 2024-03-22T07:20:30.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.405+0000 7fb53ffff640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb54006d2f0 0x7fb54006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:30.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.405+0000 7fb53ffff640 1 -- 172.21.15.81:0/4053217038 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb5480af230 con 0x7fb5580fdb40 2024-03-22T07:20:30.407 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.405+0000 7fb5577fe640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb54006d2f0 0x7fb54006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:30.408 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.406+0000 7fb5577fe640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb54006d2f0 0x7fb54006f7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb54c007fd0 tx=0x7fb54c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:30.410 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.410+0000 7fb53ffff640 1 -- 172.21.15.81:0/4053217038 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb548078d20 con 0x7fb5580fdb40 2024-03-22T07:20:30.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.589+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5581093a0 con 0x7fb5580fdb40 2024-03-22T07:20:30.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.590+0000 7fb53ffff640 1 -- 172.21.15.81:0/4053217038 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16642 (secure 0 0 0) 0x7fb54807c3e0 con 0x7fb5580fdb40 2024-03-22T07:20:30.591 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:30.591 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":85,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","created":"2024-03-22T06:59:35.415160+0000","modified":"2024-03-22T07:18:04.460156+0000","last_up_change":"2024-03-22T07:17:09.537521+0000","last_in_change":"2024-03-22T07:04:44.637120+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:03:49.371549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T07:06:20.045033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T07:06:20.879130+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69addaec-0295-4219-9e22-5ebf831a65d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6803","nonce":2753302858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6805","nonce":2753302858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6809","nonce":2753302858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6807","nonce":2753302858}]},"public_addr":"172.21.15.81:6803/2753302858","cluster_addr":"172.21.15.81:6805/2753302858","heartbeat_back_addr":"172.21.15.81:6809/2753302858","heartbeat_front_addr":"172.21.15.81:6807/2753302858","state":["exists","up"]},{"osd":1,"uuid":"cd572358-1fd5-4e96-8caf-3d00d6e0608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6811","nonce":356075948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6813","nonce":356075948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6817","nonce":356075948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6815","nonce":356075948}]},"public_addr":"172.21.15.81:6811/356075948","cluster_addr":"172.21.15.81:6813/356075948","heartbeat_back_addr":"172.21.15.81:6817/356075948","heartbeat_front_addr":"172.21.15.81:6815/356075948","state":["exists","up"]},{"osd":2,"uuid":"ba6f9b7e-712d-4f4b-8b33-6e051daf2875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6819","nonce":3743751332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6821","nonce":3743751332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6825","nonce":3743751332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6823","nonce":3743751332}]},"public_addr":"172.21.15.81:6819/3743751332","cluster_addr":"172.21.15.81:6821/3743751332","heartbeat_back_addr":"172.21.15.81:6825/3743751332","heartbeat_front_addr":"172.21.15.81:6823/3743751332","state":["exists","up"]},{"osd":3,"uuid":"e5adfd7b-e190-45b4-b8ca-da7c45d21d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6801","nonce":1325806457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6803","nonce":1325806457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6807","nonce":1325806457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6805","nonce":1325806457}]},"public_addr":"172.21.15.149:6801/1325806457","cluster_addr":"172.21.15.149:6803/1325806457","heartbeat_back_addr":"172.21.15.149:6807/1325806457","heartbeat_front_addr":"172.21.15.149:6805/1325806457","state":["exists","up"]},{"osd":4,"uuid":"a85bb681-b621-4fbf-b77a-cbd090a41a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6809","nonce":1122192339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6811","nonce":1122192339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6815","nonce":1122192339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6813","nonce":1122192339}]},"public_addr":"172.21.15.149:6809/1122192339","cluster_addr":"172.21.15.149:6811/1122192339","heartbeat_back_addr":"172.21.15.149:6815/1122192339","heartbeat_front_addr":"172.21.15.149:6813/1122192339","state":["exists","up"]},{"osd":5,"uuid":"0e83552d-4e6c-4426-9b7c-99d9ad5c45ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6817","nonce":530633899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6819","nonce":530633899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6823","nonce":530633899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6821","nonce":530633899}]},"public_addr":"172.21.15.149:6817/530633899","cluster_addr":"172.21.15.149:6819/530633899","heartbeat_back_addr":"172.21.15.149:6823/530633899","heartbeat_front_addr":"172.21.15.149:6821/530633899","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:02:52.797148+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:18.980557+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:46.975511+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:10.893766+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:36.048181+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:05:01.848799+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6829/268578316":"2024-03-23T07:17:11.575629+0000","172.21.15.149:6824/3521694486":"2024-03-23T07:17:11.575646+0000","172.21.15.81:0/1030883466":"2024-03-23T07:00:35.069960+0000","172.21.15.149:6829/18108693":"2024-03-23T07:10:12.272683+0000","172.21.15.81:0/3192935300":"2024-03-23T07:08:38.778309+0000","172.21.15.149:6828/18108693":"2024-03-23T07:10:12.272683+0000","172.21.15.81:6828/268578316":"2024-03-23T07:17:11.575629+0000","172.21.15.81:6801/742540357":"2024-03-23T07:08:38.778309+0000","172.21.15.81:0/1980154994":"2024-03-23T07:08:38.778309+0000","172.21.15.81:0/2249539268":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6800/2774978653":"2024-03-23T07:12:45.787752+0000","172.21.15.81:6827/2831536232":"2024-03-23T07:06:37.046400+0000","172.21.15.81:6801/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.149:0/3423294147":"2024-03-23T07:10:12.272683+0000","172.21.15.81:6800/742540357":"2024-03-23T07:08:38.778309+0000","172.21.15.81:0/2438362157":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/473242716":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/1138486583":"2024-03-23T07:12:45.787752+0000","172.21.15.149:6825/3521694486":"2024-03-23T07:17:11.575646+0000","172.21.15.81:6800/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6800/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.149:0/3477455875":"2024-03-23T07:10:12.272683+0000","172.21.15.81:6826/2831536232":"2024-03-23T07:06:37.046400+0000","172.21.15.81:0/683056633":"2024-03-23T07:12:45.787752+0000","172.21.15.81:6801/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/2809233037":"2024-03-23T07:08:38.778309+0000","172.21.15.81:6800/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6801/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/1939667879":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/4180230166":"2024-03-23T07:08:38.778309+0000","172.21.15.149:0/594572994":"2024-03-23T07:10:12.272683+0000","172.21.15.81:0/3756300008":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/2472307158":"2024-03-23T07:00:01.117993+0000","172.21.15.149:0/749736269":"2024-03-23T07:10:12.272683+0000","172.21.15.81:0/988274144":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/770908409":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/1704456246":"2024-03-23T07:12:45.787752+0000","172.21.15.81:6801/2774978653":"2024-03-23T07:12:45.787752+0000","172.21.15.81:0/1143393440":"2024-03-23T07:12:45.787752+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:20:30.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb54006d2f0 msgr2=0x7fb54006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb54006d2f0 0x7fb54006f7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb54c007fd0 tx=0x7fb54c009040 comp rx=0 tx=0).stop 2024-03-22T07:20:30.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 msgr2=0x7fb5580fb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 0x7fb5580fb5a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb548000c00 tx=0x7fb548009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 shutdown_connections 2024-03-22T07:20:30.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fb54006d2f0 0x7fb54006f7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5580fdb40 0x7fb5580fb5a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 --2- 172.21.15.81:0/4053217038 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5580fd1a0 0x7fb5580fb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.593 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 >> 172.21.15.81:0/4053217038 conn(0x7fb5580f6ca0 msgr2=0x7fb5580f8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:30.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 shutdown_connections 2024-03-22T07:20:30.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.592+0000 7fb55e19d640 1 -- 172.21.15.81:0/4053217038 wait complete. 2024-03-22T07:20:30.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:03:49.371549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T07:06:20.045033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T07:06:20.879130+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T07:20:30.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T07:20:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.703+0000 7fa6d64ae640 1 -- 172.21.15.81:0/2269243447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 msgr2=0x7fa6d0101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.703+0000 7fa6d64ae640 1 --2- 172.21.15.81:0/2269243447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 0x7fa6d0101510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa6c00048d0 tx=0x7fa6c0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.703+0000 7fa6d64ae640 1 -- 172.21.15.81:0/2269243447 shutdown_connections 2024-03-22T07:20:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.703+0000 7fa6d64ae640 1 --2- 172.21.15.81:0/2269243447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6d00fb110 0x7fa6d00fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.703+0000 7fa6d64ae640 1 --2- 172.21.15.81:0/2269243447 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 0x7fa6d0101510 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.703+0000 7fa6d64ae640 1 -- 172.21.15.81:0/2269243447 >> 172.21.15.81:0/2269243447 conn(0x7fa6d00710b0 msgr2=0x7fa6d00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.703+0000 7fa6d64ae640 1 -- 172.21.15.81:0/2269243447 shutdown_connections 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.703+0000 7fa6d64ae640 1 -- 172.21.15.81:0/2269243447 wait complete. 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.704+0000 7fa6d64ae640 1 Processor -- start 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.704+0000 7fa6d64ae640 1 -- start start 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.704+0000 7fa6d64ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6d00fb110 0x7fa6d0112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.704+0000 7fa6d64ae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 0x7fa6d0113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.704+0000 7fa6d64ae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0113890 con 0x7fa6d0101140 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.704+0000 7fa6d64ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0113a00 con 0x7fa6d00fb110 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.704+0000 7fa6cf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 0x7fa6d0113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.704+0000 7fa6cf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 0x7fa6d0113290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55678/0 (socket says 172.21.15.81:55678) 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.704+0000 7fa6cf7fe640 1 -- 172.21.15.81:0/1235496691 learned_addr learned my addr 172.21.15.81:0/1235496691 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6cffff640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6d00fb110 0x7fa6d0112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6cf7fe640 1 -- 172.21.15.81:0/1235496691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6d00fb110 msgr2=0x7fa6d0112d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6cf7fe640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6d00fb110 0x7fa6d0112d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6cf7fe640 1 -- 172.21.15.81:0/1235496691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c0005040 con 0x7fa6d0101140 2024-03-22T07:20:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6cffff640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6d00fb110 0x7fa6d0112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6cf7fe640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 0x7fa6d0113290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c790 tx=0x7fa6c40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6ccff9640 1 -- 172.21.15.81:0/1235496691 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6c400a820 con 0x7fa6d0101140 2024-03-22T07:20:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6ccff9640 1 -- 172.21.15.81:0/1235496691 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6c400ae50 con 0x7fa6d0101140 2024-03-22T07:20:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d010d090 con 0x7fa6d0101140 2024-03-22T07:20:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6ccff9640 1 -- 172.21.15.81:0/1235496691 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6c40115e0 con 0x7fa6d0101140 2024-03-22T07:20:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.705+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d010d5b0 con 0x7fa6d0101140 2024-03-22T07:20:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.706+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d00fc800 con 0x7fa6d0101140 2024-03-22T07:20:30.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.706+0000 7fa6ccff9640 1 -- 172.21.15.81:0/1235496691 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fa6c4012400 con 0x7fa6d0101140 2024-03-22T07:20:30.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.707+0000 7fa6ccff9640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa6b806d2f0 0x7fa6b806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:30.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.707+0000 7fa6ccff9640 1 -- 172.21.15.81:0/1235496691 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa6c408cae0 con 0x7fa6d0101140 2024-03-22T07:20:30.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.707+0000 7fa6cffff640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa6b806d2f0 0x7fa6b806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:30.709 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.707+0000 7fa6cffff640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa6b806d2f0 0x7fa6b806f7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0004540 tx=0x7fa6c0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:30.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.712+0000 7fa6ccff9640 1 -- 172.21.15.81:0/1235496691 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa6c4056590 con 0x7fa6d0101140 2024-03-22T07:20:30.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.885+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6d010d830 con 0x7fa6d0101140 2024-03-22T07:20:30.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.885+0000 7fa6ccff9640 1 -- 172.21.15.81:0/1235496691 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6c4054e60 con 0x7fa6d0101140 2024-03-22T07:20:30.886 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-03-22T07:20:30.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.887+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa6b806d2f0 msgr2=0x7fa6b806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.887+0000 7fa6d64ae640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa6b806d2f0 0x7fa6b806f7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0004540 tx=0x7fa6c0009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.887+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 msgr2=0x7fa6d0113290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:30.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.887+0000 7fa6d64ae640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 0x7fa6d0113290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c790 tx=0x7fa6c40043c0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.888+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 shutdown_connections 2024-03-22T07:20:30.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.888+0000 7fa6d64ae640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa6b806d2f0 0x7fa6b806f7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.888+0000 7fa6d64ae640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6d0101140 0x7fa6d0113290 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.888+0000 7fa6d64ae640 1 --2- 172.21.15.81:0/1235496691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6d00fb110 0x7fa6d0112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:30.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.888+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 >> 172.21.15.81:0/1235496691 conn(0x7fa6d00710b0 msgr2=0x7fa6d0064370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:30.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.888+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 shutdown_connections 2024-03-22T07:20:30.889 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.888+0000 7fa6d64ae640 1 -- 172.21.15.81:0/1235496691 wait complete. 2024-03-22T07:20:30.899 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.999+0000 7fd09191d640 1 -- 172.21.15.81:0/2970686912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 msgr2=0x7fd08c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.999+0000 7fd09191d640 1 --2- 172.21.15.81:0/2970686912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 0x7fd08c101510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd0840048d0 tx=0x7fd084009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.999+0000 7fd09191d640 1 -- 172.21.15.81:0/2970686912 shutdown_connections 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.999+0000 7fd09191d640 1 --2- 172.21.15.81:0/2970686912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd08c0fb110 0x7fd08c0fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.999+0000 7fd09191d640 1 --2- 172.21.15.81:0/2970686912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 0x7fd08c101510 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:30.999+0000 7fd09191d640 1 -- 172.21.15.81:0/2970686912 >> 172.21.15.81:0/2970686912 conn(0x7fd08c0710b0 msgr2=0x7fd08c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd09191d640 1 -- 172.21.15.81:0/2970686912 shutdown_connections 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd09191d640 1 -- 172.21.15.81:0/2970686912 wait complete. 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd09191d640 1 Processor -- start 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd09191d640 1 -- start start 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd09191d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd08c0fb110 0x7fd08c10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd09191d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 0x7fd08c10e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:31.003 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd09191d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c065330 con 0x7fd08c101140 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd09191d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c10e920 con 0x7fd08c0fb110 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd08a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 0x7fd08c10e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd08a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 0x7fd08c10e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55698/0 (socket says 172.21.15.81:55698) 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.000+0000 7fd08a7fc640 1 -- 172.21.15.81:0/33312526 learned_addr learned my addr 172.21.15.81:0/33312526 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd08affd640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd08c0fb110 0x7fd08c10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd08a7fc640 1 -- 172.21.15.81:0/33312526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd08c0fb110 msgr2=0x7fd08c10de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd08a7fc640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd08c0fb110 0x7fd08c10de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd08a7fc640 1 -- 172.21.15.81:0/33312526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd084005040 con 0x7fd08c101140 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd08affd640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd08c0fb110 0x7fd08c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd08a7fc640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 0x7fd08c10e3b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd08000cae0 tx=0x7fd080007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd0737fe640 1 -- 172.21.15.81:0/33312526 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd08000ab00 con 0x7fd08c101140 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd0737fe640 1 -- 172.21.15.81:0/33312526 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd08000ac60 con 0x7fd08c101140 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c10ec00 con 0x7fd08c101140 2024-03-22T07:20:31.004 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd0737fe640 1 -- 172.21.15.81:0/33312526 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0800116f0 con 0x7fd08c101140 2024-03-22T07:20:31.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.001+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c117190 con 0x7fd08c101140 2024-03-22T07:20:31.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.002+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08c0fc800 con 0x7fd08c101140 2024-03-22T07:20:31.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.002+0000 7fd0737fe640 1 -- 172.21.15.81:0/33312526 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fd080012440 con 0x7fd08c101140 2024-03-22T07:20:31.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.003+0000 7fd0737fe640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd07406d2f0 0x7fd07406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:31.005 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.003+0000 7fd0737fe640 1 -- 172.21.15.81:0/33312526 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd08008d170 con 0x7fd08c101140 2024-03-22T07:20:31.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.003+0000 7fd08affd640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd07406d2f0 0x7fd07406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:31.006 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.003+0000 7fd08affd640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd07406d2f0 0x7fd07406f7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd084003080 tx=0x7fd084009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:31.007 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.007+0000 7fd0737fe640 1 -- 172.21.15.81:0/33312526 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd080056c20 con 0x7fd08c101140 2024-03-22T07:20:31.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.182+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd08c04bcc0 con 0x7fd08c101140 2024-03-22T07:20:31.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.182+0000 7fd0737fe640 1 -- 172.21.15.81:0/33312526 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd08005a2e0 con 0x7fd08c101140 2024-03-22T07:20:31.183 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-03-22T07:20:31.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.184+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd07406d2f0 msgr2=0x7fd07406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.184+0000 7fd09191d640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd07406d2f0 0x7fd07406f7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd084003080 tx=0x7fd084009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.184+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 msgr2=0x7fd08c10e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.184+0000 7fd09191d640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 0x7fd08c10e3b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd08000cae0 tx=0x7fd080007e50 comp rx=0 tx=0).stop 2024-03-22T07:20:31.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.184+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 shutdown_connections 2024-03-22T07:20:31.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.184+0000 7fd09191d640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fd07406d2f0 0x7fd07406f7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.184+0000 7fd09191d640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c101140 0x7fd08c10e3b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.184+0000 7fd09191d640 1 --2- 172.21.15.81:0/33312526 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd08c0fb110 0x7fd08c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.184+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 >> 172.21.15.81:0/33312526 conn(0x7fd08c0710b0 msgr2=0x7fd08c0f9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:31.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.185+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 shutdown_connections 2024-03-22T07:20:31.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.185+0000 7fd09191d640 1 -- 172.21.15.81:0/33312526 wait complete. 2024-03-22T07:20:31.195 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:31 smithi081 ceph-mon[113965]: pgmap v271: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:31.195 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:31 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1760455544' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-03-22T07:20:31.195 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:31 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/4053217038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:20:31.195 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:31 smithi081 ceph-mon[113965]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:31.195 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:31 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/1235496691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:20:31.196 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.295+0000 7fa7bd99d640 1 -- 172.21.15.81:0/4192894330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 msgr2=0x7fa7b8101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.295+0000 7fa7bd99d640 1 --2- 172.21.15.81:0/4192894330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 0x7fa7b8101510 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80048d0 tx=0x7fa7a8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 -- 172.21.15.81:0/4192894330 shutdown_connections 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 --2- 172.21.15.81:0/4192894330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7b80fb110 0x7fa7b80fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 --2- 172.21.15.81:0/4192894330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 0x7fa7b8101510 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 -- 172.21.15.81:0/4192894330 >> 172.21.15.81:0/4192894330 conn(0x7fa7b80710b0 msgr2=0x7fa7b80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 -- 172.21.15.81:0/4192894330 shutdown_connections 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 -- 172.21.15.81:0/4192894330 wait complete. 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 Processor -- start 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 -- start start 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7b80fb110 0x7fa7b810dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 0x7fa7b810e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:31.299 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b810e960 con 0x7fa7b8101140 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.296+0000 7fa7bd99d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b810ead0 con 0x7fa7b80fb110 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7ae7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 0x7fa7b810e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7ae7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 0x7fa7b810e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55724/0 (socket says 172.21.15.81:55724) 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7ae7fc640 1 -- 172.21.15.81:0/3072989842 learned_addr learned my addr 172.21.15.81:0/3072989842 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7b6ffd640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7b80fb110 0x7fa7b810dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7ae7fc640 1 -- 172.21.15.81:0/3072989842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7b80fb110 msgr2=0x7fa7b810dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7ae7fc640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7b80fb110 0x7fa7b810dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7ae7fc640 1 -- 172.21.15.81:0/3072989842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a8005040 con 0x7fa7b8101140 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7b6ffd640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7b80fb110 0x7fa7b810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7ae7fc640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 0x7fa7b810e420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000b790 tx=0x7fa7a00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7affff640 1 -- 172.21.15.81:0/3072989842 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a000c700 con 0x7fa7b8101140 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7affff640 1 -- 172.21.15.81:0/3072989842 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7a000cd30 con 0x7fa7b8101140 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b8116f50 con 0x7fa7b8101140 2024-03-22T07:20:31.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7affff640 1 -- 172.21.15.81:0/3072989842 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a0013370 con 0x7fa7b8101140 2024-03-22T07:20:31.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.297+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b8117280 con 0x7fa7b8101140 2024-03-22T07:20:31.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.298+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b80fc800 con 0x7fa7b8101140 2024-03-22T07:20:31.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.298+0000 7fa7affff640 1 -- 172.21.15.81:0/3072989842 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fa7a0012070 con 0x7fa7b8101140 2024-03-22T07:20:31.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.299+0000 7fa7affff640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa7a406d2f0 0x7fa7a406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:31.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.299+0000 7fa7affff640 1 -- 172.21.15.81:0/3072989842 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa7a008c5b0 con 0x7fa7b8101140 2024-03-22T07:20:31.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.299+0000 7fa7b6ffd640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa7a406d2f0 0x7fa7a406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:31.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.299+0000 7fa7b6ffd640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa7a406d2f0 0x7fa7a406f7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8004540 tx=0x7fa7a8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:31.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.303+0000 7fa7affff640 1 -- 172.21.15.81:0/3072989842 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7a0055fe0 con 0x7fa7b8101140 2024-03-22T07:20:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:31 smithi149 ceph-mon[83447]: pgmap v271: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:31 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1760455544' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-03-22T07:20:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:31 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/4053217038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:20:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:31 smithi149 ceph-mon[83447]: from='mgr.34104 172.21.15.81:0/4146674514' entity='mgr.smithi081.bcxarw' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:31.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:31 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/1235496691' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:20:31.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.475+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa7b810ec90 con 0x7fa7b8101140 2024-03-22T07:20:31.475 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.475+0000 7fa7affff640 1 -- 172.21.15.81:0/3072989842 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa7a00596a0 con 0x7fa7b8101140 2024-03-22T07:20:31.476 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-03-22T07:20:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa7a406d2f0 msgr2=0x7fa7a406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa7a406d2f0 0x7fa7a406f7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8004540 tx=0x7fa7a8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 msgr2=0x7fa7b810e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 0x7fa7b810e420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000b790 tx=0x7fa7a00091d0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 shutdown_connections 2024-03-22T07:20:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fa7a406d2f0 0x7fa7a406f7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b8101140 0x7fa7b810e420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 --2- 172.21.15.81:0/3072989842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7b80fb110 0x7fa7b810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 >> 172.21.15.81:0/3072989842 conn(0x7fa7b80710b0 msgr2=0x7fa7b80f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:31.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 shutdown_connections 2024-03-22T07:20:31.479 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.477+0000 7fa7bd99d640 1 -- 172.21.15.81:0/3072989842 wait complete. 2024-03-22T07:20:31.488 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T07:20:31.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.586+0000 7f65f3d5d640 1 -- 172.21.15.81:0/623126458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec0fb110 msgr2=0x7f65ec0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.586+0000 7f65f3d5d640 1 --2- 172.21.15.81:0/623126458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec0fb110 0x7f65ec0fb560 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0048d0 tx=0x7f65dc009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.586+0000 7f65f3d5d640 1 -- 172.21.15.81:0/623126458 shutdown_connections 2024-03-22T07:20:31.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.586+0000 7f65f3d5d640 1 --2- 172.21.15.81:0/623126458 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec0fb110 0x7f65ec0fb560 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.586+0000 7f65f3d5d640 1 --2- 172.21.15.81:0/623126458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec101140 0x7f65ec101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.589 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.586+0000 7f65f3d5d640 1 -- 172.21.15.81:0/623126458 >> 172.21.15.81:0/623126458 conn(0x7f65ec0710b0 msgr2=0x7f65ec0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f3d5d640 1 -- 172.21.15.81:0/623126458 shutdown_connections 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f3d5d640 1 -- 172.21.15.81:0/623126458 wait complete. 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f3d5d640 1 Processor -- start 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f3d5d640 1 -- start start 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f3d5d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0fb110 0x7f65ec10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f3d5d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec101140 0x7f65ec10e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f3d5d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec065310 con 0x7f65ec101140 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f3d5d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec10e930 con 0x7f65ec0fb110 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f12f8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec101140 0x7f65ec10e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f12f8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec101140 0x7f65ec10e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55748/0 (socket says 172.21.15.81:55748) 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.587+0000 7f65f12f8640 1 -- 172.21.15.81:0/3337467045 learned_addr learned my addr 172.21.15.81:0/3337467045 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.588+0000 7f65f12f8640 1 -- 172.21.15.81:0/3337467045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0fb110 msgr2=0x7f65ec10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.590 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.588+0000 7f65f12f8640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0fb110 0x7f65ec10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.588+0000 7f65f12f8640 1 -- 172.21.15.81:0/3337467045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc005040 con 0x7f65ec101140 2024-03-22T07:20:31.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.588+0000 7f65f12f8640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec101140 0x7f65ec10e3f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0041d0 tx=0x7f65dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:31.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.588+0000 7f65e27fc640 1 -- 172.21.15.81:0/3337467045 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65dc030870 con 0x7f65ec101140 2024-03-22T07:20:31.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.588+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec10ebb0 con 0x7f65ec101140 2024-03-22T07:20:31.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.588+0000 7f65e27fc640 1 -- 172.21.15.81:0/3337467045 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65dc0092e0 con 0x7f65ec101140 2024-03-22T07:20:31.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.588+0000 7f65e27fc640 1 -- 172.21.15.81:0/3337467045 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65dc043400 con 0x7f65ec101140 2024-03-22T07:20:31.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.588+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec117170 con 0x7f65ec101140 2024-03-22T07:20:31.591 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.589+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec0fc800 con 0x7f65ec101140 2024-03-22T07:20:31.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.589+0000 7f65e27fc640 1 -- 172.21.15.81:0/3337467045 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f65dc03b420 con 0x7f65ec101140 2024-03-22T07:20:31.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.589+0000 7f65e27fc640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f65d806d2f0 0x7f65d806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:31.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.590+0000 7f65e27fc640 1 -- 172.21.15.81:0/3337467045 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f65dc0af7b0 con 0x7f65ec101140 2024-03-22T07:20:31.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.590+0000 7f65f1af9640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f65d806d2f0 0x7f65d806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:31.592 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.590+0000 7f65f1af9640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f65d806d2f0 0x7f65d806f7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f65e8004c80 tx=0x7f65e8009c50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:31.594 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.594+0000 7f65e27fc640 1 -- 172.21.15.81:0/3337467045 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65dc079260 con 0x7f65ec101140 2024-03-22T07:20:31.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.778+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f65ec04bcc0 con 0x7f65ec101140 2024-03-22T07:20:31.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.778+0000 7f65e27fc640 1 -- 172.21.15.81:0/3337467045 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v85) v1 ==== 81+0+2310 (secure 0 0 0) 0x7f65dc07c920 con 0x7f65ec101140 2024-03-22T07:20:31.779 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6829/268578316 2024-03-23T07:17:11.575629+0000 2024-03-22T07:20:31.779 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:6824/3521694486 2024-03-23T07:17:11.575646+0000 2024-03-22T07:20:31.779 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1030883466 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:31.779 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:6829/18108693 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:31.779 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/3192935300 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:31.779 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:6828/18108693 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:31.779 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6828/268578316 2024-03-23T07:17:11.575629+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/742540357 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1980154994 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/2249539268 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/2774978653 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6827/2831536232 2024-03-23T07:06:37.046400+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/4051369418 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:0/3423294147 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/742540357 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/2438362157 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/473242716 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1138486583 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:6825/3521694486 2024-03-23T07:17:11.575646+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/4051369418 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:31.780 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/1870922013 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:0/3477455875 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6826/2831536232 2024-03-23T07:06:37.046400+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/683056633 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/1870922013 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/2809233037 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/4111281059 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/4111281059 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1939667879 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/4180230166 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:0/594572994 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/3756300008 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/2472307158 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:31.781 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:0/749736269 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:31.782 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/988274144 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:31.782 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/770908409 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:31.782 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1704456246 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:31.782 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/2774978653 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:31.782 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1143393440 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:listed 39 entries 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f65d806d2f0 msgr2=0x7f65d806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f65d806d2f0 0x7f65d806f7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f65e8004c80 tx=0x7f65e8009c50 comp rx=0 tx=0).stop 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec101140 msgr2=0x7f65ec10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec101140 0x7f65ec10e3f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0041d0 tx=0x7f65dc009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 shutdown_connections 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f65d806d2f0 0x7f65d806f7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ec101140 0x7f65ec10e3f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 --2- 172.21.15.81:0/3337467045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0fb110 0x7f65ec10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 >> 172.21.15.81:0/3337467045 conn(0x7f65ec0710b0 msgr2=0x7f65ec0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:31.783 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 shutdown_connections 2024-03-22T07:20:31.784 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:31.780+0000 7f65f3d5d640 1 -- 172.21.15.81:0/3337467045 wait complete. 2024-03-22T07:20:31.791 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T07:20:31.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:31.792 DEBUG:teuthology.orchestra.run.smithi081:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T07:20:31.819 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:20:31.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:31.820 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T07:20:31.913 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:20:31.914 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:20:31.948 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-22T07:20:31.949 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:20:32.029 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T07:20:32.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.152+0000 7f6de4e6e640 1 -- 172.21.15.81:0/3693483524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de00fb110 msgr2=0x7f6de00fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.152+0000 7f6de4e6e640 1 --2- 172.21.15.81:0/3693483524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de00fb110 0x7f6de00fb560 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80048d0 tx=0x7f6dc8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.152+0000 7f6de4e6e640 1 -- 172.21.15.81:0/3693483524 shutdown_connections 2024-03-22T07:20:32.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.152+0000 7f6de4e6e640 1 --2- 172.21.15.81:0/3693483524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de00fb110 0x7f6de00fb560 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 --2- 172.21.15.81:0/3693483524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de0101140 0x7f6de0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.155 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 -- 172.21.15.81:0/3693483524 >> 172.21.15.81:0/3693483524 conn(0x7f6de0071130 msgr2=0x7f6de0071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 -- 172.21.15.81:0/3693483524 shutdown_connections 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 -- 172.21.15.81:0/3693483524 wait complete. 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 Processor -- start 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 -- start start 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de00fb110 0x7f6de0111ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de0101140 0x7f6de01120e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01126e0 con 0x7f6de0101140 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6de4e6e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de010bd30 con 0x7f6de00fb110 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6dd7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de0101140 0x7f6de01120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6dd7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de0101140 0x7f6de01120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55770/0 (socket says 172.21.15.81:55770) 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6dd7fff640 1 -- 172.21.15.81:0/2424990436 learned_addr learned my addr 172.21.15.81:0/2424990436 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.153+0000 7f6dde59c640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de00fb110 0x7f6de0111ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.154+0000 7f6dd7fff640 1 -- 172.21.15.81:0/2424990436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de00fb110 msgr2=0x7f6de0111ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.154+0000 7f6dd7fff640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de00fb110 0x7f6de0111ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.154+0000 7f6dd7fff640 1 -- 172.21.15.81:0/2424990436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8005040 con 0x7f6de0101140 2024-03-22T07:20:32.156 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.154+0000 7f6dd7fff640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de0101140 0x7f6de01120e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80041d0 tx=0x7f6dc8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:32.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.154+0000 7f6dd6ffd640 1 -- 172.21.15.81:0/2424990436 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dc8030870 con 0x7f6de0101140 2024-03-22T07:20:32.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.154+0000 7f6dd6ffd640 1 -- 172.21.15.81:0/2424990436 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dc80092e0 con 0x7f6de0101140 2024-03-22T07:20:32.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.154+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de010bfb0 con 0x7f6de0101140 2024-03-22T07:20:32.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.154+0000 7f6dd6ffd640 1 -- 172.21.15.81:0/2424990436 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dc8043400 con 0x7f6de0101140 2024-03-22T07:20:32.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.154+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de010c4d0 con 0x7f6de0101140 2024-03-22T07:20:32.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.155+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de00fc800 con 0x7f6de0101140 2024-03-22T07:20:32.157 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.155+0000 7f6dd6ffd640 1 -- 172.21.15.81:0/2424990436 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f6dc803b420 con 0x7f6de0101140 2024-03-22T07:20:32.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.156+0000 7f6dd6ffd640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6dcc06d2f0 0x7f6dcc06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:32.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.156+0000 7f6dd6ffd640 1 -- 172.21.15.81:0/2424990436 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6dc80af7d0 con 0x7f6de0101140 2024-03-22T07:20:32.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.156+0000 7f6dde59c640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6dcc06d2f0 0x7f6dcc06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:32.158 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.156+0000 7f6dde59c640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6dcc06d2f0 0x7f6dcc06f7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0004eb0 tx=0x7f6dd000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:32.160 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.160+0000 7f6dd6ffd640 1 -- 172.21.15.81:0/2424990436 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6dc8079340 con 0x7f6de0101140 2024-03-22T07:20:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:32 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/33312526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:20:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:32 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3072989842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:20:32.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:32 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3337467045' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T07:20:32.334 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.333+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6de0108230 con 0x7f6de0101140 2024-03-22T07:20:32.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.335+0000 7f6dd6ffd640 1 -- 172.21.15.81:0/2424990436 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16642 (secure 0 0 0) 0x7f6dc807ca00 con 0x7f6de0101140 2024-03-22T07:20:32.336 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-03-22T07:20:32.336 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":85,"fsid":"a42e8632-e819-11ee-95cd-87774f69a715","created":"2024-03-22T06:59:35.415160+0000","modified":"2024-03-22T07:18:04.460156+0000","last_up_change":"2024-03-22T07:17:09.537521+0000","last_in_change":"2024-03-22T07:04:44.637120+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:03:49.371549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T07:06:20.045033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T07:06:20.879130+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"69addaec-0295-4219-9e22-5ebf831a65d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6803","nonce":2753302858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6805","nonce":2753302858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6809","nonce":2753302858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2753302858},{"type":"v1","addr":"172.21.15.81:6807","nonce":2753302858}]},"public_addr":"172.21.15.81:6803/2753302858","cluster_addr":"172.21.15.81:6805/2753302858","heartbeat_back_addr":"172.21.15.81:6809/2753302858","heartbeat_front_addr":"172.21.15.81:6807/2753302858","state":["exists","up"]},{"osd":1,"uuid":"cd572358-1fd5-4e96-8caf-3d00d6e0608f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6811","nonce":356075948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6813","nonce":356075948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6817","nonce":356075948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":356075948},{"type":"v1","addr":"172.21.15.81:6815","nonce":356075948}]},"public_addr":"172.21.15.81:6811/356075948","cluster_addr":"172.21.15.81:6813/356075948","heartbeat_back_addr":"172.21.15.81:6817/356075948","heartbeat_front_addr":"172.21.15.81:6815/356075948","state":["exists","up"]},{"osd":2,"uuid":"ba6f9b7e-712d-4f4b-8b33-6e051daf2875","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6819","nonce":3743751332}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6821","nonce":3743751332}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6825","nonce":3743751332}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3743751332},{"type":"v1","addr":"172.21.15.81:6823","nonce":3743751332}]},"public_addr":"172.21.15.81:6819/3743751332","cluster_addr":"172.21.15.81:6821/3743751332","heartbeat_back_addr":"172.21.15.81:6825/3743751332","heartbeat_front_addr":"172.21.15.81:6823/3743751332","state":["exists","up"]},{"osd":3,"uuid":"e5adfd7b-e190-45b4-b8ca-da7c45d21d93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6801","nonce":1325806457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6803","nonce":1325806457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6807","nonce":1325806457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":1325806457},{"type":"v1","addr":"172.21.15.149:6805","nonce":1325806457}]},"public_addr":"172.21.15.149:6801/1325806457","cluster_addr":"172.21.15.149:6803/1325806457","heartbeat_back_addr":"172.21.15.149:6807/1325806457","heartbeat_front_addr":"172.21.15.149:6805/1325806457","state":["exists","up"]},{"osd":4,"uuid":"a85bb681-b621-4fbf-b77a-cbd090a41a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6809","nonce":1122192339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6811","nonce":1122192339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6815","nonce":1122192339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":1122192339},{"type":"v1","addr":"172.21.15.149:6813","nonce":1122192339}]},"public_addr":"172.21.15.149:6809/1122192339","cluster_addr":"172.21.15.149:6811/1122192339","heartbeat_back_addr":"172.21.15.149:6815/1122192339","heartbeat_front_addr":"172.21.15.149:6813/1122192339","state":["exists","up"]},{"osd":5,"uuid":"0e83552d-4e6c-4426-9b7c-99d9ad5c45ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6817","nonce":530633899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6819","nonce":530633899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6823","nonce":530633899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":530633899},{"type":"v1","addr":"172.21.15.149:6821","nonce":530633899}]},"public_addr":"172.21.15.149:6817/530633899","cluster_addr":"172.21.15.149:6819/530633899","heartbeat_back_addr":"172.21.15.149:6823/530633899","heartbeat_front_addr":"172.21.15.149:6821/530633899","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:02:52.797148+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:18.980557+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:03:46.975511+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:10.893766+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:04:36.048181+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:05:01.848799+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6829/268578316":"2024-03-23T07:17:11.575629+0000","172.21.15.149:6824/3521694486":"2024-03-23T07:17:11.575646+0000","172.21.15.81:0/1030883466":"2024-03-23T07:00:35.069960+0000","172.21.15.149:6829/18108693":"2024-03-23T07:10:12.272683+0000","172.21.15.81:0/3192935300":"2024-03-23T07:08:38.778309+0000","172.21.15.149:6828/18108693":"2024-03-23T07:10:12.272683+0000","172.21.15.81:6828/268578316":"2024-03-23T07:17:11.575629+0000","172.21.15.81:6801/742540357":"2024-03-23T07:08:38.778309+0000","172.21.15.81:0/1980154994":"2024-03-23T07:08:38.778309+0000","172.21.15.81:0/2249539268":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6800/2774978653":"2024-03-23T07:12:45.787752+0000","172.21.15.81:6827/2831536232":"2024-03-23T07:06:37.046400+0000","172.21.15.81:6801/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.149:0/3423294147":"2024-03-23T07:10:12.272683+0000","172.21.15.81:6800/742540357":"2024-03-23T07:08:38.778309+0000","172.21.15.81:0/2438362157":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/473242716":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/1138486583":"2024-03-23T07:12:45.787752+0000","172.21.15.149:6825/3521694486":"2024-03-23T07:17:11.575646+0000","172.21.15.81:6800/4051369418":"2024-03-23T07:00:35.069960+0000","172.21.15.81:6800/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.149:0/3477455875":"2024-03-23T07:10:12.272683+0000","172.21.15.81:6826/2831536232":"2024-03-23T07:06:37.046400+0000","172.21.15.81:0/683056633":"2024-03-23T07:12:45.787752+0000","172.21.15.81:6801/1870922013":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/2809233037":"2024-03-23T07:08:38.778309+0000","172.21.15.81:6800/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:6801/4111281059":"2024-03-23T07:00:01.117993+0000","172.21.15.81:0/1939667879":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/4180230166":"2024-03-23T07:08:38.778309+0000","172.21.15.149:0/594572994":"2024-03-23T07:10:12.272683+0000","172.21.15.81:0/3756300008":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/2472307158":"2024-03-23T07:00:01.117993+0000","172.21.15.149:0/749736269":"2024-03-23T07:10:12.272683+0000","172.21.15.81:0/988274144":"2024-03-23T07:01:39.049837+0000","172.21.15.81:0/770908409":"2024-03-23T07:00:35.069960+0000","172.21.15.81:0/1704456246":"2024-03-23T07:12:45.787752+0000","172.21.15.81:6801/2774978653":"2024-03-23T07:12:45.787752+0000","172.21.15.81:0/1143393440":"2024-03-23T07:12:45.787752+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:20:32.337 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6dcc06d2f0 msgr2=0x7f6dcc06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.337 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6dcc06d2f0 0x7f6dcc06f7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0004eb0 tx=0x7f6dd000b040 comp rx=0 tx=0).stop 2024-03-22T07:20:32.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de0101140 msgr2=0x7f6de01120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de0101140 0x7f6de01120e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80041d0 tx=0x7f6dc8009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 shutdown_connections 2024-03-22T07:20:32.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f6dcc06d2f0 0x7f6dcc06f7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6de0101140 0x7f6de01120e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 --2- 172.21.15.81:0/2424990436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6de00fb110 0x7f6de0111ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 >> 172.21.15.81:0/2424990436 conn(0x7f6de0071130 msgr2=0x7f6de00f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:32.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 shutdown_connections 2024-03-22T07:20:32.338 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.337+0000 7f6de4e6e640 1 -- 172.21.15.81:0/2424990436 wait complete. 2024-03-22T07:20:32.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:03:49.371549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T07:06:20.045033+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T07:06:20.879130+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T07:20:32.348 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T07:20:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:32 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/33312526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:20:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:32 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3072989842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:20:32.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:32 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3337467045' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 -- 172.21.15.81:0/1312578575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f74101c80 msgr2=0x7f2f740fb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 --2- 172.21.15.81:0/1312578575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f74101c80 0x7f2f740fb310 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2f640048d0 tx=0x7f2f64009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 -- 172.21.15.81:0/1312578575 shutdown_connections 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 --2- 172.21.15.81:0/1312578575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f74101c80 0x7f2f740fb310 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 --2- 172.21.15.81:0/1312578575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f741012e0 0x7f2f741016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 -- 172.21.15.81:0/1312578575 >> 172.21.15.81:0/1312578575 conn(0x7f2f740f6e70 msgr2=0x7f2f740f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 -- 172.21.15.81:0/1312578575 shutdown_connections 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 -- 172.21.15.81:0/1312578575 wait complete. 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 Processor -- start 2024-03-22T07:20:32.451 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.448+0000 7f2f7c59d640 1 -- start start 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7c59d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f741012e0 0x7f2f74112f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7c59d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f74101c80 0x7f2f74113490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7c59d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74113a90 con 0x7f2f741012e0 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7c59d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7410d0e0 con 0x7f2f74101c80 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7a339640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f741012e0 0x7f2f74112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7a339640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f741012e0 0x7f2f74112f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55792/0 (socket says 172.21.15.81:55792) 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7a339640 1 -- 172.21.15.81:0/3201249744 learned_addr learned my addr 172.21.15.81:0/3201249744 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f79b38640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f74101c80 0x7f2f74113490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7a339640 1 -- 172.21.15.81:0/3201249744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f74101c80 msgr2=0x7f2f74113490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7a339640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f74101c80 0x7f2f74113490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7a339640 1 -- 172.21.15.81:0/3201249744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64005040 con 0x7f2f741012e0 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f79b38640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f74101c80 0x7f2f74113490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7a339640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f741012e0 0x7f2f74112f50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c830 tx=0x7f2f6c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f6affd640 1 -- 172.21.15.81:0/3201249744 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c00a850 con 0x7f2f741012e0 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f6affd640 1 -- 172.21.15.81:0/3201249744 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f6c00ae80 con 0x7f2f741012e0 2024-03-22T07:20:32.452 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7410d3c0 con 0x7f2f741012e0 2024-03-22T07:20:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f6affd640 1 -- 172.21.15.81:0/3201249744 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c012ea0 con 0x7f2f741012e0 2024-03-22T07:20:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.449+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7410d8b0 con 0x7f2f741012e0 2024-03-22T07:20:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.450+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f740fca50 con 0x7f2f741012e0 2024-03-22T07:20:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.451+0000 7f2f6affd640 1 -- 172.21.15.81:0/3201249744 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f2f6c012400 con 0x7f2f741012e0 2024-03-22T07:20:32.453 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.451+0000 7f2f6affd640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2f6006d2f0 0x7f2f6006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:32.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.451+0000 7f2f6affd640 1 -- 172.21.15.81:0/3201249744 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2f6c08cae0 con 0x7f2f741012e0 2024-03-22T07:20:32.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.451+0000 7f2f79b38640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2f6006d2f0 0x7f2f6006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:32.454 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.452+0000 7f2f79b38640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2f6006d2f0 0x7f2f6006f7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f740f9890 tx=0x7f2f64009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:32.456 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.456+0000 7f2f6affd640 1 -- 172.21.15.81:0/3201249744 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f6c056630 con 0x7f2f741012e0 2024-03-22T07:20:32.629 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.629+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f740638c0 con 0x7f2f741012e0 2024-03-22T07:20:32.630 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.629+0000 7f2f6affd640 1 -- 172.21.15.81:0/3201249744 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f6c0126b0 con 0x7f2f741012e0 2024-03-22T07:20:32.630 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2024-03-22T07:20:32.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2f6006d2f0 msgr2=0x7f2f6006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2f6006d2f0 0x7f2f6006f7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f740f9890 tx=0x7f2f64009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f741012e0 msgr2=0x7f2f74112f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f741012e0 0x7f2f74112f50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c830 tx=0x7f2f6c004430 comp rx=0 tx=0).stop 2024-03-22T07:20:32.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 shutdown_connections 2024-03-22T07:20:32.632 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f2f6006d2f0 0x7f2f6006f7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f74101c80 0x7f2f74113490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 --2- 172.21.15.81:0/3201249744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f741012e0 0x7f2f74112f50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 >> 172.21.15.81:0/3201249744 conn(0x7f2f740f6e70 msgr2=0x7f2f740f85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:32.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.631+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 shutdown_connections 2024-03-22T07:20:32.633 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.632+0000 7f2f7c59d640 1 -- 172.21.15.81:0/3201249744 wait complete. 2024-03-22T07:20:32.642 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T07:20:32.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 -- 172.21.15.81:0/3099523943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 msgr2=0x7fc48c0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.744 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 --2- 172.21.15.81:0/3099523943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 0x7fc48c0fb560 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc4740048d0 tx=0x7fc474009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 -- 172.21.15.81:0/3099523943 shutdown_connections 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 --2- 172.21.15.81:0/3099523943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 0x7fc48c0fb560 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 --2- 172.21.15.81:0/3099523943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc48c101140 0x7fc48c101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 -- 172.21.15.81:0/3099523943 >> 172.21.15.81:0/3099523943 conn(0x7fc48c0710b0 msgr2=0x7fc48c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 -- 172.21.15.81:0/3099523943 shutdown_connections 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 -- 172.21.15.81:0/3099523943 wait complete. 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 Processor -- start 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 -- start start 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 0x7fc48c10de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc48c101140 0x7fc48c10e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c10e8d0 con 0x7fc48c0fb110 2024-03-22T07:20:32.745 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.742+0000 7fc491e6e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c10ea40 con 0x7fc48c101140 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc48b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 0x7fc48c10de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc48b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 0x7fc48c10de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55802/0 (socket says 172.21.15.81:55802) 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc48b7fe640 1 -- 172.21.15.81:0/622934025 learned_addr learned my addr 172.21.15.81:0/622934025 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc482ffd640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc48c101140 0x7fc48c10e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc48b7fe640 1 -- 172.21.15.81:0/622934025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc48c101140 msgr2=0x7fc48c10e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc48b7fe640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc48c101140 0x7fc48c10e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc48b7fe640 1 -- 172.21.15.81:0/622934025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc474005040 con 0x7fc48c0fb110 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc48b7fe640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 0x7fc48c10de50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc47c002a50 tx=0x7fc47c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc488ff9640 1 -- 172.21.15.81:0/622934025 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc47c007c30 con 0x7fc48c0fb110 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc488ff9640 1 -- 172.21.15.81:0/622934025 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc47c00f040 con 0x7fc48c0fb110 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc488ff9640 1 -- 172.21.15.81:0/622934025 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc47c00dbe0 con 0x7fc48c0fb110 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c116ec0 con 0x7fc48c0fb110 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.743+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c1171f0 con 0x7fc48c0fb110 2024-03-22T07:20:32.746 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.744+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c0fc800 con 0x7fc48c0fb110 2024-03-22T07:20:32.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.744+0000 7fc488ff9640 1 -- 172.21.15.81:0/622934025 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fc47c00dd40 con 0x7fc48c0fb110 2024-03-22T07:20:32.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.745+0000 7fc488ff9640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc47806d2f0 0x7fc47806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:32.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.745+0000 7fc488ff9640 1 -- 172.21.15.81:0/622934025 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc47c08c830 con 0x7fc48c0fb110 2024-03-22T07:20:32.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.745+0000 7fc482ffd640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc47806d2f0 0x7fc47806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:32.748 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.745+0000 7fc482ffd640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc47806d2f0 0x7fc47806f7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc474009fa0 tx=0x7fc474009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:32.749 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.749+0000 7fc488ff9640 1 -- 172.21.15.81:0/622934025 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc47c0562e0 con 0x7fc48c0fb110 2024-03-22T07:20:32.922 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.922+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fc48c04bcc0 con 0x7fc48c0fb110 2024-03-22T07:20:32.923 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.922+0000 7fc488ff9640 1 -- 172.21.15.81:0/622934025 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc47c0599a0 con 0x7fc48c0fb110 2024-03-22T07:20:32.923 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-03-22T07:20:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc47806d2f0 msgr2=0x7fc47806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc47806d2f0 0x7fc47806f7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc474009fa0 tx=0x7fc474009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 msgr2=0x7fc48c10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 0x7fc48c10de50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc47c002a50 tx=0x7fc47c004600 comp rx=0 tx=0).stop 2024-03-22T07:20:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 shutdown_connections 2024-03-22T07:20:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7fc47806d2f0 0x7fc47806f7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc48c101140 0x7fc48c10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 --2- 172.21.15.81:0/622934025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc48c0fb110 0x7fc48c10de50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:32.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 >> 172.21.15.81:0/622934025 conn(0x7fc48c0710b0 msgr2=0x7fc48c0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:32.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.924+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 shutdown_connections 2024-03-22T07:20:32.926 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:32.925+0000 7fc491e6e640 1 -- 172.21.15.81:0/622934025 wait complete. 2024-03-22T07:20:32.935 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T07:20:33.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.036+0000 7f298de2e640 1 -- 172.21.15.81:0/189607641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 msgr2=0x7f2988102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:33.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.036+0000 7f298de2e640 1 --2- 172.21.15.81:0/189607641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 0x7f2988102610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f29780048d0 tx=0x7f2978009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.036+0000 7f298de2e640 1 -- 172.21.15.81:0/189607641 shutdown_connections 2024-03-22T07:20:33.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.036+0000 7f298de2e640 1 --2- 172.21.15.81:0/189607641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29880643a0 0x7f29880647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.036+0000 7f298de2e640 1 --2- 172.21.15.81:0/189607641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 0x7f2988102610 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.036+0000 7f298de2e640 1 -- 172.21.15.81:0/189607641 >> 172.21.15.81:0/189607641 conn(0x7f29880710b0 msgr2=0x7f29880714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:33.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.036+0000 7f298de2e640 1 -- 172.21.15.81:0/189607641 shutdown_connections 2024-03-22T07:20:33.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.036+0000 7f298de2e640 1 -- 172.21.15.81:0/189607641 wait complete. 2024-03-22T07:20:33.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.036+0000 7f298de2e640 1 Processor -- start 2024-03-22T07:20:33.038 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f298de2e640 1 -- start start 2024-03-22T07:20:33.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f298de2e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29880643a0 0x7f2988112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:33.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f298de2e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 0x7f2988113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:33.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f298de2e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2988113810 con 0x7f2988102240 2024-03-22T07:20:33.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f298de2e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298810ce60 con 0x7f29880643a0 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f2986ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 0x7f2988113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f2986ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 0x7f2988113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55820/0 (socket says 172.21.15.81:55820) 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f2986ffd640 1 -- 172.21.15.81:0/347536450 learned_addr learned my addr 172.21.15.81:0/347536450 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f29877fe640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29880643a0 0x7f2988112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f2986ffd640 1 -- 172.21.15.81:0/347536450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29880643a0 msgr2=0x7f2988112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f2986ffd640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29880643a0 0x7f2988112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f2986ffd640 1 -- 172.21.15.81:0/347536450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2978005040 con 0x7f2988102240 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.037+0000 7f29877fe640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29880643a0 0x7f2988112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.038+0000 7f2986ffd640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 0x7f2988113210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f297c002a50 tx=0x7f297c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.038+0000 7f296ffff640 1 -- 172.21.15.81:0/347536450 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297c007c30 con 0x7f2988102240 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.038+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298810d140 con 0x7f2988102240 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.038+0000 7f296ffff640 1 -- 172.21.15.81:0/347536450 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f297c00f040 con 0x7f2988102240 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.038+0000 7f296ffff640 1 -- 172.21.15.81:0/347536450 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297c00db50 con 0x7f2988102240 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.038+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298810d660 con 0x7f2988102240 2024-03-22T07:20:33.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.039+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29880fd940 con 0x7f2988102240 2024-03-22T07:20:33.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.039+0000 7f296ffff640 1 -- 172.21.15.81:0/347536450 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f297c01c400 con 0x7f2988102240 2024-03-22T07:20:33.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.039+0000 7f296ffff640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f297006d2f0 0x7f297006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:33.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.040+0000 7f296ffff640 1 -- 172.21.15.81:0/347536450 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f297c08c240 con 0x7f2988102240 2024-03-22T07:20:33.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.040+0000 7f29877fe640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f297006d2f0 0x7f297006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:33.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.040+0000 7f29877fe640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f297006d2f0 0x7f297006f7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2978000c00 tx=0x7f2978009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:33.044 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.044+0000 7f296ffff640 1 -- 172.21.15.81:0/347536450 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f297c055c70 con 0x7f2988102240 2024-03-22T07:20:33.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.217+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f2988109320 con 0x7f2988102240 2024-03-22T07:20:33.217 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.217+0000 7f296ffff640 1 -- 172.21.15.81:0/347536450 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f297c059330 con 0x7f2988102240 2024-03-22T07:20:33.218 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2024-03-22T07:20:33.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f297006d2f0 msgr2=0x7f297006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:33.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f297006d2f0 0x7f297006f7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2978000c00 tx=0x7f2978009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.219 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 msgr2=0x7f2988113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:33.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 0x7f2988113210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f297c002a50 tx=0x7f297c004600 comp rx=0 tx=0).stop 2024-03-22T07:20:33.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 shutdown_connections 2024-03-22T07:20:33.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f297006d2f0 0x7f297006f7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2988102240 0x7f2988113210 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 --2- 172.21.15.81:0/347536450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29880643a0 0x7f2988112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 >> 172.21.15.81:0/347536450 conn(0x7f29880710b0 msgr2=0x7f29880f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:33.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 shutdown_connections 2024-03-22T07:20:33.220 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.219+0000 7f298de2e640 1 -- 172.21.15.81:0/347536450 wait complete. 2024-03-22T07:20:33.229 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:33 smithi081 ceph-mon[113965]: pgmap v272: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:33.229 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:33 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/2424990436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:20:33.229 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:33 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/3201249744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:20:33.229 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:33 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/622934025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:20:33.230 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T07:20:33.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.327+0000 7f197c2ae640 1 -- 172.21.15.81:0/760228113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 msgr2=0x7f19741064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:33.329 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.327+0000 7f197c2ae640 1 --2- 172.21.15.81:0/760228113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 0x7f19741064a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f19640048d0 tx=0x7f1964009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.327+0000 7f197c2ae640 1 -- 172.21.15.81:0/760228113 shutdown_connections 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.327+0000 7f197c2ae640 1 --2- 172.21.15.81:0/760228113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 0x7f19741064a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.327+0000 7f197c2ae640 1 --2- 172.21.15.81:0/760228113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19740fd430 0x7f19740fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.327+0000 7f197c2ae640 1 -- 172.21.15.81:0/760228113 >> 172.21.15.81:0/760228113 conn(0x7f19740f6eb0 msgr2=0x7f19740f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.327+0000 7f197c2ae640 1 -- 172.21.15.81:0/760228113 shutdown_connections 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.327+0000 7f197c2ae640 1 -- 172.21.15.81:0/760228113 wait complete. 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f197c2ae640 1 Processor -- start 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f197c2ae640 1 -- start start 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f197c2ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19740fd430 0x7f1974112fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f197c2ae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 0x7f1974113510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f197c2ae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1974113ba0 con 0x7f19740fdd40 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f197c2ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197410d160 con 0x7f19740fd430 2024-03-22T07:20:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f1979849640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 0x7f1974113510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f1979849640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 0x7f1974113510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55842/0 (socket says 172.21.15.81:55842) 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f1979849640 1 -- 172.21.15.81:0/4185798840 learned_addr learned my addr 172.21.15.81:0/4185798840 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f197a04a640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19740fd430 0x7f1974112fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f1979849640 1 -- 172.21.15.81:0/4185798840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19740fd430 msgr2=0x7f1974112fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f1979849640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19740fd430 0x7f1974112fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f1979849640 1 -- 172.21.15.81:0/4185798840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1964005040 con 0x7f19740fdd40 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f197a04a640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19740fd430 0x7f1974112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.328+0000 7f1979849640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 0x7f1974113510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1964004930 tx=0x7f1964009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.329+0000 7f196affd640 1 -- 172.21.15.81:0/4185798840 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19640092b0 con 0x7f19740fdd40 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.329+0000 7f196affd640 1 -- 172.21.15.81:0/4185798840 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f196403b400 con 0x7f19740fdd40 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.329+0000 7f196affd640 1 -- 172.21.15.81:0/4185798840 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19640435b0 con 0x7f19740fdd40 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.329+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197410d3e0 con 0x7f19740fdd40 2024-03-22T07:20:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.329+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197410d8a0 con 0x7f19740fdd40 2024-03-22T07:20:33.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.330+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1974065d90 con 0x7f19740fdd40 2024-03-22T07:20:33.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.330+0000 7f196affd640 1 -- 172.21.15.81:0/4185798840 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 39) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f196402f070 con 0x7f19740fdd40 2024-03-22T07:20:33.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.331+0000 7f196affd640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f196006d2f0 0x7f196006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:33.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.331+0000 7f196affd640 1 -- 172.21.15.81:0/4185798840 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f19640ae7d0 con 0x7f19740fdd40 2024-03-22T07:20:33.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.331+0000 7f197a04a640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f196006d2f0 0x7f196006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:33.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.331+0000 7f197a04a640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f196006d2f0 0x7f196006f7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1970006fd0 tx=0x7f1970008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:33.335 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.335+0000 7f196affd640 1 -- 172.21.15.81:0/4185798840 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1964078200 con 0x7f19740fdd40 2024-03-22T07:20:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:33 smithi149 ceph-mon[83447]: pgmap v272: 65 pgs: 65 active+clean; 298 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:20:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:33 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/2424990436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:20:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:33 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/3201249744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:20:33.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:33 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/622934025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:20:33.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.520+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1974109620 con 0x7f19740fdd40 2024-03-22T07:20:33.521 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.520+0000 7f196affd640 1 -- 172.21.15.81:0/4185798840 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v85) v1 ==== 81+0+2310 (secure 0 0 0) 0x7f196407b8c0 con 0x7f19740fdd40 2024-03-22T07:20:33.521 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6829/268578316 2024-03-23T07:17:11.575629+0000 2024-03-22T07:20:33.521 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:6824/3521694486 2024-03-23T07:17:11.575646+0000 2024-03-22T07:20:33.521 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1030883466 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:33.521 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:6829/18108693 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:33.521 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/3192935300 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:6828/18108693 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6828/268578316 2024-03-23T07:17:11.575629+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/742540357 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1980154994 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/2249539268 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/2774978653 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6827/2831536232 2024-03-23T07:06:37.046400+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/4051369418 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:0/3423294147 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/742540357 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/2438362157 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/473242716 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:33.522 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1138486583 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:6825/3521694486 2024-03-23T07:17:11.575646+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/4051369418 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/1870922013 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:0/3477455875 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6826/2831536232 2024-03-23T07:06:37.046400+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/683056633 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/1870922013 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/2809233037 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6800/4111281059 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/4111281059 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1939667879 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/4180230166 2024-03-23T07:08:38.778309+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:0/594572994 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:33.523 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/3756300008 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:33.524 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/2472307158 2024-03-23T07:00:01.117993+0000 2024-03-22T07:20:33.524 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.149:0/749736269 2024-03-23T07:10:12.272683+0000 2024-03-22T07:20:33.524 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/988274144 2024-03-23T07:01:39.049837+0000 2024-03-22T07:20:33.524 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/770908409 2024-03-23T07:00:35.069960+0000 2024-03-22T07:20:33.524 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1704456246 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:33.524 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6801/2774978653 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:33.524 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:0/1143393440 2024-03-23T07:12:45.787752+0000 2024-03-22T07:20:33.525 INFO:teuthology.orchestra.run.smithi081.stderr:listed 39 entries 2024-03-22T07:20:33.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.522+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f196006d2f0 msgr2=0x7f196006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:33.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.522+0000 7f197c2ae640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f196006d2f0 0x7f196006f7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1970006fd0 tx=0x7f1970008040 comp rx=0 tx=0).stop 2024-03-22T07:20:33.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.522+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 msgr2=0x7f1974113510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:33.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.522+0000 7f197c2ae640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 0x7f1974113510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1964004930 tx=0x7f1964009da0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.522+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 shutdown_connections 2024-03-22T07:20:33.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.522+0000 7f197c2ae640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:6800/2090479429,v1:172.21.15.81:6801/2090479429] conn(0x7f196006d2f0 0x7f196006f7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.525 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.522+0000 7f197c2ae640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19740fdd40 0x7f1974113510 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.522+0000 7f197c2ae640 1 --2- 172.21.15.81:0/4185798840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19740fd430 0x7f1974112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:33.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.522+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 >> 172.21.15.81:0/4185798840 conn(0x7f19740f6eb0 msgr2=0x7f19740f8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:33.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.523+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 shutdown_connections 2024-03-22T07:20:33.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-03-22T07:20:33.523+0000 7f197c2ae640 1 -- 172.21.15.81:0/4185798840 wait complete. 2024-03-22T07:20:33.533 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-22T07:20:33.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:33.534 DEBUG:teuthology.orchestra.run.smithi149:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-22T07:20:33.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi149.front.sepia.ceph.com 2024-03-22T07:20:33.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:33.568 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-22T07:20:33.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:33.649 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2024-03-22T07:20:33.664 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T07:20:33.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:33.664 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T07:20:33.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:33.731 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2024-03-22T07:20:33.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:33.833 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-03-22T07:20:33.851 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-22T07:20:33.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:33.851 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T07:20:33.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:20:33.924 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link delete ceph-brx 2024-03-22T07:20:34.022 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-22T07:20:34.033 INFO:tasks.cephadm:Teardown begin 2024-03-22T07:20:34.033 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:20:34.063 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:20:34.092 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-22T07:20:34.093 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T07:20:34.119 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T07:20:34.148 INFO:tasks.cephadm:Stopping all daemons... 2024-03-22T07:20:34.148 INFO:tasks.cephadm.mon.smithi081:Stopping mon.smithi081... 2024-03-22T07:20:34.149 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081 2024-03-22T07:20:34.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:34 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/347536450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:20:34.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:34 smithi081 ceph-mon[113965]: from='client.? 172.21.15.81:0/4185798840' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T07:20:34.319 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:34 smithi081 systemd[1]: Stopping Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:20:34.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:34 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/347536450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:20:34.352 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:34 smithi149 ceph-mon[83447]: from='client.? 172.21.15.81:0/4185798840' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T07:20:34.795 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:34 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081[113961]: 2024-03-22T07:20:34.468+0000 7f1030d3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi081 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:20:34.795 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:34 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081[113961]: 2024-03-22T07:20:34.468+0000 7f1030d3c700 -1 mon.smithi081@0(leader) e3 *** Got Signal Terminated *** 2024-03-22T07:20:34.795 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:34 smithi081 podman[155438]: 2024-03-22 07:20:34.540276806 +0000 UTC m=+0.194804705 container died 8f07a753bc9464cdde3ad4390ee028b92c3c614c5e4229ec5f117eb439360c50 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-22T07:20:35.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:34 smithi081 podman[155438]: 2024-03-22 07:20:34.986324727 +0000 UTC m=+0.640852674 container cleanup 8f07a753bc9464cdde3ad4390ee028b92c3c614c5e4229ec5f117eb439360c50 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:20:35.069 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:34 smithi081 bash[155438]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081 2024-03-22T07:20:35.404 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service' 2024-03-22T07:20:35.430 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:35 smithi081 podman[155463]: 2024-03-22 07:20:35.179668608 +0000 UTC m=+0.632093446 container remove 8f07a753bc9464cdde3ad4390ee028b92c3c614c5e4229ec5f117eb439360c50 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi081, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-22T07:20:35.431 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:35 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service: Deactivated successfully. 2024-03-22T07:20:35.431 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:35 smithi081 systemd[1]: Stopped Ceph mon.smithi081 for a42e8632-e819-11ee-95cd-87774f69a715. 2024-03-22T07:20:35.431 INFO:journalctl@ceph.mon.smithi081.smithi081.stdout:Mar 22 07:20:35 smithi081 systemd[1]: ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi081.service: Consumed 7.853s CPU time. 2024-03-22T07:20:35.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:20:35.473 INFO:tasks.cephadm.mon.smithi081:Stopped mon.smithi081 2024-03-22T07:20:35.473 INFO:tasks.cephadm.mon.smithi149:Stopping mon.smithi149... 2024-03-22T07:20:35.473 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi149 2024-03-22T07:20:35.638 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:35 smithi149 systemd[1]: Stopping Ceph mon.smithi149 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:20:35.900 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:35 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149[83443]: 2024-03-22T07:20:35.797+0000 7f6355574700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi149 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:20:35.900 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:35 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149[83443]: 2024-03-22T07:20:35.797+0000 7f6355574700 -1 mon.smithi149@1(peon) e3 *** Got Signal Terminated *** 2024-03-22T07:20:36.256 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:35 smithi149 podman[115246]: 2024-03-22 07:20:35.900198394 +0000 UTC m=+0.220535674 container died 2c438525329ca95fc6bea7c9dd650eb10db4efd5a7e76f4545465f8e2d5c916b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=squid-db0330b, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-22T07:20:36.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:36 smithi149 podman[115246]: 2024-03-22 07:20:36.448453992 +0000 UTC m=+0.768791270 container cleanup 2c438525329ca95fc6bea7c9dd650eb10db4efd5a7e76f4545465f8e2d5c916b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True) 2024-03-22T07:20:36.602 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:36 smithi149 bash[115246]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149 2024-03-22T07:20:36.863 INFO:journalctl@ceph.mon.smithi149.smithi149.stdout:Mar 22 07:20:36 smithi149 podman[115267]: 2024-03-22 07:20:36.741828622 +0000 UTC m=+0.842636727 container remove 2c438525329ca95fc6bea7c9dd650eb10db4efd5a7e76f4545465f8e2d5c916b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-mon-smithi149, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-22T07:20:36.883 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@mon.smithi149.service' 2024-03-22T07:20:36.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:20:36.932 INFO:tasks.cephadm.mon.smithi149:Stopped mon.smithi149 2024-03-22T07:20:36.933 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-22T07:20:36.933 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.0 2024-03-22T07:20:37.233 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:36 smithi081 systemd[1]: Stopping Ceph osd.0 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:20:37.569 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:37 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[125830]: 2024-03-22T07:20:37.232+0000 7f3a906bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:20:37.569 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:37 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[125830]: 2024-03-22T07:20:37.232+0000 7f3a906bb700 -1 osd.0 85 *** Got signal Terminated *** 2024-03-22T07:20:37.569 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:37 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0[125830]: 2024-03-22T07:20:37.232+0000 7f3a906bb700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:20:43.069 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:42 smithi081 podman[155570]: 2024-03-22 07:20:42.779173359 +0000 UTC m=+5.647647950 container died 033c080308b6a70f9f5613a0cee5a3a56425efcaff2a19743b2be785bf6ebb4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=squid-db0330b) 2024-03-22T07:20:43.442 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:43 smithi081 podman[155570]: 2024-03-22 07:20:43.166641123 +0000 UTC m=+6.035115714 container cleanup 033c080308b6a70f9f5613a0cee5a3a56425efcaff2a19743b2be785bf6ebb4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b) 2024-03-22T07:20:43.443 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:43 smithi081 bash[155570]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0 2024-03-22T07:20:43.792 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:43 smithi081 podman[155591]: 2024-03-22 07:20:43.443105237 +0000 UTC m=+0.663091314 container remove 033c080308b6a70f9f5613a0cee5a3a56425efcaff2a19743b2be785bf6ebb4e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:20:44.047 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:43 smithi081 podman[155684]: 2024-03-22 07:20:43.695638378 +0000 UTC m=+0.018767116 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:20:44.047 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:43 smithi081 podman[155684]: 2024-03-22 07:20:43.810776464 +0000 UTC m=+0.133905196 container create b738a665ea6549d315d204acf5b97c5ab6f990150706c3e7310e19efc0eb9bf2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-22T07:20:44.047 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:44 smithi081 podman[155684]: 2024-03-22 07:20:44.047306983 +0000 UTC m=+0.370435721 container init b738a665ea6549d315d204acf5b97c5ab6f990150706c3e7310e19efc0eb9bf2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-22T07:20:44.316 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:44 smithi081 podman[155684]: 2024-03-22 07:20:44.051001193 +0000 UTC m=+0.374129930 container start b738a665ea6549d315d204acf5b97c5ab6f990150706c3e7310e19efc0eb9bf2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-22T07:20:44.316 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:44 smithi081 podman[155684]: 2024-03-22 07:20:44.130135039 +0000 UTC m=+0.453263790 container attach b738a665ea6549d315d204acf5b97c5ab6f990150706c3e7310e19efc0eb9bf2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:20:44.569 INFO:journalctl@ceph.osd.0.smithi081.stdout:Mar 22 07:20:44 smithi081 podman[155684]: 2024-03-22 07:20:44.354553606 +0000 UTC m=+0.677682347 container died b738a665ea6549d315d204acf5b97c5ab6f990150706c3e7310e19efc0eb9bf2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.build-date=20240307) 2024-03-22T07:20:45.099 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.0.service' 2024-03-22T07:20:45.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:20:45.141 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-22T07:20:45.141 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-22T07:20:45.141 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.1 2024-03-22T07:20:45.504 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:45 smithi081 systemd[1]: Stopping Ceph osd.1 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:20:45.819 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:45 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[131796]: 2024-03-22T07:20:45.503+0000 7f0643b6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:20:45.819 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:45 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[131796]: 2024-03-22T07:20:45.503+0000 7f0643b6b700 -1 osd.1 85 *** Got signal Terminated *** 2024-03-22T07:20:45.819 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:45 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1[131796]: 2024-03-22T07:20:45.503+0000 7f0643b6b700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:20:51.297 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:51 smithi081 podman[155802]: 2024-03-22 07:20:51.006727197 +0000 UTC m=+5.623005728 container died 8341fe32f3fa1503ef948553b33fec7a5086c6493786664abf074eb09dc4ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.license=GPLv2) 2024-03-22T07:20:51.569 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:51 smithi081 podman[155802]: 2024-03-22 07:20:51.379141933 +0000 UTC m=+5.995420466 container cleanup 8341fe32f3fa1503ef948553b33fec7a5086c6493786664abf074eb09dc4ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-22T07:20:51.569 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:51 smithi081 bash[155802]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1 2024-03-22T07:20:52.049 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:51 smithi081 podman[155819]: 2024-03-22 07:20:51.705699862 +0000 UTC m=+0.694619690 container remove 8341fe32f3fa1503ef948553b33fec7a5086c6493786664abf074eb09dc4ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:20:52.319 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:52 smithi081 podman[155909]: 2024-03-22 07:20:51.953772499 +0000 UTC m=+0.018662736 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:20:52.319 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:52 smithi081 podman[155909]: 2024-03-22 07:20:52.073364103 +0000 UTC m=+0.138254323 container create 1961582ca7497b70bbc2cbdbe47340267ad9cf44d332b8180556da5ce833a63d (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-22T07:20:52.319 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:52 smithi081 podman[155909]: 2024-03-22 07:20:52.293168051 +0000 UTC m=+0.358058285 container init 1961582ca7497b70bbc2cbdbe47340267ad9cf44d332b8180556da5ce833a63d (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:20:52.319 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:52 smithi081 podman[155909]: 2024-03-22 07:20:52.296543703 +0000 UTC m=+0.361433923 container start 1961582ca7497b70bbc2cbdbe47340267ad9cf44d332b8180556da5ce833a63d (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-22T07:20:52.819 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:52 smithi081 podman[155909]: 2024-03-22 07:20:52.384725635 +0000 UTC m=+0.449615870 container attach 1961582ca7497b70bbc2cbdbe47340267ad9cf44d332b8180556da5ce833a63d (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-22T07:20:52.819 INFO:journalctl@ceph.osd.1.smithi081.stdout:Mar 22 07:20:52 smithi081 podman[155909]: 2024-03-22 07:20:52.590326119 +0000 UTC m=+0.655216347 container died 1961582ca7497b70bbc2cbdbe47340267ad9cf44d332b8180556da5ce833a63d (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-22T07:20:53.253 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.1.service' 2024-03-22T07:20:53.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:20:53.295 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-22T07:20:53.296 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-22T07:20:53.296 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.2 2024-03-22T07:20:53.569 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:20:53 smithi081 systemd[1]: Stopping Ceph osd.2 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:20:54.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:20:53 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[137200]: 2024-03-22T07:20:53.649+0000 7fcc24d4c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:20:54.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:20:53 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[137200]: 2024-03-22T07:20:53.649+0000 7fcc24d4c700 -1 osd.2 85 *** Got signal Terminated *** 2024-03-22T07:20:54.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:20:53 smithi081 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2[137200]: 2024-03-22T07:20:53.649+0000 7fcc24d4c700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:20:59.422 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:20:59 smithi081 podman[156013]: 2024-03-22 07:20:59.164384914 +0000 UTC m=+5.638497386 container died 0b8c514dced1e190bfbeaaf930ab6d853a291e717877eef6e79af60cfa22fadf (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-22T07:20:59.701 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:20:59 smithi081 podman[156013]: 2024-03-22 07:20:59.492210407 +0000 UTC m=+5.966322878 container cleanup 0b8c514dced1e190bfbeaaf930ab6d853a291e717877eef6e79af60cfa22fadf (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:20:59.701 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:20:59 smithi081 bash[156013]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2 2024-03-22T07:21:00.069 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:20:59 smithi081 podman[156030]: 2024-03-22 07:20:59.701764334 +0000 UTC m=+0.535832983 container remove 0b8c514dced1e190bfbeaaf930ab6d853a291e717877eef6e79af60cfa22fadf (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-22T07:21:00.380 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:21:00 smithi081 podman[156121]: 2024-03-22 07:21:00.018258634 +0000 UTC m=+0.018218735 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:21:00.380 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:21:00 smithi081 podman[156121]: 2024-03-22 07:21:00.144453749 +0000 UTC m=+0.144413834 container create 1dfafd835ce4e8f15cbab475560317668734d004f515f98bf16f5be1ea0e4c4f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-22T07:21:00.380 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:21:00 smithi081 podman[156121]: 2024-03-22 07:21:00.380432044 +0000 UTC m=+0.380392141 container init 1dfafd835ce4e8f15cbab475560317668734d004f515f98bf16f5be1ea0e4c4f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-db0330b) 2024-03-22T07:21:00.671 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:21:00 smithi081 podman[156121]: 2024-03-22 07:21:00.383808096 +0000 UTC m=+0.383768184 container start 1dfafd835ce4e8f15cbab475560317668734d004f515f98bf16f5be1ea0e4c4f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=squid-db0330b, ceph=True) 2024-03-22T07:21:00.672 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:21:00 smithi081 podman[156121]: 2024-03-22 07:21:00.48043405 +0000 UTC m=+0.480394157 container attach 1dfafd835ce4e8f15cbab475560317668734d004f515f98bf16f5be1ea0e4c4f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , ceph=True) 2024-03-22T07:21:00.975 INFO:journalctl@ceph.osd.2.smithi081.stdout:Mar 22 07:21:00 smithi081 podman[156121]: 2024-03-22 07:21:00.672656712 +0000 UTC m=+0.672616796 container died 1dfafd835ce4e8f15cbab475560317668734d004f515f98bf16f5be1ea0e4c4f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:21:01.270 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.2.service' 2024-03-22T07:21:01.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:21:01.312 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-22T07:21:01.312 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-22T07:21:01.312 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.3 2024-03-22T07:21:01.602 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:01 smithi149 systemd[1]: Stopping Ceph osd.3 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:21:01.958 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[92789]: 2024-03-22T07:21:01.609+0000 7fb1eb769700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:21:01.958 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[92789]: 2024-03-22T07:21:01.609+0000 7fb1eb769700 -1 osd.3 85 *** Got signal Terminated *** 2024-03-22T07:21:01.958 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:01 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3[92789]: 2024-03-22T07:21:01.609+0000 7fb1eb769700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:21:07.227 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:06 smithi149 podman[115377]: 2024-03-22 07:21:06.954334194 +0000 UTC m=+5.448363875 container died c8234f7ed003772449ea65125060011d609cf556e3fc36906eecbefd1409a54f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:21:07.227 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:06 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:06.881+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.144520+0000 front 2024-03-22T07:20:42.144434+0000 (oldest deadline 2024-03-22T07:21:06.244319+0000) 2024-03-22T07:21:07.228 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:06 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:06.900+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:07.602 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:07 smithi149 podman[115377]: 2024-03-22 07:21:07.409581234 +0000 UTC m=+5.903610963 container cleanup c8234f7ed003772449ea65125060011d609cf556e3fc36906eecbefd1409a54f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:21:07.602 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:07 smithi149 bash[115377]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3 2024-03-22T07:21:08.034 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:07 smithi149 podman[115406]: 2024-03-22 07:21:07.694201312 +0000 UTC m=+0.739940620 container remove c8234f7ed003772449ea65125060011d609cf556e3fc36906eecbefd1409a54f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-22T07:21:08.034 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:07 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:07.923+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.144520+0000 front 2024-03-22T07:20:42.144434+0000 (oldest deadline 2024-03-22T07:21:06.244319+0000) 2024-03-22T07:21:08.035 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:07 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:07.867+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:08.289 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:08 smithi149 podman[115496]: 2024-03-22 07:21:07.93702087 +0000 UTC m=+0.017845617 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:21:08.289 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:08 smithi149 podman[115496]: 2024-03-22 07:21:08.061956764 +0000 UTC m=+0.142781547 container create bde51b8b6bc29595a9be069895a52da985c0f8ce9e2dd50ce26f1a76a18856ac (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:21:08.289 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:08 smithi149 podman[115496]: 2024-03-22 07:21:08.289579393 +0000 UTC m=+0.370404137 container init bde51b8b6bc29595a9be069895a52da985c0f8ce9e2dd50ce26f1a76a18856ac (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-03-22T07:21:08.581 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:08 smithi149 podman[115496]: 2024-03-22 07:21:08.292735024 +0000 UTC m=+0.373559770 container start bde51b8b6bc29595a9be069895a52da985c0f8ce9e2dd50ce26f1a76a18856ac (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:21:08.581 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:08 smithi149 podman[115496]: 2024-03-22 07:21:08.373041724 +0000 UTC m=+0.453866473 container attach bde51b8b6bc29595a9be069895a52da985c0f8ce9e2dd50ce26f1a76a18856ac (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-03-22T07:21:08.852 INFO:journalctl@ceph.osd.3.smithi149.stdout:Mar 22 07:21:08 smithi149 podman[115496]: 2024-03-22 07:21:08.582161023 +0000 UTC m=+0.662985767 container died bde51b8b6bc29595a9be069895a52da985c0f8ce9e2dd50ce26f1a76a18856ac (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:21:08.852 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:08 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:08.823+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:09.200 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:08 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:08.954+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.144520+0000 front 2024-03-22T07:20:42.144434+0000 (oldest deadline 2024-03-22T07:21:06.244319+0000) 2024-03-22T07:21:09.221 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.3.service' 2024-03-22T07:21:09.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:21:09.265 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-22T07:21:09.266 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-22T07:21:09.266 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.4 2024-03-22T07:21:09.597 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:09 smithi149 systemd[1]: Stopping Ceph osd.4 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:21:09.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:09 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:09.596+0000 7f5f847d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:21:09.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:09 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:09.596+0000 7f5f847d1700 -1 osd.4 85 *** Got signal Terminated *** 2024-03-22T07:21:09.852 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:09 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:09.596+0000 7f5f847d1700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:21:10.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:09 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:09.917+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.144520+0000 front 2024-03-22T07:20:42.144434+0000 (oldest deadline 2024-03-22T07:21:06.244319+0000) 2024-03-22T07:21:10.352 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:09 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:09.852+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:11.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:10 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:10.901+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.144520+0000 front 2024-03-22T07:20:42.144434+0000 (oldest deadline 2024-03-22T07:21:06.244319+0000) 2024-03-22T07:21:11.352 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:10 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:10.858+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:12.219 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:11 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:11.940+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.144520+0000 front 2024-03-22T07:20:42.144434+0000 (oldest deadline 2024-03-22T07:21:06.244319+0000) 2024-03-22T07:21:12.219 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:11 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:11.940+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:47.945215+0000 front 2024-03-22T07:20:47.945291+0000 (oldest deadline 2024-03-22T07:21:11.445162+0000) 2024-03-22T07:21:12.220 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:11 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:11.863+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:12.220 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:11 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:11.863+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:13.102 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:12 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:12.926+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.144520+0000 front 2024-03-22T07:20:42.144434+0000 (oldest deadline 2024-03-22T07:21:06.244319+0000) 2024-03-22T07:21:13.102 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:12 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:12.926+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:47.945215+0000 front 2024-03-22T07:20:47.945291+0000 (oldest deadline 2024-03-22T07:21:11.445162+0000) 2024-03-22T07:21:13.103 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:12 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:12.845+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:13.103 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:12 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:12.845+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:14.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:13 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:13.971+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.144520+0000 front 2024-03-22T07:20:42.144434+0000 (oldest deadline 2024-03-22T07:21:06.244319+0000) 2024-03-22T07:21:14.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:13 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4[97933]: 2024-03-22T07:21:13.971+0000 7f5f7d5be700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:47.945215+0000 front 2024-03-22T07:20:47.945291+0000 (oldest deadline 2024-03-22T07:21:11.445162+0000) 2024-03-22T07:21:14.353 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:13 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:13.869+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:14.353 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:13 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:13.869+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:15.102 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:15 smithi149 podman[115600]: 2024-03-22 07:21:15.010804437 +0000 UTC m=+5.521049228 container died 437455ba97c044a7e40a8e75a20fb38f262d46d8d10617f36528e5078880d990 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-22T07:21:15.103 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:14 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:14.840+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:15.103 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:14 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:14.840+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:15.539 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:15 smithi149 podman[115600]: 2024-03-22 07:21:15.346941092 +0000 UTC m=+5.857185897 container cleanup 437455ba97c044a7e40a8e75a20fb38f262d46d8d10617f36528e5078880d990 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-db0330b, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:21:15.539 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:15 smithi149 bash[115600]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4 2024-03-22T07:21:15.834 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:15 smithi149 podman[115620]: 2024-03-22 07:21:15.539525753 +0000 UTC m=+0.518874065 container remove 437455ba97c044a7e40a8e75a20fb38f262d46d8d10617f36528e5078880d990 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:21:16.091 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:15 smithi149 podman[115709]: 2024-03-22 07:21:15.862825269 +0000 UTC m=+0.018007285 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:21:16.091 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:15 smithi149 podman[115709]: 2024-03-22 07:21:15.973714596 +0000 UTC m=+0.128896595 container create 30a2a0b39634ea7634101a277dcb156f215b00a7f3cffe5dc0dbf4279bd0b388 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-22T07:21:16.091 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:15 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:15.833+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:16.091 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:15 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:15.833+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:16.352 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:16 smithi149 podman[115709]: 2024-03-22 07:21:16.17605625 +0000 UTC m=+0.331238273 container init 30a2a0b39634ea7634101a277dcb156f215b00a7f3cffe5dc0dbf4279bd0b388 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-22T07:21:16.353 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:16 smithi149 podman[115709]: 2024-03-22 07:21:16.179276203 +0000 UTC m=+0.334458209 container start 30a2a0b39634ea7634101a277dcb156f215b00a7f3cffe5dc0dbf4279bd0b388 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, io.buildah.version=1.33.5) 2024-03-22T07:21:16.353 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:16 smithi149 podman[115709]: 2024-03-22 07:21:16.251004851 +0000 UTC m=+0.406186868 container attach 30a2a0b39634ea7634101a277dcb156f215b00a7f3cffe5dc0dbf4279bd0b388 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:21:16.761 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 22 07:21:16 smithi149 podman[115709]: 2024-03-22 07:21:16.468602032 +0000 UTC m=+0.623784033 container died 30a2a0b39634ea7634101a277dcb156f215b00a7f3cffe5dc0dbf4279bd0b388 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-03-22T07:21:17.020 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:16 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:16.817+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:17.021 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:16 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:16.817+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:17.039 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.4.service' 2024-03-22T07:21:17.080 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:21:17.080 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-22T07:21:17.081 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-22T07:21:17.081 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.5 2024-03-22T07:21:17.352 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:17 smithi149 systemd[1]: Stopping Ceph osd.5 for a42e8632-e819-11ee-95cd-87774f69a715... 2024-03-22T07:21:17.809 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:17 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:17.440+0000 7fa198442700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:21:17.810 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:17 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:17.440+0000 7fa198442700 -1 osd.5 85 *** Got signal Terminated *** 2024-03-22T07:21:17.810 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:17 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:17.440+0000 7fa198442700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:21:17.810 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:17 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:17.809+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:17.810 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:17 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:17.809+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:19.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:18 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:18.791+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:19.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:18 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:18.791+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:20.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:19 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:19.758+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:20.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:19 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:19.758+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:20.103 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:19 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:19.758+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6822 osd.2 since back 2024-03-22T07:20:57.176664+0000 front 2024-03-22T07:20:57.176679+0000 (oldest deadline 2024-03-22T07:21:19.476458+0000) 2024-03-22T07:21:21.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:20 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:20.766+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:21.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:20 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:20.766+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:21.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:20 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:20.766+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6822 osd.2 since back 2024-03-22T07:20:57.176664+0000 front 2024-03-22T07:20:57.176679+0000 (oldest deadline 2024-03-22T07:21:19.476458+0000) 2024-03-22T07:21:22.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:21 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:21.717+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6806 osd.0 since back 2024-03-22T07:20:42.174541+0000 front 2024-03-22T07:20:42.174529+0000 (oldest deadline 2024-03-22T07:21:06.274351+0000) 2024-03-22T07:21:22.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:21 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:21.717+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6814 osd.1 since back 2024-03-22T07:20:50.375280+0000 front 2024-03-22T07:20:50.375298+0000 (oldest deadline 2024-03-22T07:21:11.475181+0000) 2024-03-22T07:21:22.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:21 smithi149 ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5[102975]: 2024-03-22T07:21:21.717+0000 7fa195237700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.81:6822 osd.2 since back 2024-03-22T07:20:57.176664+0000 front 2024-03-22T07:20:57.176679+0000 (oldest deadline 2024-03-22T07:21:19.476458+0000) 2024-03-22T07:21:23.102 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:22 smithi149 podman[115813]: 2024-03-22 07:21:22.775115944 +0000 UTC m=+5.469462355 container died 0d8d31a867d9bb3b9cd8376978336480a433f96771011c8527823a44be9036d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:21:23.602 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:23 smithi149 podman[115813]: 2024-03-22 07:21:23.190292387 +0000 UTC m=+5.884638797 container cleanup 0d8d31a867d9bb3b9cd8376978336480a433f96771011c8527823a44be9036d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-22T07:21:23.602 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:23 smithi149 bash[115813]: ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5 2024-03-22T07:21:23.602 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:23 smithi149 podman[115832]: 2024-03-22 07:21:23.383221793 +0000 UTC m=+0.605783633 container remove 0d8d31a867d9bb3b9cd8376978336480a433f96771011c8527823a44be9036d2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-22T07:21:24.094 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:23 smithi149 podman[115920]: 2024-03-22 07:21:23.710553403 +0000 UTC m=+0.018151156 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:21:24.094 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:23 smithi149 podman[115920]: 2024-03-22 07:21:23.81741459 +0000 UTC m=+0.125012328 container create 224e69ac517ca9b536e8b4324f53224bf4ae827c870b5534f363cf92b6e1ec99 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:21:24.094 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:24 smithi149 podman[115920]: 2024-03-22 07:21:24.011374542 +0000 UTC m=+0.318972290 container init 224e69ac517ca9b536e8b4324f53224bf4ae827c870b5534f363cf92b6e1ec99 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-db0330b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-22T07:21:24.095 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:24 smithi149 podman[115920]: 2024-03-22 07:21:24.014533919 +0000 UTC m=+0.322131658 container start 224e69ac517ca9b536e8b4324f53224bf4ae827c870b5534f363cf92b6e1ec99 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307) 2024-03-22T07:21:24.095 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:24 smithi149 podman[115920]: 2024-03-22 07:21:24.094729549 +0000 UTC m=+0.402327285 container attach 224e69ac517ca9b536e8b4324f53224bf4ae827c870b5534f363cf92b6e1ec99 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-22T07:21:24.352 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:24 smithi149 podman[115920]: 2024-03-22 07:21:24.308882341 +0000 UTC m=+0.616480079 container died 224e69ac517ca9b536e8b4324f53224bf4ae827c870b5534f363cf92b6e1ec99 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b) 2024-03-22T07:21:24.832 INFO:journalctl@ceph.osd.5.smithi149.stdout:Mar 22 07:21:24 smithi149 podman[115946]: 2024-03-22 07:21:24.81478711 +0000 UTC m=+0.500303890 container remove 224e69ac517ca9b536e8b4324f53224bf4ae827c870b5534f363cf92b6e1ec99 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-a42e8632-e819-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:21:24.836 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a42e8632-e819-11ee-95cd-87774f69a715@osd.5.service' 2024-03-22T07:21:24.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:21:24.879 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-22T07:21:24.879 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a42e8632-e819-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T07:21:47.767 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a42e8632-e819-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T07:22:02.080 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:22:02.115 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:22:02.145 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-22T07:22:02.147 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012/remote/smithi081/crash 2024-03-22T07:22:02.148 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/crash -- . 2024-03-22T07:22:02.184 INFO:teuthology.orchestra.run.smithi081.stderr:tar: /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T07:22:02.184 INFO:teuthology.orchestra.run.smithi081.stderr:tar: Error is not recoverable: exiting now 2024-03-22T07:22:02.186 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012/remote/smithi149/crash 2024-03-22T07:22:02.187 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/crash -- . 2024-03-22T07:22:02.213 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/a42e8632-e819-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T07:22:02.213 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-03-22T07:22:02.214 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-22T07:22:02.214 DEBUG:teuthology.orchestra.run.smithi081:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-22T07:22:02.257 INFO:teuthology.orchestra.run.smithi081.stdout:1711091188.6883206 mon.smithi081 (mon.0) 681 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:22:02.258 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-22T07:22:02.258 DEBUG:teuthology.orchestra.run.smithi081:> sudo egrep '\[SEC\]' /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-22T07:22:02.331 DEBUG:teuthology.orchestra.run.smithi081:> sudo egrep '\[ERR\]' /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-22T07:22:02.403 DEBUG:teuthology.orchestra.run.smithi081:> sudo egrep '\[WRN\]' /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-22T07:22:02.476 INFO:teuthology.orchestra.run.smithi081.stdout:1711091188.6883206 mon.smithi081 (mon.0) 681 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:22:02.477 INFO:tasks.cephadm:Compressing logs... 2024-03-22T07:22:02.477 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T07:22:02.520 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T07:22:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mgr.smithi081.bcxarw.log 2024-03-22T07:22:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:: No such file or directory 2024-03-22T07:22:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mon.smithi081.log 2024-03-22T07:22:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-22T07:22:02.545 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mgr.smithi081.bcxarw.log: /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mon.smithi081.log: gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi081.zfgxzy.log 2024-03-22T07:22:02.546 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-22T07:22:02.546 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi081.zfgxzy.log: gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi081.bnzfrv.log 2024-03-22T07:22:02.546 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi149.ffewvr.log: No such file or directory 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mgr.smithi149.sgmtoa.log 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi149.ffewvr.log: /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mon.smithi149.log 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mgr.smithi149.sgmtoa.log: /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.4.log: /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.5.log: /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-22T07:22:02.548 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mon.smithi149.log: gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi149.kdtxfw.log 2024-03-22T07:22:02.553 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi081.bnzfrv.log: gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-22T07:22:02.553 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.0.log: 90.9% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-22T07:22:02.553 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log 2024-03-22T07:22:02.553 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.cephadm.log: 85.3% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-22T07:22:02.554 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-22T07:22:02.554 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.cephadm.log: 85.3% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-22T07:22:02.554 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log 2024-03-22T07:22:02.554 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi149.kdtxfw.log: 91.1% 93.9% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi149.ffewvr.log.gz 2024-03-22T07:22:02.554 INFO:teuthology.orchestra.run.smithi149.stderr: -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-22T07:22:02.554 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-22T07:22:02.554 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-22T07:22:02.555 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-volume.log: 87.4% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-22T07:22:02.556 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log: 87.3% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-22T07:22:02.556 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-22T07:22:02.570 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-22T07:22:02.570 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-22T07:22:02.573 INFO:teuthology.orchestra.run.smithi149.stderr: 90.9% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-22T07:22:02.602 INFO:teuthology.orchestra.run.smithi149.stderr: 88.7% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mgr.smithi149.sgmtoa.log.gz 2024-03-22T07:22:02.604 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/cephadm.log: 90.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-22T07:22:02.615 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-22T07:22:02.912 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mgr.smithi081.bcxarw.log.gz 2024-03-22T07:22:02.933 INFO:teuthology.orchestra.run.smithi149.stderr: 92.5% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mon.smithi149.log.gz 2024-03-22T07:22:03.602 INFO:teuthology.orchestra.run.smithi081.stderr: 90.6% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mon.smithi081.log.gz 2024-03-22T07:22:06.514 INFO:teuthology.orchestra.run.smithi149.stderr: 93.6% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-22T07:22:06.928 INFO:teuthology.orchestra.run.smithi081.stderr: 93.6% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-22T07:22:07.658 INFO:teuthology.orchestra.run.smithi081.stderr: 93.6% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-22T07:22:07.770 INFO:teuthology.orchestra.run.smithi149.stderr: 93.5% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-22T07:22:07.884 INFO:teuthology.orchestra.run.smithi081.stderr: 93.7% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-22T07:22:08.139 INFO:teuthology.orchestra.run.smithi081.stderr: 94.7% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi081.bnzfrv.log.gz 2024-03-22T07:22:08.440 INFO:teuthology.orchestra.run.smithi149.stderr: 93.5% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-22T07:22:26.032 INFO:teuthology.orchestra.run.smithi149.stderr: 93.6% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi149.kdtxfw.log.gz 2024-03-22T07:22:26.034 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-03-22T07:22:26.034 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m23.500s 2024-03-22T07:22:26.034 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m37.564s 2024-03-22T07:22:26.034 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m1.433s 2024-03-22T07:23:10.083 INFO:teuthology.orchestra.run.smithi081.stderr: 93.1% -- replaced with /var/log/ceph/a42e8632-e819-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi081.zfgxzy.log.gz 2024-03-22T07:23:10.085 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-03-22T07:23:10.085 INFO:teuthology.orchestra.run.smithi081.stderr:real 1m7.553s 2024-03-22T07:23:10.085 INFO:teuthology.orchestra.run.smithi081.stderr:user 1m26.415s 2024-03-22T07:23:10.085 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m2.824s 2024-03-22T07:23:10.085 INFO:tasks.cephadm:Archiving logs... 2024-03-22T07:23:10.085 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012/remote/smithi081/log 2024-03-22T07:23:10.087 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T07:23:23.047 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012/remote/smithi149/log 2024-03-22T07:23:23.048 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T07:23:28.765 INFO:tasks.cephadm:Removing cluster... 2024-03-22T07:23:28.765 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a42e8632-e819-11ee-95cd-87774f69a715 --force 2024-03-22T07:23:29.350 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a42e8632-e819-11ee-95cd-87774f69a715 --force 2024-03-22T07:23:29.876 INFO:tasks.cephadm:Removing cephadm ... 2024-03-22T07:23:29.877 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-22T07:23:29.893 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-22T07:23:29.908 INFO:tasks.cephadm:Teardown complete 2024-03-22T07:23:29.909 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-22T07:23:29.921 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-22T07:23:29.921 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T07:23:29.936 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T07:23:30.028 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-22T07:23:30.038 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-22T07:23:30.039 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T07:23:30.041 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T07:23:30.053 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-03-22T07:23:30.055 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-22T07:23:30.057 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T07:23:30.057 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-03-22T07:23:30.057 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 +1568us[+1568us] +/- 104ms 2024-03-22T07:23:30.057 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv02.front.sepia.ceph.com 3 6 377 64 -1035us[-1050us] +/- 50ms 2024-03-22T07:23:30.058 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +331us[ +331us] +/- 75ms 2024-03-22T07:23:30.058 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:23:30.060 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T07:23:30.060 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-22T07:23:30.060 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 3 +1584us[+1584us] +/- 105ms 2024-03-22T07:23:30.060 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 -1033us[-1045us] +/- 50ms 2024-03-22T07:23:30.060 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +319us[ +308us] +/- 75ms 2024-03-22T07:23:30.060 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:23:30.060 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-22T07:23:30.071 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-22T07:23:30.072 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-22T07:23:30.083 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-22T07:23:30.184 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-22T07:23:30.262 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T07:23:30.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:23:30.293 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T07:23:30.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:23:30.324 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-22T07:23:30.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-22T07:23:30.374 INFO:teuthology.task.internal:Duration was 1888.258825 seconds 2024-03-22T07:23:30.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-22T07:23:30.436 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-22T07:23:30.436 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T07:23:30.438 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T07:23:30.479 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T07:23:30.482 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T07:23:30.900 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-22T07:23:30.900 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-03-22T07:23:30.900 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T07:23:30.920 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-03-22T07:23:30.920 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T07:23:30.944 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-22T07:23:30.944 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T07:23:30.962 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T07:23:31.013 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-22T07:23:31.013 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T07:23:31.031 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T07:23:31.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-22T07:23:31.650 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-22T07:23:31.651 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T07:23:31.677 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T07:23:31.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-22T07:23:31.717 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T07:23:31.719 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T07:23:31.742 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-03-22T07:23:31.770 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-03-22T07:23:31.787 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T07:23:31.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:23:31.812 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T07:23:31.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:23:31.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-22T07:23:31.853 INFO:teuthology.task.internal:Transferring archived files... 2024-03-22T07:23:31.854 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012/remote/smithi081 2024-03-22T07:23:31.854 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T07:23:31.904 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012/remote/smithi149 2024-03-22T07:23:31.904 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T07:23:31.944 INFO:teuthology.task.internal:Removing archive directory... 2024-03-22T07:23:31.944 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T07:23:31.946 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T07:23:31.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-22T07:23:32.007 INFO:teuthology.task.internal:Not uploading archives. 2024-03-22T07:23:32.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-22T07:23:32.041 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-22T07:23:32.042 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T07:23:32.044 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T07:23:32.056 INFO:teuthology.orchestra.run.smithi081.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 22 07:23 /home/ubuntu/cephtest 2024-03-22T07:23:32.058 INFO:teuthology.orchestra.run.smithi149.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 22 07:23 /home/ubuntu/cephtest 2024-03-22T07:23:32.059 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-22T07:23:32.071 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-22T07:23:32.096 INFO:teuthology.nuke:Checking targets against current locks 2024-03-22T07:23:32.122 DEBUG:teuthology.nuke:shortname: smithi081 2024-03-22T07:23:32.123 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T07:23:32.145 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 06:42:18.102830', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwNgaw9IvHANiy323QcUSEv0eajn27jRpU8ksYOYRdnRnSKvguX+wrRKylK2EcXxprsG1eAliPTRlvrgTCpyAU='} 2024-03-22T07:23:32.148 DEBUG:teuthology.nuke:shortname: smithi149 2024-03-22T07:23:32.148 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T07:23:32.169 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 06:42:18.104714', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVRdHgXkB1iKextmBEkFFRabpyekhTPCapgDe+3fOyrSJRkuY90+1wptaQFt/JnWt2FpZO/lWsSAOcgDApsGmY='} 2024-03-22T07:23:32.195 INFO:teuthology.orchestra.console.smithi081:Power off 2024-03-22T07:23:32.195 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T07:23:32.219 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off 2024-03-22T07:23:32.219 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:23:32.232 INFO:teuthology.orchestra.console.smithi149:Power off 2024-03-22T07:23:32.233 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T07:23:32.240 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-03-22T07:23:32.254 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-03-22T07:23:32.254 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:23:32.276 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-22T07:23:36.241 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:23:36.278 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:23:36.352 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-03-22T07:23:36.388 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-22T07:23:40.353 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:23:40.389 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:23:40.463 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off 2024-03-22T07:23:40.498 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-22T07:23:40.563 INFO:teuthology.orchestra.console.smithi081:Power off completed 2024-03-22T07:23:44.500 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:23:44.610 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-03-22T07:23:44.711 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-03-22T07:23:44.815 INFO:teuthology.run:Summary data: description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1888.258825302124 failure_reason: '"1711091188.6883206 mon.smithi081 (mon.0) 681 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-03-22T07:23:44.815 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T07:23:44.895 INFO:teuthology.run:FAIL